Re: [Geopriv] Location measurement error

["Thomson, Martin" <Martin.Thomson at andrew.com>]

Ah Brian

Familiar old ground, made interesting by your particular rhetorical flair.

> -----Original Message-----
> From: Brian Rosen [mailto:br at brianrosen.net]
> Sent: Tuesday, 15 July 2008 11:05 PM
> To: Thomson, Martin
> Cc: geopriv at ietf.org
> Subject: RE: [Geopriv] Location measurement error
> 
> Ah Martin
> 
> Allow me to go over some old ground
> 1. No watcher knows what to do with confidence and uncertainty.  I've
> asked
> literally dozens of watchers from all kinds of emergency and
> "pizza-delivery" equivalent services.  They all understood their
> measuring
> device had error.  They ignore it.

Debatable, but hardly relevant, so I wont belabour the point.

> 2. When pressed to deal with it, all watchers want confidence set to at
> least 95%.  It would be okay with them, at least today, if it was fixed
> at
> 95%.  Watchers HATE two measurements with different confidence.
> Allowing
> specification of uncertainty is completely unnecessary, fix it at 95%
> and
> all watchers will be at least content (they won't be happy until it's
> 99%).

I agree on the different confidence - given the non-linear relationship, managing two variables is a pain.

Everyone wants 100%, but fail to acknowledge the tradeoffs.  I think that I can see where you are going - you are making points about what people _want_.  I can appreciate that.

> 3. Watchers specify uncertainty in meters.  That means, to you,
> whatever the
> shape that has the same error value for x and y and a different one for
> z.
> They don't want any shape that doesn't look like that.  There is no
> need,
> from a watcher point of view, to specify uncertainty any other way.

The point that people understand the expression of uncertainty as "within x metres" is relevant.  But what they want is no uncertainty, failing that, they want simple.  I've describe how to accommodate these wishes, while not ignoring the real constraints: <http://tools.ietf.org/html/draft-thomson-geopriv-uncertainty>

> 4. Existing systems don't allow the watcher to specify confidence or
> uncertainty.  They simply report what the owner of the measuring system
> decides to report.  The notion that the error spec goes in either a
> request
> or a filter is a novel idea, and I'm not sure it will be implemented.

Now you've lost me.  Specifying target uncertainty is not especially novel, we've been handling it for almost 10 years now.  Quality of position parameters are a fixture of most existing LCS protocols.  They get pushed down the whole chain.  For example: Application -> GMLC -> MSC -> RNC -> SAS - all the protocols have QoP parameters of various sorts.

I've explained why I think that adding an "error spec" to filters is not useful.

> 5. All PSAPs and all responders in the U.S. have a very well defined
> value
> for confidence and uncertainty for a filter: within 100 meters (x and
> y),
> 95% of the time.

If you are talking FCC mandates, you are still getting them wrong.  In any case, these numbers cannot be applied to an individual call - they are statistical figures that are applied across a large set of calls.  Again, this falls into the "want" basket.

> 6. With a minRate, you cover the case of "best you can do in x
> seconds".
> Without a way to specify error in some way, you can only say that.  If
> you
> want "best you can do", there is no way to say that.  If you want to
> say
> "within 10 meters is good enough" there is no way to say that.

Actually, with a minRate, you specify that the LS update you every x seconds.  Unless there is a way for the LS to push the LG, you only get a better estimate if the LG happens to get one.  That is my point - and one that you seem to be reinforcing.  I'm officially confused now.

> 7. Event notification is a much more acceptable way to get a
> measurement
> that takes seconds to compute.  The error spec in the filter is much
> more
> useful that the error spec in the request.  If you want an error return
> that
> is effectively "you'll never get one with this filter" I would be okay
> with
> that, but see below.

OK, say that I subscribe for location information.  You are saying that specifying a filter is better than cramming some extra parameters in the subscription request.  I can't contest that.  It misses the point, but in and of itself, the argument is sound.

The problem with a filter (or indeed anything related to the subscription and only the subscription) is that whatever you do there, you have no influence over the information.  You get whatever the LG generates.  Without some influence over the location generation, anything you do is essentially pointless.  Unless the LG is given instructions on what to do, you get whatever you get.  Assuming that the LG is independently motivated to generate an extremely accurate result isn't going to work.  The model can't assume that the LS has some secret means to influence the LG.  Unless you specify a means for the LS to do so, which I'd be very interested in.

> 8. The biggest problem with all of the mechanisms that have so far been
> proposed is that they don't include any way for a watcher to determine
> what
> an achievable error could be from a particular LG (or LS, since that is
> what
> it deals with).  Until you come up with a way for the watcher to find
> what
> the measurement mechanism's achievable error spec is, you don't have a
> useful set of mechanisms.

That's because noone knows what the achievable error is.  It's not related to the "mechanism", but to the specific situation: time of day, wind speed and direction, battery levels, indoors/outdoors, access technology, what the owner had for lunch, solar winds, etc.
 
> It occurs to me to ask if "within a 100 meters, 95% of the time" is
> actually
> the same as a circular error shape with a 100 meter radius and a 95%
> confidence.  It sounds like the same thing.  Is it exactly the same
> thing?

Yes.  I'm surprised that this realization took so long, given that the exact statement has been made many times in the past.

> 
> Brian
> 
> > -----Original Message-----
> > From: geopriv-bounces at ietf.org [mailto:geopriv-bounces at ietf.org] On
> Behalf
> > Of Thomson, Martin
> > Sent: Tuesday, July 15, 2008 12:15 AM
> > To: Rosen, Brian
> > Cc: geopriv at ietf.org
> > Subject: Re: [Geopriv] Location measurement error
> >
> > Hi Brian,
> >
> > Once again, I'm pretty sure that we disagree because we were talking
> about
> > subtly different things.  And giving me a lesson on uncertainty and
> > confidence isn't really necessary :)
> >
> > I really needed to understand the use case for this, and I think that
> I am
> > seeing where you are going.  What you are talking about is just the
> > trigger that trips at a certain uncertainty.  I don't think you
> realise
> > how little help this would be.  Setting an absolute goal for
> uncertainty
> > would be detrimental in the PSAP case, since certain values might not
> be
> > achievable.  The PSAP could wait indefinitely for a 10m uncertainty
> when a
> > 15m result (which is much better than the 500m result it has) sits
> there
> > waiting.  It would be better to have a rule that simply states "let
> me
> > know when you get a more accurate estimate".  How you codify such a
> rule
> > is tricky, but it's probably a worthwhile goal.
> >
> > On the other hand, a target uncertainty sounds reasonable as a
> general
> > application.  However, I struggle with finding a real-world
> application
> > that couldn't be solved using more concrete methods.  The problem
> with
> > specifying target uncertainty is that you are never really sure when,
> or
> > even _if_, that uncertainty can be achieved.  Not being sure means
> that
> > you need other criteria to base your application decision on, lest
> you be
> > left waiting indefinitely.  Application developers will ultimately
> find
> > their means: triggers based on <method>, polling, time constraints,
> or
> > something entirely else.  I can't see how specifying uncertainty
> helps.
> >
> > That doesn't mean that an application wont have a preference, but the
> > difference between *preferring* 10m uncertainty and *requiring* it is
> a
> > great deal.  Applying a filter is a strictly Boolean process.  That
> is why
> > the location-quality draft focuses on the location generation side.
> > Having the consumer of the location directly affect the location
> > generation process is of great advantage to them in terms of the
> resulting
> > estimate.  Particularly where there are multiple means of location
> > determination.
> >
> > The other aspect to this is how a trigger or preference is applied.
> I
> > would expect that a loc-filter would be applied by any LS, in the
> sense
> > that the LS could be detached entirely from the LG (my rhetorical GPS
> > phone publishing a position to a presence server is a good example).
> > Without having a direct impact on location generation, there is no
> sense
> > in expressing a preference.  A strictly Boolean filter is the only
> option
> > when you don't have any means of affecting the source data.
> >
> > So, I've probably given you a similarly unnecessary lesson, but the
> upshot
> > is that I don't see the use case for target uncertainty in loc-
> filters.
> >
> > What loc-filters needs most is to be implementable.  It is currently
> > overly difficult to implement.  If I get some time, I'll give you a
> more
> > complete critique, but most of what needs to be said has already been
> > said.  Sad to say, but the draft has been neglected until now.
> >
> > Cheers,
> > Martin
> >
> >
> > > -----Original Message-----
> > > From: Rosen, Brian [mailto:Brian.Rosen at neustar.biz]
> > > Sent: Tuesday, 15 July 2008 11:17 AM
> > > To: Thomson, Martin; Brian Rosen; Hannes Tschofenig; Carl Reed
> > > Cc: geopriv at ietf.org
> > > Subject: RE: [Geopriv] Location measurement error
> > >
> > > I don't agree with most of this.
> > >
> > > Error is of interest to anyone who uses a measurement.  If you
> don't
> > > know the error, you don't know what you have.  Error can be
> expressed
> > > in
> > > many ways, one of which is confidence and uncertainty.  Error can
> be
> > > passed through an LS without the LS looking at it.  On the other
> hand,
> > > if you ask an LS to give you location within some error band, it
> does
> > > have to understand the error it gets from the LG in order to meet
> the
> > > request.
> > >
> > > The notion that you ask for a location update when you are within
> some
> > > error bound is a pretty reasonable request.  The usual emergency
> case
> > > for wireless would use this; routing would be completed with cell
> or
> > > coarse or quick fix.  Rather than poll for an accurate result, the
> PSAP
> > > would request a notification of update when good (i.e. low error)
> > > information was available.  That is about as useful an operation as
> you
> > > could imagine, and much better than repeated bids or spin locks on
> > > completion when it takes many seconds.
> > >
> > > As with the time issue, I'm willing to have distinguished values
> that
> > > essentially mean "best you can do" for the error spec
> (corresponding to
> > > "EmergencyDispatch"), but the general case is filtering on a real
> error
> > > bound.
> > >
> > > Brian
> > > > -----Original Message-----
> > > > From: Thomson, Martin [mailto:Martin.Thomson at andrew.com]
> > > > Sent: Monday, July 14, 2008 8:59 PM
> > > > To: Brian Rosen; Hannes Tschofenig; Carl Reed
> > > > Cc: geopriv at ietf.org; Rosen, Brian
> > > > Subject: RE: [Geopriv] Location measurement error
> > > >
> > > > The problem with location quality (draft plug: draft-thomson-
> geopriv-
> > > > location-quality) is that the criteria therein are not designed
> for
> > > > consumption by an LS.  The information is intended to be consumed
> by
> > > an
> > > > LG.  The distinction is important.
> > > >
> > > > Based on this reasoning, the situation is complicated.  Filters
> as
> > > they
> > > > stand apply well to filtering of information (i.e. don't let me
> know
> > > > unless...), location quality issues are related to the generation
> of
> > > > location information.  Adding location quality parameters to a
> filter
> > > only
> > > > result in the filter being invalidated when the LS is not also an
> LG.
> > > >
> > > > Dealing with the issue separately is the best approach and I am
> > > against
> > > > adding more complexity to loc-filters.  Quality parameters are
> enough
> > > > different to warrant having another document.
> > > >
> > > >
> > > > I can remember getting some (positive) OGC feedback on draft-
> thomson-
> > > > geopriv-uncertainty, but I'm open to receiving more.
> > > >
> > > >
> > > > Cheers,
> > > > M
> > > >
> > > > > -----Original Message-----
> > > > > From: geopriv-bounces at ietf.org [mailto:geopriv-
> bounces at ietf.org] On
> > > > > Behalf Of Brian Rosen
> > > > > Sent: Tuesday, 15 July 2008 4:04 AM
> > > > > To: 'Hannes Tschofenig'; 'Carl Reed'
> > > > > Cc: geopriv at ietf.org; Rosen, Brian
> > > > > Subject: Re: [Geopriv] Location measurement error
> > > > >
> > > > > Although I am happy to have OGC do the actual work, I think we
> need
> > > to
> > > > > drive
> > > > > the effort towards a (relatively) swift conclusion.
> > > > >
> > > > > Towards that end, perhaps we could develop requirements and
> liaison
> > > > > them to
> > > > > OGC?
> > > > >
> > > > > Loc-filters does not touch this issue yet.  My hope is that
> there
> > > is
> > > a
> > > > > result from whatever comes of this discussion which makes an
> > > "obvious"
> > > > > addition to loc-filters to request a notification when a
> location
> > > with
> > > > > a
> > > > > specified error ("quality") was available.  I certainly want to
> > > copy
> > > > > something else.
> > > > >
> > > > > Brian
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: geopriv-bounces at ietf.org [mailto:geopriv-
> bounces at ietf.org]
> > > On
> > > > > Behalf
> > > > > > Of Hannes Tschofenig
> > > > > > Sent: Monday, July 14, 2008 1:26 PM
> > > > > > To: Carl Reed
> > > > > > Cc: geopriv at ietf.org; Rosen, Brian
> > > > > > Subject: Re: [Geopriv] Location measurement error
> > > > > >
> > > > > > I would feel much better if we push these type of things to
> the
> > > OGC
> > > > > (as
> > > > > > much as we can).
> > > > > >
> > > > > > We have so many other things todo in our group. Additionally,
> > > > > starting
> > > > > > work on these topics within the IETF smells like a recipe for
> > > > > disaster.
> > > > > > Finally, we don't have the experts on this subject in the
> group
> > > in
> > > > > order
> > > > > > to produce a high-quality document.
> > > > > >
> > > > > > I agree with Brian that we should have something in our
> > > documents.
> > > > > The
> > > > > > easiest way for us to get such a work done is to reference
> > > something
> > > > > > that was already discussed and finished in OGC.
> > > > > >
> > > > > > There is only a slight problem with that approach: there are
> a
> > > few
> > > > > > pieces that reach into our work as well. For example, with
> the
> > > > > following
> > > > > > documents there are aspect where uncertainty might reach into
> our
> > > > > > protocol work:
> > > > > >
> > > http://www.ietf.org/internet-drafts/draft-ietf-geopriv-policy-
> 17.txt
> > > > > > http://tools.ietf.org/wg/geopriv/draft-ietf-geopriv-loc-
> filters/
> > > > > >
> > > > > > I strongly hope that the former document is fine now. I
> haven't
> > > > > checked
> > > > > > the loc-filters since it was re-submitted today.
> > > > > >
> > > > > > I would also suggest Martin and Jams to forward their
> uncertainty
> > > > > > document to the OGC for review:
> > > > > > http://tools.ietf.org/id/draft-thomson-geopriv-uncertainty-
> 01.txt
> > > > > > Maybe Carl has an idea on how to tackle that document.
> > > > > >
> > > > > > Ciao
> > > > > > Hannes
> > > > > >
> > > > > >
> > > > > > Carl Reed wrote:
> > > > > > > Brian -
> > > > > > >
> > > > > > > There is some relevant work done (and being done) in the
> OGC
> > > > > related
> > > > > > > to this topic. Let me check on the current status and
> provide
> > > some
> > > > > > > feedback.  Also, the OGC is totally open to accepting
> > > > > > > requirements/change requests from the community and using
> these
> > > > > > > requirements to help drive enhancements to the OGC
> standards
> > > > > baseline.
> > > > > > >
> > > > > > > FYI, there is an open call for change requests for GML.
> This
> > > open
> > > > > call
> > > > > > > closes in September. Any individual and/or organization can
> > > submit
> > > > > > > such change requests.
> > > > > > >
> > > > > > > So, if the IETF community has any requirements that need to
> be
> > > > > > > considered for the next version of GML, please visit
> > > > > > > http://www.opengeospatial.org/pressroom/pressreleases/879
> > > > > > >
> > > > > > > Thanks and regards
> > > > > > >
> > > > > > > Carl
> > > > > > >
> > > > > > > ----- Original Message ----- From: "Rosen, Brian"
> > > > > > > <Brian.Rosen at neustar.biz>
> > > > > > > To: <geopriv at ietf.org>
> > > > > > > Sent: Monday, July 14, 2008 7:56 AM
> > > > > > > Subject: [Geopriv] Location measurement error
> > > > > > >
> > > > > > >
> > > > > > >> I'd like to start a discussion on how we deal with, in the
> big
> > > > > picture,
> > > > > > >> the notion of measurement error in general and
> "confidence"
> > > and
> > > > > > >> "uncertainty" in specific.
> > > > > > >>
> > > > > > >> I am aware that there are a set of views that go roughly
> like:
> > > > > "there
> > > > > > >> really is no such thing as confidence and uncertainty,
> some
> > > > > vendors are
> > > > > > >> pushing this idea because they can't do what 'real' GPS
> > > devices
> > > do
> > > > > and
> > > > > > >> give you a 3 dimensional error in meters".
> > > > > > >>
> > > > > > >> Nevertheless, in one of the most commonly encountered
> location
> > > > > > >> determination mechanisms (A-GPS and similar systems),
> > > confidence
> > > > > and
> > > > > > >> uncertainty are reported, and are the only error indicator
> > > > > available.
> > > > > > >>
> > > > > > >> In other systems, notably commercial GPS systems, you get
> a
> > > > > different
> > > > > > >> kind of error specification.  Sometimes it's simply error
> in
> > > > > meters,
> > > > > > >> often with a different value for Z than for x and y.
> Other
> > > times
> > > > > you
> > > > > > >> get a more complex representation of error.
> > > > > > >>
> > > > > > >> It does seem to me that when we convey measured location,
> that
> > > it
> > > > > would
> > > > > > >> be appropriate to convey the error of the measurement, and
> > > that
> > > > > error
> > > > > > >> indication should have the ability to represent confidence
> and
> > > > > > >> uncertainty.  I think we need to standardize the
> > > representation
> > > of
> > > > > > >> error.  It does occur to me to ask if such standardization
> > > would
> > > > > better
> > > > > > >> be done in OGC, and Carl may wish to comment.  It's also
> > > possible
> > > > > for
> > > > > > >> the geopriv crowd to come up with something and then OGC
> to
> > > > > incorporate
> > > > > > >> it in a future version of GML.
> > > > > > >>
> > > > > > >> If we had a standardized representation of error, then we
> > > could
> > > > > also
> > > > > > use
> > > > > > >> that in protocols like HELD to request a location within a
> > > > > specified
> > > > > > >> error bound, and we could also have a filter that
> requested
> > > > > location
> > > > > > >> within such a bound.  It would of course, be best to
> specify
> > > the
> > > > > XML
> > > > > > for
> > > > > > >> such a bound once.
> > > > > > >>
> > > > > > >> I'm specifically proposing that we allow error to be
> specified
> > > one
> > > > > of
> > > > > > >> two ways initially: with an actual error measurement, in
> > > meters
> > > > > for X,
> > > > > > Y
> > > > > > >> and Z or as a shape (as in our other shape definitions)
> with
> > > an
> > > > > > >> "uncertainty" metric in percentage terms.  I would add it
> to
> > > GML
> > > > > if we
> > > > > > >> could do that, or additional elements in PIDF-LO.  I would
> > > then
> > > > > > describe
> > > > > > >> an XML data structure to specify a bound, again, either in
> > > meters
> > > > > on X,
> > > > > > >> Y, and Z or confidence as a shape and uncertainty as a
> > > percentage.
> > > > > I'd
> > > > > > >> allow the representation of both to be extended for other
> ways
> > > to
> > > > > > >> represent error.
> > > > > > >>
> > > > > > >> Brian
> >
> > ---------------------------------------------------------------------
> -----
> > ----------------------
> > This message is for the designated recipient only and may
> > contain privileged, proprietary, or otherwise private information.
> > If you have received it in error, please notify the sender
> > immediately and delete the original.  Any unauthorized use of
> > this email is prohibited.
> > ---------------------------------------------------------------------
> -----
> > ----------------------
> > [mf2]
> > _______________________________________________
> > Geopriv mailing list
> > Geopriv at ietf.org
> > https://www.ietf.org/mailman/listinfo/geopriv
> 

------------------------------------------------------------------------------------------------
This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise private information.  
If you have received it in error, please notify the sender
immediately and delete the original.  Any unauthorized use of
this email is prohibited.
------------------------------------------------------------------------------------------------
[mf2]
_______________________________________________
Geopriv mailing list
Geopriv at ietf.org
https://www.ietf.org/mailman/listinfo/geopriv