Re: [Geopriv] Location measurement error

["Brian Rosen" <br at brianrosen.net>]

Ah Martin

Allow me to go over some old ground
1. No watcher knows what to do with confidence and uncertainty.  I've asked
literally dozens of watchers from all kinds of emergency and
"pizza-delivery" equivalent services.  They all understood their measuring
device had error.  They ignore it.
2. When pressed to deal with it, all watchers want confidence set to at
least 95%.  It would be okay with them, at least today, if it was fixed at
95%.  Watchers HATE two measurements with different confidence.  Allowing
specification of uncertainty is completely unnecessary, fix it at 95% and
all watchers will be at least content (they won't be happy until it's 99%).
3. Watchers specify uncertainty in meters.  That means, to you, whatever the
shape that has the same error value for x and y and a different one for z.
They don't want any shape that doesn't look like that.  There is no need,
from a watcher point of view, to specify uncertainty any other way.  
4. Existing systems don't allow the watcher to specify confidence or
uncertainty.  They simply report what the owner of the measuring system
decides to report.  The notion that the error spec goes in either a request
or a filter is a novel idea, and I'm not sure it will be implemented.
5. All PSAPs and all responders in the U.S. have a very well defined value
for confidence and uncertainty for a filter: within 100 meters (x and y),
95% of the time. 
6. With a minRate, you cover the case of "best you can do in x seconds".
Without a way to specify error in some way, you can only say that.  If you
want "best you can do", there is no way to say that.  If you want to say
"within 10 meters is good enough" there is no way to say that.  
7. Event notification is a much more acceptable way to get a measurement
that takes seconds to compute.  The error spec in the filter is much more
useful that the error spec in the request.  If you want an error return that
is effectively "you'll never get one with this filter" I would be okay with
that, but see below.
8. The biggest problem with all of the mechanisms that have so far been
proposed is that they don't include any way for a watcher to determine what
an achievable error could be from a particular LG (or LS, since that is what
it deals with).  Until you come up with a way for the watcher to find what
the measurement mechanism's achievable error spec is, you don't have a
useful set of mechanisms.

It occurs to me to ask if "within a 100 meters, 95% of the time" is actually
the same as a circular error shape with a 100 meter radius and a 95%
confidence.  It sounds like the same thing.  Is it exactly the same thing?

Brian 

> -----Original Message-----
> From: geopriv-bounces at ietf.org [mailto:geopriv-bounces at ietf.org] On Behalf
> Of Thomson, Martin
> Sent: Tuesday, July 15, 2008 12:15 AM
> To: Rosen, Brian
> Cc: geopriv at ietf.org
> Subject: Re: [Geopriv] Location measurement error
> 
> Hi Brian,
> 
> Once again, I'm pretty sure that we disagree because we were talking about
> subtly different things.  And giving me a lesson on uncertainty and
> confidence isn't really necessary :)
> 
> I really needed to understand the use case for this, and I think that I am
> seeing where you are going.  What you are talking about is just the
> trigger that trips at a certain uncertainty.  I don't think you realise
> how little help this would be.  Setting an absolute goal for uncertainty
> would be detrimental in the PSAP case, since certain values might not be
> achievable.  The PSAP could wait indefinitely for a 10m uncertainty when a
> 15m result (which is much better than the 500m result it has) sits there
> waiting.  It would be better to have a rule that simply states "let me
> know when you get a more accurate estimate".  How you codify such a rule
> is tricky, but it's probably a worthwhile goal.
> 
> On the other hand, a target uncertainty sounds reasonable as a general
> application.  However, I struggle with finding a real-world application
> that couldn't be solved using more concrete methods.  The problem with
> specifying target uncertainty is that you are never really sure when, or
> even _if_, that uncertainty can be achieved.  Not being sure means that
> you need other criteria to base your application decision on, lest you be
> left waiting indefinitely.  Application developers will ultimately find
> their means: triggers based on <method>, polling, time constraints, or
> something entirely else.  I can't see how specifying uncertainty helps.
> 
> That doesn't mean that an application wont have a preference, but the
> difference between *preferring* 10m uncertainty and *requiring* it is a
> great deal.  Applying a filter is a strictly Boolean process.  That is why
> the location-quality draft focuses on the location generation side.
> Having the consumer of the location directly affect the location
> generation process is of great advantage to them in terms of the resulting
> estimate.  Particularly where there are multiple means of location
> determination.
> 
> The other aspect to this is how a trigger or preference is applied.  I
> would expect that a loc-filter would be applied by any LS, in the sense
> that the LS could be detached entirely from the LG (my rhetorical GPS
> phone publishing a position to a presence server is a good example).
> Without having a direct impact on location generation, there is no sense
> in expressing a preference.  A strictly Boolean filter is the only option
> when you don't have any means of affecting the source data.
> 
> So, I've probably given you a similarly unnecessary lesson, but the upshot
> is that I don't see the use case for target uncertainty in loc-filters.
> 
> What loc-filters needs most is to be implementable.  It is currently
> overly difficult to implement.  If I get some time, I'll give you a more
> complete critique, but most of what needs to be said has already been
> said.  Sad to say, but the draft has been neglected until now.
> 
> Cheers,
> Martin
> 
> 
> > -----Original Message-----
> > From: Rosen, Brian [mailto:Brian.Rosen at neustar.biz]
> > Sent: Tuesday, 15 July 2008 11:17 AM
> > To: Thomson, Martin; Brian Rosen; Hannes Tschofenig; Carl Reed
> > Cc: geopriv at ietf.org
> > Subject: RE: [Geopriv] Location measurement error
> >
> > I don't agree with most of this.
> >
> > Error is of interest to anyone who uses a measurement.  If you don't
> > know the error, you don't know what you have.  Error can be expressed
> > in
> > many ways, one of which is confidence and uncertainty.  Error can be
> > passed through an LS without the LS looking at it.  On the other hand,
> > if you ask an LS to give you location within some error band, it does
> > have to understand the error it gets from the LG in order to meet the
> > request.
> >
> > The notion that you ask for a location update when you are within some
> > error bound is a pretty reasonable request.  The usual emergency case
> > for wireless would use this; routing would be completed with cell or
> > coarse or quick fix.  Rather than poll for an accurate result, the PSAP
> > would request a notification of update when good (i.e. low error)
> > information was available.  That is about as useful an operation as you
> > could imagine, and much better than repeated bids or spin locks on
> > completion when it takes many seconds.
> >
> > As with the time issue, I'm willing to have distinguished values that
> > essentially mean "best you can do" for the error spec (corresponding to
> > "EmergencyDispatch"), but the general case is filtering on a real error
> > bound.
> >
> > Brian
> > > -----Original Message-----
> > > From: Thomson, Martin [mailto:Martin.Thomson at andrew.com]
> > > Sent: Monday, July 14, 2008 8:59 PM
> > > To: Brian Rosen; Hannes Tschofenig; Carl Reed
> > > Cc: geopriv at ietf.org; Rosen, Brian
> > > Subject: RE: [Geopriv] Location measurement error
> > >
> > > The problem with location quality (draft plug: draft-thomson-geopriv-
> > > location-quality) is that the criteria therein are not designed for
> > > consumption by an LS.  The information is intended to be consumed by
> > an
> > > LG.  The distinction is important.
> > >
> > > Based on this reasoning, the situation is complicated.  Filters as
> > they
> > > stand apply well to filtering of information (i.e. don't let me know
> > > unless...), location quality issues are related to the generation of
> > > location information.  Adding location quality parameters to a filter
> > only
> > > result in the filter being invalidated when the LS is not also an LG.
> > >
> > > Dealing with the issue separately is the best approach and I am
> > against
> > > adding more complexity to loc-filters.  Quality parameters are enough
> > > different to warrant having another document.
> > >
> > >
> > > I can remember getting some (positive) OGC feedback on draft-thomson-
> > > geopriv-uncertainty, but I'm open to receiving more.
> > >
> > >
> > > Cheers,
> > > M
> > >
> > > > -----Original Message-----
> > > > From: geopriv-bounces at ietf.org [mailto:geopriv-bounces at ietf.org] On
> > > > Behalf Of Brian Rosen
> > > > Sent: Tuesday, 15 July 2008 4:04 AM
> > > > To: 'Hannes Tschofenig'; 'Carl Reed'
> > > > Cc: geopriv at ietf.org; Rosen, Brian
> > > > Subject: Re: [Geopriv] Location measurement error
> > > >
> > > > Although I am happy to have OGC do the actual work, I think we need
> > to
> > > > drive
> > > > the effort towards a (relatively) swift conclusion.
> > > >
> > > > Towards that end, perhaps we could develop requirements and liaison
> > > > them to
> > > > OGC?
> > > >
> > > > Loc-filters does not touch this issue yet.  My hope is that there
> > is
> > a
> > > > result from whatever comes of this discussion which makes an
> > "obvious"
> > > > addition to loc-filters to request a notification when a location
> > with
> > > > a
> > > > specified error ("quality") was available.  I certainly want to
> > copy
> > > > something else.
> > > >
> > > > Brian
> > > >
> > > > > -----Original Message-----
> > > > > From: geopriv-bounces at ietf.org [mailto:geopriv-bounces at ietf.org]
> > On
> > > > Behalf
> > > > > Of Hannes Tschofenig
> > > > > Sent: Monday, July 14, 2008 1:26 PM
> > > > > To: Carl Reed
> > > > > Cc: geopriv at ietf.org; Rosen, Brian
> > > > > Subject: Re: [Geopriv] Location measurement error
> > > > >
> > > > > I would feel much better if we push these type of things to the
> > OGC
> > > > (as
> > > > > much as we can).
> > > > >
> > > > > We have so many other things todo in our group. Additionally,
> > > > starting
> > > > > work on these topics within the IETF smells like a recipe for
> > > > disaster.
> > > > > Finally, we don't have the experts on this subject in the group
> > in
> > > > order
> > > > > to produce a high-quality document.
> > > > >
> > > > > I agree with Brian that we should have something in our
> > documents.
> > > > The
> > > > > easiest way for us to get such a work done is to reference
> > something
> > > > > that was already discussed and finished in OGC.
> > > > >
> > > > > There is only a slight problem with that approach: there are a
> > few
> > > > > pieces that reach into our work as well. For example, with the
> > > > following
> > > > > documents there are aspect where uncertainty might reach into our
> > > > > protocol work:
> > > > >
> > http://www.ietf.org/internet-drafts/draft-ietf-geopriv-policy-17.txt
> > > > > http://tools.ietf.org/wg/geopriv/draft-ietf-geopriv-loc-filters/
> > > > >
> > > > > I strongly hope that the former document is fine now. I haven't
> > > > checked
> > > > > the loc-filters since it was re-submitted today.
> > > > >
> > > > > I would also suggest Martin and Jams to forward their uncertainty
> > > > > document to the OGC for review:
> > > > > http://tools.ietf.org/id/draft-thomson-geopriv-uncertainty-01.txt
> > > > > Maybe Carl has an idea on how to tackle that document.
> > > > >
> > > > > Ciao
> > > > > Hannes
> > > > >
> > > > >
> > > > > Carl Reed wrote:
> > > > > > Brian -
> > > > > >
> > > > > > There is some relevant work done (and being done) in the OGC
> > > > related
> > > > > > to this topic. Let me check on the current status and provide
> > some
> > > > > > feedback.  Also, the OGC is totally open to accepting
> > > > > > requirements/change requests from the community and using these
> > > > > > requirements to help drive enhancements to the OGC standards
> > > > baseline.
> > > > > >
> > > > > > FYI, there is an open call for change requests for GML. This
> > open
> > > > call
> > > > > > closes in September. Any individual and/or organization can
> > submit
> > > > > > such change requests.
> > > > > >
> > > > > > So, if the IETF community has any requirements that need to be
> > > > > > considered for the next version of GML, please visit
> > > > > > http://www.opengeospatial.org/pressroom/pressreleases/879
> > > > > >
> > > > > > Thanks and regards
> > > > > >
> > > > > > Carl
> > > > > >
> > > > > > ----- Original Message ----- From: "Rosen, Brian"
> > > > > > <Brian.Rosen at neustar.biz>
> > > > > > To: <geopriv at ietf.org>
> > > > > > Sent: Monday, July 14, 2008 7:56 AM
> > > > > > Subject: [Geopriv] Location measurement error
> > > > > >
> > > > > >
> > > > > >> I'd like to start a discussion on how we deal with, in the big
> > > > picture,
> > > > > >> the notion of measurement error in general and "confidence"
> > and
> > > > > >> "uncertainty" in specific.
> > > > > >>
> > > > > >> I am aware that there are a set of views that go roughly like:
> > > > "there
> > > > > >> really is no such thing as confidence and uncertainty, some
> > > > vendors are
> > > > > >> pushing this idea because they can't do what 'real' GPS
> > devices
> > do
> > > > and
> > > > > >> give you a 3 dimensional error in meters".
> > > > > >>
> > > > > >> Nevertheless, in one of the most commonly encountered location
> > > > > >> determination mechanisms (A-GPS and similar systems),
> > confidence
> > > > and
> > > > > >> uncertainty are reported, and are the only error indicator
> > > > available.
> > > > > >>
> > > > > >> In other systems, notably commercial GPS systems, you get a
> > > > different
> > > > > >> kind of error specification.  Sometimes it's simply error in
> > > > meters,
> > > > > >> often with a different value for Z than for x and y.  Other
> > times
> > > > you
> > > > > >> get a more complex representation of error.
> > > > > >>
> > > > > >> It does seem to me that when we convey measured location, that
> > it
> > > > would
> > > > > >> be appropriate to convey the error of the measurement, and
> > that
> > > > error
> > > > > >> indication should have the ability to represent confidence and
> > > > > >> uncertainty.  I think we need to standardize the
> > representation
> > of
> > > > > >> error.  It does occur to me to ask if such standardization
> > would
> > > > better
> > > > > >> be done in OGC, and Carl may wish to comment.  It's also
> > possible
> > > > for
> > > > > >> the geopriv crowd to come up with something and then OGC to
> > > > incorporate
> > > > > >> it in a future version of GML.
> > > > > >>
> > > > > >> If we had a standardized representation of error, then we
> > could
> > > > also
> > > > > use
> > > > > >> that in protocols like HELD to request a location within a
> > > > specified
> > > > > >> error bound, and we could also have a filter that requested
> > > > location
> > > > > >> within such a bound.  It would of course, be best to specify
> > the
> > > > XML
> > > > > for
> > > > > >> such a bound once.
> > > > > >>
> > > > > >> I'm specifically proposing that we allow error to be specified
> > one
> > > > of
> > > > > >> two ways initially: with an actual error measurement, in
> > meters
> > > > for X,
> > > > > Y
> > > > > >> and Z or as a shape (as in our other shape definitions) with
> > an
> > > > > >> "uncertainty" metric in percentage terms.  I would add it to
> > GML
> > > > if we
> > > > > >> could do that, or additional elements in PIDF-LO.  I would
> > then
> > > > > describe
> > > > > >> an XML data structure to specify a bound, again, either in
> > meters
> > > > on X,
> > > > > >> Y, and Z or confidence as a shape and uncertainty as a
> > percentage.
> > > > I'd
> > > > > >> allow the representation of both to be extended for other ways
> > to
> > > > > >> represent error.
> > > > > >>
> > > > > >> Brian
> 
> --------------------------------------------------------------------------
> ----------------------
> This message is for the designated recipient only and may
> contain privileged, proprietary, or otherwise private information.
> If you have received it in error, please notify the sender
> immediately and delete the original.  Any unauthorized use of
> this email is prohibited.
> --------------------------------------------------------------------------
> ----------------------
> [mf2]
> _______________________________________________
> Geopriv mailing list
> Geopriv at ietf.org
> https://www.ietf.org/mailman/listinfo/geopriv

_______________________________________________
Geopriv mailing list
Geopriv at ietf.org
https://www.ietf.org/mailman/listinfo/geopriv