Re: [Geopriv] Location measurement error

["Brian Rosen" <br at brianrosen.net>]

I'm not arguing that measurements have no error.
I'm not disputing that the LBS vendors have settled on confidence and
uncertainty as the way they will express error.
I'm not arguing IETF protocols should ignore error.

I am suggesting that apps don't know what values to use for uncertainty and
confidence.  They consistently want "best you can do".

If you really, truly have multiple mechanisms that can be used pretty much
at an apps whim (even if there are cost implications to the app), which is
pretty rare so far, then the notion that you insist the app tell you what
uncertainty it will accept without having the LG tell the app what
uncertainty it can achieve makes no engineering sense to me.

What I see out there is much simpler.  The system has a spec, the app
expects the system to meet the spec, and that's the end of the story.  We
should be able to make our protocols work in such simple systems with
minimal fuss.

We're getting closer on confidence.  The NENA/APCO filing suggests everyone
use the same confidence value, which I believe will be 95%.  That's the
suggestion we seem to be heading towards here (with some exceptions).

I'm not opposed to allowing an app to request a specific uncertainty,
especially if it can use simple metrics (e.g. within 3 meters) as long as
there is a distinguished value or other mechanism that means "best you can
do".

It has always made sense to me to return the error of the measurement.

I want to be able to filter on error, although 90% of the problem is solved
with "send me an update when you have the best answer you can get", as long
as we can specify a max time for that.  I think it does make as much sense
to filter on the error as it makes sense to send the error spec to the LG.
In fact, since I don't distinguish between the LG and the LS, since we don't
currently specify protocols between the LG and the LS, I think they are
nearly the same thing.

Brian

> -----Original Message-----
> From: Dawson, Martin [mailto:Martin.Dawson at andrew.com]
> Sent: Thursday, July 17, 2008 8:23 PM
> To: Brian Rosen; Thomson, Martin
> Cc: geopriv at ietf.org
> Subject: RE: [Geopriv] Location measurement error
> 
> I might hate the fact that sunshine can give me skin cancer, but the
> physics is undeniable - even if I put "hate" in upper case.
> 
> All location-based applications that get geodetic areas of uncertainty
> have to deal with the fact that:
> 
> a) The location of the target can only be provided as an area that the
> target may be in (uncertainty)
> b) There is a possibility that the target isn't even in that area (100%
> - confidence)
> 
> If we're honest, the same kind of "uncertainty" exists for civic
> location as well.
> 
> An application that ignores this information isn't as good as one that
> optimizes its procedures based on this information. Despite what you
> say, I don't actually believe that PSAPs don't have procedures that deal
> with this fact. They may not be formalised through the ODC, they may be
> by convention, and might not even be documented. However, I'm confident
> they don't just dispatch to the centre point and then give up if the
> caller isn't there. In any case, an emergency application that doesn't
> proceduralise this fact isn't as good as one that does.
> 
> Actually, this week's filing from APCO and NENA in response to the FCC
> NPRM demonstrates that they well recognize that accuracy can't be
> arbitrarily demanded and, indeed, are recommending that the FCC back off
> on arbitrary accuracy demands. I've attached the filing - in the,
> perhaps vain, hope that it will survive the list server...
> 
> Bottom line - I don't think "hate" qualifies as reasonable justification
> for a requirement.
> 
> With respect to the FCC mandate versus what the query protocol might
> support, there is an obvious difference. The mandate just states that
> the cellular network is statistically within the uncertainty requirement
> across all the locations they return. There is no requirement to
> "tailor" the confidence on a per-request basis. As such, you'll find
> that, for example, the radius of uncertainty returned from a cell-based
> location may just be based on the distance from the tower at which the
> signal nominally drops by 3dB (half power). This has no particular
> relationship with where in that area of coverage most calls actually
> originate or, more subtly, where most 9-1-1 calls originate. If the
> carrier performance was evaluated on how accurately their provided
> uncertainty actually matched a nominal confidence (rather than just the
> statistical variance based on the centre points provided) then they
> would need to tailor those cell boundaries more carefully. However, that
> would have to be against a specific confidence, it's not like their
> systems are designed to generate a location based on an arbitrary
> required level of confidence. The algorithms in some of their LG
> components my have the necessary wherewithal embedded in them (e.g.
> UTDOA, fingerprinting, GPS) since convergence threshold parameters in
> those algorithms can be driven by confidence. However, it's by no means
> always possible to do that and it's by no means common that
> implementations allow these convergence parameters to be controlled on a
> per-request basis.
> 
> Cheers,
> Martin
> 
> -----Original Message-----
> From: geopriv-bounces at ietf.org [mailto:geopriv-bounces at ietf.org] On
> Behalf Of Brian Rosen
> Sent: Friday, 18 July 2008 8:01 AM
> To: Thomson, Martin
> Cc: geopriv at ietf.org
> Subject: Re: [Geopriv] Location measurement error
> 
> Sorry for the delay, but I am still wallowing
> 
> > > 2. When pressed to deal with it, all watchers want confidence set to
> at
> > > least 95%.  It would be okay with them, at least today, if it was
> fixed
> > > at
> > > 95%.  Watchers HATE two measurements with different confidence.
> > > Allowing
> > > specification of uncertainty is completely unnecessary, fix it at
> 95%
> > > and
> > > all watchers will be at least content (they won't be happy until
> it's
> > > 99%).
> >
> > I agree on the different confidence - given the non-linear
> relationship,
> > managing two variables is a pain.
> >
> > Everyone wants 100%, but fail to acknowledge the tradeoffs.  I think
> that
> > I can see where you are going - you are making points about what
> people
> > _want_.  I can appreciate that.
> Well, it's more of a requirement.  It's also very related to "they
> ignore
> it".  Since the data they get has variable confidence, the uncertainty
> doesn't mean the same thing.
> 
> I'm specifically suggesting, somewhat naively, that there not be a
> variable
> for confidence, but it simply be set to 95%.
> 
> The reality might be that the text in all the documents say:
> Confidence MUST be set to 95% except for certain existing devices which
> will
> not currently provide measurements with such a value.
> >
> > > 3. Watchers specify uncertainty in meters.  That means, to you,
> > > whatever the
> > > shape that has the same error value for x and y and a different one
> for
> > > z.
> > > They don't want any shape that doesn't look like that.  There is no
> > > need,
> > > from a watcher point of view, to specify uncertainty any other way.
> >
> > The point that people understand the expression of uncertainty as
> "within
> > x metres" is relevant.  But what they want is no uncertainty, failing
> > that, they want simple.  I've describe how to accommodate these
> wishes,
> > while not ignoring the real constraints:
> > <http://tools.ietf.org/html/draft-thomson-geopriv-uncertainty>
> Yeah, pretty nice piece of work.  I do need to send you some comments,
> but I
> still like it a lot.
> 
> You could reasonably ask, if the users only want simple, is it better to
> push the calculations described back to the LS?
> 
> >
> > > 4. Existing systems don't allow the watcher to specify confidence or
> > > uncertainty.  They simply report what the owner of the measuring
> system
> > > decides to report.  The notion that the error spec goes in either a
> > > request
> > > or a filter is a novel idea, and I'm not sure it will be
> implemented.
> >
> > Now you've lost me.  Specifying target uncertainty is not especially
> > novel, we've been handling it for almost 10 years now.  Quality of
> > position parameters are a fixture of most existing LCS protocols.
> They
> > get pushed down the whole chain.  For example: Application -> GMLC ->
> MSC
> > -> RNC -> SAS - all the protocols have QoP parameters of various
> sorts.
> In the protocols with which I am familiar, you cannot specify
> uncertainty,
> you can only receive it when you get a measurement.  OTOH, I have
> limited
> experience outside the emergency call system.  In that system, both the
> E2
> and ALI protocols have a way to return uncertainty, but no way to
> request a
> specific uncertainty (or confidence).
> 
> >
> > I've explained why I think that adding an "error spec" to filters is
> not
> > useful.
> Yeah, but we are disagreeing on that point still
> 
> >
> > > 5. All PSAPs and all responders in the U.S. have a very well defined
> > > value
> > > for confidence and uncertainty for a filter: within 100 meters (x
> and
> > > y),
> > > 95% of the time.
> >
> > If you are talking FCC mandates, you are still getting them wrong.  In
> any
> > case, these numbers cannot be applied to an individual call - they are
> > statistical figures that are applied across a large set of calls.
> Again,
> > this falls into the "want" basket.
> It's the only thing we have.  This is the source of endless frustration
> to
> me and the PSAP folks.  We want to put in "6 cm uncertainty, 100%
> confidence".  It's the only spec we actually have.   We have a
> regulation
> that says 100 meters, 95% of the time.  What do you want a PSAP to put
> in
> the request?
> 
> This is the nut of the problem I have.  The user goal is (nearly always)
> infinite precision, 100% confidence.  If the system didn't deliver them
> location that met some purchase, legal or other spec, they wouldn't be
> using
> it in the first place.  There IS no other spec other than "best you can
> do".
> 
> Clearly, as we have discussed several times, you sometimes have an
> expressable time constraint (best you can do in 10 seconds).  I can't
> think
> of any application where there is a meaningful value that could be put
> into
> a request other than "best you can do", but you insist we come up with
> some
> numbers.
> 
> I'd really like to get off this merry go round.  Your end says "tell me
> your
> spec".  My end says "best you can do" or the spec is zero uncertainty
> and
> 100% confidence is all we got.
> 
> >
> > > 6. With a minRate, you cover the case of "best you can do in x
> > > seconds".
> > > Without a way to specify error in some way, you can only say that.
> If
> > > you
> > > want "best you can do", there is no way to say that.  If you want to
> > > say
> > > "within 10 meters is good enough" there is no way to say that.
> >
> > Actually, with a minRate, you specify that the LS update you every x
> > seconds.  Unless there is a way for the LS to push the LG, you only
> get a
> > better estimate if the LG happens to get one.  That is my point - and
> one
> > that you seem to be reinforcing.  I'm officially confused now.
> I refuse to get drawn into the difference between an LS to watcher and
> LG to
> watcher and/or LS to LG to watcher.  We may need another protocol for LS
> to
> LG.  So far, we don't have one.  When we are in that circumstance, we
> ignore
> the difference between LG and LS; assume a proprietary protocol, or a
> single
> box.  The watcher tells the LS, the LS tells the LG.
> 
> We talked about a similar problem in a NENA call today: if you use the
> enter/exit boundary filter, at what rate should the LG take measurements
> to
> give you the desired result (how long after you entered/left the
> boundary
> are you willing to wait to be told you entered or left)?  I suggest that
> you
> can use minRate as a hint, and perhaps we should say that.  We could
> also
> have a specific parameter for the purpose.
> 
> 
> 
> >
> > > 7. Event notification is a much more acceptable way to get a
> > > measurement
> > > that takes seconds to compute.  The error spec in the filter is much
> > > more
> > > useful that the error spec in the request.  If you want an error
> return
> > > that
> > > is effectively "you'll never get one with this filter" I would be
> okay
> > > with
> > > that, but see below.
> >
> > OK, say that I subscribe for location information.  You are saying
> that
> > specifying a filter is better than cramming some extra parameters in
> the
> > subscription request.  I can't contest that.  It misses the point, but
> in
> > and of itself, the argument is sound.
> >
> > The problem with a filter (or indeed anything related to the
> subscription
> > and only the subscription) is that whatever you do there, you have no
> > influence over the information.  You get whatever the LG generates.
> > Without some influence over the location generation, anything you do
> is
> > essentially pointless.  Unless the LG is given instructions on what to
> do,
> > you get whatever you get.  Assuming that the LG is independently
> motivated
> > to generate an extremely accurate result isn't going to work.  The
> model
> > can't assume that the LS has some secret means to influence the LG.
> > Unless you specify a means for the LS to do so, which I'd be very
> > interested in.
> See above
> 
> >
> > > 8. The biggest problem with all of the mechanisms that have so far
> been
> > > proposed is that they don't include any way for a watcher to
> determine
> > > what
> > > an achievable error could be from a particular LG (or LS, since that
> is
> > > what
> > > it deals with).  Until you come up with a way for the watcher to
> find
> > > what
> > > the measurement mechanism's achievable error spec is, you don't have
> a
> > > useful set of mechanisms.
> >
> > That's because noone knows what the achievable error is.  It's not
> related
> > to the "mechanism", but to the specific situation: time of day, wind
> speed
> > and direction, battery levels, indoors/outdoors, access technology,
> what
> > the owner had for lunch, solar winds, etc.
> 
> Okay, so what you are saying is this:
> We haven't a clue what we can do.
> The app has no clue what you can do.
> The app wants zero error and 100% confidence
> The app wants the best you can do (with perhaps a time constraint)
> 
> What the heck do you want it to say?
> 
> Oh, let's see, let's try 6cm and 99% confidence.  Gee, no answer.  Darn.
> 
> Okay, let's try 6cm and 95% confidence.  Gee, no answer.  Darn again.
> 
> Okay, let's try 1m and 95% confidence
> 
> You wouldn't go the other way:
> 
> Okay, let's give it 2km uncertainty and 10% confidence.  Oh, great, an
> answer.
> 
> I wonder if it will do better.  Let's try 1km and 30%...
> 
> That's the real, honest situation.
> 
> It's the reality for pizza hut
> It's the reality for 1-1-2
> It's the reality for turn by turn directions
> It's the reality for "where is the nearest Starbucks"
> It's the reality for "tell me when my kid goes outside the mall"
> 
> What the heck do you want the app to do?
> 
> >
> > > It occurs to me to ask if "within a 100 meters, 95% of the time" is
> > > actually
> > > the same as a circular error shape with a 100 meter radius and a 95%
> > > confidence.  It sounds like the same thing.  Is it exactly the same
> > > thing?
> >
> > Yes.  I'm surprised that this realization took so long, given that the
> > exact statement has been made many times in the past.
> 
> Actually, you just said it wasn't the same thing above.  You said that
> 100 m
> 95% of the time (the FCC mandate) was not the same as 95 meter circular
> radius uncertainty with 95% confidence.  You said it was a statistical
> calculation over a large number of measurements (implying it's not the
> spec
> you use for a single measurement).
> 
> Okay, so if you are a U.S. PSAP, what DO you ask for?
> 
> Brian
> 
> _______________________________________________
> Geopriv mailing list
> Geopriv at ietf.org
> https://www.ietf.org/mailman/listinfo/geopriv
> 
> --------------------------------------------------------------------------
> ----------------------
> This message is for the designated recipient only and may
> contain privileged, proprietary, or otherwise private information.
> If you have received it in error, please notify the sender
> immediately and delete the original.  Any unauthorized use of
> this email is prohibited.
> --------------------------------------------------------------------------
> ----------------------
> [mf2]

_______________________________________________
Geopriv mailing list
Geopriv at ietf.org
https://www.ietf.org/mailman/listinfo/geopriv