Hi Phil, Michael,

Based on our own internal study here are a few highlights that can shed
light on the ECMP accuracy:

1) once the number of flows (at hash granularity) becomes large
(~100,000), most hashes perform very well at a single hop for a wide
range of hash inputs.
2) when the number of flows in a hash is smaller (not that small
actually - on the order of 1000!) practically all hashes display large
statistical variation for some distributions of the hash inputs (where
large is on the order 20-40% or more of relative error per bucket).
This appears to be true for a wide set of hashes, including those shown
very robust in many academic studies.

The above is for single hop only, and for heterogeneous traffic rates.
If flow rates distribution is skewed (i.e. there are a few large flows),
it can become much worse wrt actual bandwidth distribution. Some of the
known ways to fix this issue is to employ hashes that look deeper into
the packets so that large flows (e.g. based on (src, dst) hash) get
split into smaller flows (e.g. look not only at src,dst, but also at
layer 4 info) - but note that these same methods cause, for our PCN
purposes, a problem that rsvp messages may no longer follow the same
path as the data packets. 

So I think realistically, for truly large aggregation at the core, ECMP
imbalances are likely to be not much of an issue, but for relatively
lower levels of aggregation one might expect that fairly large ECMP
inaccuracies in some cases are quite possible.  Note that "relatively
low aggregation" may occur for tunneled traffic (i.e. lots of small
flows are aggregated into a relatively small number of large tunnels). 

Anna 
 

> -----Original Message-----
> From: philip.eardley@bt.com [mailto:philip.eardley@bt.com] 
> Sent: Tuesday, November 06, 2007 4:13 AM
> To: menth@informatik.uni-wuerzburg.de; lars.eggert@nokia.com
> Cc: pcn@ietf.org
> Subject: RE: [PCN] Re: 5.5. Probing functions
> 
> Michael
> 
> Do you have an idea of how likely this is? will it be the 
> case that for a small increase in load all/most of the ECMP 
> paths become full? Does it depend strongly on topology? If 
> each link is shared by mnay ecmp paths from many different 
> ingress-egress-aggregates is this less of a risk?
> 
> phil
> 
> > -----Original Message-----
> > From: Michael Menth [mailto:menth@informatik.uni-wuerzburg.de]
> > Sent: 06 November 2007 08:29
> > To: Lars Eggert
> > Cc: Eardley,PL,Philip,CXR9 R; pcn@ietf.org
> > Subject: Re: [PCN] Re: 5.5. Probing functions
> > 
> > Hi Lars and Phil,
> > 
> > I think that one major motivation for probing is to make PCN capable
> to
> > deal with multipath routing like ECMP. Some path are empty, 
> others are 
> > full. Some flows must be rejected, others can proceed although they
> all
> > belong to the same ingress-egress-aggregate.
> > 
> > Regards,
> > 
> >     Michael
> > 
> > Lars Eggert wrote:
> > > Hi,
> > >
> > > On 2007-11-5, at 21:30, ext philip.eardley@bt.com wrote:
> > >> My impression is that probing advocates have 2 reasons:
> > >
> > > thanks for summarizing this! I think this does capture the main
> points
> > > made.
> > >
> > >> - the on-going ingress-egress-aggregate measurement of 
> marked pkts 
> > >> doesn't work. Because there's often no traffic on this 
> > >> ingress-egress-aggregate, or no traffic on the ECMP path; AND
> simply
> > >> admitting the flow has a significant risk of leading to overload
> [pkts
> > >> dropped &/or flows terminated]
> > >> - there's no signalling state on the PCN-boundary-nodes for this 
> > >> ingress-egress-aggregate (and no traffic already on it); AND the
> router
> > >> behaviour is such that every pkt is marked if the flow should be 
> > >> blocked; AND just admitting the call is dangerous 
> because there's a
> > good
> > >> chance of overload in some parts of the nw (near the 
> edge where the 
> > >> capacity is low). Hence probing creates no extra delay (since you
> have
> > >> to send a signalling set-up msg anyway). Often this scenario is
> people
> > >> who envisage PCN going to end terminals or at least further out
> than
> > >> core nw [I think].
> > >
> > > Although I agree that we can construct scenarios where "simply 
> > > admitting the flow has a significant risk of leading to 
> overload", I 
> > > still remain unclear on how likely such cases are in whatever we 
> > > envision the most likely deployment scenarios to be. Basically, I 
> > > worry that we're making the architecture complex for what will
> provide
> > > little real benefit in the end.
> > >
> > > Also, don't forget that using probing will mean dropping 
> the packets 
> > > of the flow waiting for admission for an RTT *every time* 
> probing is 
> > > used. Whereas "just admitting" may only cause overload sometimes.
> > > There is a tradeoff here - I believe it's possible to construct a
> case
> > > where probing hurts more than "just admitting."
> > >
> > > And I'm still waiting for someone to convince me that 
> dropping the 
> > > initial packets of a flow at the ingress during the 
> probing RTT will 
> > > result in something that is acceptable for apps.
> > >
> > >> The first scenario will only happen in narrow conditions [flash
> crowds,
> > >> many ingress-egress-aggregates without any traffic]. An 
> alternative
> way
> > >> of handling this is to limit the rate at which admit new 
> flows [at
> > least
> > >> this makes the conditions when the problem occurs even narrower].
> > >
> > > And we should try to determine how narrow these 
> conditions are. If 
> > > they're very narrow, we may be able to avoid them through 
> > > configuration suggestions (sufficient overprovisioning, 
> sufficiently 
> > > broad marking regions, etc.)
> > >
> > >>  The second scenario seems to break the (link) aggregation
> assumption?
> > >> presumably it could be solved by running different 
> mechanism at the
> > edge
> > >> compared with at the core. [joe, sorry if you've already 
> explained,
> the
> > >> late hour => even poorer memory than usual.]
> > >
> > > Yes, we seem to be running into the "the number of flows 
> across any 
> > > potential aggregation bottleneck is sufficiently large for
> stateless,
> > > statistical mechanisms to be effective" restriction from 
> the charter 
> > > here. These scenarios seems to require a deterministic mechanism, 
> > > while PCN is supposed to provide a probabilistic one.
> > >
> > > Lars
> > >
> --------------------------------------------------------------
> ----------
> > >
> > > _______________________________________________
> > > PCN mailing list
> > > PCN@ietf.org
> > > https://www1.ietf.org/mailman/listinfo/pcn
> > >
> > 
> > --
> > Dr. Michael Menth, Assistant Professor University of Wuerzburg, 
> > Institute of Computer Science Am Hubland, D-97074 
> Wuerzburg, Germany, 
> > room B206
> > phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 
> > mailto:menth@informatik.uni-wuerzburg.de
> > http://www3.informatik.uni-wuerzburg.de/research/ngn
> 
> 
> 
> _______________________________________________
> PCN mailing list
> PCN@ietf.org
> https://www1.ietf.org/mailman/listinfo/pcn
> 


_______________________________________________
PCN mailing list
PCN@ietf.org
https://www1.ietf.org/mailman/listinfo/pcn