[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[IPFIX] WG last call on draft-ietf-ipfix-export-per-sctp-stream-00
Dear all,
I did not have enough time for reviewing for the draft in Aug.
Although the time limit exceeds, I have reviewed the draft.
Please see my questions and comments and so on in lines.
Q1)
I think that the following condition is configurable choice.
- One stream treats one Template Record and associated Data Records.
- One stream treats some Template Records and associated Data Records
related to same Observation Domain, or same Observation Points.
It can deduce Data Record Loss per observation domain or observation points.
Does this draft cover the latter pattern ?
Q2)
This draft does not describe how to export some statistics option
template and data. Should they be exported on same stream exporting
associated Template and Data Records ?
>
>
>
> IPFIX Working Group B. Claise
> Internet-Draft P. Aitken
> Intended Status: Informational A. Johnson
> Expires: January 1, 2009 Cisco Systems, Inc.
> G. Muenz
> University of Tuebingen
> July 1, 2008
>
> IPFIX Export per SCTP Stream
> draft-ietf-ipfix-export-per-sctp-stream-00
>
>
> Status of this Memo
>
> By submitting this Internet-Draft, each author represents
> that any applicable patent or other IPR claims of which he
> or she is aware have been or will be disclosed, and any of
> which he or she becomes aware will be disclosed, in
> accordance with Section 6 of BCP 79.
>
> Internet-Drafts are working documents of the Internet
> Engineering Task Force (IETF), its areas, and its working
> groups. Note that other groups may also distribute
> working documents as Internet-Drafts.
>
snip
>
> 1. Terminology
>
> IPFIX-specific terminology used in this document is defined
> in section 2 of [RFC5101]. As in [RFC5101], these IPFIX-
> specific terms have the first letter of a word capitalized
> when used in this document.
>
> Template Reuse Delay
>
> The configurable timeout to allow the Collecting Process
> to receive and process the last Data Record using this
> Template information before which the Template Withdrawal
> Message MUST NOT be sent. [RFC5101] specifies a default
> value of 5 seconds.
>
Q3)
In first reading, I could not understand which process uses this timer,
Collecting process or Exporting Process.
I think that the description of RFC5101 is more easily understandable.
RFC5101 say as follows.
The Template Withdrawal Message MUST NOT be sent until sufficient
time has elapsed to allow the Collecting Process to receive and
process the last Data Record using this Template information.
>
> 1.1. IPFIX Documents Overview
>
> The IPFIX Protocol [RFC5101] provides network administrators
> with access to IP Flow information.
snip
>
> 2. Introduction
>
> The IPFIX working group has specified a protocol to export IP
> Flow information [RFC5101]. This protocol is designed to
> export information about IP traffic Flows and related
> measurement data, where a Flow is defined by a set of key
> attributes (e.g. source and destination IP address, source
> and destination port, etc.). However, thanks to its template
> mechanism, the IPFIX protocol can export any type of
> information, as long as the relevant Information Element is
> specified in the IPFIX Information Model [RFC5102],
> registered with IANA, or specified as an enterprise-specific
> Information Element.
>
> The IPFIX protocol [RFC5101] specifies that IP traffic
> measurements for Flows are exported using a TLV (type,
> length, value) format. The information is exported using a
> Template Record that is sent once to export the {type,
> length} pairs that define the data format for the Information
> Elements in a Flow. The Data Records specify values for each
> Flow.
>
> The IPFIX protocol [RFC5101] is flexible: it foresees the usage
> of the multiple SCTP streams per association; it allows the
> transmission of Data Sets, Template Sets, and/or Options
> Template Sets on any stream; it offers the full or partial
> reliability export of Data Sets; it proposes the ordered or out-
> of-order delivery of Data Sets. However, due to bandwidth
> restrictions and packet losses in the network as well as
> resource constraints on the Exporter and Collector (e.g.,
> limited buffer sizes), it is not always possible to export all
> Data Sets in a reliable way.
>
>
> Without delving into the details of the specifications described
> later on in this document, the basic idea is to export the
> Template Record and its associated Data Sets into a single
> unique SCTP stream, ideally to limit the Template ID to a single
> stream, while imposing in-order transmission.
>
> The specification in this document offers several advantages
> such as: calculation of Data Record losses in case of partially-
> reliable SCTP export, immediate export of the Template
> Withdrawal Message, immediate reuse of template ID within a
> stream, reduced likelihood of losing Data Record, and the
> Collecting Process's job is easier.
Q4)
This feature seems to be able to be applied to SCTP and PR-SCTP. But,
Introduction emphasizes the usage of PR-SCTP. What is advantages of
PR-SCTP situation over normal SCTP ?
Is it to deduce the loss count related to Template ID ?
>
>
> 2.1. Relationship with IPFIX and PSAMP
>
> The specification in this document applies to the IPFIX
> protocol specifications [RFC5101]. However, it only applies
> to the SCTP transport protocol [RFC4960] option of the IPFIX
> protocol specifications, specifically in the case of the
> partial reliability extension [RFC3758]. All specifications
> from [RFC5101] apply unless specified otherwise in this
> document.
>
> As the Packet Sampling (PSAMP) protocol specifications
> [PSAMP-PROTO] are based on the IPFIX protocol specifications,
> the specifications in this document are also valid for the
> PSAMP protocol. Therefore, the advantages specified by this
> document also apply to PSAMP.
>
>
> 2.2. Applicability
>
> The specifications are required in cases where we must know how
> many Data Records of a certain type (i.e. from a certain
> Template ID) were lost. Furthermore, they apply in cases where
> the Exporter can not afford to export all the Flow Records
> reliably, due to the limited resources to buffer the huge amount
> of flow records. Such situations may occur if Data Sets are
> generated at a higher rate at the Exporter than can be
> transferred to the Collector because of bandwidth limitations in
> the network or slow reception at the Collector. A typically
> example is a router exporting billing records.
>
> To be more precise, the specification applicability is the case
> where multiple Template IDs are sent within a SCTP Transport
> Session and the calculation of the Data Record loss for a
> particular one Template ID is required. Indeed, with the
> current IPFIX specifications [RFC5101], if an IPFIX Message is
> lost (UDP or SCTP partially reliable), it is not possible to
> determine to which Template ID of the Transport Session the lost
> Data Records belong to.
>
> In terms of Collector, there is backwards compatibility: the
> Collecting Process does not require any changes to support an
> Exporter that complies to the specifications in this document.
>
>
> 2.3. Limitations
>
> To be compliant with the specifications in this document, the
> Transport Session must support multiple SCTP streams.
> Furthermore, if the SCTP Transport Session does not support
> enough streams for the increasing number of Template ID in the
> Transport Session, the addition of streams must be supported
> according to [SCTP-RESET]. Alternatively, the new Template ID
> and associated Data Records may be added to an existing stream
> at the cost of diluting the granularity of Data Records loss.
> The other alternatives, which is not practical in operational
> networks, is to restart the SCTP association with an increase
> number of streams.
>
>
> 3. IPFIX Protocol Specifications Limitations and Improvements
>
snip
> 4. Specifications
>
> 4.1. Template Management
>
> This section introduces modifications compared to the Template
> Management section 8 in [RFC5101].
> As specified in [RFC5101], Template Sets and Options Template
> Sets MUST be sent reliably. In other words, any IPFIX Message
> containing an (Options) Template Set MUST be sent reliably.
>
> Any Data Sets associated with a Template Record MUST be sent on
> the same stream on which the Template Record was sent.
>
> The Exporter SHOULD send a single Template and associated Data
> Sets within a single stream in order to calculate the potential
> Data Record loss for this Template ID. However, the Exporter
> MAY group related Templates and their associated Data Sets
> within a single stream so loss statistics are calculated for the
> group. This may be suitable in cases where there is
> insufficient SCTP streams to send each Template on its own
> stream and/or the case where there are slight variations on a
> single Template to show that some fields were unavailable at the
> time of monitoring.
>
> If a SCTP stream contains a mixture of Data Records defined by a
> Template Record and Options Template Record(s), the Data Records
> defined by the Options Template Record(s) SHOULD be sent
> reliably within the same stream so that the Collector does not
> consider any loss to be associated with the Options Data.
> Indeed, if the Collector does not have the guarantee that the
Q5)
It seems Exporter, not Collector. Is it correct?
> Data Records defined by the Options Template Record are sent
> reliably, the Collector can not determine whether the loss in
> that stream belongs to the Data Records defined by the Template
> Record, defined by the Option Template Record, or by both of
> them. By sending the Options Data reliably (which is usually
> required to interpret the Data Records correctly), any loss will
> be limited to the non-option Data Record and loss can still be
> calculated on a per Template basis.
>
> For each (Options) Template Record, the Exporting Process MUST
> send the Data Record Reliability Option Template using an Option
> Template with the following Information Elements:
>
> SCOPE: Template ID
> NOT-SCOPE: dataRecordsReliability
>
> The Data Record Reliability Option Template MUST be sent
> reliably.
>
Q6)
I cannot clearly understand how to use this field
"dataRecordsReliability".
"Data Record Reliability Option Template" might be applied to Option
Template only? How is it handled by Collector for consecutive Data
Record after receiving the option template?
If Collector does not receive this template, what's going to happen?
>
> The Option Data Record SHOULD be sent before the Data Record
> that needs it so that it arrives first and is available for the
> Collector to use.
>
>
> 4.2. New Information Element
>
> dataRecordsReliability
>
> Description:
> The Data Records reliability associated with this
> Template ID. The integer value 1 means that the Data
> Records are sent reliably, while the integer value 2
> means that the Data Records are not sent reliably.
> Abstract Data Type: boolean
> Data Type Semantics: identifier
> ElementId: xxx
> Status: current
>
>
> 4.3. SCTP
>
> This section introduces modifications compared to the "SCTP"
> section 10.2 (and subsections) in [RFC5101]. More specifically
> the "Stream" section 10.2.4.3
>
> PR-SCTP [RFC3758] MUST be implemented by all compliant
> implementations.
>
> All IPFIX Messages MUST be sent in order within a stream.
>
> Depending on the application requirement, the Exporting Process
> MAY send Data Sets with full or partial reliability. Unreliable
> data transfer MAY be used where the application does not require
> reliable transmission or the use of a retransmission queue is
> impractical due to resource restrictions at the Exporter.
>
>
> If the Exporting Process requires to export a new Template but
> there are no more free SCTP streams available, it SHOULD attempt
> to increase the number of outbound streams it is able to send
> to, per [SCTP-RESET]. Alternatively, the Exporting Process MAY
> add the Template Set and Data Records to an existing stream at
> the cost of diluting the granularity of Data Records loss.
>
>
>
> 4.4. Template Withdrawal Message
>
> This section introduces Template Withdrawal Message-related
> modifications compared to the Template Management section 8 in
> [RFC5101].
>
> Templates that are not used anymore SHOULD be deleted. Before
> reusing a Template ID, the Template MUST be deleted. In order
> to delete an allocated Template, the Template is withdrawn
> through the use of a Template Withdrawal Message. The Template
> Withdrawal Message MUST be sent on the same stream as the
> Template Record.
>
> As the Template Withdrawal Message MUST be sent reliably, using
> SCTP-ordered delivery per [RFC5101], and as all IPFIX Messages
> are sent in order within a stream (per the specifications in
> this document), the IPFIX Message containing the Template
> Withdrawal Message will not arrive at the Collecting Process
> before any associated and previously sent Data Record. As a
> consequence, no Data Records will be lost due to delayed arrival
> at the Collector.
>
> The Template ID from a withdrawn Template MAY be reused on the
> same stream immediately after the Template Withdrawal Message is
> sent. This case is equivalent to the use of a Template Reuse
> Delay value of 0.
>
> If the new definition of the Template ID is to be reused on a
> different stream, the Template Withdrawal Message MUST NOT be
> sent before the Template Reuse Delay.
>
> A Template Withdrawal Message to withdraw all Templates for the
> Observation Domain ID specified in the IPFIX Message header MUST
> NOT be used.
>
> Multiple Template IDs MAY be withdrawn with a single Template
> Withdrawal Message at the condition that all the Template IDs in
> the Template Withdrawal Message are used on the same SCTP
> stream.
>
>
> 4.5. The Collecting Process's Side
>
> This section introduces modifications to the Collection Process
> as compared to section 9 in [RFC5101].
>
>
> The Collecting Process SHOULD listen for a new association
> request from the Exporting Process. The Exporting Process will
> request a number of streams to use for export: the number of
> streams SHOULD be equivalent to the number of simultaneous
> Template Records used in the association. Note that the Template
> Records don't include the Options Template Records.
>
> A Collecting Process SHOULD support the procedure for the
> addition of an SCTP stream [SCTP-RESET].
>
> The IPFIX protocol has a Sequence Number field in the IPFIX
> Message header that increases with the number of IPFIX Data
> Records in the IPFIX Message. A Collector may detect out-of-
> sequence, dropped, or duplicate IPFIX Messages by tracking the
> Sequence Number. As this Sequence Number is per SCTP stream,
> the loss for the Data Records sent in that stream can be
> calculated in case of partially-reliable export.
>
> If the Collecting Process receives a Template Withdrawal Message
> on a different stream than the one on which the Template ID is
> used, then the Collecting Process MUST reset the association and
> SHOULD log an error message.
Q7)
I think it is difficult to distinguish the session to use the export
per stream function. Because RFC5101 said the Template Withdrawal
Message may be sent on any SCTP stream.
>
> The following sentences from [RFC5101] are not applicable in
> this specification:
>
> "The Collecting Process normally receives Template Records
> from the Exporting Process before receiving Data Records.
> The Data Records are then decoded and stored by the
> Collector. If the Template Records have not been received at
> the time Data Records are received, the Collecting Process
> MAY store the Data Records for a short period of time and
> decode them after the Template Records are received."
>
>
> 5. Examples
Q8)
I hope that the examples include "Data Record Reliability Option
Template".
>
> Figure 1 shows an example where the stream 10 carries a Template
> with the Template ID 256 transmitted with full reliability (FR),
> together with associated Data Records transmitted with partial
> reliability (PR). Note that, because all IPFIX Messages are
> sent in order within a stream, the Template 256 will always be
> processed before the Data Records by the Collecting Process.
> Therefore, the Collecting Process job is simplified.
> Furthermore, the Data Record loss for the Template 256 can
> easily be calculated on the Collecting Process.
>
>
> +--------+ +---------+ +----------+
> | | | | | |
> stream 10 ----| Data | . . . | Data |---| Template |--->
> | 256 | | 256 | | 256 |
> | PR| | PR| | FR|
> +--------+ +---------+ +----------+
> Figure 1
>
> If an Option Template is necessary to understand the content of
> a Data Record (i.e. the scope in the Options Template Record is
> an Information Element contained in the Data Record), the
> Options Template Record may be sent in the same stream, as
> displayed in figure 2.
>
> +--------+ +--------+ +----------+
> | | | | | |
> stream 20 ... ---| Data |...| Data |-----| Template |---
> | 258 | | 258 | | 258 |
> | PR| | PR| | FR|
> +--------+ +--------+ +----------+
>
> +--------+ +----------+
> | | | Options |
> ...---| Data |-------| Template |------>
> | 257 | | 257 |
> | FR| | FR|
> +--------+ +----------+
> Figure 2
>
> Figure 2 shows an example where stream 20 carries an Options
> Template with Template ID 257 transmitted with full reliability
> (FR), an associated Data Record transmitted with full
> reliability (FR), a Template with Template ID 258 transmitted
> with full reliability (FR), and associated Data Records
> transmitted with partial reliability (PR). In this example the
> Option Template Record contains information required to decode
> the latter Data Records, such as Common Properties information
> [IPFIX-RED-RED]. So it makes sense to export the Data Sets 257
> reliably. If some Data Record loss is observed from the
> Sequence Number , the loss can only stem from the Data Sets with
> the Template ID 258, as these are the only Sets not exported
> reliably. Therefore, the calculation of loss per Template ID
> 258 is possible.
>
>
> Note that, because all IPFIX Messages must be sent in order
> within a stream, the Options Template 257 will always arrive
> before its associated Data Records, and that the Template 259
Q9)
Is it correct, 259 or 258?
> will always arrive before the its associated Data Records.
>
> Figure 3 shows an example where stream 30 carries a Template
> with Template ID 259 transmitted with full reliability (FR), an
> associated Data Record transmitted with partial reliability
> (PR), a Template Withdrawal Message, followed by a redefinition
> of the Template ID 259, and finally the new definition of Data
> Record transmitted with partial reliability. The Template
> Withdrawal Message and the new definition of the Template ID 259
> are sent immediately, without waiting for the Template Reuse
> Delay.
>
>
> +--------+ +----------+ +----------+
> | | | | | Template |
> stream 30 ... ---| Data |...| Template |-----| Withdraw.|---
> | 259 | | 259 | | 259 |
> | PR| | FR| | FR|
> +--------+ +----------+ +----------+
>
> +--------+ +----------+
> | | | |
> ...---| Data |-------| Template |------>
> | 259 | | 259 |
> | PR| | FR|
> +--------+ +----------+
>
> Figure 3
>
> 6. IANA Considerations
>
> The dataRecordsReliability Information Element must be requested
> from IANA, following the process in [RFC5102].
>
>
---
Atsushi KOBAYASHI <akoba at nttv6.net>
NTT Information Sharing Platform Lab.
tel:+81-(0)422-59-3978 fax:+81-(0)422-59-5637
_______________________________________________
IPFIX mailing list
IPFIX at ietf.org
https://www.ietf.org/mailman/listinfo/ipfix