Internet Engineering Task Force (IETF)                       B. Trammell
Request for Comments: 7015                                    ETH Zurich
Category: Standards Track                                      A. Wagner
ISSN: 2070-1721                                              Consecom AG
                                                               B. Claise
                                                     Cisco Systems, Inc.
                                                          September 2013
        
Internet Engineering Task Force (IETF)                       B. Trammell
Request for Comments: 7015                                    ETH Zurich
Category: Standards Track                                      A. Wagner
ISSN: 2070-1721                                              Consecom AG
                                                               B. Claise
                                                     Cisco Systems, Inc.
                                                          September 2013
        

Flow Aggregation for the IP Flow Information Export (IPFIX) Protocol

IP流信息导出(IPFIX)协议的流聚合

Abstract

摘要

This document provides a common implementation-independent basis for the interoperable application of the IP Flow Information Export (IPFIX) protocol to the handling of Aggregated Flows, which are IPFIX Flows representing packets from multiple Original Flows sharing some set of common properties. It does this through a detailed terminology and a descriptive Intermediate Aggregation Process architecture, including a specification of methods for Original Flow counting and counter distribution across intervals.

本文档为IP流信息导出(IPFIX)协议的互操作应用程序提供了一个通用的独立于实现的基础,该协议用于处理聚合流,聚合流是IPFIX流,表示来自多个原始流的数据包,共享一些公共属性集。它通过一个详细的术语和一个描述性的中间聚合过程体系结构来实现这一点,包括一个原始流计数方法规范和跨间隔的计数器分布。

Status of This Memo

关于下段备忘

This is an Internet Standards Track document.

这是一份互联网标准跟踪文件。

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741.

本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。有关互联网标准的更多信息,请参见RFC 5741第2节。

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc7015.

有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问http://www.rfc-editor.org/info/rfc7015.

Copyright Notice

版权公告

Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.

版权所有(c)2013 IETF信托基金和确定为文件作者的人员。版权所有。

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(http://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。

Table of Contents
   1. Introduction ....................................................3
      1.1. IPFIX Protocol Overview ....................................4
      1.2. IPFIX Documents Overview ...................................5
   2. Terminology .....................................................5
   3. Use Cases for IPFIX Aggregation .................................7
   4. Architecture for Flow Aggregation ...............................8
      4.1. Aggregation within the IPFIX Architecture ..................8
      4.2. Intermediate Aggregation Process Architecture .............12
           4.2.1. Correlation and Normalization ......................14
   5. IP Flow Aggregation Operations .................................15
      5.1. Temporal Aggregation through Interval Distribution ........15
           5.1.1. Distributing Values across Intervals ...............16
           5.1.2. Time Composition ...................................18
           5.1.3. External Interval Distribution .....................19
      5.2. Spatial Aggregation of Flow Keys ..........................19
           5.2.1. Counting Original Flows ............................21
           5.2.1. Counting Distinct Key Values .......................22
      5.3. Spatial Aggregation of Non-key Fields .....................22
           5.3.1. Counter Statistics .................................22
           5.3.2. Derivation of New Values from Flow Keys and
                  Non-key fields .....................................23
      5.4. Aggregation Combination ...................................23
   6. Additional Considerations and Special Cases in Flow
      Aggregation ....................................................24
      6.1. Exact versus Approximate Counting during Aggregation ......24
      6.2. Delay and Loss Introduced by the IAP ......................24
      6.3. Considerations for Aggregation of Sampled Flows ...........24
      6.4. Considerations for Aggregation of Heterogeneous Flows .....25
   7. Export of Aggregated IP Flows Using IPFIX ......................25
      7.1. Time Interval Export ......................................25
      7.2. Flow Count Export .........................................25
           7.2.1. originalFlowsPresent ...............................26
        
Table of Contents
   1. Introduction ....................................................3
      1.1. IPFIX Protocol Overview ....................................4
      1.2. IPFIX Documents Overview ...................................5
   2. Terminology .....................................................5
   3. Use Cases for IPFIX Aggregation .................................7
   4. Architecture for Flow Aggregation ...............................8
      4.1. Aggregation within the IPFIX Architecture ..................8
      4.2. Intermediate Aggregation Process Architecture .............12
           4.2.1. Correlation and Normalization ......................14
   5. IP Flow Aggregation Operations .................................15
      5.1. Temporal Aggregation through Interval Distribution ........15
           5.1.1. Distributing Values across Intervals ...............16
           5.1.2. Time Composition ...................................18
           5.1.3. External Interval Distribution .....................19
      5.2. Spatial Aggregation of Flow Keys ..........................19
           5.2.1. Counting Original Flows ............................21
           5.2.1. Counting Distinct Key Values .......................22
      5.3. Spatial Aggregation of Non-key Fields .....................22
           5.3.1. Counter Statistics .................................22
           5.3.2. Derivation of New Values from Flow Keys and
                  Non-key fields .....................................23
      5.4. Aggregation Combination ...................................23
   6. Additional Considerations and Special Cases in Flow
      Aggregation ....................................................24
      6.1. Exact versus Approximate Counting during Aggregation ......24
      6.2. Delay and Loss Introduced by the IAP ......................24
      6.3. Considerations for Aggregation of Sampled Flows ...........24
      6.4. Considerations for Aggregation of Heterogeneous Flows .....25
   7. Export of Aggregated IP Flows Using IPFIX ......................25
      7.1. Time Interval Export ......................................25
      7.2. Flow Count Export .........................................25
           7.2.1. originalFlowsPresent ...............................26
        
           7.2.2. originalFlowsInitiated .............................26
           7.2.3. originalFlowsCompleted .............................26
           7.2.4. deltaFlowCount .....................................26
      7.3. Distinct Host Export ......................................27
           7.3.1. distinctCountOfSourceIPAddress .....................27
           7.3.2. distinctCountOfDestinationIPAddress ................27
           7.3.3. distinctCountOfSourceIPv4Address ...................27
           7.3.4. distinctCountOfDestinationIPv4Address ..............28
           7.3.5. distinctCountOfSourceIPv6Address ...................28
           7.3.6. distinctCountOfDestinationIPv6Address ..............28
      7.4. Aggregate Counter Distribution Export .....................28
           7.4.1. Aggregate Counter Distribution Options Template ....29
           7.4.2. valueDistributionMethod Information Element ........29
   8. Examples .......................................................31
      8.1. Traffic Time Series per Source ............................32
      8.2. Core Traffic Matrix .......................................37
      8.3. Distinct Source Count per Destination Endpoint ............42
      8.4. Traffic Time Series per Source with Counter Distribution ..44
   9. Security Considerations ........................................46
   10. IANA Considerations ...........................................46
   11. Acknowledgments ...............................................46
   12. References ....................................................47
      12.1. Normative References .....................................47
      12.2. Informative References ...................................47
        
           7.2.2. originalFlowsInitiated .............................26
           7.2.3. originalFlowsCompleted .............................26
           7.2.4. deltaFlowCount .....................................26
      7.3. Distinct Host Export ......................................27
           7.3.1. distinctCountOfSourceIPAddress .....................27
           7.3.2. distinctCountOfDestinationIPAddress ................27
           7.3.3. distinctCountOfSourceIPv4Address ...................27
           7.3.4. distinctCountOfDestinationIPv4Address ..............28
           7.3.5. distinctCountOfSourceIPv6Address ...................28
           7.3.6. distinctCountOfDestinationIPv6Address ..............28
      7.4. Aggregate Counter Distribution Export .....................28
           7.4.1. Aggregate Counter Distribution Options Template ....29
           7.4.2. valueDistributionMethod Information Element ........29
   8. Examples .......................................................31
      8.1. Traffic Time Series per Source ............................32
      8.2. Core Traffic Matrix .......................................37
      8.3. Distinct Source Count per Destination Endpoint ............42
      8.4. Traffic Time Series per Source with Counter Distribution ..44
   9. Security Considerations ........................................46
   10. IANA Considerations ...........................................46
   11. Acknowledgments ...............................................46
   12. References ....................................................47
      12.1. Normative References .....................................47
      12.2. Informative References ...................................47
        
1. Introduction
1. 介绍

The assembly of packet data into Flows serves a variety of different purposes, as noted in the requirements [RFC3917] and applicability statement [RFC5472] for the IP Flow Information Export (IPFIX) protocol [RFC7011]. Aggregation beyond the Flow level, into records representing multiple Flows, is a common analysis and data reduction technique as well, with applicability to large-scale network data analysis, archiving, and inter-organization exchange. This applicability in large-scale situations, in particular, led to the inclusion of aggregation as part of the IPFIX Mediation Problem Statement [RFC5982], and the definition of an Intermediate Aggregation Process in the Mediator framework [RFC6183].

如IP流信息导出(IPFIX)协议[RFC7011]的要求[RFC3917]和适用性声明[RFC5472]中所述,将分组数据组装成流可用于各种不同的目的。超出流级别的聚合到表示多个流的记录中,也是一种常见的分析和数据缩减技术,适用于大规模网络数据分析、归档和组织间交换。特别是在大规模情况下的这种适用性,导致将聚合作为IPFIX调解问题声明[RFC5982]的一部分,并在中介框架[RFC6183]中定义中间聚合过程。

Aggregation is used for analysis and data reduction in a wide variety of applications, for example, in traffic matrix calculation, generation of time series data for visualizations or anomaly detection, or data reduction for long-term trending and storage. Depending on the keys used for aggregation, it may additionally have an anonymizing effect on the data: for example, aggregation operations that eliminate IP addresses make it impossible to later directly identify nodes using those addresses.

聚合用于各种应用中的分析和数据缩减,例如,在流量矩阵计算中,生成用于可视化或异常检测的时间序列数据,或用于长期趋势分析和存储的数据缩减。根据用于聚合的密钥,它还可能对数据产生匿名效应:例如,消除IP地址的聚合操作使得以后无法使用这些地址直接识别节点。

Aggregation, as defined and described in this document, covers the applications defined in [RFC5982], including Sections 5.1 "Adjusting Flow Granularity", 5.4 "Time Composition", and 5.5 "Spatial Composition". However, Section 4.2 of this document specifies a more flexible architecture for an Intermediate Aggregation Process than that envisioned by the original Mediator work [RFC5982]. Instead of a focus on these specific limited use cases, the Intermediate Aggregation Process is specified to cover any activity commonly described as "Flow aggregation". This architecture is intended to describe any such activity without reference to the specific implementation of aggregation.

本文件中定义和描述的聚合涵盖了[RFC5982]中定义的应用,包括第5.1节“调整流粒度”、第5.4节“时间组成”和第5.5节“空间组成”。然而,本文件第4.2节规定了一种比原始中介工作[RFC5982]设想的更灵活的中间聚合过程体系结构。与关注这些特定的有限用例不同,中间聚合过程被指定为涵盖通常称为“流聚合”的任何活动。该体系结构旨在描述任何此类活动,而不参考聚合的具体实现。

An Intermediate Aggregation Process may be applied to data collected from multiple Observation Points, as it is natural to use aggregation for data reduction when concentrating measurement data. This document specifically does not address the protocol issues that arise when combining IPFIX data from multiple Observation Points and exporting from a single Mediator, as these issues are general to IPFIX Mediation; they are therefore treated in detail in the Mediation Protocol document [IPFIX-MED-PROTO].

中间聚合过程可应用于从多个观测点收集的数据,因为在集中测量数据时,使用聚合进行数据缩减是很自然的。本文件并未具体说明组合来自多个观测点的IPFIX数据并从单个中介导出时出现的协议问题,因为这些问题对于IPFIX中介而言是一般性的;因此,在调解协议文档[IPFIX-MED-PROTO]中对它们进行了详细的处理。

Since Aggregated Flows as defined in the following section are essentially Flows, the IPFIX protocol [RFC7011] can be used to export, and the IPFIX File Format [RFC5655] can be used to store, aggregated data "as is"; there are no changes necessary to the protocol. This document provides a common basis for the application of IPFIX to the handling of aggregated data, through a detailed terminology, Intermediate Aggregation Process architecture, and methods for Original Flow counting and counter distribution across intervals. Note that Sections 5, 6, and 7 of this document are normative.

由于下节中定义的聚合流本质上是流,因此可以使用IPFIX协议[RFC7011]导出,并且可以使用IPFIX文件格式[RFC5655]按“原样”存储聚合数据;协议无需更改。本文档通过详细的术语、中间聚合过程体系结构以及原始流计数和计数器跨时间间隔分布的方法,为IPFIX在聚合数据处理中的应用提供了通用基础。请注意,本文件第5、6和7节是规范性的。

1.1. IPFIX Protocol Overview
1.1. IPFIX协议概述

In the IPFIX protocol, { type, length, value } tuples are expressed in Templates containing { type, length } pairs, specifying which { value } fields are present in data records conforming to the Template, giving great flexibility as to what data is transmitted. Since Templates are sent very infrequently compared with Data Records, this results in significant bandwidth savings. Various different data formats may be transmitted simply by sending new Templates specifying the { type, length } pairs for the new data format. See [RFC7011] for more information.

在IPFIX协议中,{type,length,value}元组在包含{type,length}对的模板中表示,指定符合模板的数据记录中存在哪些{value}字段,从而在传输什么数据方面提供了极大的灵活性。由于与数据记录相比,发送模板的频率非常低,因此可以显著节省带宽。通过发送指定新数据格式的{type,length}对的新模板,可以简单地传输各种不同的数据格式。有关更多信息,请参阅[RFC7011]。

The IPFIX Information Element Registry [IANA-IPFIX] defines a large number of standard Information Elements that provide the necessary { type } information for Templates. The use of standard elements enables interoperability among different vendors' implementations.

IPFIX信息元素注册表[IANA-IPFIX]定义了大量标准信息元素,为模板提供必要的{type}信息。使用标准元素可以实现不同供应商实现之间的互操作性。

Additionally, non-standard enterprise-specific elements may be defined for private use.

此外,非标准企业特定元素可定义为私人使用。

1.2. IPFIX Documents Overview
1.2. IPFIX文档概述

"Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information" [RFC7011] and its associated documents define the IPFIX protocol, which provides network engineers and administrators with access to IP traffic Flow information.

“交换流量信息的IP流量信息导出(IPFIX)协议规范”[RFC7011]及其相关文件定义了IPFIX协议,该协议为网络工程师和管理员提供了访问IP流量信息的权限。

IPFIX has a formal description of IPFIX Information Elements, their names, types, and additional semantic information, as specified in the IPFIX Information Model [RFC7012]. The IPFIX Information Element registry [IANA-IPFIX] is maintained by IANA. New Information Element definitions can be added to this registry subject to an Expert Review [RFC5226], with additional process considerations described in [RFC7013].

按照IPFIX信息模型[RFC7012]的规定,IPFIX对IPFIX信息元素、它们的名称、类型和附加语义信息有一个正式的描述。IPFIX信息元素注册表[IANA-IPFIX]由IANA维护。经专家审查[RFC5226]后,可将新的信息元素定义添加到此注册表中,并在[RFC7013]中说明其他过程注意事项。

"Architecture for IP Flow Information Export" [RFC5470] defines the architecture for the export of measured IP Flow information out of an IPFIX Exporting Process to an IPFIX Collecting Process and the basic terminology used to describe the elements of this architecture, per the requirements defined in "Requirements for IP Flow Information Export" [RFC3917]. The IPFIX protocol document [RFC7011] covers the details of the method for transporting IPFIX Data Records and Templates via a congestion-aware transport protocol from an IPFIX Exporting Process to an IPFIX Collecting Process.

“IP流信息导出体系结构”[RFC5470]根据“IP流信息导出要求”中定义的要求,定义了将测得的IP流信息从IPFIX导出过程导出到IPFIX收集过程的体系结构,以及用于描述此体系结构元素的基本术语[RFC3917]。IPFIX协议文档[RFC7011]详细介绍了通过拥塞感知传输协议将IPFIX数据记录和模板从IPFIX导出进程传输到IPFIX收集进程的方法。

"IP Flow Information Export (IPFIX) Mediation: Problem Statement" [RFC5982] introduces the concept of IPFIX Mediators, and defines the use cases for which they were designed; "IP Flow Information Export (IPFIX) Mediation: Framework" [RFC6183] then provides an architectural framework for Mediators. Protocol-level issues (e.g., Template and Observation Domain handling across Mediators) are covered by "Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX Mediators" [IPFIX-MED-PROTO].

“IP流信息导出(IPFIX)中介:问题陈述”[RFC5982]介绍了IPFIX中介的概念,并定义了为其设计的用例;“IP流信息导出(IPFIX)中介:框架”[RFC6183]然后为中介提供了一个体系结构框架。协议级问题(例如,跨中介体的模板和观察域处理)由“IPFIX中介体上IP流信息导出(IPFIX)协议的操作”[IPFIX-MED-PROTO]涵盖。

This document specifies an Intermediate Process for Flow aggregation that may be applied at an IPFIX Mediator, as well as at an original Observation Point prior to export, or for analysis and data reduction purposes after receipt at a Collecting Process.

本文档规定了流聚合的中间过程,可在IPFIX中介处应用,也可在导出前在原始观察点应用,或在收集过程中接收后用于分析和数据缩减目的。

2. Terminology
2. 术语

Terms used in this document that are defined in the Terminology section of the IPFIX protocol document [RFC7011] are to be interpreted as defined there.

IPFIX协议文件[RFC7011]术语部分中定义的本文件中使用的术语应按照此处定义进行解释。

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

本文件中的关键词“必须”、“不得”、“必需”、“应”、“不应”、“应”、“不应”、“建议”、“可”和“可选”应按照[RFC2119]中所述进行解释。

In addition, this document defines the following terms:

此外,本文件定义了以下术语:

Aggregated Flow: A Flow, as defined by [RFC7011], derived from a set of zero or more Original Flows within a defined Aggregation Interval. Note that an Aggregated Flow is defined in the context of an Intermediate Aggregation Process only. Once an Aggregated Flow is exported, it is essentially a Flow as in [RFC7011] and can be treated as such.

聚合流:由[RFC7011]定义的流,从定义的聚合间隔内的一组零或多个原始流派生而来。请注意,聚合流仅在中间聚合过程的上下文中定义。一旦导出聚合流,它本质上就是[RFC7011]中所述的流,可以这样处理。

Intermediate Aggregation Process: an Intermediate Aggregation Process (IAP), as in [RFC6183], that aggregates records, based upon a set of Flow Keys or functions applied to fields from the record.

中间聚合过程:一种中间聚合过程(IAP),如[RFC6183]中所述,它根据应用于记录字段的一组流键或函数聚合记录。

Aggregation Interval: A time interval imposed upon an Aggregated Flow. Intermediate Aggregation Processes may use a regular Aggregation Interval (e.g., "every five minutes", "every calendar month"), though regularity is not necessary. Aggregation intervals may also be derived from the time intervals of the Original Flows being aggregated.

聚合间隔:对聚合流施加的时间间隔。中间聚合过程可以使用规则的聚合间隔(例如,“每五分钟”、“每个日历月”),但不需要规则性。聚合间隔也可以从被聚合的原始流的时间间隔派生。

Partially Aggregated Flow: A Flow during processing within an Intermediate Aggregation Process; refers to an intermediate data structure during aggregation within the Intermediate Aggregation Process architecture detailed in Section 4.2.

部分聚合流:中间聚合过程中处理期间的流;指第4.2节详述的中间聚合过程体系结构中聚合期间的中间数据结构。

Original Flow: A Flow given as input to an Intermediate Aggregation Process in order to generate Aggregated Flows.

原始流:作为中间聚合过程输入的流,用于生成聚合流。

Contributing Flow: An Original Flow that is partially or completely represented within an Aggregated Flow. Each Aggregated Flow is made up of zero or more Contributing Flows, and an Original Flow may contribute to zero or more Aggregated Flows.

贡献流:在聚合流中部分或完全表示的原始流。每个聚合流由零个或多个贡献流组成,原始流可能贡献零个或多个聚合流。

Original Exporter: The Exporter from which the Original Flows are received; meaningful only when an IAP is deployed at a Mediator.

原始出口商:收到原始流量的出口商;仅当IAP部署在中介时才有意义。

The terminology presented herein improves the precision of, but does not supersede or contradict the terms related to, Mediation and aggregation defined in the Mediation Problem Statement [RFC5982] and the Mediation Framework [RFC6183] documents. Within this document, the terminology defined in this section is to be considered normative.

本文提供的术语提高了调解问题声明[RFC5982]和调解框架[RFC6183]文档中定义的调解和聚合的精度,但不取代或与之相矛盾。在本文件中,本节中定义的术语应视为规范性术语。

3. Use Cases for IPFIX Aggregation
3. IPFIX聚合的用例

Aggregation, as a common data reduction method used in traffic data analysis, has many applications. When used with a regular Aggregation Interval and Original Flows containing timing information, it generates time series data from a collection of Flows with discrete intervals, as in the example in Section 8.1. This time series data is itself useful for a wide variety of analysis tasks, such as generating input for network anomaly detection systems or driving visualizations of volume per time for traffic with specific characteristics. As a second example, traffic matrix calculation from Flow data, as shown in Section 8.2 is inherently an aggregation action, by spatially aggregating the Flow Key down to input or output interface, address prefix, or autonomous system (AS).

聚合作为交通数据分析中常用的数据约简方法,有着广泛的应用。当与规则聚合间隔和包含计时信息的原始流一起使用时,它从具有离散间隔的流集合中生成时间序列数据,如第8.1节中的示例所示。该时间序列数据本身可用于各种分析任务,例如为网络异常检测系统生成输入,或驱动具有特定特征的流量每次体积的可视化。作为第二个示例,如第8.2节所示,从流量数据计算流量矩阵本质上是一种聚合行为,通过将流量键向下空间聚合到输入或输出接口、地址前缀或自治系统(As)。

Irregular or data-dependent Aggregation Intervals and key aggregation operations can also be used to provide adaptive aggregation of network Flow data. Here, full Flow Records can be kept for Flows of interest, while Flows deemed "less interesting" to a given application can be aggregated. For example, in an IPFIX Mediator equipped with traffic classification capabilities for security purposes, potentially malicious Flows could be exported directly, while known-good or probably-good Flows (e.g., normal web browsing) could be exported simply as time series volumes per web server.

不规则或依赖数据的聚合间隔和关键聚合操作也可用于提供网络流数据的自适应聚合。在这里,可以为感兴趣的流保留完整的流记录,而对给定应用程序来说“不太感兴趣”的流可以聚合。例如,在配备了用于安全目的的流量分类功能的IPFIX中介中,可以直接导出潜在的恶意流,而已知良好或可能良好的流(例如,正常web浏览)可以简单地导出为每个web服务器的时间序列卷。

Aggregation can also be applied to final analysis of stored Flow data, as shown in the example in Section 8.3. All such aggregation applications in which timing information is not available or not important can be treated as if an infinite Aggregation Interval applies.

如第8.3节中的示例所示,聚合也可应用于存储流量数据的最终分析。在所有此类聚合应用程序中,如果时间信息不可用或不重要,则可以将其视为无限聚合间隔。

Note that an Intermediate Aggregation Process that removes potentially sensitive information as identified in [RFC6235] may tend to have an anonymizing effect on the Aggregated Flows as well; however, any application of aggregation as part of a data protection scheme should ensure that all the issues raised in [RFC6235] are addressed, specifically Sections 4 ("Anonymization of IP Flow Data"), 7.2 ("IPFIX-Specific Anonymization Guidelines"), and 9 ("Security Considerations").

注意,删除[RFC6235]中标识的潜在敏感信息的中间聚合过程也可能对聚合流产生匿名效应;但是,作为数据保护方案一部分的任何聚合应用应确保[RFC6235]中提出的所有问题都得到解决,特别是第4节(“IP流数据的匿名化”)、第7.2节(“特定于IPFIX的匿名化指南”)和第9节(“安全注意事项”)。

While much of the discussion in this document, and all of the examples, apply to the common case that the Original Flows to be aggregated are all of the same underlying type (i.e., are represented with identical Templates or compatible Templates containing a core set Information Elements that can be freely converted to one another), and that each packet observed by the Metering Process associated with the Original Exporter is represented, this is not a necessary assumption. Aggregation can also be applied as part of a

虽然本文档中的大部分讨论和所有示例适用于以下常见情况:要聚合的原始流都是相同的基础类型(即,用相同的模板或包含可自由转换为彼此的核心集信息元素的兼容模板表示),并且,与原始导出器相关联的计量过程所观察到的每个数据包都被表示,这不是一个必要的假设。聚合也可以作为

technique using both aggregation and correlation to pull together multiple views of the same traffic from different Observation Points using different Templates. For example, consider a set of applications running at different Observation Points for different purposes -- one generating Flows with round-trip times for passive performance measurement, and one generating billing records. Once correlated, these Flows could be used to produce Aggregated Flows containing both volume and performance information together. The correlation and normalization operation described in Section 4.2.1 handles this specific case of correlation. Flow correlation in the general case is outside the scope of this document.

使用聚合和关联的技术,使用不同的模板将来自不同观测点的相同流量的多个视图汇集在一起。例如,考虑在不同的观察点运行的一组不同目的的应用程序——一个具有用于被动性能测量的往返时间的生成流,以及一个生成计费记录的流程。一旦关联起来,这些流就可以用来生成包含卷和性能信息的聚合流。第4.2.1节中描述的关联和规范化操作处理这种特定的关联情况。一般情况下的流量关联不在本文件范围内。

4. Architecture for Flow Aggregation
4. 流聚合的体系结构

This section specifies the architecture of the Intermediate Aggregation Process and how it fits into the IPFIX architecture.

本节指定了中间聚合过程的体系结构,以及它如何适应IPFIX体系结构。

4.1. Aggregation within the IPFIX Architecture
4.1. IPFIX体系结构中的聚合

An Intermediate Aggregation Process could be deployed at any of three places within the IPFIX architecture. While aggregation is most commonly done within a Mediator that collects Original Flows from an Original Exporter and exports Aggregated Flows, aggregation can also occur before initial export, or after final collection, as shown in Figure 1. The presence of an IAP at any of these points is, of course, optional.

中间聚合过程可以部署在IPFIX体系结构中的三个位置中的任意一个。虽然聚合通常在从原始导出器收集原始流并导出聚合流的中介中完成,但聚合也可以在初始导出之前或最终收集之后进行,如图1所示。当然,在这些点中的任何一点上存在IAP都是可选的。

   +===========================================+
   |  IPFIX Exporter        +----------------+ |
   |                        | Metering Proc. | |
   | +-----------------+    +----------------+ |
   | | Metering Proc.  | or |      IAP       | |
   | +-----------------+----+----------------+ |
   | |           Exporting Process           | |
   | +-|----------------------------------|--+ |
   +===|==================================|====+
       |                                  |
   +===|===========================+      |
   |   |  Aggregating Mediator     |      |
   + +-V-------------------+       |      |
   | | Collecting Process  |       |      |
   + +---------------------+       |      |
   | |         IAP         |       |      |
   + +---------------------+       |      |
   | |  Exporting Process  |       |      |
   + +-|-------------------+       |      |
   +===|===========================+      |
       |                                  |
   +===|==================================|=====+
   |   | Collector                        |     |
   | +-V----------------------------------V-+   |
   | |         Collecting Process           |   |
   | +------------------+-------------------+   |
   |                    |        IAP        |   |
   |                    +-------------------+   |
   |  (Aggregation      |   File Writer     |   |
       for Storage)     +-----------|-------+   |
   +================================|===========+
                                    |
                             +------V-----------+
                             |    IPFIX File    |
                             +------------------+
        
   +===========================================+
   |  IPFIX Exporter        +----------------+ |
   |                        | Metering Proc. | |
   | +-----------------+    +----------------+ |
   | | Metering Proc.  | or |      IAP       | |
   | +-----------------+----+----------------+ |
   | |           Exporting Process           | |
   | +-|----------------------------------|--+ |
   +===|==================================|====+
       |                                  |
   +===|===========================+      |
   |   |  Aggregating Mediator     |      |
   + +-V-------------------+       |      |
   | | Collecting Process  |       |      |
   + +---------------------+       |      |
   | |         IAP         |       |      |
   + +---------------------+       |      |
   | |  Exporting Process  |       |      |
   + +-|-------------------+       |      |
   +===|===========================+      |
       |                                  |
   +===|==================================|=====+
   |   | Collector                        |     |
   | +-V----------------------------------V-+   |
   | |         Collecting Process           |   |
   | +------------------+-------------------+   |
   |                    |        IAP        |   |
   |                    +-------------------+   |
   |  (Aggregation      |   File Writer     |   |
       for Storage)     +-----------|-------+   |
   +================================|===========+
                                    |
                             +------V-----------+
                             |    IPFIX File    |
                             +------------------+
        

Figure 1: Potential Aggregation Locations

图1:潜在聚合位置

The Mediator use case is further shown in Figures A and B in [RFC6183].

[RFC6183]中的图A和图B进一步显示了中介器用例。

Aggregation can be applied for either intermediate or final analytic purposes. In certain circumstances, it may make sense to export Aggregated Flows directly after metering, for example, if the Exporting Process is applied to drive a time series visualization, or when Flow data export bandwidth is restricted and Flow or packet sampling is not an option. Note that this case, where the Aggregation Process is essentially integrated into the Metering

聚合可用于中间或最终分析目的。在某些情况下,在计量之后直接导出聚合流可能是有意义的,例如,如果导出过程被应用于驱动时间序列可视化,或者当流数据导出带宽受到限制并且流或包采样不是选项时。注意,在这种情况下,聚合过程基本上集成到计量中

Process, is basically covered by the IPFIX architecture [RFC5470]: the Flow Keys used are simply a subset of those that would normally be used, and time intervals may be chosen other than those available from the cache policies customarily offered by the Metering Process. A Metering Process in this arrangement MAY choose to simulate the generation of larger Flows in order to generate Original Flow counts, if the application calls for compatibility with an Intermediate Aggregation Process deployed in a separate location.

IPFIX体系结构[RFC5470]基本上涵盖了进程:使用的流键只是通常使用的流键的子集,并且可以选择时间间隔,而不是从计量进程通常提供的缓存策略中选择的时间间隔。如果应用程序要求与部署在单独位置的中间聚合过程兼容,则这种安排中的计量过程可以选择模拟较大流的生成,以生成原始流计数。

In the specific case that an Intermediate Aggregation Process is employed for data reduction for storage purposes, it can take Original Flows from a Collecting Process or File Reader and pass Aggregated Flows to a File Writer for storage.

在特定情况下,中间聚合进程用于存储目的的数据缩减,它可以从收集进程或文件读取器获取原始流,并将聚合流传递给文件编写器进行存储。

Deployment of an Intermediate Aggregation Process within a Mediator [RFC5982] is a much more flexible arrangement. Here, the Mediator consumes Original Flows and produces Aggregated Flows; this arrangement is suited to any of the use cases detailed in Section 3. In a Mediator, Original Flows from multiple sources can also be aggregated into a single stream of Aggregated Flows. The architectural specifics of this arrangement are not addressed in this document, which is concerned only with the aggregation operation itself. See [IPFIX-MED-PROTO] for details.

在中介中部署中间聚合过程[RFC5982]是一种更灵活的安排。在这里,中介使用原始流并生成聚合流;这种安排适用于第3节中详述的任何用例。在中介器中,来自多个源的原始流也可以聚合为聚合流的单个流。本文档中未涉及此安排的体系结构细节,仅涉及聚合操作本身。有关详细信息,请参见[IPFIX-MED-PROTO]。

The data paths into and out of an Intermediate Aggregation Process are shown in Figure 2.

进入和离开中间聚合过程的数据路径如图2所示。

   packets --+               IPFIX Messages      IPFIX Files
             |                     |                  |
             V                     V                  V
   +==================+ +====================+ +=============+
   | Metering Process | | Collecting Process | | File Reader |
   |                  | +====================+ +=============+
   | (Original Flows  |            |                  |
   |    or direct     |            |  Original Flows  |
   |   aggregation)   |            V                  V
   + - - - - - - - - -+======================================+
   |           Intermediate Aggregation Process (IAP)        |
   +=========================================================+
             | Aggregated                  Aggregated |
             | Flows                            Flows |
             V                                        V
   +===================+                       +=============+
   | Exporting Process |                       | File Writer |
   +===================+                       +=============+
             |                                        |
             V                                        V
       IPFIX Messages                            IPFIX Files
        
   packets --+               IPFIX Messages      IPFIX Files
             |                     |                  |
             V                     V                  V
   +==================+ +====================+ +=============+
   | Metering Process | | Collecting Process | | File Reader |
   |                  | +====================+ +=============+
   | (Original Flows  |            |                  |
   |    or direct     |            |  Original Flows  |
   |   aggregation)   |            V                  V
   + - - - - - - - - -+======================================+
   |           Intermediate Aggregation Process (IAP)        |
   +=========================================================+
             | Aggregated                  Aggregated |
             | Flows                            Flows |
             V                                        V
   +===================+                       +=============+
   | Exporting Process |                       | File Writer |
   +===================+                       +=============+
             |                                        |
             V                                        V
       IPFIX Messages                            IPFIX Files
        

Figure 2: Data Paths through the Aggregation Process

图2:通过聚合过程的数据路径

Note that as Aggregated Flows are IPFIX Flows, an Intermediate Aggregation Process may aggregate already Aggregated Flows from an upstream IAP as well as Original Flows from an upstream Original Exporter or Metering Process.

请注意,由于聚合流是IPFIX流,中间聚合过程可能会聚合来自上游IAP的已聚合流以及来自上游原始出口商或计量过程的原始流。

Aggregation may also need to correlate Original Flows from multiple Metering Processes, each according to a different Template with different Flow Keys and values. This arrangement is shown in Figure 3; in this case, the correlation and normalization operation described in Section 4.2.1 handles merging the Original Flows before aggregation.

聚合还可能需要关联来自多个计量流程的原始流,每个流程根据具有不同流键和值的不同模板。这种安排如图3所示;在这种情况下,第4.2.1节中描述的关联和规范化操作处理在聚合之前合并原始流。

   packets --+---------------------+------------------+
             |                     |                  |
             V                     V                  V
   +====================+ +====================+ +====================+
   | Metering Process 1 | | Metering Process 2 | | Metering Process n |
   +====================+ +====================+ +====================+
             |                     |  Original Flows  |
             V                     V                  V
   +==================================================================+
   | Intermediate Aggregation Process  +  correlation / normalization |
   +==================================================================+
             | Aggregated                  Aggregated |
             | Flows                            Flows |
             V                                        V
   +===================+                       +=============+
   | Exporting Process |                       | File Writer |
   +===================+                       +=============+
             |                                        |
             +------------> IPFIX Messages <----------+
        
   packets --+---------------------+------------------+
             |                     |                  |
             V                     V                  V
   +====================+ +====================+ +====================+
   | Metering Process 1 | | Metering Process 2 | | Metering Process n |
   +====================+ +====================+ +====================+
             |                     |  Original Flows  |
             V                     V                  V
   +==================================================================+
   | Intermediate Aggregation Process  +  correlation / normalization |
   +==================================================================+
             | Aggregated                  Aggregated |
             | Flows                            Flows |
             V                                        V
   +===================+                       +=============+
   | Exporting Process |                       | File Writer |
   +===================+                       +=============+
             |                                        |
             +------------> IPFIX Messages <----------+
        

Figure 3: Aggregating Original Flows from Multiple Metering Processes

图3:聚合来自多个计量流程的原始流

4.2. Intermediate Aggregation Process Architecture
4.2. 中间聚合过程体系结构

Within this document, an Intermediate Aggregation Process can be seen as hosting a function composed of four types of operations on Partially Aggregated Flows, as illustrated in Figure 4: interval distribution (temporal), key aggregation (spatial), value aggregation (spatial), and aggregate combination. "Partially Aggregated Flows", as defined in Section 2, are essentially the intermediate results of aggregation, internal to the Intermediate Aggregation Process.

在本文档中,可以将中间聚合过程视为承载一个函数,该函数由部分聚合流上的四种类型的操作组成,如图4所示:间隔分布(时间)、键聚合(空间)、值聚合(空间)和聚合组合。第2节中定义的“部分聚合流”本质上是聚合的中间结果,是中间聚合过程的内部结果。

           Original Flows  /   Original Flows requiring correlation
   +=============|===================|===================|=============+
   |             |   Intermediate    |    Aggregation    |   Process   |
   |             |                   V                   V             |
   |             |   +-----------------------------------------------+ |
   |             |   |   (optional) correlation and normalization    | |
   |             |   +-----------------------------------------------+ |
   |             |                          |                          |
   |             V                          V                          |
   |  +--------------------------------------------------------------+ |
   |  |                interval distribution (temporal)              | |
   |  +--------------------------------------------------------------+ |
   |           | ^                         | ^                |        |
   |           | |  Partially Aggregated   | |                |        |
   |           V |         Flows           V |                |        |
   |  +-------------------+       +--------------------+      |        |
   |  |  key aggregation  |<------|  value aggregation |      |        |
   |  |     (spatial)     |------>|      (spatial)     |      |        |
   |  +-------------------+       +--------------------+      |        |
   |            |                          |                  |        |
   |            |   Partially Aggregated   |                  |        |
   |            V          Flows           V                  V        |
   |  +--------------------------------------------------------------+ |
   |  |                     aggregate combination                    | |
   |  +--------------------------------------------------------------+ |
   |                                       |                           |
   +=======================================|===========================+
                                           V
                                   Aggregated Flows
        
           Original Flows  /   Original Flows requiring correlation
   +=============|===================|===================|=============+
   |             |   Intermediate    |    Aggregation    |   Process   |
   |             |                   V                   V             |
   |             |   +-----------------------------------------------+ |
   |             |   |   (optional) correlation and normalization    | |
   |             |   +-----------------------------------------------+ |
   |             |                          |                          |
   |             V                          V                          |
   |  +--------------------------------------------------------------+ |
   |  |                interval distribution (temporal)              | |
   |  +--------------------------------------------------------------+ |
   |           | ^                         | ^                |        |
   |           | |  Partially Aggregated   | |                |        |
   |           V |         Flows           V |                |        |
   |  +-------------------+       +--------------------+      |        |
   |  |  key aggregation  |<------|  value aggregation |      |        |
   |  |     (spatial)     |------>|      (spatial)     |      |        |
   |  +-------------------+       +--------------------+      |        |
   |            |                          |                  |        |
   |            |   Partially Aggregated   |                  |        |
   |            V          Flows           V                  V        |
   |  +--------------------------------------------------------------+ |
   |  |                     aggregate combination                    | |
   |  +--------------------------------------------------------------+ |
   |                                       |                           |
   +=======================================|===========================+
                                           V
                                   Aggregated Flows
        

Figure 4: Conceptual Model of Aggregation Operations within an IAP

图4:IAP中聚合操作的概念模型

Interval distribution: a temporal aggregation operation that imposes an Aggregation Interval on the Partially Aggregated Flow. This Aggregation Interval may be regular, irregular, or derived from the timing of the Original Flows themselves. Interval distribution is discussed in detail in Section 5.1.

区间分布:对部分聚合流施加聚合区间的临时聚合操作。该聚合间隔可以是规则的、不规则的,也可以是从原始流本身的计时中派生出来的。第5.1节详细讨论了区间分布。

Key aggregation: a spatial aggregation operation that results in the addition, modification, or deletion of Flow Key fields in the Partially Aggregated Flows. New Flow Keys may be derived from existing Flow Keys (e.g., looking up an AS number (ASN) for an IP address), or "promoted" from specific non-key fields (e.g., when aggregating Flows by packet count per Flow). Key aggregation can also add new non-key fields derived from Flow Keys that are deleted during key aggregation: mainly counters of unique reduced keys. Key aggregation is discussed in detail in Section 5.2.

密钥聚合:导致在部分聚合流中添加、修改或删除流密钥字段的空间聚合操作。新的流密钥可以从现有的流密钥(例如,查找IP地址的AS编号(ASN))派生,或者从特定的非密钥字段(例如,当按每个流的数据包计数聚合流时)派生。密钥聚合还可以添加从密钥聚合期间删除的流密钥派生的新非密钥字段:主要是唯一缩减密钥的计数器。第5.2节详细讨论了密钥聚合。

Value aggregation: a spatial aggregation operation that results in the addition, modification, or deletion of non-key fields in the Partially Aggregated Flows. These non-key fields may be "demoted" from existing key fields, or derived from existing key or non-key fields. Value aggregation is discussed in detail in Section 5.3.

值聚合:导致在部分聚合流中添加、修改或删除非关键字段的空间聚合操作。这些非关键字段可以从现有关键字段“降级”,也可以从现有关键字段或非关键字段派生。第5.3节详细讨论了价值汇总。

Aggregate combination: an operation combining multiple Partially Aggregated Flows having undergone interval distribution, key aggregation, and value aggregation that share Flow Keys and Aggregation Intervals into a single Aggregated Flow per set of Flow Key values and Aggregation Interval. Aggregate combination is discussed in detail in Section 5.4.

聚合组合:一种操作,将经过间隔分布、密钥聚合和值聚合的多个部分聚合流(共享流密钥和聚合间隔)组合为单个聚合流(每组流键值和聚合间隔)。第5.4节详细讨论了骨料组合。

Correlation and normalization: an optional operation that applies when accepting Original Flows from Metering Processes that export different views of essentially the same Flows before aggregation. The details of correlation and normalization are specified in Section 4.2.1, below.

关联和规范化:一种可选操作,当接受来自计量流程的原始流时适用,这些流程在聚合之前导出基本相同流的不同视图。下文第4.2.1节规定了相关性和归一化的详细信息。

The first three of these operations may be carried out any number of times in any order, either on Original Flows or on the results of one of the operations above, with one caveat: since Flows carry their own interval data, any spatial aggregation operation implies a temporal aggregation operation, so at least one interval distribution step, even if implicit, is required by this architecture. This is shown as the first step for the sake of simplicity in the diagram above. Once all aggregation operations are complete, aggregate combination ensures that for a given Aggregation Interval, set of Flow Key values, and Observation Domain, only one Flow is produced by the Intermediate Aggregation Process.

这些操作中的前三个可以按任何顺序执行任意次数,无论是在原始流上还是在上述一个操作的结果上,但有一个警告:由于流携带其自己的间隔数据,任何空间聚合操作都意味着时间聚合操作,因此至少有一个间隔分布步骤,即使是隐式的,也是此体系结构所必需的。为了简单起见,在上图中这是第一步。一旦所有聚合操作完成,聚合组合将确保对于给定的聚合间隔、流键值集和观察域,中间聚合过程只生成一个流。

This model describes the operations within a single Intermediate Aggregation Process, and it is anticipated that most aggregation will be applied within a single process. However, as the steps in the model may be applied in any order and aggregate combination is idempotent, any number of Intermediate Aggregation Processes operating in series can be modeled as a single process. This allows aggregation operations to be flexibly distributed across any number of processes, should application or deployment considerations so dictate.

该模型描述单个中间聚合过程中的操作,预计大多数聚合将应用于单个过程中。然而,由于模型中的步骤可以以任何顺序应用,并且聚合组合是幂等的,因此可以将任何数量的串联操作的中间聚合过程建模为单个过程。这使得聚合操作可以灵活地分布在任意数量的进程中,如果应用程序或部署考虑因素需要这样做的话。

4.2.1. Correlation and Normalization
4.2.1. 相关性和规范化

When accepting Original Flows from multiple Metering Processes, each of which provides a different view of the Original Flow as seen from the point of view of the IAP, an optional correlation and normalization operation combines each of these single Flow Records

当接受来自多个计量过程的原始流时,每个计量过程都提供了从IAP的角度看原始流的不同视图,可选的关联和标准化操作组合了这些单个流记录

into a set of unified Partially Aggregated Flows before applying interval distribution. These unified Flows appear as if they had been measured at a single Metering Process that used the union of the set of Flow Keys and non-key fields of all Metering Processes sending Original Flows to the IAP.

在应用区间分布之前,转换为一组统一的部分聚合流。这些统一的流似乎是在单个计量过程中测量的,该计量过程使用所有计量过程的流键集和非键字段的并集,将原始流发送到IAP。

Since, due to export errors or other slight irregularities in Flow metering, the multiple views may not be completely consistent; normalization involves applying a set of corrections that are specific to the aggregation application in order to ensure consistency in the unified Flows.

因为,由于导出错误或流量计量中的其他轻微不规则,多个视图可能不完全一致;规范化涉及应用一组特定于聚合应用程序的更正,以确保统一流中的一致性。

In general, correlation and normalization should take multiple views of essentially the same Flow, as determined by the configuration of the operation itself, and render them into a single unified Flow. Flows that are essentially different should not be unified by the correlation and normalization operation. This operation therefore requires enough information about the configuration and deployment of Metering Processes from which it correlates Original Flows in order to make this distinction correctly and consistently.

一般来说,关联和规范化应该采用基本相同流的多个视图(由操作本身的配置确定),并将它们呈现为单个统一流。本质上不同的流不应该通过关联和规范化操作来统一。因此,此操作需要有关计量流程配置和部署的足够信息,以便正确且一致地进行此区分,并将原始流程与之关联。

The exact steps performed to correlate and normalize Flows in this step are application, implementation, and deployment specific, and will not be further specified in this document.

在此步骤中关联和规范化流所执行的确切步骤是特定于应用程序、实现和部署的,本文档中将不再详细说明。

5. IP Flow Aggregation Operations
5. IP流聚合操作

As stated in Section 2, an Aggregated Flow is simply an IPFIX Flow generated from Original Flows by an Intermediate Aggregation Process. Here, we detail the operations by which this is achieved within an Intermediate Aggregation Process.

如第2节所述,聚合流只是由中间聚合过程从原始流生成的IPFIX流。这里,我们详细介绍了在中间聚合过程中实现这一点的操作。

5.1. Temporal Aggregation through Interval Distribution
5.1. 通过区间分布的时间聚集

Interval distribution imposes a time interval on the resulting Aggregated Flows. The selection of an interval is specific to the given aggregation application. Intervals may be derived from the Original Flows themselves (e.g., an interval may be selected to cover the entire time containing the set of all Flows sharing a given Key, as in Time Composition, described in Section 5.1.2) or externally imposed; in the latter case the externally imposed interval may be regular (e.g., every five minutes) or irregular (e.g., to allow for different time resolutions at different times of day, under different network conditions, or indeed for different sets of Original Flows).

间隔分布对生成的聚合流施加时间间隔。间隔的选择特定于给定的聚合应用程序。间隔可以从原始流本身派生(例如,可以选择一个间隔,以覆盖包含共享给定密钥的所有流集的整个时间,如第5.1.2节所述的时间组成)或外部施加的间隔;在后一种情况下,外部施加的间隔可以是规则的(例如,每五分钟一次)或不规则的(例如,允许在一天的不同时间、不同网络条件下,或者实际上对于不同的原始流集,有不同的时间分辨率)。

The length of the imposed interval itself has trade-offs. Shorter intervals allow higher-resolution aggregated data and, in streaming applications, faster reaction time. Longer intervals generally lead

强制间隔的长度本身具有权衡。更短的时间间隔允许更高分辨率的聚合数据,并且在流式应用程序中,更快的反应时间。较长的间隔通常会导致

to greater data reduction and simplified counter distribution. Specifically, counter distribution is greatly simplified by the choice of an interval longer than the duration of longest Original Flow, itself generally determined by the Original Flow's Metering Process active timeout; in this case, an Original Flow can contribute to at most two Aggregated Flows, and the more complex value distribution methods become inapplicable.

以实现更大的数据缩减和简化计数器分布。具体地说,通过选择比最长原始流持续时间更长的间隔,计数器分布大大简化,其本身通常由原始流的计量过程活动超时决定;在这种情况下,一个原始流最多可以有两个聚合流,更复杂的值分布方法将不适用。

   |                |                |                |
   | |<--Flow A-->| |                |                |
   |        |<--Flow B-->|           |                |
   |          |<-------------Flow C-------------->|   |
   |                |                |                |
   |   interval 0   |   interval 1   |   interval 2   |
        
   |                |                |                |
   | |<--Flow A-->| |                |                |
   |        |<--Flow B-->|           |                |
   |          |<-------------Flow C-------------->|   |
   |                |                |                |
   |   interval 0   |   interval 1   |   interval 2   |
        

Figure 5: Illustration of Interval Distribution

图5:区间分布示意图

In Figure 5, we illustrate three common possibilities for interval distribution as applies with regular intervals to a set of three Original Flows. For Flow A, the start and end times lie within the boundaries of a single interval 0; therefore, Flow A contributes to only one Aggregated Flow. Flow B, by contrast, has the same duration but crosses the boundary between intervals 0 and 1; therefore, it will contribute to two Aggregated Flows, and its counters must be distributed among these Flows; though, in the two-interval case, this can be simplified somewhat simply by picking one of the two intervals or proportionally distributing between them. Only Flows like Flow A and Flow B will be produced when the interval is chosen to be longer than the duration of longest Original Flow, as above. More complicated is the case of Flow C, which contributes to more than two Aggregated Flows and must have its counters distributed according to some policy as in Section 5.1.1.

在图5中,我们展示了适用于三个原始流集合的规则间隔的间隔分布的三种常见可能性。对于流A,开始和结束时间位于单个间隔0的边界内;因此,流A只贡献一个聚合流。相比之下,流量B具有相同的持续时间,但穿过间隔0和1之间的边界;因此,它将有助于两个聚合流,其计数器必须分布在这些流中;不过,在两个区间的情况下,只需从两个区间中选择一个或按比例分布,就可以简化这一过程。如上文所述,当选择的时间间隔大于最长原始流量的持续时间时,仅会产生流量A和流量B之类的流量。更复杂的是流C的情况,它产生两个以上的聚合流,并且必须按照第5.1.1节中的某些策略分配其计数器。

5.1.1. Distributing Values across Intervals
5.1.1. 跨区间分布值

In general, counters in Aggregated Flows are treated the same as in any Flow. Each counter is independently calculated as if it were derived from the set of packets in the Original Flow. For example, delta counters are summed, the most recent total count for each Original Flow taken then summed across Flows, and so on.

通常,聚合流中的计数器与任何流中的计数器一样处理。每个计数器都是独立计算的,就好像它是从原始流中的一组数据包派生出来的一样。例如,增量计数器求和,每个原始流的最新总计数,然后在流之间求和,依此类推。

When the Aggregation Interval is guaranteed to be longer than the longest Original Flow, a Flow can cross at most one Interval boundary, and will therefore contribute to at most two Aggregated Flows. Most common in this case is to arbitrarily but consistently choose to account the Original Flow's counters either to the first or to the last Aggregated Flow to which it could contribute.

当聚合间隔保证比最长的原始流长时,一个流最多可以跨越一个间隔边界,因此最多会产生两个聚合流。在这种情况下,最常见的是任意但一致地选择将原始流的计数器计算到它可能贡献的第一个或最后一个聚合流。

However, this becomes more complicated when the Aggregation Interval is shorter than the longest Original Flow in the source data. In such cases, each Original Flow can incompletely cover one or more time intervals, and apply to one or more Aggregated Flows. In this case, the Intermediate Aggregation Process must distribute the counters in the Original Flows across one or more resulting Aggregated Flows. There are several methods for doing this, listed here in roughly increasing order of complexity and accuracy; most of these are necessary only in specialized cases.

但是,当聚合间隔小于源数据中最长的原始流时,这将变得更加复杂。在这种情况下,每个原始流可以不完全覆盖一个或多个时间间隔,并应用于一个或多个聚合流。在这种情况下,中间聚合进程必须将原始流中的计数器分布到一个或多个生成的聚合流中。有几种方法可以做到这一点,这里按复杂性和准确性的大致递增顺序列出;其中大多数只有在特殊情况下才是必要的。

End Interval: The counters for an Original Flow are added to the counters of the appropriate Aggregated Flow containing the end time of the Original Flow.

结束间隔:原始流的计数器添加到包含原始流结束时间的适当聚合流的计数器中。

Start Interval: The counters for an Original Flow are added to the counters of the appropriate Aggregated Flow containing the start time of the Original Flow.

开始间隔:原始流的计数器添加到包含原始流开始时间的适当聚合流的计数器中。

Mid Interval: The counters for an Original Flow are added to the counters of a single appropriate Aggregated Flow containing some timestamp between start and end time of the Original Flow.

中间间隔:原始流的计数器添加到单个适当聚合流的计数器中,该聚合流包含原始流的开始时间和结束时间之间的时间戳。

Simple Uniform Distribution: Each counter for an Original Flow is divided by the number of time intervals the Original Flow covers (i.e., of appropriate Aggregated Flows sharing the same Flow Keys), and this number is added to each corresponding counter in each Aggregated Flow.

简单均匀分布:原始流的每个计数器除以原始流覆盖的时间间隔数(即,共享相同流键的适当聚合流的时间间隔数),并将该数添加到每个聚合流中的每个对应计数器。

Proportional Uniform Distribution: This is like simple uniform distribution, but accounts for the fractional portions of a time interval covered by an Original Flow in the first and last time interval. Each counter for an Original Flow is divided by the number of time _units_ the Original Flow covers, to derive a mean count rate. This rate is then multiplied by the number of time units in the intersection of the duration of the Original Flow and the time interval of each Aggregated Flow.

比例均匀分布:这类似于简单的均匀分布,但考虑了第一个和最后一个时间间隔中原始流覆盖的时间间隔的分数部分。原始流的每个计数器除以原始流覆盖的时间单位数,得出平均计数率。然后将该速率乘以原始流的持续时间与每个聚合流的时间间隔相交处的时间单位数。

Simulated Process: Each counter of the Original Flow is distributed among the intervals of the Aggregated Flows according to some function the Intermediate Aggregation Process uses based upon properties of Flows presumed to be like the Original Flow. For example, Flow Records representing bulk transfer might follow a more or less proportional uniform distribution, while interactive processes are far more bursty.

模拟过程:原始流的每个计数器根据中间聚合过程基于假定与原始流相似的流的属性使用的某些函数分布在聚合流的间隔中。例如,表示批量传输的流记录可能遵循或多或少的比例均匀分布,而交互过程则更具突发性。

Direct: The Intermediate Aggregation Process has access to the original packet timings from the packets making up the Original Flow, and uses these to distribute or recalculate the counters.

直接:中间聚合进程可以从组成原始流的数据包访问原始数据包定时,并使用这些数据包来分配或重新计算计数器。

A method for exporting the distribution of counters across multiple Aggregated Flows is detailed in Section 7.4. In any case, counters MUST be distributed across the multiple Aggregated Flows in such a way that the total count is preserved, within the limits of accuracy of the implementation. This property allows data to be aggregated and re-aggregated with negligible loss of original count information. To avoid confusion in interpretation of the aggregated data, all the counters in a given Aggregated Flow MUST be distributed via the same method.

第7.4节详细介绍了在多个聚合流中导出计数器分布的方法。在任何情况下,计数器必须分布在多个聚合流中,以便在实现的精度范围内保留总计数。此属性允许对数据进行聚合和重新聚合,而原始计数信息的损失可以忽略不计。为了避免在解释聚合数据时出现混淆,给定聚合流中的所有计数器必须通过相同的方法进行分配。

More complex counter distribution methods generally require that the interval distribution process track multiple "current" time intervals at once. This may introduce some delay into the aggregation operation, as an interval should only expire and be available for export when no additional Original Flows applying to the interval are expected to arrive at the Intermediate Aggregation Process.

更复杂的计数器分配方法通常要求间隔分配过程一次跟踪多个“当前”时间间隔。这可能会给聚合操作带来一些延迟,因为只有当应用于该间隔的其他原始流预计不会到达中间聚合过程时,该间隔才会过期并可用于导出。

Note, however, that since there is no guarantee that Flows from the Original Exporter will arrive in any given order, whether for transport-specific reasons (i.e., UDP reordering) or reasons specific to the implementation of the Metering Process or Exporting Process, even simpler distribution methods may need to deal with Flows arriving in an order other than start time or end time. Therefore, the use of larger intervals does not obviate the need to buffer Partially Aggregated Flows within "current" time intervals, to ensure the IAP can accept Flow time intervals in any arrival order. More generally, the interval distribution process SHOULD accept Flow start and end times in the Original Flows in any reasonable order. The expiration of intervals in interval distribution operations is dependent on implementation and deployment requirements, and it MUST be made configurable in contexts in which "reasonable order" is not obvious at implementation time. This operation may lead to delay and loss introduced by the IAP, as detailed in Section 6.2.

但是,请注意,由于无法保证来自原始导出器的流将以任何给定的顺序到达,无论是出于特定于传输的原因(即UDP重新排序)还是由于特定于计量过程或导出过程的实施的原因,甚至更简单的分布方法也可能需要处理以开始时间或结束时间以外的顺序到达的流。因此,使用较大的时间间隔并不排除需要在“当前”时间间隔内缓冲部分聚合的流,以确保IAP可以接受任何到达顺序中的流时间间隔。更一般地说,间隔分布过程应该以任何合理的顺序接受原始流中的流开始和结束时间。间隔分发操作中间隔的过期取决于实施和部署需求,并且必须在实施时“合理顺序”不明显的上下文中进行配置。如第6.2节所述,该操作可能导致IAP引入的延迟和损失。

5.1.2. Time Composition
5.1.2. 时间构成

Time Composition, as in Section 5.4 of [RFC5982] (or interval combination), is a special case of aggregation, where interval distribution imposes longer intervals on Flows with matching keys and "chained" start and end times, without any key reduction, in order to join long-lived Flows that may have been split (e.g., due to an active timeout shorter than the actual duration of the Flow). Here, no Key aggregation is applied, and the Aggregation Interval is chosen on a per-Flow basis to cover the interval spanned by the set of Aggregated Flows. This may be applied alone in order to normalize split Flows, or it may be applied in combination with other aggregation functions in order to obtain more accurate Original Flow counts.

时间组合,如[RFC5982]第5.4节(或区间组合)所述,是聚合的一种特殊情况,其中区间分布对具有匹配键和“链式”开始和结束时间的流施加更长的区间,而不减少任何键,以便连接可能已拆分的长寿命流(例如,由于活动超时时间短于流的实际持续时间)。此处,不应用密钥聚合,聚合间隔是基于每个流选择的,以覆盖聚合流集所跨越的间隔。这可以单独应用以规范拆分流,也可以与其他聚合函数结合应用以获得更准确的原始流计数。

5.1.3. External Interval Distribution
5.1.3. 外部区间分布

Note that much of the difficulty of interval distribution at an IAP can be avoided simply by configuring the original Exporters to synchronize the time intervals in the Original Flows with the desired aggregation interval. The resulting Original Flows would then be split to align perfectly with the time intervals imposed during interval imposition, as shown in Figure 6, though this may reduce their usefulness for non-aggregation purposes. This approach allows the Intermediate Aggregation Process to use Start Interval or End Interval distribution, while having equivalent information to that available to direct interval distribution.

请注意,只要将原始导出器配置为将原始流中的时间间隔与所需的聚合间隔同步,就可以避免在IAP中进行间隔分布的许多困难。由此产生的原始流将被分割,以与在间隔施加期间施加的时间间隔完全一致,如图6所示,尽管这可能会降低它们对非聚合目的的有用性。这种方法允许中间聚合过程使用开始间隔或结束间隔分布,同时具有与直接间隔分布可用的信息相同的信息。

   |                |                |                |
   |<----Flow D---->|<----Flow E---->|<----Flow F---->|
   |                |                |                |
   |   interval 0   |   interval 1   |   interval 2   |
        
   |                |                |                |
   |<----Flow D---->|<----Flow E---->|<----Flow F---->|
   |                |                |                |
   |   interval 0   |   interval 1   |   interval 2   |
        

Figure 6: Illustration of External Interval Distribution

图6:外部间隔分布示意图

5.2. Spatial Aggregation of Flow Keys
5.2. 流键的空间聚合

Key aggregation generates a new set of Flow Key values for the Aggregated Flows from the Original Flow Key and non-key fields in the Original Flows or from correlation of the Original Flow information with some external source. There are two basic operations here. First, Aggregated Flow Keys may be derived directly from Original Flow Keys through reduction, or they may be derived by the dropping of fields or precision in the Original Flow Keys. Second, Aggregated Flow Keys may be derived through replacement, e.g., by removing one or more fields from the Original Flow and replacing them with fields derived from the removed fields. Replacement may refer to external information (e.g., IP to AS number mappings). Replacement may apply to Flow Keys as well as non-key fields. For example, consider an application that aggregates Original Flows by packet count (i.e., generating an Aggregated Flow for all one-packet Flows, one for all two-packet Flows, and so on). This application would promote the packet count to a Flow Key.

密钥聚合从原始流中的原始流密钥和非密钥字段,或从原始流信息与某些外部源的关联,为聚合流生成一组新的流密钥值。这里有两个基本操作。首先,聚合流键可以通过缩减直接从原始流键派生,或者可以通过删除原始流键中的字段或精度来派生。第二,可以通过替换来导出聚合流密钥,例如,通过从原始流中移除一个或多个字段并用从移除的字段派生的字段替换它们。替换可能涉及外部信息(例如,IP到AS编号映射)。替换可能适用于流键以及非键字段。例如,考虑通过分组计数来聚合原始流的应用(即,为所有一个分组流生成聚合流,一个用于所有两个分组流,等等)。此应用程序将数据包计数提升为流密钥。

Key aggregation may also result in the addition of new non-key fields to the Aggregated Flows, namely, Original Flow counters and unique reduced key counters. These are treated in more detail in Sections 5.2.1 and 5.2.2, respectively.

密钥聚合还可能导致向聚合流中添加新的非密钥字段,即原始流计数器和唯一缩减密钥计数器。第5.2.1节和第5.2.2节分别对其进行了更详细的处理。

In any key aggregation operation, reduction and/or replacement may be applied any number of times in any order. Which of these operations are supported by a given implementation is implementation and application dependent.

在任何密钥聚合操作中,可以以任何顺序应用任何次数的减少和/或替换。给定实现支持哪些操作取决于实现和应用程序。

Original Flow Keys

原始流键

   +---------+---------+----------+----------+-------+-----+
   | src ip4 | dst ip4 | src port | dst port | proto | tos |
   +---------+---------+----------+----------+-------+-----+
        |         |         |          |         |      |
     retain   mask /24      X          X         X      X
        |         |
        V         V
   +---------+-------------+
   | src ip4 | dst ip4 /24 |
   +---------+-------------+
        
   +---------+---------+----------+----------+-------+-----+
   | src ip4 | dst ip4 | src port | dst port | proto | tos |
   +---------+---------+----------+----------+-------+-----+
        |         |         |          |         |      |
     retain   mask /24      X          X         X      X
        |         |
        V         V
   +---------+-------------+
   | src ip4 | dst ip4 /24 |
   +---------+-------------+
        

Aggregated Flow Keys (by source address and destination /24 network)

聚合流密钥(按源地址和目标/24网络)

Figure 7: Illustration of Key Aggregation by Reduction

图7:按减少的键聚合说明

Figure 7 illustrates an example reduction operation, aggregation by source address and destination /24 network. Here, the port, protocol, and type-of-service information is removed from the Flow Key, the source address is retained, and the destination address is masked by dropping the lower 8 bits.

图7展示了一个示例缩减操作,按源地址和目标/24网络进行聚合。这里,从流密钥中删除端口、协议和服务类型信息,保留源地址,并通过删除较低的8位来屏蔽目标地址。

Original Flow Keys

原始流键

   +---------+---------+----------+----------+-------+-----+
   | src ip4 | dst ip4 | src port | dst port | proto | tos |
   +---------+---------+----------+----------+-------+-----+
        |         |         |          |         |      |
        V         V         |          |         |      |
   +-------------------+    X          X         X      X
   | ASN lookup table  |
   +-------------------+
        |         |
        V         V
   +---------+---------+
   | src asn | dst asn |
   +---------+---------+
        
   +---------+---------+----------+----------+-------+-----+
   | src ip4 | dst ip4 | src port | dst port | proto | tos |
   +---------+---------+----------+----------+-------+-----+
        |         |         |          |         |      |
        V         V         |          |         |      |
   +-------------------+    X          X         X      X
   | ASN lookup table  |
   +-------------------+
        |         |
        V         V
   +---------+---------+
   | src asn | dst asn |
   +---------+---------+
        

Aggregated Flow Keys (by source and destination ASN)

聚合流密钥(按源和目标ASN)

Figure 8: Illustration of Key Aggregation by Reduction and Replacement

图8:按减少和替换的键聚合说明

Figure 8 illustrates an example reduction and replacement operation, aggregation by source and destination Border Gateway Protocol (BGP) Autonomous System Number (ASN) without ASN information available in the Original Flow. Here, the port, protocol, and type-of-service

图8说明了一个示例性的缩减和替换操作,即按源和目标边界网关协议(BGP)自治系统号(ASN)进行聚合,而原始流中没有可用的ASN信息。这里是端口、协议和服务类型

information is removed from the Flow Keys, while the source and destination addresses are run though an IP address to ASN lookup table, and the Aggregated Flow Keys are made up of the resulting source and destination ASNs.

从流密钥中删除信息,而源和目标地址通过IP地址到ASN查找表运行,聚合流密钥由生成的源和目标ASN组成。

5.2.1. Counting Original Flows
5.2.1. 计算原始流

When aggregating multiple Original Flows into an Aggregated Flow, it is often useful to know how many Original Flows are present in the Aggregated Flow. Section 7.2 introduces four new Information Elements to export these counters.

将多个原始流聚合为聚合流时,了解聚合流中存在多少原始流通常很有用。第7.2节介绍了导出这些计数器的四个新信息元素。

There are two possible ways to count Original Flows, which we call conservative and non-conservative. Conservative Flow counting has the property that each Original Flow contributes exactly one to the total Flow count within a set of Aggregated Flows. In other words, conservative Flow counters are distributed just as any other counter during interval distribution, except each Original Flow is assumed to have a Flow count of one. When a count for an Original Flow must be distributed across a set of Aggregated Flows, and a distribution method is used that does not account for that Original Flow completely within a single Aggregated Flow, conservative Flow counting requires a fractional representation.

计算原始流量有两种可能的方法,我们称之为保守和非保守。保守流计数的特性是,在一组聚合流中,每个原始流对总流计数的贡献正好为一。换句话说,在间隔分布期间,保守流计数器的分布与任何其他计数器的分布相同,但假设每个原始流的流量计数为1。当原始流的计数必须分布在一组聚合流中,并且使用的分布方法不能在单个聚合流中完全解释原始流时,保守流计数需要分数表示。

By contrast, non-conservative Flow counting is used to count how many Contributing Flows are represented in an Aggregated Flow. Flow counters are not distributed in this case. An Original Flow that is present within N Aggregated Flows would add N to the sum of non-conservative Flow counts, one to each Aggregated Flow. In other words, the sum of conservative Flow counts over a set of Aggregated Flows is always equal to the number of Original Flows, while the sum of non-conservative Flow counts is strictly greater than or equal to the number of Original Flows.

相比之下,非保守流计数用于计算聚合流中表示了多少贡献流。在这种情况下,不分布流计数器。存在于N个聚合流中的原始流将向非保守流计数之和添加N,每个聚合流添加一个。换句话说,一组聚合流上的保守流计数之和始终等于原始流的数量,而非保守流计数之和严格大于或等于原始流的数量。

For example, consider Flows A, B, and C as illustrated in Figure 5. Assume that the key aggregation step aggregates the keys of these three Flows to the same aggregated Flow Key, and that start interval counter distribution is in effect. The conservative Flow count for interval 0 is 3 (since Flows A, B, and C all begin in this interval), and for the other two intervals is 0. The non-conservative Flow count for interval 0 is also 3 (due to the presence of Flows A, B, and C), for interval 1 is 2 (Flows B and C), and for interval 2 is 1 (Flow C). The sum of the conservative counts 3 + 0 + 0 = 3, the number of Original Flows; while the sum of the non-conservative counts 3 + 2 + 1 = 6.

例如,考虑流A、B和C,如图5所示。假设密钥聚合步骤将这三个流的密钥聚合到相同的聚合流密钥,并且开始间隔计数器分布有效。区间0的保守流量计数为3(因为流量A、B和C均在此区间内开始),而其他两个区间的保守流量计数为0。区间0的非保守流量计数也为3(由于存在流量A、B和C),区间1为2(流量B和C),区间2为1(流量C)。保守计数之和为3+0+0=3,即原始流量的数量;而非保守计数之和为3+2+1=6。

Note that the active and inactive timeouts used to generate Original Flows, as well as the cache policy used to generate those Flows, have an effect on how meaningful either the conservative or non-conservative Flow count will be during aggregation. In general, Original Exporters using the IPFIX Configuration Model SHOULD be configured to export Flows with equal or similar activeTimeout and inactiveTimeout configuration values, and the same cacheMode, as defined in [RFC6728]. Original Exporters not using the IPFIX Configuration Model SHOULD be configured equivalently.

请注意,用于生成原始流的活动和非活动超时,以及用于生成这些流的缓存策略,都会影响聚合期间保守或非保守流计数的意义。通常,应将使用IPFIX配置模型的原始导出器配置为导出具有相同或类似activeTimeout和inactiveTimeout配置值以及相同缓存模式的流,如[RFC6728]中所定义。不使用IPFIX配置模型的原始导出程序应进行等效配置。

5.2.2. Counting Distinct Key Values
5.2.2. 计算不同的键值

One common case in aggregation is counting distinct key values that were reduced away during key aggregation. The most common use case for this is counting distinct hosts per Flow Key; for example, in host characterization or anomaly detection, distinct sources per destination or distinct destinations per source are common metrics. These new non-key fields are added during key aggregation.

聚合中的一种常见情况是计算在键聚合期间减少的不同键值。最常见的用例是计算每个流键的不同主机数;例如,在主机特性描述或异常检测中,每个目的地的不同来源或每个来源的不同目的地是常见的度量。这些新的非键字段是在键聚合期间添加的。

For such applications, Information Elements for distinct counts of IPv4 and IPv6 addresses are defined in Section 7.3. These are named distinctCountOf(KeyName). Additional such Information Elements should be registered with IANA on an as-needed basis.

对于此类应用,IPv4和IPv6地址的不同计数的信息元素在第7.3节中定义。它们被命名为distinctcontof(KeyName)。其他此类信息要素应根据需要向IANA注册。

5.3. Spatial Aggregation of Non-key Fields
5.3. 非关键字段的空间聚合

Aggregation operations may also lead to the addition of value fields that are demoted from key fields or are derived from other value fields in the Original Flows. Specific cases of this are treated in the subsections below.

聚合操作还可能导致添加从关键字段降级或从原始流中的其他值字段派生的值字段。具体情况将在以下小节中处理。

5.3.1. Counter Statistics
5.3.1. 反统计

Some applications of aggregation may benefit from computing different statistics than those native to each non-key field (e.g., flags are natively combined via union and delta counters by summing). For example, minimum and maximum packet counts per Flow, mean bytes per packet per Contributing Flow, and so on. Certain Information Elements for these applications are already provided in the IANA IPFIX Information Elements registry [IANA-IPFIX] (e.g., minimumIpTotalLength).

某些聚合应用程序可能会受益于计算不同于每个非关键字段本机统计数据的统计数据(例如,通过求和通过union和delta计数器本机组合标志)。例如,每个流的最小和最大数据包计数,每个贡献流的每个数据包的平均字节数,等等。IANA IPFIX信息元素注册表[IANA-IPFIX]中已经提供了这些应用程序的某些信息元素(例如,MinimumiPotallength)。

A complete specification of additional aggregate counter statistics is outside the scope of this document, and should be added in the future to the IANA IPFIX Information Elements registry on a per-application, as-needed basis.

附加聚合计数器统计信息的完整规范不在本文档的范围内,将来应根据需要按每个应用程序添加到IANA IPFIX信息元素注册表中。

5.3.2. Derivation of New Values from Flow Keys and Non-key fields
5.3.2. 从流键和非键字段派生新值

More complex operations may lead to other derived fields being generated from the set of values or Flow Keys reduced away during aggregation. A prime example of this is sample entropy calculation. This counts distinct values and frequency, so it is similar to distinct key counting as in Section 5.2.2; however, it may be applied to the distribution of values for any Flow field.

更复杂的操作可能会导致从聚合期间减少的值集或流键生成其他派生字段。这方面的一个主要例子是样本熵计算。这对不同的值和频率进行计数,因此与第5.2.2节中的不同键计数类似;然而,它可以应用于任何流场的值分布。

Sample entropy calculation provides a one-number normalized representation of the value spread and is useful for anomaly detection. The behavior of entropy statistics is such that a small number of keys showing up very often drives the entropy value down towards zero, while a large number of keys, each showing up with lower frequency, drives the entropy value up.

样本熵计算提供了一个值扩散的一个数字标准化表示,对于异常检测非常有用。熵统计的行为是这样的,一小部分出现的关键点通常会使熵值下降到零,而大量出现的关键点(每个出现的频率较低)会使熵值上升。

Entropy statistics are generally useful for identifier keys, such as IP addresses, port numbers, AS numbers, etc. They can also be calculated on Flow length, Flow duration fields, and the like, even if this generally yields less distinct value shifts when the traffic mix changes.

熵统计通常对标识符密钥(如IP地址、端口号、as号等)有用。它们也可以根据流长度、流持续时间字段等进行计算,即使这通常在流量混合变化时产生不太明显的值偏移。

As a practical example, one host scanning a lot of other hosts will drive source IP entropy down and target IP entropy up. A similar effect can be observed for ports. This pattern can also be caused by the scan-traffic of a fast Internet worm. A second example would be a Distributed Denial of Service (DDoS) flooding attack against a single target (or small number of targets) that drives source IP entropy up and target IP entropy down.

作为一个实际的例子,一个主机扫描许多其他主机将导致源IP熵降低,目标IP熵升高。对于端口也可以观察到类似的效果。这种模式也可能是由快速互联网蠕虫的扫描流量造成的。第二个例子是针对单个目标(或少量目标)的分布式拒绝服务(DDoS)洪泛攻击,该攻击导致源IP熵上升,目标IP熵下降。

A complete specification of additional derived values or entropy Information Elements is outside the scope of this document. Any such Information Elements should be added in the future to the IANA IPFIX Information Elements registry on a per-application, as-needed basis.

附加衍生值或熵信息元素的完整规范不在本文件范围内。任何此类信息元素都应根据需要,按每个应用程序添加到IANA IPFIX信息元素注册表中。

5.4. Aggregation Combination
5.4. 聚合组合

Interval distribution and key aggregation together may generate multiple Partially Aggregated Flows covering the same time interval with the same set of Flow Key values. The process of combining these Partially Aggregated Flows into a single Aggregated Flow is called aggregation combination. In general, non-Key values from multiple Contributing Flows are combined using the same operation by which values are combined from packets to form Flows for each Information Element. Delta counters are summed, flags are unioned, and so on.

间隔分布和密钥聚合一起可以生成多个部分聚合的流,这些流覆盖具有相同流键值集的相同时间间隔。将这些部分聚合的流组合成单个聚合流的过程称为聚合组合。通常,来自多个贡献流的非键值使用相同的操作进行组合,通过该操作,来自分组的值被组合以形成每个信息元素的流。增量计数器被求和,标志被联合,等等。

6. Additional Considerations and Special Cases in Flow Aggregation
6. 流聚合中的其他注意事项和特殊情况
6.1. Exact versus Approximate Counting during Aggregation
6.1. 聚合期间的精确计数与近似计数

In certain circumstances, particularly involving aggregation by devices with limited resources, and in situations where exact aggregated counts are less important than relative magnitudes (e.g., driving graphical displays), counter distribution during key aggregation may be performed by approximate counting means (e.g., Bloom filters). The choice to use approximate counting is implementation and application dependent.

在某些情况下,特别是涉及由具有有限资源的设备进行聚合的情况下,以及在精确聚合计数不如相对大小(例如,驱动图形显示)重要的情况下,可通过近似计数装置(例如,Bloom过滤器)执行键聚合期间的计数器分配。使用近似计数的选择取决于实现和应用程序。

6.2. Delay and Loss Introduced by the IAP
6.2. IAP引入的延迟和损失

When accepting Original Flows in export order from traffic captured live, the Intermediate Aggregation Process waits for all Original Flows that may contribute to a given interval during interval distribution. This is generally dominated by the active timeout of the Metering Process measuring the Original Flows. For example, with Metering Processes configured with a five-minute active timeout, the Intermediate Aggregation Process introduces a delay of at least five minutes to all exported Aggregated Flows to ensure it has received all Original Flows. Note that when aggregating Flows from multiple Metering Processes with different active timeouts, the delay is determined by the maximum active timeout.

当从实时捕获的流量中以导出顺序接受原始流时,中间聚合过程将等待所有原始流,这些原始流可能在间隔分布期间贡献给给定的间隔。这通常由测量原始流量的计量过程的活动超时决定。例如,对于配置了5分钟活动超时的计量进程,中间聚合进程向所有导出的聚合流引入至少5分钟的延迟,以确保它已接收到所有原始流。请注意,当聚合来自具有不同活动超时的多个计量进程的流时,延迟由最大活动超时确定。

In certain circumstances, additional delay at the original Exporter may cause an IAP to close an interval before the last Original Flow(s) accountable to the interval arrives. In this case, the IAP MAY drop the late Original Flow(s). Accounting of Flows lost at an Intermediate Process due to such issues is covered in [IPFIX-MED-PROTO].

在某些情况下,原始导出器的额外延迟可能导致IAP在负责间隔的最后一个原始流到达之前关闭间隔。在这种情况下,IAP可能会丢弃延迟的原始流。[IPFIX-MED-PROTO]中介绍了由于此类问题在中间过程中损失的流量核算。

6.3. Considerations for Aggregation of Sampled Flows
6.3. 采样流聚合的注意事项

The accuracy of Aggregated Flows may also be affected by sampling of the Original Flows, or sampling of packets making up the Original Flows. At the time of writing, the effect of sampling on Flow aggregation is still an open research question. However, to maximize the comparability of Aggregated Flows, aggregation of sampled Flows should only be applied to Original Flows sampled using the same sampling rate and sampling algorithm, Flows created from packets sampled using the same sampling rate and sampling algorithm, or Original Flows that have been normalized as if they had the same sampling rate and algorithm before aggregation. For more on packet sampling within IPFIX, see [RFC5476]. For more on Flow sampling within the IPFIX Mediator framework, see [RFC7014].

聚合流的准确性也可能受到原始流的采样或构成原始流的分组的采样的影响。在撰写本文时,采样对流聚集的影响仍然是一个开放的研究问题。但是,为了最大化聚合流的可比性,应仅将采样流聚合应用于使用相同采样率和采样算法采样的原始流,以及使用相同采样率和采样算法采样的数据包创建的流,或原始流,这些流已被规范化,就好像它们在聚合之前具有相同的采样率和算法一样。有关IPFIX内数据包采样的更多信息,请参阅[RFC5476]。有关IPFIX Mediator框架内的流采样的更多信息,请参阅[RFC7014]。

6.4. Considerations for Aggregation of Heterogeneous Flows
6.4. 异构流聚合的考虑因素

Aggregation may be applied to Original Flows from different sources and of different types (i.e., represented using different, perhaps wildly different Templates). When the goal is to separate the heterogeneous Original Flows and aggregate them into heterogeneous Aggregated Flows, each aggregation should be done at its own Intermediate Aggregation Process. The Observation Domain ID on the Messages containing the output Aggregated Flows can be used to identify the different Processes and to segregate the output.

聚合可以应用于来自不同来源和不同类型的原始流(即,使用不同的、可能完全不同的模板表示)。当目标是分离异构原始流并将其聚合为异构聚合流时,每个聚合都应该在其自己的中间聚合过程中完成。包含输出聚合流的消息上的观察域ID可用于标识不同的进程并隔离输出。

However, when the goal is to aggregate these Flows into a single stream of Aggregated Flows representing one type of data, and if the Original Flows may represent the same original packet at two different Observation Points, the Original Flows should be correlated by the correlation and normalization operation within the IAP to ensure that each packet is only represented in a single Aggregated Flow or set of Aggregated Flows differing only by aggregation interval.

然而,当目标是将这些流聚合为表示一种数据类型的聚合流的单个流时,并且如果原始流可能表示两个不同观测点处的相同原始数据包,原始流应通过IAP内的相关和规范化操作进行相关,以确保每个数据包仅在单个聚合流或仅在聚合间隔不同的聚合流集合中表示。

7. Export of Aggregated IP Flows Using IPFIX
7. 使用IPFIX导出聚合IP流

In general, Aggregated Flows are exported in IPFIX as any other Flow. However, certain aspects of Aggregated Flow export benefit from additional guidelines or new Information Elements to represent aggregation metadata or information generated during aggregation. These are detailed in the following subsections.

通常,聚合流与任何其他流一样在IPFIX中导出。然而,聚合流导出的某些方面得益于额外的准则或新的信息元素,以表示聚合元数据或聚合期间生成的信息。这些在以下小节中详细说明。

7.1. Time Interval Export
7.1. 时间间隔导出

Since an Aggregated Flow is simply a Flow, the existing timestamp Information Elements in the IPFIX Information Model (e.g., flowStartMilliseconds, flowEndNanoseconds) are sufficient to specify the time interval for aggregation. Therefore, no new aggregation-specific Information Elements for exporting time interval information are necessary.

由于聚合流只是一个流,因此IPFIX信息模型中现有的时间戳信息元素(例如flowStartMilliseconds、flowEndNanoseconds)足以指定聚合的时间间隔。因此,导出时间间隔信息不需要新的特定于聚合的信息元素。

Each Aggregated Flow carrying timing information SHOULD contain both an interval start and interval end timestamp.

每个携带计时信息的聚合流都应该包含间隔开始和间隔结束时间戳。

7.2. Flow Count Export
7.2. 流量计数导出

The following four Information Elements are defined to count Original Flows as discussed in Section 5.2.1.

如第5.2.1节所述,定义以下四个信息元素以计算原始流量。

7.2.1. originalFlowsPresent
7.2.1. 原始流动存在

Description: The non-conservative count of Original Flows contributing to this Aggregated Flow. Non-conservative counts need not sum to the original count on re-aggregation.

描述:对该聚合流起作用的原始流的非保守计数。非保守计数无需与重新聚合时的原始计数求和。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: deltaCounter

数据类型语义:deltaCounter

ElementID: 375

元素ID:375

7.2.2. originalFlowsInitiated
7.2.2. 原始流中国化

Description: The conservative count of Original Flows whose first packet is represented within this Aggregated Flow. Conservative counts must sum to the original count on re-aggregation.

描述:其第一个数据包在此聚合流中表示的原始流的保守计数。保守计数必须与重新聚合时的原始计数相加。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: deltaCounter

数据类型语义:deltaCounter

ElementID: 376

元素ID:376

7.2.3. originalFlowsCompleted
7.2.3. 原始流已完成

Description: The conservative count of Original Flows whose last packet is represented within this Aggregated Flow. Conservative counts must sum to the original count on re-aggregation.

描述:其最后一个数据包在此聚合流中表示的原始流的保守计数。保守计数必须与重新聚合时的原始计数相加。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: deltaCounter

数据类型语义:deltaCounter

ElementID: 377

元素ID:377

7.2.4. deltaFlowCount
7.2.4. 三角洲流量计

Description: The conservative count of Original Flows contributing to this Aggregated Flow; may be distributed via any of the methods expressed by the valueDistributionMethod Information Element.

说明:促成此聚合流的原始流的保守计数;可以通过valueDistributionMethod信息元素表示的任何方法进行分发。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: deltaCounter

数据类型语义:deltaCounter

ElementID: 3

元素ID:3

7.3. Distinct Host Export
7.3. 不同主机导出

The following six Information Elements represent the distinct counts of source and destination network-layer addresses used to export distinct host counts reduced away during key aggregation.

以下六个信息元素表示源和目标网络层地址的不同计数,用于导出在密钥聚合期间减少的不同主机计数。

7.3.1. distinctCountOfSourceIPAddress
7.3.1. DistinctCountofSourcePaddress

Description: The count of distinct source IP address values for Original Flows contributing to this Aggregated Flow, without regard to IP version. This Information Element is preferred to the IP-version-specific counters, unless it is important to separate the counts by version.

Description:对该聚合流起作用的原始流的不同源IP地址值的计数,不考虑IP版本。此信息元素优先于特定于IP版本的计数器,除非按版本区分计数很重要。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: totalCounter

数据类型语义:totalCounter

ElementID: 378

元素ID:378

7.3.2. distinctCountOfDestinationIPAddress
7.3.2. DistinctCountofDestinationPaddress

Description: The count of distinct destination IP address values for Original Flows contributing to this Aggregated Flow, without regard to IP version. This Information Element is preferred to the version-specific counters below, unless it is important to separate the counts by version.

Description:参与此聚合流的原始流的不同目标IP地址值的计数,不考虑IP版本。此信息元素优先于以下特定于版本的计数器,除非按版本区分计数很重要。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: totalCounter

数据类型语义:totalCounter

ElementID: 379

元素ID:379

7.3.3. distinctCountOfSourceIPv4Address
7.3.3. distinctcontofsourceipv4地址

Description: The count of distinct source IPv4 address values for Original Flows contributing to this Aggregated Flow.

描述:参与此聚合流的原始流的不同源IPv4地址值的计数。

Abstract Data Type: unsigned32

抽象数据类型:unsigned32

Data Type Semantics: totalCounter

数据类型语义:totalCounter

ElementID: 380

元素ID:380

7.3.4. distinctCountOfDestinationIPv4Address
7.3.4. DistinctCountofDestinationPv4地址

Description: The count of distinct destination IPv4 address values for Original Flows contributing to this Aggregated Flow.

描述:参与此聚合流的原始流的不同目标IPv4地址值的计数。

Abstract Data Type: unsigned32

抽象数据类型:unsigned32

Data Type Semantics: totalCounter

数据类型语义:totalCounter

ElementID: 381

元素ID:381

7.3.5. distinctCountOfSourceIPv6Address
7.3.5. distinctcontofsourceipv6地址

Description: The count of distinct source IPv6 address values for Original Flows contributing to this Aggregated Flow.

描述:参与此聚合流的原始流的不同源IPv6地址值的计数。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: totalCounter

数据类型语义:totalCounter

ElementID: 382

元素ID:382

7.3.6. distinctCountOfDestinationIPv6Address
7.3.6. DistinctCountofDestinationPv6地址

Description: The count of distinct destination IPv6 address values for Original Flows contributing to this Aggregated Flow.

描述:参与此聚合流的原始流的不同目标IPv6地址值的计数。

Abstract Data Type: unsigned64

抽象数据类型:unsigned64

Data Type Semantics: totalCounter

数据类型语义:totalCounter

ElementID: 383

元素ID:383

7.4. Aggregate Counter Distribution Export
7.4. 柜台分销出口总额

When exporting counters distributed among Aggregated Flows, as described in Section 5.1.1, the Exporting Process MAY export an Aggregate Counter Distribution Option Record for each Template describing Aggregated Flow records; this Options Template is described below. It uses the valueDistributionMethod Information Element, also defined below. Since, in many cases, distribution is simple, accounting the counters from Contributing Flows to the first Interval to which they contribute, this is the default situation, for which no Aggregate Counter Distribution Record is necessary; Aggregate Counter Distribution Records are only applicable in more exotic situations, such as using an Aggregation Interval smaller than the durations of Original Flows.

如第5.1.1节所述,当导出分布在聚合流中的计数器时,导出过程可以为描述聚合流记录的每个模板导出聚合计数器分配选项记录;下面介绍此选项模板。它使用valueDistributionMethod信息元素,也定义如下。由于在许多情况下,分配很简单,将计数器从贡献流计算到它们贡献的第一个间隔,这是默认情况,不需要汇总计数器分配记录;聚合计数器分布记录仅适用于更特殊的情况,例如使用小于原始流持续时间的聚合间隔。

7.4.1. Aggregate Counter Distribution Options Template
7.4.1. 聚合计数器分发选项模板

This Options Template defines the Aggregate Counter Distribution Record, which allows the binding of a value distribution method to a Template ID. The scope is the Template ID, whose uniqueness, per [RFC7011], is local to the Transport Session and Observation Domain that generated the Template ID. This is used to signal to the Collecting Process how the counters were distributed. The fields are as below:

此选项模板定义聚合计数器分布记录,该记录允许将值分布方法绑定到模板ID。范围是模板ID,根据[RFC7011],其唯一性,是生成模板ID的传输会话和观察域的本地。它用于向收集进程通知计数器是如何分布的。字段如下所示:

   +-----------------------------+-------------------------------------+
   | IE                          | Description                         |
   +-----------------------------+-------------------------------------+
   | templateId [scope]          | The Template ID of the Template     |
   |                             | defining the Aggregated Flows to    |
   |                             | which this distribution option      |
   |                             | applies.  This Information Element |
   |                             | MUST be defined as a Scope field.   |
   | valueDistributionMethod     | The method used to distribute the   |
   |                             | counters for the Aggregated Flows   |
   |                             | defined by the associated Template. |
   +-----------------------------+-------------------------------------+
        
   +-----------------------------+-------------------------------------+
   | IE                          | Description                         |
   +-----------------------------+-------------------------------------+
   | templateId [scope]          | The Template ID of the Template     |
   |                             | defining the Aggregated Flows to    |
   |                             | which this distribution option      |
   |                             | applies.  This Information Element |
   |                             | MUST be defined as a Scope field.   |
   | valueDistributionMethod     | The method used to distribute the   |
   |                             | counters for the Aggregated Flows   |
   |                             | defined by the associated Template. |
   +-----------------------------+-------------------------------------+
        
7.4.2. valueDistributionMethod Information Element
7.4.2. 值分配方法信息元素

Description: A description of the method used to distribute the counters from Contributing Flows into the Aggregated Flow records described by an associated scope, generally a Template. The method is deemed to apply to all the non-Key Information Elements in the referenced scope for which value distribution is a valid operation; if the originalFlowsInitiated and/or originalFlowsCompleted Information Elements appear in the Template, they are not subject to this distribution method, as they each infer their own distribution method. This is intended to be a complete set of possible value distribution methods; it is encoded as follows:

描述:用于将计数器从贡献流分配到由关联作用域(通常为模板)描述的聚合流记录的方法的描述。该方法被视为适用于参考范围内的所有非关键信息元素,其值分布是有效操作;如果模板中出现原始流动的和/或原始流动的完整信息元素,则它们不受此分发方法的约束,因为它们各自推断出自己的分发方法。这是一整套可能的价值分配方法;其编码如下:

   +-------+-----------------------------------------------------------+
   | Value | Description                                               |
   +-------+-----------------------------------------------------------+
   | 0     | Unspecified: The counters for an Original Flow are        |
   |       | explicitly not distributed according to any other method  |
   |       | defined for this Information Element; use for arbitrary   |
   |       | distribution, or distribution algorithms not described by |
   |       | any other codepoint.                                      |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 1     | Start Interval: The counters for an Original Flow are     |
   |       | added to the counters of the appropriate Aggregated Flow  |
   |       | containing the start time of the Original Flow.  This     |
   |       | should be assumed the default if value distribution       |
   |       | information is not available at a Collecting Process for  |
   |       | an Aggregated Flow.                                       |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 2     | End Interval: The counters for an Original Flow are added |
   |       | to the counters of the appropriate Aggregated Flow        |
   |       | containing the end time of the Original Flow.             |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 3     | Mid Interval: The counters for an Original Flow are added |
   |       | to the counters of a single appropriate Aggregated Flow   |
   |       | containing some timestamp between start and end time of   |
   |       | the Original Flow.                                        |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 4     | Simple Uniform Distribution: Each counter for an Original |
   |       | Flow is divided by the number of time intervals the       |
   |       | Original Flow covers (i.e., of appropriate Aggregated     |
   |       | Flows sharing the same Flow Key), and this number is      |
   |       | added to each corresponding counter in each Aggregated    |
   |       | Flow.                                                     |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 5     | Proportional Uniform Distribution: Each counter for an    |
   |       | Original Flow is divided by the number of time units the  |
   |       | Original Flow covers, to derive a mean count rate.  This  |
   |       | mean count rate is then multiplied by the number of time  |
   |       | units in the intersection of the duration of the Original |
   |       | Flow and the time interval of each Aggregated Flow.       |
   |       |  This is like simple uniform distribution, but accounts   |
   |       | for the fractional portions of a time interval covered by |
   |       | an Original Flow in the first and last time interval.     |
   |       | --------------------------------------------------------- |
        
   +-------+-----------------------------------------------------------+
   | Value | Description                                               |
   +-------+-----------------------------------------------------------+
   | 0     | Unspecified: The counters for an Original Flow are        |
   |       | explicitly not distributed according to any other method  |
   |       | defined for this Information Element; use for arbitrary   |
   |       | distribution, or distribution algorithms not described by |
   |       | any other codepoint.                                      |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 1     | Start Interval: The counters for an Original Flow are     |
   |       | added to the counters of the appropriate Aggregated Flow  |
   |       | containing the start time of the Original Flow.  This     |
   |       | should be assumed the default if value distribution       |
   |       | information is not available at a Collecting Process for  |
   |       | an Aggregated Flow.                                       |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 2     | End Interval: The counters for an Original Flow are added |
   |       | to the counters of the appropriate Aggregated Flow        |
   |       | containing the end time of the Original Flow.             |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 3     | Mid Interval: The counters for an Original Flow are added |
   |       | to the counters of a single appropriate Aggregated Flow   |
   |       | containing some timestamp between start and end time of   |
   |       | the Original Flow.                                        |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 4     | Simple Uniform Distribution: Each counter for an Original |
   |       | Flow is divided by the number of time intervals the       |
   |       | Original Flow covers (i.e., of appropriate Aggregated     |
   |       | Flows sharing the same Flow Key), and this number is      |
   |       | added to each corresponding counter in each Aggregated    |
   |       | Flow.                                                     |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 5     | Proportional Uniform Distribution: Each counter for an    |
   |       | Original Flow is divided by the number of time units the  |
   |       | Original Flow covers, to derive a mean count rate.  This  |
   |       | mean count rate is then multiplied by the number of time  |
   |       | units in the intersection of the duration of the Original |
   |       | Flow and the time interval of each Aggregated Flow.       |
   |       |  This is like simple uniform distribution, but accounts   |
   |       | for the fractional portions of a time interval covered by |
   |       | an Original Flow in the first and last time interval.     |
   |       | --------------------------------------------------------- |
        
   |       | --------------------------------------------------------- |
   | 6     | Simulated Process: Each counter of the Original Flow is   |
   |       | distributed among the intervals of the Aggregated Flows   |
   |       | according to some function the Intermediate Aggregation   |
   |       | Process uses based upon properties of Flows presumed to   |
   |       | be like the Original Flow.  This is essentially an        |
   |       | assertion that the Intermediate Aggregation Process has   |
   |       | no direct packet timing information but is nevertheless   |
   |       | not using one of the other simpler distribution methods.  |
   |       | The Intermediate Aggregation Process specifically makes   |
   |       | no assertion as to the correctness of the simulation.     |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 7     | Direct: The Intermediate Aggregation Process has access   |
   |       | to the original packet timings from the packets making up |
   |       | the Original Flow, and uses these to distribute or        |
   |       | recalculate the counters.                                 |
   +-------+-----------------------------------------------------------+
        
   |       | --------------------------------------------------------- |
   | 6     | Simulated Process: Each counter of the Original Flow is   |
   |       | distributed among the intervals of the Aggregated Flows   |
   |       | according to some function the Intermediate Aggregation   |
   |       | Process uses based upon properties of Flows presumed to   |
   |       | be like the Original Flow.  This is essentially an        |
   |       | assertion that the Intermediate Aggregation Process has   |
   |       | no direct packet timing information but is nevertheless   |
   |       | not using one of the other simpler distribution methods.  |
   |       | The Intermediate Aggregation Process specifically makes   |
   |       | no assertion as to the correctness of the simulation.     |
   |       | --------------------------------------------------------- |
   |       |                                                           |
   | 7     | Direct: The Intermediate Aggregation Process has access   |
   |       | to the original packet timings from the packets making up |
   |       | the Original Flow, and uses these to distribute or        |
   |       | recalculate the counters.                                 |
   +-------+-----------------------------------------------------------+
        

Abstract Data Type: unsigned8

抽象数据类型:unsigned8

ElementID: 384

元素ID:384

8. Examples
8. 例子

In these examples, the same data, described by the same Template, will be aggregated multiple different ways; this illustrates the various different functions that could be implemented by Intermediate Aggregation Processes. Templates are shown in IESpec format as introduced in [RFC7013]. The source data format is a simplified Flow: timestamps, traditional 5-tuple, and octet count; the Flow Key fields are the 5-tuple. The Template is shown in Figure 9.

在这些示例中,由相同模板描述的相同数据将以多种不同方式聚合;这说明了中间聚合过程可以实现的各种不同功能。模板以[RFC7013]中介绍的IESpec格式显示。源数据格式是一个简化的流:时间戳、传统的5元组和八位字节计数;流键字段是5元组。该模板如图9所示。

   flowStartMilliseconds(152)[8]
   flowEndMilliseconds(153)[8]
   sourceIPv4Address(8)[4]{key}
   destinationIPv4Address(12)[4]{key}
   sourceTransportPort(7)[2]{key}
   destinationTransportPort(11)[2]{key}
   protocolIdentifier(4)[1]{key}
   octetDeltaCount(1)[8]
        
   flowStartMilliseconds(152)[8]
   flowEndMilliseconds(153)[8]
   sourceIPv4Address(8)[4]{key}
   destinationIPv4Address(12)[4]{key}
   sourceTransportPort(7)[2]{key}
   destinationTransportPort(11)[2]{key}
   protocolIdentifier(4)[1]{key}
   octetDeltaCount(1)[8]
        

Figure 9: Input Template for Examples

图9:示例的输入模板

   The data records given as input to the examples in this section are
   shown below; timestamps are given in H:MM:SS.sss format.  In this and
   subsequent figures, flowStartMilliseconds is shown in H:MM:SS.sss
   format as 'start time', flowEndMilliseconds is shown in H:MM:SS.sss
        
   The data records given as input to the examples in this section are
   shown below; timestamps are given in H:MM:SS.sss format.  In this and
   subsequent figures, flowStartMilliseconds is shown in H:MM:SS.sss
   format as 'start time', flowEndMilliseconds is shown in H:MM:SS.sss
        

format as 'end time', sourceIPv4Address is shown as 'source ip4' with the following 'port' representing sourceTransportPort, destinationIPv4Address is shown as 'dest ip4' with the following 'port' representing destinationTransportPort, protocolIdentifier is shown as 'pt', and octetDeltaCount as 'oct'.

格式为“结束时间”,sourceIPv4Address显示为“source ip4”,以下“port”表示sourceTransportPort,DestinationIP4Address显示为“dest ip4”,以下“port”表示destinationTransportPort,protocolIdentifier显示为“pt”,octetDeltaCount显示为“oct”。

  start time |end time   |source ip4 |port |dest ip4      |port|pt|  oct
  9:00:00.138 9:00:00.138 192.0.2.2   47113 192.0.2.131    53   17   119
  9:00:03.246 9:00:03.246 192.0.2.2   22153 192.0.2.131    53   17    83
  9:00:00.478 9:00:03.486 192.0.2.2   52420 198.51.100.2   443  6   1637
  9:00:07.172 9:00:07.172 192.0.2.3   56047 192.0.2.131    53   17   111
  9:00:07.309 9:00:14.861 192.0.2.3   41183 198.51.100.67  80   6  16838
  9:00:03.556 9:00:19.876 192.0.2.2   17606 198.51.100.68  80   6  11538
  9:00:25.210 9:00:25.210 192.0.2.3   47113 192.0.2.131    53   17   119
  9:00:26.358 9:00:30.198 192.0.2.3   48458 198.51.100.133 80   6   2973
  9:00:29.213 9:01:00.061 192.0.2.4   61295 198.51.100.2   443  6   8350
  9:04:00.207 9:04:04.431 203.0.113.3 41256 198.51.100.133 80   6    778
  9:03:59.624 9:04:06.984 203.0.113.3 51662 198.51.100.3   80   6    883
  9:00:30.532 9:06:15.402 192.0.2.2   37581 198.51.100.2   80   6  15420
  9:06:56.813 9:06:59.821 203.0.113.3 52572 198.51.100.2   443  6   1637
  9:06:30.565 9:07:00.261 203.0.113.3 49914 198.51.100.133 80   6    561
  9:06:55.160 9:07:05.208 192.0.2.2   50824 198.51.100.2   443  6   1899
  9:06:49.322 9:07:05.322 192.0.2.3   34597 198.51.100.3   80   6   1284
  9:07:05.849 9:07:09.625 203.0.113.3 58907 198.51.100.4   80   6   2670
  9:10:45.161 9:10:45.161 192.0.2.4   22478 192.0.2.131    53   17    75
  9:10:45.209 9:11:01.465 192.0.2.4   49513 198.51.100.68  80   6   3374
  9:10:57.094 9:11:00.614 192.0.2.4   64832 198.51.100.67  80   6    138
  9:10:59.770 9:11:02.842 192.0.2.3   60833 198.51.100.69  443  6   2325
  9:02:18.390 9:13:46.598 203.0.113.3 39586 198.51.100.17  80   6  11200
  9:13:53.933 9:14:06.605 192.0.2.2   19638 198.51.100.3   80   6   2869
  9:13:02.864 9:14:08.720 192.0.2.3   40429 198.51.100.4   80   6  18289
        
  start time |end time   |source ip4 |port |dest ip4      |port|pt|  oct
  9:00:00.138 9:00:00.138 192.0.2.2   47113 192.0.2.131    53   17   119
  9:00:03.246 9:00:03.246 192.0.2.2   22153 192.0.2.131    53   17    83
  9:00:00.478 9:00:03.486 192.0.2.2   52420 198.51.100.2   443  6   1637
  9:00:07.172 9:00:07.172 192.0.2.3   56047 192.0.2.131    53   17   111
  9:00:07.309 9:00:14.861 192.0.2.3   41183 198.51.100.67  80   6  16838
  9:00:03.556 9:00:19.876 192.0.2.2   17606 198.51.100.68  80   6  11538
  9:00:25.210 9:00:25.210 192.0.2.3   47113 192.0.2.131    53   17   119
  9:00:26.358 9:00:30.198 192.0.2.3   48458 198.51.100.133 80   6   2973
  9:00:29.213 9:01:00.061 192.0.2.4   61295 198.51.100.2   443  6   8350
  9:04:00.207 9:04:04.431 203.0.113.3 41256 198.51.100.133 80   6    778
  9:03:59.624 9:04:06.984 203.0.113.3 51662 198.51.100.3   80   6    883
  9:00:30.532 9:06:15.402 192.0.2.2   37581 198.51.100.2   80   6  15420
  9:06:56.813 9:06:59.821 203.0.113.3 52572 198.51.100.2   443  6   1637
  9:06:30.565 9:07:00.261 203.0.113.3 49914 198.51.100.133 80   6    561
  9:06:55.160 9:07:05.208 192.0.2.2   50824 198.51.100.2   443  6   1899
  9:06:49.322 9:07:05.322 192.0.2.3   34597 198.51.100.3   80   6   1284
  9:07:05.849 9:07:09.625 203.0.113.3 58907 198.51.100.4   80   6   2670
  9:10:45.161 9:10:45.161 192.0.2.4   22478 192.0.2.131    53   17    75
  9:10:45.209 9:11:01.465 192.0.2.4   49513 198.51.100.68  80   6   3374
  9:10:57.094 9:11:00.614 192.0.2.4   64832 198.51.100.67  80   6    138
  9:10:59.770 9:11:02.842 192.0.2.3   60833 198.51.100.69  443  6   2325
  9:02:18.390 9:13:46.598 203.0.113.3 39586 198.51.100.17  80   6  11200
  9:13:53.933 9:14:06.605 192.0.2.2   19638 198.51.100.3   80   6   2869
  9:13:02.864 9:14:08.720 192.0.2.3   40429 198.51.100.4   80   6  18289
        

Figure 10: Input Data for Examples

图10:示例的输入数据

8.1. Traffic Time Series per Source
8.1. 每个源的流量时间序列

Aggregating Flows by source IP address in time series (i.e., with a regular interval) can be used in subsequent heavy-hitter analysis and as a source parameter for statistical anomaly detection techniques. Here, the Intermediate Aggregation Process imposes an interval, aggregates the key to remove all key fields other than the source IP address, then combines the result into a stream of Aggregated Flows. The imposed interval of five minutes is longer than the majority of Flows; for those Flows crossing interval boundaries, the entire Flow is accounted to the interval containing the start time of the Flow.

按源IP地址按时间序列(即有规律的间隔)聚合流可用于后续重击分析,并作为统计异常检测技术的源参数。在这里,中间聚合过程施加一个间隔,聚合密钥以删除源IP地址以外的所有密钥字段,然后将结果合并到聚合流流中。施加的5分钟间隔比大多数流量长;对于跨越区间边界的流量,整个流量计入包含流量开始时间的区间。

In this example, the Partially Aggregated Flows after each conceptual operation in the Intermediate Aggregation Process are shown. These are meant to be illustrative of the conceptual operations only, and not to suggest an implementation (indeed, the example shown here would not necessarily be the most efficient method for performing these operations). Subsequent examples will omit the Partially Aggregated Flows for brevity.

在此示例中,显示了中间聚合过程中每个概念操作之后的部分聚合流。这些只是为了说明概念性操作,而不是建议实现(实际上,这里显示的示例不一定是执行这些操作的最有效方法)。为简洁起见,后续示例将省略部分聚合流。

The input to this process could be any Flow Record containing a source IP address and octet counter; consider for this example the Template and data from the introduction. The Intermediate Aggregation Process would then output records containing just timestamps, source IP, and octetDeltaCount, as in Figure 11.

这个过程的输入可以是任何包含源IP地址和八位字节计数器的流记录;从这个例子中考虑模板和数据的介绍。然后,中间聚合过程将输出只包含时间戳、源IP和octetDeltaCount的记录,如图11所示。

   flowStartMilliseconds(152)[8]
   flowEndMilliseconds(153)[8]
   sourceIPv4Address(8)[4]
   octetDeltaCount(1)[8]
        
   flowStartMilliseconds(152)[8]
   flowEndMilliseconds(153)[8]
   sourceIPv4Address(8)[4]
   octetDeltaCount(1)[8]
        

Figure 11: Output Template for Time Series per Source

图11:每个源的时间序列的输出模板

Assume the goal is to get 5-minute (300 s) time series of octet counts per source IP address. The aggregation operations would then be arranged as in Figure 12.

假设目标是获得每个源IP地址5分钟(300秒)的八位组计数时间序列。聚合操作将如图12所示进行安排。

                    Original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * impose uniform     |
              |    300s time interval |
              +-----------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +------------------------+
   |  key aggregation       |
   |   * reduce key to only |
   |     sourceIPv4Address  |
   +------------------------+
                  |
                  | Partially Aggregated Flows
                  V
             +-------------------------+
             |  aggregate combination  |
             |   * sum octetDeltaCount |
             +-------------------------+
                          |
                          V
                  Aggregated Flows
        
                    Original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * impose uniform     |
              |    300s time interval |
              +-----------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +------------------------+
   |  key aggregation       |
   |   * reduce key to only |
   |     sourceIPv4Address  |
   +------------------------+
                  |
                  | Partially Aggregated Flows
                  V
             +-------------------------+
             |  aggregate combination  |
             |   * sum octetDeltaCount |
             +-------------------------+
                          |
                          V
                  Aggregated Flows
        

Figure 12: Aggregation Operations for Time Series per Source

图12:每个源的时间序列的聚合操作

After applying the interval distribution step to the source data in Figure 10, only the time intervals have changed; the Partially Aggregated Flows are shown in Figure 13. Note that interval distribution follows the default Start Interval policy; that is, the entire Flow is accounted to the interval containing the Flow's start time.

对图10中的源数据应用间隔分布步骤后,只有时间间隔发生了变化;部分聚合流如图13所示。请注意,间隔分布遵循默认的开始间隔策略;也就是说,整个流将计入包含流的开始时间的间隔。

  start time |end time   |source ip4 |port |dest ip4      |port|pt|  oct
  9:00:00.000 9:05:00.000 192.0.2.2   47113 192.0.2.131    53   17   119
  9:00:00.000 9:05:00.000 192.0.2.2   22153 192.0.2.131    53   17    83
  9:00:00.000 9:05:00.000 192.0.2.2   52420 198.51.100.2   443  6   1637
  9:00:00.000 9:05:00.000 192.0.2.3   56047 192.0.2.131    53   17   111
  9:00:00.000 9:05:00.000 192.0.2.3   41183 198.51.100.67  80   6  16838
  9:00:00.000 9:05:00.000 192.0.2.2   17606 198.51.100.68  80   6  11538
  9:00:00.000 9:05:00.000 192.0.2.3   47113 192.0.2.131    53   17   119
  9:00:00.000 9:05:00.000 192.0.2.3   48458 198.51.100.133 80   6   2973
  9:00:00.000 9:05:00.000 192.0.2.4   61295 198.51.100.2   443  6   8350
  9:00:00.000 9:05:00.000 203.0.113.3 41256 198.51.100.133 80   6    778
  9:00:00.000 9:05:00.000 203.0.113.3 51662 198.51.100.3   80   6    883
  9:00:00.000 9:05:00.000 192.0.2.2   37581 198.51.100.2   80   6  15420
  9:00:00.000 9:05:00.000 203.0.113.3 39586 198.51.100.17  80   6  11200
  9:05:00.000 9:10:00.000 203.0.113.3 52572 198.51.100.2   443  6   1637
  9:05:00.000 9:10:00.000 203.0.113.3 49914 197.51.100.133 80   6    561
  9:05:00.000 9:10:00.000 192.0.2.2   50824 198.51.100.2   443  6   1899
  9:05:00.000 9:10:00.000 192.0.2.3   34597 198.51.100.3   80   6   1284
  9:05:00.000 9:10:00.000 203.0.113.3 58907 198.51.100.4   80   6   2670
  9:10:00.000 9:15:00.000 192.0.2.4   22478 192.0.2.131    53   17    75
  9:10:00.000 9:15:00.000 192.0.2.4   49513 198.51.100.68  80   6   3374
  9:10:00.000 9:15:00.000 192.0.2.4   64832 198.51.100.67  80   6    138
  9:10:00.000 9:15:00.000 192.0.2.3   60833 198.51.100.69  443  6   2325
  9:10:00.000 9:15:00.000 192.0.2.2   19638 198.51.100.3   80   6   2869
  9:10:00.000 9:15:00.000 192.0.2.3   40429 198.51.100.4   80   6  18289
        
  start time |end time   |source ip4 |port |dest ip4      |port|pt|  oct
  9:00:00.000 9:05:00.000 192.0.2.2   47113 192.0.2.131    53   17   119
  9:00:00.000 9:05:00.000 192.0.2.2   22153 192.0.2.131    53   17    83
  9:00:00.000 9:05:00.000 192.0.2.2   52420 198.51.100.2   443  6   1637
  9:00:00.000 9:05:00.000 192.0.2.3   56047 192.0.2.131    53   17   111
  9:00:00.000 9:05:00.000 192.0.2.3   41183 198.51.100.67  80   6  16838
  9:00:00.000 9:05:00.000 192.0.2.2   17606 198.51.100.68  80   6  11538
  9:00:00.000 9:05:00.000 192.0.2.3   47113 192.0.2.131    53   17   119
  9:00:00.000 9:05:00.000 192.0.2.3   48458 198.51.100.133 80   6   2973
  9:00:00.000 9:05:00.000 192.0.2.4   61295 198.51.100.2   443  6   8350
  9:00:00.000 9:05:00.000 203.0.113.3 41256 198.51.100.133 80   6    778
  9:00:00.000 9:05:00.000 203.0.113.3 51662 198.51.100.3   80   6    883
  9:00:00.000 9:05:00.000 192.0.2.2   37581 198.51.100.2   80   6  15420
  9:00:00.000 9:05:00.000 203.0.113.3 39586 198.51.100.17  80   6  11200
  9:05:00.000 9:10:00.000 203.0.113.3 52572 198.51.100.2   443  6   1637
  9:05:00.000 9:10:00.000 203.0.113.3 49914 197.51.100.133 80   6    561
  9:05:00.000 9:10:00.000 192.0.2.2   50824 198.51.100.2   443  6   1899
  9:05:00.000 9:10:00.000 192.0.2.3   34597 198.51.100.3   80   6   1284
  9:05:00.000 9:10:00.000 203.0.113.3 58907 198.51.100.4   80   6   2670
  9:10:00.000 9:15:00.000 192.0.2.4   22478 192.0.2.131    53   17    75
  9:10:00.000 9:15:00.000 192.0.2.4   49513 198.51.100.68  80   6   3374
  9:10:00.000 9:15:00.000 192.0.2.4   64832 198.51.100.67  80   6    138
  9:10:00.000 9:15:00.000 192.0.2.3   60833 198.51.100.69  443  6   2325
  9:10:00.000 9:15:00.000 192.0.2.2   19638 198.51.100.3   80   6   2869
  9:10:00.000 9:15:00.000 192.0.2.3   40429 198.51.100.4   80   6  18289
        

Figure 13: Interval Imposition for Time Series per Source

图13:每个源的时间序列的间隔强制

After the key aggregation step, all Flow Keys except the source IP address have been discarded, as shown in Figure 14. This leaves duplicate Partially Aggregated Flows to be combined in the final operation.

在密钥聚合步骤之后,除了源IP地址之外的所有流密钥都被丢弃,如图14所示。这将在最终操作中合并重复的部分聚合流。

   start time |end time   |source ip4 |octets
   9:00:00.000 9:05:00.000 192.0.2.2      119
   9:00:00.000 9:05:00.000 192.0.2.2       83
   9:00:00.000 9:05:00.000 192.0.2.2     1637
   9:00:00.000 9:05:00.000 192.0.2.3      111
   9:00:00.000 9:05:00.000 192.0.2.3    16838
   9:00:00.000 9:05:00.000 192.0.2.2    11538
   9:00:00.000 9:05:00.000 192.0.2.3      119
   9:00:00.000 9:05:00.000 192.0.2.3     2973
   9:00:00.000 9:05:00.000 192.0.2.4     8350
   9:00:00.000 9:05:00.000 203.0.113.3    778
   9:00:00.000 9:05:00.000 203.0.113.3    883
   9:00:00.000 9:05:00.000 192.0.2.2    15420
   9:00:00.000 9:05:00.000 203.0.113.3  11200
   9:05:00.000 9:10:00.000 203.0.113.3   1637
   9:05:00.000 9:10:00.000 203.0.113.3    561
   9:05:00.000 9:10:00.000 192.0.2.2     1899
   9:05:00.000 9:10:00.000 192.0.2.3     1284
   9:05:00.000 9:10:00.000 203.0.113.3   2670
   9:10:00.000 9:15:00.000 192.0.2.4       75
   9:10:00.000 9:15:00.000 192.0.2.4     3374
   9:10:00.000 9:15:00.000 192.0.2.4      138
   9:10:00.000 9:15:00.000 192.0.2.3     2325
   9:10:00.000 9:15:00.000 192.0.2.2     2869
   9:10:00.000 9:15:00.000 192.0.2.3    18289
        
   start time |end time   |source ip4 |octets
   9:00:00.000 9:05:00.000 192.0.2.2      119
   9:00:00.000 9:05:00.000 192.0.2.2       83
   9:00:00.000 9:05:00.000 192.0.2.2     1637
   9:00:00.000 9:05:00.000 192.0.2.3      111
   9:00:00.000 9:05:00.000 192.0.2.3    16838
   9:00:00.000 9:05:00.000 192.0.2.2    11538
   9:00:00.000 9:05:00.000 192.0.2.3      119
   9:00:00.000 9:05:00.000 192.0.2.3     2973
   9:00:00.000 9:05:00.000 192.0.2.4     8350
   9:00:00.000 9:05:00.000 203.0.113.3    778
   9:00:00.000 9:05:00.000 203.0.113.3    883
   9:00:00.000 9:05:00.000 192.0.2.2    15420
   9:00:00.000 9:05:00.000 203.0.113.3  11200
   9:05:00.000 9:10:00.000 203.0.113.3   1637
   9:05:00.000 9:10:00.000 203.0.113.3    561
   9:05:00.000 9:10:00.000 192.0.2.2     1899
   9:05:00.000 9:10:00.000 192.0.2.3     1284
   9:05:00.000 9:10:00.000 203.0.113.3   2670
   9:10:00.000 9:15:00.000 192.0.2.4       75
   9:10:00.000 9:15:00.000 192.0.2.4     3374
   9:10:00.000 9:15:00.000 192.0.2.4      138
   9:10:00.000 9:15:00.000 192.0.2.3     2325
   9:10:00.000 9:15:00.000 192.0.2.2     2869
   9:10:00.000 9:15:00.000 192.0.2.3    18289
        

Figure 14: Key Aggregation for Time Series per Source

图14:每个源的时间序列的键聚合

Aggregate combination sums the counters per key and interval; the summations of the first two keys and intervals are shown in detail in Figure 15.

聚合组合对每个键和间隔的计数器求和;图15详细显示了前两个键和间隔的总和。

     start time |end time   |source ip4 |octets
     9:00:00.000 9:05:00.000 192.0.2.2      119
     9:00:00.000 9:05:00.000 192.0.2.2       83
     9:00:00.000 9:05:00.000 192.0.2.2     1637
     9:00:00.000 9:05:00.000 192.0.2.2    11538
   + 9:00:00.000 9:05:00.000 192.0.2.2    15420
                                          -----
   = 9:00:00.000 9:05:00.000 192.0.2.2    28797
        
     start time |end time   |source ip4 |octets
     9:00:00.000 9:05:00.000 192.0.2.2      119
     9:00:00.000 9:05:00.000 192.0.2.2       83
     9:00:00.000 9:05:00.000 192.0.2.2     1637
     9:00:00.000 9:05:00.000 192.0.2.2    11538
   + 9:00:00.000 9:05:00.000 192.0.2.2    15420
                                          -----
   = 9:00:00.000 9:05:00.000 192.0.2.2    28797
        
     9:00:00.000 9:05:00.000 192.0.2.3      111
     9:00:00.000 9:05:00.000 192.0.2.3    16838
     9:00:00.000 9:05:00.000 192.0.2.3      119
   + 9:00:00.000 9:05:00.000 192.0.2.3     2973
                                          -----
   = 9:00:00.000 9:05:00.000 192.0.2.3    20041
        
     9:00:00.000 9:05:00.000 192.0.2.3      111
     9:00:00.000 9:05:00.000 192.0.2.3    16838
     9:00:00.000 9:05:00.000 192.0.2.3      119
   + 9:00:00.000 9:05:00.000 192.0.2.3     2973
                                          -----
   = 9:00:00.000 9:05:00.000 192.0.2.3    20041
        

Figure 15: Summation during Aggregate Combination

图15:骨料组合期间的总和

This can be applied to each set of Partially Aggregated Flows to produce the final Aggregated Flows that are shown in Figure 16, as exported by the Template in Figure 11.

这可以应用于每一组部分聚合的流,以生成最终的聚合流,如图16所示,如图11中的模板所导出。

   start time |end time   |source ip4 |octets
   9:00:00.000 9:05:00.000 192.0.2.2    28797
   9:00:00.000 9:05:00.000 192.0.2.3    20041
   9:00:00.000 9:05:00.000 192.0.2.4     8350
   9:00:00.000 9:05:00.000 203.0.113.3  12861
   9:05:00.000 9:10:00.000 192.0.2.2     1899
   9:05:00.000 9:10:00.000 192.0.2.3     1284
   9:05:00.000 9:10:00.000 203.0.113.3   4868
   9:10:00.000 9:15:00.000 192.0.2.2     2869
   9:10:00.000 9:15:00.000 192.0.2.3    20614
   9:10:00.000 9:15:00.000 192.0.2.4     3587
        
   start time |end time   |source ip4 |octets
   9:00:00.000 9:05:00.000 192.0.2.2    28797
   9:00:00.000 9:05:00.000 192.0.2.3    20041
   9:00:00.000 9:05:00.000 192.0.2.4     8350
   9:00:00.000 9:05:00.000 203.0.113.3  12861
   9:05:00.000 9:10:00.000 192.0.2.2     1899
   9:05:00.000 9:10:00.000 192.0.2.3     1284
   9:05:00.000 9:10:00.000 203.0.113.3   4868
   9:10:00.000 9:15:00.000 192.0.2.2     2869
   9:10:00.000 9:15:00.000 192.0.2.3    20614
   9:10:00.000 9:15:00.000 192.0.2.4     3587
        

Figure 16: Aggregated Flows for Time Series per Source

图16:每个源的时间序列的聚合流

8.2. Core Traffic Matrix
8.2. 核心流量矩阵

Aggregating Flows by source and destination ASN in time series is used to generate core traffic matrices. The core traffic matrix provides a view of the state of the routes within a network, and it can be used for long-term planning of changes to network design based on traffic demand. Here, imposed time intervals are generally much longer than active Flow timeouts. The traffic matrix is reported in terms of octets, packets, and flows, as each of these values may have a subtly different effect on capacity planning.

将源ASN和目标ASN的流量按时间序列进行聚合,以生成核心流量矩阵。核心交通矩阵提供了网络内路线状态的视图,可用于根据交通需求对网络设计变更进行长期规划。这里,施加的时间间隔通常比主动流超时长得多。流量矩阵以八位字节、数据包和流的形式报告,因为这些值中的每一个都可能对容量规划产生细微不同的影响。

This example demonstrates key aggregation using derived keys and Original Flow counting. While some Original Flows may be generated by Exporting Processes on forwarding devices, and therefore contain the bgpSourceAsNumber and bgpDestinationAsNumber Information Elements, Original Flows from Exporting Processes on dedicated measurement devices without routing data contain only a destinationIPv[46]Address. For these Flows, the Mediator must look up a next-hop AS from an IP-to-AS table, replacing source and destination addresses with ASNs. The table used in this example is shown in Figure 17. (Note that due to limited example address space, in this example we ignore the common practice of routing only blocks of /24 or larger.)

此示例演示使用派生密钥和原始流计数的密钥聚合。虽然一些原始流可能是通过在转发设备上导出进程生成的,因此包含bgpSourceAsNumber和bgpDestinationAsNumber信息元素,但在没有路由数据的专用测量设备上导出进程的原始流仅包含DestinationPV[46]地址。对于这些流,中介器必须查找从IP到AS表的下一个跃点AS,用ASN替换源地址和目标地址。本例中使用的表格如图17所示。(注意,由于示例地址空间有限,在此示例中,我们忽略了只路由/24或更大的块的常见做法。)

prefix |ASN 192.0.2.0/25 64496 192.0.2.128/25 64497 198.51.100/24 64498 203.0.113.0/24 64499

前缀| ASN 192.0.2.0/25 64496 192.0.2.128/25 64497 198.51.100/24 64498 203.0.113.0/24 64499

Figure 17: Example ASN Map

图17:ASN映射示例

The Template for Aggregated Flows produced by this example is shown in Figure 18.

此示例生成的聚合流模板如图18所示。

   flowStartMilliseconds(152)[8]
   flowEndMilliseconds(153)[8]
   bgpSourceAsNumber(16)[4]
   bgpDestinationAsNumber(17)[4]
   octetDeltaCount(1)[8]
        
   flowStartMilliseconds(152)[8]
   flowEndMilliseconds(153)[8]
   bgpSourceAsNumber(16)[4]
   bgpDestinationAsNumber(17)[4]
   octetDeltaCount(1)[8]
        

Figure 18: Output Template for Traffic Matrix

图18:流量矩阵的输出模板

Assume the goal is to get 60-minute time series of octet counts per source/destination ASN pair. The aggregation operations would then be arranged as in Figure 19.

假设目标是获得每个源/目标ASN对的60分钟八位组计数时间序列。聚合操作将如图19所示进行安排。

                    Original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * impose uniform     |
              |    3600s time interval|
              +-----------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +------------------------+
   |  key aggregation       |
   |  * reduce key to only  |
   |    sourceIPv4Address + |
   |    destIPv4Address     |
   +------------------------+
                  |
                  V
   +------------------------+
   |  key aggregation       |
   |  * replace addresses   |
   |    with ASN from map   |
   +------------------------+
                  |
                  | Partially Aggregated Flows
                  V
             +-------------------------+
             |  aggregate combination  |
             |   * sum octetDeltaCount |
             +-------------------------+
                          |
                          V
                  Aggregated Flows
        
                    Original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * impose uniform     |
              |    3600s time interval|
              +-----------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +------------------------+
   |  key aggregation       |
   |  * reduce key to only  |
   |    sourceIPv4Address + |
   |    destIPv4Address     |
   +------------------------+
                  |
                  V
   +------------------------+
   |  key aggregation       |
   |  * replace addresses   |
   |    with ASN from map   |
   +------------------------+
                  |
                  | Partially Aggregated Flows
                  V
             +-------------------------+
             |  aggregate combination  |
             |   * sum octetDeltaCount |
             +-------------------------+
                          |
                          V
                  Aggregated Flows
        

Figure 19: Aggregation Operations for Traffic Matrix

图19:流量矩阵的聚合操作

After applying the interval distribution step to the source data in Figure 10, the Partially Aggregated Flows are shown in Figure 20. Note that the Flows are identical to those in the interval distribution step in the previous example, except the chosen interval (1 hour, 3600 seconds) is different; therefore, all the Flows fit into a single interval.

对图10中的源数据应用interval distribution步骤后,部分聚合的流如图20所示。注意,除了所选择的间隔(1小时3600秒)不同之外,流量与前一示例中的间隔分配步骤中的流量相同;因此,所有的流都适合于单个间隔。

   start time |end time |source ip4 |port |dest ip4      |port|pt|  oct
   9:00:00     10:00:00  192.0.2.2   47113 192.0.2.131    53   17   119
   9:00:00     10:00:00  192.0.2.2   22153 192.0.2.131    53   17    83
   9:00:00     10:00:00  192.0.2.2   52420 198.51.100.2   443  6   1637
   9:00:00     10:00:00  192.0.2.3   56047 192.0.2.131    53   17   111
   9:00:00     10:00:00  192.0.2.3   41183 198.51.100.67  80   6  16838
   9:00:00     10:00:00  192.0.2.2   17606 198.51.100.68  80   6  11538
   9:00:00     10:00:00  192.0.2.3   47113 192.0.2.131    53   17   119
   9:00:00     10:00:00  192.0.2.3   48458 198.51.100.133 80   6   2973
   9:00:00     10:00:00  192.0.2.4   61295 198.51.100.2   443  6   8350
   9:00:00     10:00:00  203.0.113.3 41256 198.51.100.133 80   6    778
   9:00:00     10:00:00  203.0.113.3 51662 198.51.100.3   80   6    883
   9:00:00     10:00:00  192.0.2.2   37581 198.51.100.2   80   6  15420
   9:00:00     10:00:00  203.0.113.3 52572 198.51.100.2   443  6   1637
   9:00:00     10:00:00  203.0.113.3 49914 197.51.100.133 80   6    561
   9:00:00     10:00:00  192.0.2.2   50824 198.51.100.2   443  6   1899
   9:00:00     10:00:00  192.0.2.3   34597 198.51.100.3   80   6   1284
   9:00:00     10:00:00  203.0.113.3 58907 198.51.100.4   80   6   2670
   9:00:00     10:00:00  192.0.2.4   22478 192.0.2.131    53   17    75
   9:00:00     10:00:00  192.0.2.4   49513 198.51.100.68  80   6   3374
   9:00:00     10:00:00  192.0.2.4   64832 198.51.100.67  80   6    138
   9:00:00     10:00:00  192.0.2.3   60833 198.51.100.69  443  6   2325
   9:00:00     10:00:00  203.0.113.3 39586 198.51.100.17  80   6  11200
   9:00:00     10:00:00  192.0.2.2   19638 198.51.100.3   80   6   2869
   9:00:00     10:00:00  192.0.2.3   40429 198.51.100.4   80   6  18289
        
   start time |end time |source ip4 |port |dest ip4      |port|pt|  oct
   9:00:00     10:00:00  192.0.2.2   47113 192.0.2.131    53   17   119
   9:00:00     10:00:00  192.0.2.2   22153 192.0.2.131    53   17    83
   9:00:00     10:00:00  192.0.2.2   52420 198.51.100.2   443  6   1637
   9:00:00     10:00:00  192.0.2.3   56047 192.0.2.131    53   17   111
   9:00:00     10:00:00  192.0.2.3   41183 198.51.100.67  80   6  16838
   9:00:00     10:00:00  192.0.2.2   17606 198.51.100.68  80   6  11538
   9:00:00     10:00:00  192.0.2.3   47113 192.0.2.131    53   17   119
   9:00:00     10:00:00  192.0.2.3   48458 198.51.100.133 80   6   2973
   9:00:00     10:00:00  192.0.2.4   61295 198.51.100.2   443  6   8350
   9:00:00     10:00:00  203.0.113.3 41256 198.51.100.133 80   6    778
   9:00:00     10:00:00  203.0.113.3 51662 198.51.100.3   80   6    883
   9:00:00     10:00:00  192.0.2.2   37581 198.51.100.2   80   6  15420
   9:00:00     10:00:00  203.0.113.3 52572 198.51.100.2   443  6   1637
   9:00:00     10:00:00  203.0.113.3 49914 197.51.100.133 80   6    561
   9:00:00     10:00:00  192.0.2.2   50824 198.51.100.2   443  6   1899
   9:00:00     10:00:00  192.0.2.3   34597 198.51.100.3   80   6   1284
   9:00:00     10:00:00  203.0.113.3 58907 198.51.100.4   80   6   2670
   9:00:00     10:00:00  192.0.2.4   22478 192.0.2.131    53   17    75
   9:00:00     10:00:00  192.0.2.4   49513 198.51.100.68  80   6   3374
   9:00:00     10:00:00  192.0.2.4   64832 198.51.100.67  80   6    138
   9:00:00     10:00:00  192.0.2.3   60833 198.51.100.69  443  6   2325
   9:00:00     10:00:00  203.0.113.3 39586 198.51.100.17  80   6  11200
   9:00:00     10:00:00  192.0.2.2   19638 198.51.100.3   80   6   2869
   9:00:00     10:00:00  192.0.2.3   40429 198.51.100.4   80   6  18289
        

Figure 20: Interval Imposition for Traffic Matrix

图20:流量矩阵的间隔施加

The next steps are to discard irrelevant key fields and to replace the source and destination addresses with source and destination ASNs in the map; the results of these key aggregation steps are shown in Figure 21.

接下来的步骤是丢弃不相关的键字段,并用映射中的源和目标ASN替换源和目标地址;这些关键聚合步骤的结果如图21所示。

   start time |end time |source ASN |dest ASN |octets
   9:00:00     10:00:00  AS64496     AS64497      119
   9:00:00     10:00:00  AS64496     AS64497       83
   9:00:00     10:00:00  AS64496     AS64498     1637
   9:00:00     10:00:00  AS64496     AS64497      111
   9:00:00     10:00:00  AS64496     AS64498    16838
   9:00:00     10:00:00  AS64496     AS64498    11538
   9:00:00     10:00:00  AS64496     AS64497      119
   9:00:00     10:00:00  AS64496     AS64498     2973
   9:00:00     10:00:00  AS64496     AS64498     8350
   9:00:00     10:00:00  AS64499     AS64498      778
   9:00:00     10:00:00  AS64499     AS64498      883
   9:00:00     10:00:00  AS64496     AS64498    15420
   9:00:00     10:00:00  AS64499     AS64498     1637
   9:00:00     10:00:00  AS64499     AS64498      561
   9:00:00     10:00:00  AS64496     AS64498     1899
   9:00:00     10:00:00  AS64496     AS64498     1284
   9:00:00     10:00:00  AS64499     AS64498     2670
   9:00:00     10:00:00  AS64496     AS64497       75
   9:00:00     10:00:00  AS64496     AS64498     3374
   9:00:00     10:00:00  AS64496     AS64498      138
   9:00:00     10:00:00  AS64496     AS64498     2325
   9:00:00     10:00:00  AS64499     AS64498    11200
   9:00:00     10:00:00  AS64496     AS64498     2869
   9:00:00     10:00:00  AS64496     AS64498    18289
        
   start time |end time |source ASN |dest ASN |octets
   9:00:00     10:00:00  AS64496     AS64497      119
   9:00:00     10:00:00  AS64496     AS64497       83
   9:00:00     10:00:00  AS64496     AS64498     1637
   9:00:00     10:00:00  AS64496     AS64497      111
   9:00:00     10:00:00  AS64496     AS64498    16838
   9:00:00     10:00:00  AS64496     AS64498    11538
   9:00:00     10:00:00  AS64496     AS64497      119
   9:00:00     10:00:00  AS64496     AS64498     2973
   9:00:00     10:00:00  AS64496     AS64498     8350
   9:00:00     10:00:00  AS64499     AS64498      778
   9:00:00     10:00:00  AS64499     AS64498      883
   9:00:00     10:00:00  AS64496     AS64498    15420
   9:00:00     10:00:00  AS64499     AS64498     1637
   9:00:00     10:00:00  AS64499     AS64498      561
   9:00:00     10:00:00  AS64496     AS64498     1899
   9:00:00     10:00:00  AS64496     AS64498     1284
   9:00:00     10:00:00  AS64499     AS64498     2670
   9:00:00     10:00:00  AS64496     AS64497       75
   9:00:00     10:00:00  AS64496     AS64498     3374
   9:00:00     10:00:00  AS64496     AS64498      138
   9:00:00     10:00:00  AS64496     AS64498     2325
   9:00:00     10:00:00  AS64499     AS64498    11200
   9:00:00     10:00:00  AS64496     AS64498     2869
   9:00:00     10:00:00  AS64496     AS64498    18289
        

Figure 21: Key Aggregation for Traffic Matrix: Reduction and Replacement

图21:流量矩阵的关键聚合:减少和替换

Finally, aggregate combination sums the counters per key and interval. The resulting Aggregated Flows containing the traffic matrix, shown in Figure 22, are then exported using the Template in Figure 18. Note that these Aggregated Flows represent a sparse matrix: AS pairs for which no traffic was received have no corresponding record in the output.

最后,聚合组合对每个键和间隔的计数器求和。然后使用图18中的模板导出包含流量矩阵的聚合流,如图22所示。请注意,这些聚合流表示一个稀疏矩阵:因为没有收到任何通信量的对在输出中没有相应的记录。

   start time  end time  source ASN  dest ASN  octets
   9:00:00     10:00:00  AS64496     AS64497      507
   9:00:00     10:00:00  AS64496     AS64498    86934
   9:00:00     10:00:00  AS64499     AS64498    17729
        
   start time  end time  source ASN  dest ASN  octets
   9:00:00     10:00:00  AS64496     AS64497      507
   9:00:00     10:00:00  AS64496     AS64498    86934
   9:00:00     10:00:00  AS64499     AS64498    17729
        

Figure 22: Aggregated Flows for Traffic Matrix

图22:流量矩阵的聚合流量

The output of this operation is suitable for re-aggregation: that is, traffic matrices from single links or Observation Points can be aggregated through the same interval imposition and aggregate combination steps in order to build a traffic matrix for an entire network.

此操作的输出适用于重新聚合:即,可以通过相同的间隔施加和聚合组合步骤聚合来自单个链路或观测点的流量矩阵,以便为整个网络构建流量矩阵。

8.3. Distinct Source Count per Destination Endpoint
8.3. 每个目标端点的不同源计数

Aggregating Flows by destination address and port, and counting distinct sources aggregated away, can be used as part of passive service inventory and host characterization. This example shows aggregation as an analysis technique, performed on source data stored in an IPFIX File. As the Transport Session in this File is bounded, removal of all timestamp information allows summarization of the entire time interval contained within the interval. Removal of timing information during interval imposition is equivalent to an infinitely long imposed time interval. This demonstrates both how infinite intervals work, and how unique counters work. The aggregation operations are summarized in Figure 23.

通过目标地址和端口聚合流,并计算聚合掉的不同源,可以用作被动服务清单和主机特性描述的一部分。此示例将聚合显示为一种分析技术,对存储在IPFIX文件中的源数据执行。由于此文件中的传输会话是有界的,因此删除所有时间戳信息可以汇总包含在时间间隔内的整个时间间隔。在间隔施加期间移除定时信息相当于无限长的施加时间间隔。这演示了无限间隔的工作方式以及唯一计数器的工作方式。图23总结了聚合操作。

                    Original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * discard timestamps |
              +-----------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +----------------------------+
   |  value aggregation         |
   |  * discard octetDeltaCount |
   +----------------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +----------------------------+
   |  key aggregation           |
   |   * reduce key to only     |
   |     destIPv4Address +      |
   |     destTransportPort,     |
   |   * count distinct sources |
   +----------------------------+
                  |
                  | Partially Aggregated Flows
                  V
       +----------------------------------------------+
       |  aggregate combination                       |
       |   * no-op (distinct sources already counted) |
       +----------------------------------------------+
                          |
                          V
                  Aggregated Flows
        
                    Original Flows
                          |
                          V
              +-----------------------+
              | interval distribution |
              |  * discard timestamps |
              +-----------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +----------------------------+
   |  value aggregation         |
   |  * discard octetDeltaCount |
   +----------------------------+
                  |
                  | Partially Aggregated Flows
                  V
   +----------------------------+
   |  key aggregation           |
   |   * reduce key to only     |
   |     destIPv4Address +      |
   |     destTransportPort,     |
   |   * count distinct sources |
   +----------------------------+
                  |
                  | Partially Aggregated Flows
                  V
       +----------------------------------------------+
       |  aggregate combination                       |
       |   * no-op (distinct sources already counted) |
       +----------------------------------------------+
                          |
                          V
                  Aggregated Flows
        

Figure 23: Aggregation Operations for Source Count

图23:源计数的聚合操作

The Template for Aggregated Flows produced by this example is shown in Figure 24.

此示例生成的聚合流模板如图24所示。

   destinationIPv4Address(12)[4]
   destinationTransportPort(11)[2]
   distinctCountOfSourceIPAddress(378)[8]
        
   destinationIPv4Address(12)[4]
   destinationTransportPort(11)[2]
   distinctCountOfSourceIPAddress(378)[8]
        

Figure 24: Output Template for Source Count

图24:源计数的输出模板

Interval distribution, in this case, merely discards the timestamp information from the Original Flows in Figure 10, and as such is not shown. Likewise, the value aggregation step simply discards the octetDeltaCount value field. The key aggregation step reduces the key to the destinationIPv4Address and destinationTransportPort, counting the distinct source addresses. Since this is essentially the output of this aggregation function, the aggregate combination operation is a no-op; the resulting Aggregated Flows are shown in Figure 25.

在本例中,间隔分布仅丢弃图10中原始流中的时间戳信息,因此未显示。类似地,值聚合步骤简单地丢弃octetDeltaCount值字段。密钥聚合步骤将密钥减少到destinationIPv4Address和destinationTransportPort,并计算不同的源地址。由于这本质上是该聚合函数的输出,因此聚合组合操作是无操作的;由此产生的聚合流如图25所示。

dest ip4 |port |dist src 192.0.2.131 53 3 198.51.100.2 80 1 198.51.100.2 443 3 198.51.100.67 80 2 198.51.100.68 80 2 198.51.100.133 80 2 198.51.100.3 80 3 198.51.100.4 80 2 198.51.100.17 80 1 198.51.100.69 443 1

目的地ip4 |港口区src 192.0.2.131 53 3 198.51.100.2 80 1 198.51.100.2 443 3 198.51.100.67 80 2 198.51.100.68 80 2 198.51.100.133 80 2 198.51.100.3 80 3 198.51.100.17 80 1 198.51.100.69 443 1

Figure 25: Aggregated Flows for Source Count

图25:源计数的聚合流

8.4. Traffic Time Series per Source with Counter Distribution
8.4. 具有计数器分布的每个源的流量时间序列

Returning to the example in Section 8.1, note that our source data contains some Flows with durations longer than the imposed interval of five minutes. The default method for dealing with such Flows is to account them to the interval containing the Flow's start time.

回到第8.1节中的示例,请注意,我们的源数据包含一些持续时间长于规定的5分钟间隔的流。处理此类流的默认方法是将它们计入包含流开始时间的间隔。

In this example, the same data is aggregated using the same arrangement of operations and the same output Template as in Section 8.1, but using a different counter distribution policy, Simple Uniform Distribution, as described in Section 5.1.1. In order to do this, the Exporting Process first exports the Aggregate Counter Distribution Options Template, as in Figure 26.

在本例中,使用与第8.1节中相同的操作安排和相同的输出模板,但使用不同的计数器分配策略,即第5.1.1节中所述的简单均匀分配,聚合相同的数据。为此,导出过程首先导出聚合计数器分发选项模板,如图26所示。

   templateId(12)[2]{scope}
   valueDistributionMethod(384)[1]
        
   templateId(12)[2]{scope}
   valueDistributionMethod(384)[1]
        

Figure 26: Aggregate Counter Distribution Options Template

图26:聚合计数器分发选项模板

This Template is followed by an Aggregate Counter Distribution Record described by this Template; assuming the output Template in Figure 11 has ID 257, this record would appear as in Figure 27.

此模板后面是此模板描述的汇总计数器分布记录;假设图11中的输出模板具有ID257,该记录将如图27所示。

template ID | value distribution method 257 4 (simple uniform)

模板ID |值分布方法257 4(简单均匀)

Figure 27: Aggregate Counter Distribution Record

图27:合计计数器分布记录

Following metadata export, the aggregation steps follow as before. However, two long Flows are distributed across multiple intervals in the interval imposition step, as indicated with "*" in Figure 28. Note the uneven distribution of the three-interval, 11200-octet Flow into three Partially Aggregated Flows of 3733, 3733, and 3734 octets; this ensures no cumulative error is injected by the interval distribution step.

元数据导出后,聚合步骤与前面一样。然而,如图28中的“*”所示,在间隔施加步骤中,两个长流分布在多个间隔上。注意三个区间的11200个八位元流不均匀分布为3733、3733和3734个八位元的三个部分聚合流;这确保了间隔分布步骤不会注入累积误差。

 start time |end time   |source ip4 |port |dest ip4      |port|pt|  oct
 9:00:00.000 9:05:00.000 192.0.2.2   47113 192.0.2.131    53   17   119
 9:00:00.000 9:05:00.000 192.0.2.2   22153 192.0.2.131    53   17    83
 9:00:00.000 9:05:00.000 192.0.2.2   52420 198.51.100.2   443  6   1637
 9:00:00.000 9:05:00.000 192.0.2.3   56047 192.0.2.131    53   17   111
 9:00:00.000 9:05:00.000 192.0.2.3   41183 198.51.100.67  80   6  16838
 9:00:00.000 9:05:00.000 192.0.2.2   17606 198.51.100.68  80   6  11538
 9:00:00.000 9:05:00.000 192.0.2.3   47113 192.0.2.131    53   17   119
 9:00:00.000 9:05:00.000 192.0.2.3   48458 198.51.100.133 80   6   2973
 9:00:00.000 9:05:00.000 192.0.2.4   61295 198.51.100.2   443  6   8350
 9:00:00.000 9:05:00.000 203.0.113.3 41256 198.51.100.133 80   6    778
 9:00:00.000 9:05:00.000 203.0.113.3 51662 198.51.100.3   80   6    883
 9:00:00.000 9:05:00.000 192.0.2.2   37581 198.51.100.2   80   6   7710*
 9:00:00.000 9:05:00.000 203.0.113.3 39586 198.51.100.17  80   6   3733*
 9:05:00.000 9:10:00.000 203.0.113.3 52572 198.51.100.2   443  6   1637
 9:05:00.000 9:10:00.000 203.0.113.3 49914 197.51.100.133 80   6    561
 9:05:00.000 9:10:00.000 192.0.2.2   50824 198.51.100.2   443  6   1899
 9:05:00.000 9:10:00.000 192.0.2.3   34597 198.51.100.3   80   6   1284
 9:05:00.000 9:10:00.000 203.0.113.3 58907 198.51.100.4   80   6   2670
 9:05:00.000 9:10:00.000 192.0.2.2   37581 198.51.100.2   80   6   7710*
 9:05:00.000 9:10:00.000 203.0.113.3 39586 198.51.100.17  80   6   3733*
 9:10:00.000 9:15:00.000 192.0.2.4   22478 192.0.2.131    53   17    75
 9:10:00.000 9:15:00.000 192.0.2.4   49513 198.51.100.68  80   6   3374
 9:10:00.000 9:15:00.000 192.0.2.4   64832 198.51.100.67  80   6    138
 9:10:00.000 9:15:00.000 192.0.2.3   60833 198.51.100.69  443  6   2325
 9:10:00.000 9:15:00.000 192.0.2.2   19638 198.51.100.3   80   6   2869
 9:10:00.000 9:15:00.000 192.0.2.3   40429 198.51.100.4   80   6  18289
 9:10:00.000 9:15:00.000 203.0.113.3 39586 198.51.100.17  80   6   3734*
        
 start time |end time   |source ip4 |port |dest ip4      |port|pt|  oct
 9:00:00.000 9:05:00.000 192.0.2.2   47113 192.0.2.131    53   17   119
 9:00:00.000 9:05:00.000 192.0.2.2   22153 192.0.2.131    53   17    83
 9:00:00.000 9:05:00.000 192.0.2.2   52420 198.51.100.2   443  6   1637
 9:00:00.000 9:05:00.000 192.0.2.3   56047 192.0.2.131    53   17   111
 9:00:00.000 9:05:00.000 192.0.2.3   41183 198.51.100.67  80   6  16838
 9:00:00.000 9:05:00.000 192.0.2.2   17606 198.51.100.68  80   6  11538
 9:00:00.000 9:05:00.000 192.0.2.3   47113 192.0.2.131    53   17   119
 9:00:00.000 9:05:00.000 192.0.2.3   48458 198.51.100.133 80   6   2973
 9:00:00.000 9:05:00.000 192.0.2.4   61295 198.51.100.2   443  6   8350
 9:00:00.000 9:05:00.000 203.0.113.3 41256 198.51.100.133 80   6    778
 9:00:00.000 9:05:00.000 203.0.113.3 51662 198.51.100.3   80   6    883
 9:00:00.000 9:05:00.000 192.0.2.2   37581 198.51.100.2   80   6   7710*
 9:00:00.000 9:05:00.000 203.0.113.3 39586 198.51.100.17  80   6   3733*
 9:05:00.000 9:10:00.000 203.0.113.3 52572 198.51.100.2   443  6   1637
 9:05:00.000 9:10:00.000 203.0.113.3 49914 197.51.100.133 80   6    561
 9:05:00.000 9:10:00.000 192.0.2.2   50824 198.51.100.2   443  6   1899
 9:05:00.000 9:10:00.000 192.0.2.3   34597 198.51.100.3   80   6   1284
 9:05:00.000 9:10:00.000 203.0.113.3 58907 198.51.100.4   80   6   2670
 9:05:00.000 9:10:00.000 192.0.2.2   37581 198.51.100.2   80   6   7710*
 9:05:00.000 9:10:00.000 203.0.113.3 39586 198.51.100.17  80   6   3733*
 9:10:00.000 9:15:00.000 192.0.2.4   22478 192.0.2.131    53   17    75
 9:10:00.000 9:15:00.000 192.0.2.4   49513 198.51.100.68  80   6   3374
 9:10:00.000 9:15:00.000 192.0.2.4   64832 198.51.100.67  80   6    138
 9:10:00.000 9:15:00.000 192.0.2.3   60833 198.51.100.69  443  6   2325
 9:10:00.000 9:15:00.000 192.0.2.2   19638 198.51.100.3   80   6   2869
 9:10:00.000 9:15:00.000 192.0.2.3   40429 198.51.100.4   80   6  18289
 9:10:00.000 9:15:00.000 203.0.113.3 39586 198.51.100.17  80   6   3734*
        

Figure 28: Distributed Interval Imposition for Time Series per Source

图28:每个源的时间序列的分布式间隔强制

Subsequent steps are as in Section 8.1; the results, to be exported using the Template shown in Figure 11, are shown in Figure 29, with Aggregated Flows differing from the example in Section 8.1 indicated by "*".

后续步骤如第8.1节所述;使用图11所示模板导出的结果如图29所示,聚合流与第8.1节中的示例不同,用“*”表示。

   start time |end time   |source ip4 |octets
   9:00:00.000 9:05:00.000 192.0.2.2    21087*
   9:00:00.000 9:05:00.000 192.0.2.3    20041
   9:00:00.000 9:05:00.000 192.0.2.4     8350
   9:00:00.000 9:05:00.000 203.0.113.3   5394*
   9:05:00.000 9:10:00.000 192.0.2.2     9609*
   9:05:00.000 9:10:00.000 192.0.2.3     1284
   9:05:00.000 9:10:00.000 203.0.113.3   8601*
   9:10:00.000 9:15:00.000 192.0.2.2     2869
   9:10:00.000 9:15:00.000 192.0.2.3    20614
   9:10:00.000 9:15:00.000 192.0.2.4     3587
   9:10:00.000 9:15:00.000 203.0.113.3   3734*
        
   start time |end time   |source ip4 |octets
   9:00:00.000 9:05:00.000 192.0.2.2    21087*
   9:00:00.000 9:05:00.000 192.0.2.3    20041
   9:00:00.000 9:05:00.000 192.0.2.4     8350
   9:00:00.000 9:05:00.000 203.0.113.3   5394*
   9:05:00.000 9:10:00.000 192.0.2.2     9609*
   9:05:00.000 9:10:00.000 192.0.2.3     1284
   9:05:00.000 9:10:00.000 203.0.113.3   8601*
   9:10:00.000 9:15:00.000 192.0.2.2     2869
   9:10:00.000 9:15:00.000 192.0.2.3    20614
   9:10:00.000 9:15:00.000 192.0.2.4     3587
   9:10:00.000 9:15:00.000 203.0.113.3   3734*
        

Figure 29: Aggregated Flows for Time Series per Source with Counter Distribution

图29:具有计数器分布的每个源的时间序列的聚合流

9. Security Considerations
9. 安全考虑

This document specifies the operation of an Intermediate Aggregation Process with the IPFIX protocol; the Security Considerations for the protocol itself in Section 11 of [RFC7011] therefore apply. In the common case that aggregation is performed on a Mediator, the Security Considerations for Mediators in Section 9 of [RFC6183] apply as well.

本文件规定了使用IPFIX协议的中间聚合过程的操作;因此,[RFC7011]第11节中关于协议本身的安全注意事项适用。在对中介执行聚合的常见情况下,[RFC6183]第9节中的中介安全注意事项也适用。

As mentioned in Section 3, certain aggregation operations may tend to have an anonymizing effect on Flow data by obliterating sensitive identifiers. Aggregation may also be combined with anonymization within a Mediator, or as part of a chain of Mediators, to further leverage this effect. In any case in which an Intermediate Aggregation Process is applied as part of a data anonymization or protection scheme, or is used together with anonymization as described in [RFC6235], the Security Considerations in Section 9 of [RFC6235] apply.

如第3节所述,某些聚合操作可能会通过删除敏感标识符对流数据产生匿名效应。聚合也可以与中介内的匿名化相结合,或者作为中介链的一部分,以进一步利用这种效果。如果中间聚合过程作为数据匿名或保护方案的一部分应用,或与[RFC6235]中所述的匿名一起使用,则[RFC6235]第9节中的安全注意事项适用。

10. IANA Considerations
10. IANA考虑

This document specifies the creation of new IPFIX Information Elements in the IPFIX Information Element registry [IANA-IPFIX], as defined in Section 7 above. IANA has assigned Information Element numbers to these Information Elements, and entered them into the registry.

本文件规定在IPFIX信息元素注册表[IANA-IPFIX]中创建新的IPFIX信息元素,如上文第7节所定义。IANA已将信息元素编号分配给这些信息元素,并将其输入注册表。

11. Acknowledgments
11. 致谢

Special thanks to Elisa Boschi for early work on the concepts laid out in this document. Thanks to Lothar Braun, Christian Henke, and Rahul Patel for their reviews and valuable feedback, with special

特别感谢Elisa Boschi在本文件中阐述的概念方面的早期工作。感谢Lothar Braun、Christian Henke和Rahul Patel的评论和宝贵反馈,特别是

thanks to Paul Aitken for his multiple detailed reviews. This work is materially supported by the European Union Seventh Framework Programme under grant agreement 257315 (DEMONS).

感谢Paul Aitken的多篇详细评论。这项工作得到了欧盟第七框架计划257315赠款协议(DEMONS)的实质性支持。

12. References
12. 工具书类
12.1. Normative References
12.1. 规范性引用文件

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2119]Bradner,S.,“RFC中用于表示需求水平的关键词”,BCP 14,RFC 2119,1997年3月。

[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.

[RFC5226]Narten,T.和H.Alvestrand,“在RFCs中编写IANA注意事项部分的指南”,BCP 26,RFC 5226,2008年5月。

[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information", STD 77, RFC 7011, September 2013.

[RFC7011]Claise,B.,Ed.,Trammell,B.,Ed.,和P.Aitken,“流量信息交换的IP流量信息导出(IPFIX)协议规范”,STD 77,RFC 7011,2013年9月。

12.2. Informative References
12.2. 资料性引用

[RFC3917] Quittek, J., Zseby, T., Claise, B., and S. Zander, "Requirements for IP Flow Information Export (IPFIX)", RFC 3917, October 2004.

[RFC3917]Quitek,J.,Zseby,T.,Claise,B.,和S.Zander,“IP流信息导出(IPFIX)的要求”,RFC 39172004年10月。

[RFC5470] Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, "Architecture for IP Flow Information Export", RFC 5470, March 2009.

[RFC5470]Sadasivan,G.,Brownlee,N.,Claise,B.,和J.Quitek,“IP流信息导出架构”,RFC 54702009年3月。

[RFC5472] Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP Flow Information Export (IPFIX) Applicability", RFC 5472, March 2009.

[RFC5472]Zseby,T.,Boschi,E.,Brownlee,N.,和B.Claise,“IP流信息导出(IPFIX)适用性”,RFC 54722009年3月。

[RFC5476] Claise, B., Johnson, A., and J. Quittek, "Packet Sampling (PSAMP) Protocol Specifications", RFC 5476, March 2009.

[RFC5476]Claise,B.,Johnson,A.,和J.Quittek,“数据包采样(PSAMP)协议规范”,RFC 54762009年3月。

[RFC5655] Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. Wagner, "Specification of the IP Flow Information Export (IPFIX) File Format", RFC 5655, October 2009.

[RFC5655]Trammell,B.,Boschi,E.,Mark,L.,Zseby,T.,和A.Wagner,“IP流信息导出(IPFIX)文件格式规范”,RFC 56552009年10月。

[RFC5982] Kobayashi, A. and B. Claise, "IP Flow Information Export (IPFIX) Mediation: Problem Statement", RFC 5982, August 2010.

[RFC5982]Kobayashi,A.和B.Claise,“IP流信息导出(IPFIX)调解:问题陈述”,RFC 59822010年8月。

[RFC6183] Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, "IP Flow Information Export (IPFIX) Mediation: Framework", RFC 6183, April 2011.

[RFC6183]Kobayashi,A.,Claise,B.,Muenz,G.,和K.Ishibashi,“IP流信息导出(IPFIX)中介:框架”,RFC 6183,2011年4月。

[RFC6235] Boschi, E. and B. Trammell, "IP Flow Anonymization Support", RFC 6235, May 2011.

[RFC6235]Boschi,E.和B.Trammell,“IP流匿名化支持”,RFC 62352011年5月。

[RFC6728] Muenz, G., Claise, B., and P. Aitken, "Configuration Data Model for the IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols", RFC 6728, October 2012.

[RFC6728]Muenz,G.,Claise,B.,和P.Aitken,“IP流信息导出(IPFIX)和数据包采样(PSAMP)协议的配置数据模型”,RFC 6728,2012年10月。

[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model for IP Flow Information Export (IPFIX)", RFC 7012, September 2013.

[RFC7012]Claise,B.,Ed.和B.Trammell,Ed.,“IP流信息导出(IPFIX)的信息模型”,RFC 7012,2013年9月。

[RFC7013] Trammell, B. and B. Claise, "Guidelines for Authors and Reviewers of IP Flow Information Export (IPFIX) Information Elements", BCP 184, RFC 7013, September 2013.

[RFC7013]Trammell,B.和B.Claise,“IP流信息导出(IPFIX)信息元素的作者和评审员指南”,BCP 184,RFC 7013,2013年9月。

[RFC7014] D'Antonio, S., Zseby, T., Henke, C., and L. Peluso, "Flow Selection Techniques", RFC 7014, September 2013.

[RFC7014]D'Antonio,S.,Zseby,T.,Henke,C.,和L.Peluso,“流量选择技术”,RFC 70142013年9月。

[IANA-IPFIX] IANA, "IP Flow Information Export (IPFIX) Entities", <http://www.iana.org/assignments/ipfix>.

[IANA-IPFIX]IANA,“IP流信息导出(IPFIX)实体”<http://www.iana.org/assignments/ipfix>.

[IPFIX-MED-PROTO] Claise, B., Kobayashi, A., and B. Trammell, "Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX Mediators", Work in Progress, July 2013.

[IPFIX-MED-PROTO]Claise,B.,Kobayashi,A.,和B.Trammell,“IPFIX中介上IP流信息导出(IPFIX)协议的操作”,正在进行的工作,2013年7月。

Authors' Addresses

作者地址

Brian Trammell Swiss Federal Institute of Technology Zurich Gloriastrasse 35 8092 Zurich Switzerland

Brian Trammell瑞士联邦理工学院苏黎世Gloriastrasse 35 8092瑞士苏黎世

   Phone: +41 44 632 70 13
   EMail: trammell@tik.ee.ethz.ch
        
   Phone: +41 44 632 70 13
   EMail: trammell@tik.ee.ethz.ch
        

Arno Wagner Consecom AG Bleicherweg 64a 8002 Zurich Switzerland

瑞士苏黎世阿诺·瓦格纳康塞姆股份有限公司Bleicherweg 64a 8002

   EMail: arno@wagner.name
        
   EMail: arno@wagner.name
        

Benoit Claise Cisco Systems, Inc. De Kleetlaan 6a b1 1831 Diegem Belgium

Benoit Claise Cisco Systems,Inc.De Kleetlaan 6a b1 1831 Diegem比利时

   Phone: +32 2 704 5622
   EMail: bclaise@cisco.com
        
   Phone: +32 2 704 5622
   EMail: bclaise@cisco.com