Internet Engineering Task Force (IETF)                    A. Morton, Ed.
Request for Comments: 5835                                     AT&T Labs
Category: Informational                           S. Van den Berghe, Ed.
ISSN: 2070-1721                                           Alcatel-Lucent
                                                              April 2010
        
Internet Engineering Task Force (IETF)                    A. Morton, Ed.
Request for Comments: 5835                                     AT&T Labs
Category: Informational                           S. Van den Berghe, Ed.
ISSN: 2070-1721                                           Alcatel-Lucent
                                                              April 2010
        

Framework for Metric Composition

度量组合框架

Abstract

摘要

This memo describes a detailed framework for composing and aggregating metrics (both in time and in space) originally defined by the IP Performance Metrics (IPPM), RFC 2330, and developed by the IETF. This new framework memo describes the generic composition and aggregation mechanisms. The memo provides a basis for additional documents that implement the framework to define detailed compositions and aggregations of metrics that are useful in practice.

本备忘录描述了最初由IP性能度量(IPPM)、RFC 2330定义并由IETF开发的用于组合和聚合度量(时间和空间)的详细框架。这个新的框架备忘录描述了通用的组合和聚合机制。该备忘录为实现该框架的其他文档提供了基础,以定义在实践中有用的度量的详细组成和聚合。

Status of This Memo

关于下段备忘

This document is not an Internet Standards Track specification; it is published for informational purposes.

本文件不是互联网标准跟踪规范;它是为了提供信息而发布的。

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 5741.

本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。并非IESG批准的所有文件都适用于任何级别的互联网标准;见RFC 5741第2节。

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc5835.

有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问http://www.rfc-editor.org/info/rfc5835.

Copyright Notice

版权公告

Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved.

版权所有(c)2010 IETF信托基金和确定为文件作者的人员。版权所有。

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(http://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。

This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English.

本文件可能包含2008年11月10日之前发布或公开的IETF文件或IETF贡献中的材料。控制某些材料版权的人员可能未授予IETF信托允许在IETF标准流程之外修改此类材料的权利。在未从控制此类材料版权的人员处获得充分许可的情况下,不得在IETF标准流程之外修改本文件,也不得在IETF标准流程之外创建其衍生作品,除了将其格式化以RFC形式发布或将其翻译成英语以外的其他语言。

Table of Contents

目录

   1. Introduction ....................................................4
      1.1. Motivation .................................................4
           1.1.1. Reducing Measurement Overhead .......................4
           1.1.2. Measurement Re-Use ..................................5
           1.1.3. Data Reduction and Consolidation ....................5
           1.1.4. Implications on Measurement Design and Reporting ....6
   2. Requirements Language ...........................................6
   3. Purpose and Scope ...............................................6
   4. Terminology .....................................................7
      4.1. Measurement Point ..........................................7
      4.2. Complete Path ..............................................7
      4.3. Complete Path Metric .......................................7
      4.4. Complete Time Interval .....................................7
      4.5. Composed Metric ............................................7
      4.6. Composition Function .......................................7
      4.7. Ground Truth ...............................................8
      4.8. Interval ...................................................8
      4.9. Sub-Interval ...............................................8
      4.10. Sub-Path ..................................................8
      4.11. Sub-Path Metrics ..........................................8
   5. Description of Metric Types .....................................9
      5.1. Temporal Aggregation Description ...........................9
      5.2. Spatial Aggregation Description ............................9
      5.3. Spatial Composition Description ...........................10
      5.4. Help Metrics ..............................................10
      5.5. Higher-Order Composition ..................................11
   6. Requirements for Composed Metrics ..............................11
      6.1. Note on Intellectual Property Rights (IPR) ................12
   7. Guidelines for Defining Composed Metrics .......................12
      7.1. Ground Truth: Comparison with Other IPPM Metrics ..........12
           7.1.1. Ground Truth for Temporal Aggregation ..............14
           7.1.2. Ground Truth for Spatial Aggregation ...............15
      7.2. Deviation from the Ground Truth ...........................15
      7.3. Incomplete Information ....................................15
      7.4. Time-Varying Metrics ......................................15
   8. Security Considerations ........................................16
   9. Acknowledgements ...............................................16
   10. References ....................................................16
      10.1. Normative References .....................................16
      10.2. Informative References ...................................17
        
   1. Introduction ....................................................4
      1.1. Motivation .................................................4
           1.1.1. Reducing Measurement Overhead .......................4
           1.1.2. Measurement Re-Use ..................................5
           1.1.3. Data Reduction and Consolidation ....................5
           1.1.4. Implications on Measurement Design and Reporting ....6
   2. Requirements Language ...........................................6
   3. Purpose and Scope ...............................................6
   4. Terminology .....................................................7
      4.1. Measurement Point ..........................................7
      4.2. Complete Path ..............................................7
      4.3. Complete Path Metric .......................................7
      4.4. Complete Time Interval .....................................7
      4.5. Composed Metric ............................................7
      4.6. Composition Function .......................................7
      4.7. Ground Truth ...............................................8
      4.8. Interval ...................................................8
      4.9. Sub-Interval ...............................................8
      4.10. Sub-Path ..................................................8
      4.11. Sub-Path Metrics ..........................................8
   5. Description of Metric Types .....................................9
      5.1. Temporal Aggregation Description ...........................9
      5.2. Spatial Aggregation Description ............................9
      5.3. Spatial Composition Description ...........................10
      5.4. Help Metrics ..............................................10
      5.5. Higher-Order Composition ..................................11
   6. Requirements for Composed Metrics ..............................11
      6.1. Note on Intellectual Property Rights (IPR) ................12
   7. Guidelines for Defining Composed Metrics .......................12
      7.1. Ground Truth: Comparison with Other IPPM Metrics ..........12
           7.1.1. Ground Truth for Temporal Aggregation ..............14
           7.1.2. Ground Truth for Spatial Aggregation ...............15
      7.2. Deviation from the Ground Truth ...........................15
      7.3. Incomplete Information ....................................15
      7.4. Time-Varying Metrics ......................................15
   8. Security Considerations ........................................16
   9. Acknowledgements ...............................................16
   10. References ....................................................16
      10.1. Normative References .....................................16
      10.2. Informative References ...................................17
        
1. Introduction
1. 介绍

The IP Performance Metrics (IPPM) framework [RFC2330] describes two forms of metric composition, spatial and temporal. The text also suggests that the concepts of the analytical framework (or A-frame) would help to develop useful relationships to derive the composed metrics from real metrics. The effectiveness of composed metrics is dependent on their usefulness in analysis and applicability to practical measurement circumstances.

IP性能度量(IPPM)框架[RFC2330]描述了度量组合的两种形式,即空间和时间。本文还建议,分析框架(或A框架)的概念将有助于发展有用的关系,以从实际度量中导出组合度量。组合度量的有效性取决于其在分析中的有用性以及对实际测量环境的适用性。

This memo expands on the notion of composition, and provides a detailed framework for several classes of metrics that were described in the original IPPM framework. The classes include temporal aggregation, spatial aggregation, and spatial composition.

本备忘录扩展了组合的概念,并为原始IPPM框架中描述的几类度量提供了详细的框架。这些类包括时间聚合、空间聚合和空间组合。

1.1. Motivation
1.1. 动机

Network operators have deployed measurement systems to serve many purposes, including performance monitoring, maintenance support, network engineering, and reporting performance to customers. The collection of elementary measurements alone is not enough to understand a network's behaviour. In general, measurements need to be post-processed to present the most relevant information for each purpose. The first step is often a process of "composition" of single measurements or measurement sets into other forms. Composition and aggregation present several more post-processing opportunities to the network operator, and we describe the key motivations below.

网络运营商已经部署了测量系统,用于多种用途,包括性能监控、维护支持、网络工程和向客户报告性能。仅收集基本测量值不足以理解网络的行为。一般来说,需要对测量值进行后处理,以便为每个目的提供最相关的信息。第一步通常是将单个测量或测量集“组合”成其他形式的过程。组合和聚合为网络运营商提供了更多的后处理机会,我们将在下面描述主要动机。

1.1.1. Reducing Measurement Overhead
1.1.1. 减少测量开销

A network's measurement possibilities scale upward with the square of the number of nodes. But each measurement implies overhead, in terms of the storage for the results, the traffic on the network (assuming active methods), and the operation and administration of the measurement system itself. In a large network, it is impossible to perform measurements from each node to all others.

网络的测量可能性随着节点数的平方而上升。但每次测量都意味着开销,包括结果存储、网络流量(假设采用主动方法)以及测量系统本身的操作和管理。在大型网络中,不可能执行从每个节点到所有其他节点的测量。

An individual network operator should be able to organize their measurement paths along the lines of physical topology, or routing areas/Autonomous Systems, and thus minimize dependencies and overlap between different measurement paths. This way, the sheer number of measurements can be reduced, as long as the operator has a set of methods to estimate performance between any particular pair of nodes when needed.

单个网络运营商应能够沿着物理拓扑或路由区域/自治系统的线路组织其测量路径,从而最小化不同测量路径之间的依赖性和重叠。这样,只要操作员在需要时有一组方法来估计任何特定节点对之间的性能,就可以减少测量的绝对数量。

Composition and aggregation play a key role when the path of interest spans multiple networks, and where each operator conducts their own measurements. Here, the complete path performance may be estimated from measurements on the component parts.

当感兴趣的路径跨越多个网络,并且每个运营商进行自己的测量时,组合和聚合起着关键作用。在这里,可以通过对组件的测量来估计完整的路径性能。

Operators that take advantage of the composition and aggregation methods recognize that the estimates may exhibit some additional error beyond that inherent in the measurements themselves, and so they are making a trade-off to achieve reasonable measurement system overhead.

利用组合和聚合方法的运营商认识到,除了测量本身固有的误差外,估计值可能会表现出一些额外的误差,因此他们正在进行权衡,以实现合理的测量系统开销。

1.1.2. Measurement Re-Use
1.1.2. 计量再利用

There are many different measurement users, each bringing specific requirements for the reporting timescale. Network managers and maintenance forces prefer to see results presented very rapidly, to detect problems quickly or see if their action has corrected a problem. On the other hand, network capacity planners and even network users sometimes prefer a long-term view of performance, for example to check trends. How can one set of measurements serve both needs?

有许多不同的测量用户,每个用户都对报告时间表提出了具体要求。网络管理人员和维护人员更喜欢快速查看结果、快速检测问题或查看他们的行动是否纠正了问题。另一方面,网络容量规划者甚至网络用户有时更喜欢从长远角度看性能,例如检查趋势。一套测量如何满足这两种需求?

The answer lies in temporal aggregation, where the short-term measurements needed by the operations community are combined to estimate a longer-term result for others. Also, problems with the measurement system itself may be isolated to one or more of the short-term measurements, rather than possibly invalidating an entire long-term measurement if the problem was undetected.

答案在于时间聚合,其中操作社区所需的短期度量被结合起来,以估计其他人的长期结果。此外,测量系统本身的问题可能被隔离到一个或多个短期测量中,而不是在未检测到问题的情况下使整个长期测量失效。

1.1.3. Data Reduction and Consolidation
1.1.3. 数据缩减和整合

Another motivation is data reduction. Assume there is a network in which delay measurements are performed among a subset of its nodes. A network manager might ask whether there is a problem with the network delay in general. It would be desirable to obtain a single value that gives an indication of the overall network delay. Spatial aggregation methods would address this need, and can produce the desired "single figure of merit" asked for, which may also be useful in trend analysis.

另一个动机是数据缩减。假设存在一个网络,其中在其节点子集之间执行延迟测量。网络管理员可能会问,一般来说,网络延迟是否存在问题。希望获得一个单一的值,该值给出总体网络延迟的指示。空间聚合方法将满足这一需求,并可产生所需的“单一优值”,这在趋势分析中也可能有用。

The overall value would be calculated from the elementary delay measurements, but it is not obvious how: for example, it may not be reasonable to average all delay measurements, as some paths (e.g., those having a higher bandwidth or more important customers) might be considered more critical than others.

总体值将根据基本延迟测量值进行计算,但如何计算并不明显:例如,对所有延迟测量值进行平均可能不合理,因为某些路径(例如,具有更高带宽或更重要客户的路径)可能被认为比其他路径更关键。

Metric composition can help to provide, from raw measurement data, some tangible, well-understood and agreed-upon information about the service guarantees provided by a network. Such information can be used in the Service Level Agreement/Service Level Specification (SLA/SLS) contracts between a service provider and its customers.

度量组合有助于从原始测量数据中提供有关网络提供的服务保证的一些有形、充分理解和一致同意的信息。此类信息可用于服务提供商与其客户之间的服务级别协议/服务级别规范(SLA/SLS)合同中。

1.1.4. Implications on Measurement Design and Reporting
1.1.4. 对计量设计和报告的影响

If a network measurement system operator anticipates needing to produce overall metrics by composition, then it is prudent to keep that requirement in mind when considering the measurement design and sampling plan. Also, certain summary statistics are more conducive to composition than others, and this figures prominently in the design of measurements and when reporting the results.

如果网络测量系统运营商预计需要按组成生成总体指标,则在考虑测量设计和采样计划时,应谨慎考虑该要求。此外,某些汇总统计数据比其他统计数据更有利于合成,这在测量设计和报告结果时尤为重要。

2. Requirements Language
2. 需求语言

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

本文件中的关键词“必须”、“不得”、“要求”、“应”、“不应”、“应”、“不应”、“建议”、“可”和“可选”应按照RFC 2119[RFC2119]中所述进行解释。

3. Purpose and Scope
3. 目的和范围

The purpose of this memo is to provide a common framework for the various classes of metrics that are composed from primary metrics. The scope is limited to the definitions of metrics that are composed from primary metrics using a deterministic function. Key information about each composed metric is included, such as the assumptions under which the relationship holds and possible sources of error/circumstances where the composition may fail.

本备忘录旨在为主要指标构成的各类指标提供通用框架。范围仅限于使用确定性函数由主要度量组成的度量定义。包括关于每个组合指标的关键信息,例如关系保持的假设以及组合可能失败的错误/情况的可能来源。

At this time, the scope of effort is limited to composed metrics for packet loss, delay, and delay variation, as defined in [RFC2679], [RFC2680], [RFC2681], [RFC3393], [RFC5481], and the comparable metrics in [Y.1540]. Composition of packet reordering metrics [RFC4737] and duplication metrics [RFC5560] are considered research topics at the time this memo was prepared, and are beyond the scope of this document.

此时,工作范围仅限于[RFC2679]、[RFC2680]、[RFC2681]、[RFC3393]、[RFC5481]中定义的分组丢失、延迟和延迟变化的组合度量以及[Y.1540]中的可比度量。在编制本备忘录时,分组重新排序度量[RFC4737]和复制度量[RFC5560]的组成被视为研究主题,不在本文档的范围之内。

This memo will retain the terminology of the IPPM Framework [RFC2330] as much as possible, but will extend the terminology when necessary. It is assumed that the reader is familiar with the concepts introduced in [RFC2330], as they will not be repeated here.

本备忘录将尽可能保留IPPM框架[RFC2330]的术语,但必要时将扩展术语。假定读者熟悉[RFC2330]中介绍的概念,因为此处不再重复。

4. Terminology
4. 术语

This section defines the terminology applicable to the processes of metric composition and aggregation.

本节定义了适用于度量组合和聚合过程的术语。

4.1. Measurement Point
4.1. 测量点

A measurement point is the logical or physical location where packet observations are made. The term "measurement point" is synonymous with the term "observation position" used in [RFC2330] when describing the notion of wire time. A measurement point may be at the boundary between a host and an adjacent link (physical), or it may be within a host (logical) that performs measurements where the difference between host time and wire time is understood.

测量点是进行数据包观察的逻辑或物理位置。术语“测量点”与[RFC2330]中描述导线时间概念时使用的术语“观测位置”同义。测量点可能位于主机和相邻链路(物理链路)之间的边界,也可能位于执行测量的主机(逻辑链路)内,其中主机时间和连线时间之间的差异是可以理解的。

4.2. Complete Path
4.2. 完整路径

The complete path is the actual path that a packet would follow as it travels from the packet's Source to its Destination. A complete path may span the administrative boundaries of one or more networks.

完整路径是数据包从数据包源到目的地的实际路径。完整路径可以跨越一个或多个网络的管理边界。

4.3. Complete Path Metric
4.3. 完全路径度量

The complete path metric is the Source-to-Destination metric that a composed metric attempts to estimate. A complete path metric represents the ground-truth for a composed metric.

完整路径度量是合成度量尝试估计的源到目标度量。完整路径度量表示合成度量的基本真值。

4.4. Complete Time Interval
4.4. 完整时间间隔

The complete time interval is comprised of two or more contiguous sub-intervals, and is the interval whose performance will be estimated through temporal aggregation.

完整时间间隔由两个或多个连续的子间隔组成,是通过时间聚合来估计其性能的间隔。

4.5. Composed Metric
4.5. 合成度量

A composed metric is an estimate of an actual metric describing the performance of a path over some time interval. A composed metric is derived from other metrics by applying a deterministic process or function (e.g., a composition function). The process may use metrics that are identical to the metric being composed, or metrics that are dissimilar, or some combination of both types.

合成度量是对描述某个时间间隔内路径性能的实际度量的估计。组合度量是通过应用确定性过程或函数(例如,组合函数)从其他度量派生而来的。该过程可以使用与所组成的度量相同的度量,或者使用不同的度量,或者使用这两种类型的某种组合。

4.6. Composition Function
4.6. 合成函数

A composition function is a deterministic process applied to individual metrics to derive another metric (such as a composed metric).

组合函数是应用于单个度量以导出另一个度量(如组合度量)的确定性过程。

4.7. Ground Truth
4.7. 基本事实

As applied here, the notion of "ground truth" is defined as the actual performance of a network path over some time interval. The ground truth is a metric on the (unavailable) packet transfer information for the desired path and time interval that a composed metric seeks to estimate.

正如这里所应用的,“地面真相”的概念被定义为网络路径在某个时间间隔内的实际性能。地面真值是合成度量寻求估计的所需路径和时间间隔的(不可用)分组传输信息的度量。

4.8. Interval
4.8. 间隔

An interval refers to a span of time.

间隔指的是一段时间。

4.9. Sub-Interval
4.9. 子区间

A sub-interval is a time interval that is included in another interval.

子间隔是包含在另一个间隔中的时间间隔。

4.10. Sub-Path
4.10. 子路径

A sub-path is a portion of the complete path where at least the sub-path Source and Destination hosts are constituents of the complete path. We say that such a sub-path is "involved" in the complete path.

子路径是完整路径的一部分,其中至少子路径源主机和目标主机是完整路径的组成部分。我们说这样的子路径“涉及”完整路径。

Since sub-paths terminate on hosts, it is important to describe how sub-paths are considered to be joined. In practice, the Source and Destination hosts may perform the function of measurement points.

由于子路径在主机上终止,因此描述如何将子路径视为连接非常重要。实际上,源主机和目的主机可以执行测量点的功能。

If the Destination and Source hosts of two adjoining paths are co-located and the link between them would contribute negligible performance, then these hosts can be considered equivalent (even if there is no physical link between them, this is a practical concession).

如果两个相邻路径的目标主机和源主机位于同一位置,并且它们之间的链路对性能的影响可以忽略不计,则可以认为这些主机是等效的(即使它们之间没有物理链路,这也是一种实际的让步)。

If the Destination and Source hosts of two adjoining paths have a link between them that contributes to the complete path performance, then the link and hosts constitute another sub-path that is involved in the complete path, and should be characterized and included in the composed metric.

如果两个相邻路径的目标主机和源主机之间有一个有助于完整路径性能的链路,则该链路和主机构成了完整路径中涉及的另一个子路径,并且应该在组合度量中进行特征化和包括。

4.11. Sub-Path Metrics
4.11. 子路径度量

A sub-path path metric is an element of the process to derive a composed metric, quantifying some aspect of the performance of a particular sub-path from its Source to Destination.

子路径度量是导出组合度量的过程的一个元素,量化从源到目标的特定子路径性能的某些方面。

5. Description of Metric Types
5. 公制类型说明

This section defines the various classes of composition. There are two classes more accurately described as aggregation over time and space, and the third involves concatenation in space.

本节定义了各种类型的组合。有两类更准确地描述为时间和空间上的聚合,第三类涉及空间上的串联。

5.1. Temporal Aggregation Description
5.1. 时间聚合描述

Aggregation in time is defined as the composition of metrics with the same type and scope obtained in different time instants or time windows. For example, starting from a time series of the measurements of maximum and minimum one-way delay (OWD) on a certain network path obtained over 5-minute intervals, we obtain a time series measurement with a coarser resolution (60 minutes) by taking the maximum of 12 consecutive 5-minute maxima and the minimum of 12 consecutive 5-minute minima.

时间聚合定义为在不同的时间瞬间或时间窗口中获得的具有相同类型和范围的度量的组合。例如,从在5分钟间隔内获得的特定网络路径上的最大和最小单向延迟(OWD)测量的时间序列开始,我们通过取12个连续5分钟的最大值和12个连续5分钟的最小值,获得具有更粗分辨率(60分钟)的时间序列测量。

The main reason for doing time aggregation is to reduce the amount of data that has to be stored, and make the visualization/spotting of regular cycles and/or growing or decreasing trends easier. Another useful application is to detect anomalies or abnormal changes in the network characteristics.

进行时间聚合的主要原因是减少必须存储的数据量,并使常规周期和/或增长或下降趋势的可视化/定位更容易。另一个有用的应用是检测网络特性中的异常或异常变化。

In RFC 2330, the term "temporal composition" is introduced and differs from temporal aggregation in that it refers to methodologies to predict future metrics on the basis of past observations (of the same metrics), exploiting the time correlation that certain metrics can exhibit. We do not consider this type of composition here.

在RFC 2330中,引入了术语“时间组合”,与时间聚合的不同之处在于,它指的是基于过去的观察(相同度量的)预测未来度量的方法,利用某些度量可以显示的时间相关性。在这里我们不考虑这种类型的构图。

5.2. Spatial Aggregation Description
5.2. 空间聚合描述

Aggregation in space is defined as the combination of metrics of the same type and different scope, in order to estimate the overall performance of a larger network. This combination may involve weighing the contributions of the input metrics.

空间聚合定义为相同类型和不同范围的度量的组合,以估计更大网络的总体性能。这种组合可能涉及权衡输入指标的贡献。

Suppose we want to compose the average one-way delay (OWD) experienced by flows traversing all the origin-destination (OD) pairs of a network (where the inputs are already metric "statistics"). Since we wish to include the effect of the traffic matrix on the result, it makes sense to weight each metric according to the traffic carried on the corresponding OD pair:

假设我们想要合成通过网络的所有起始-目的地(OD)对的流所经历的平均单向延迟(OWD)(其中输入已经是度量“统计”)。由于我们希望包括流量矩阵对结果的影响,因此根据相应OD对上承载的流量对每个度量进行加权是有意义的:

   OWD_sum=f1*OWD_1+f2*OWD_2+...+fn*OWD_n
        
   OWD_sum=f1*OWD_1+f2*OWD_2+...+fn*OWD_n
        

where fi=load_OD_i/total_load.

式中,fi=荷载/总荷载。

A simple average OWD across all network OD pairs would not use the traffic weighting.

所有网络OD对的简单平均OWD不会使用流量权重。

Another example metric that is "aggregated in space" is the maximum edge-to-edge delay across a single network. Assume that a Service Provider wants to advertise the maximum delay that transit traffic will experience while passing through his/her network. There can be multiple edge-to-edge paths across a network, and the Service Provider chooses either to publish a list of delays (each corresponding to a specific edge-to-edge path), or publish a single maximum value. The latter approach simplifies the publication of measurement information, and may be sufficient for some purposes. Similar operations can be provided to other metrics, e.g., "maximum edge-to-edge packet loss", etc.

“空间聚合”的另一个示例度量是单个网络中的最大边到边延迟。假设服务提供商希望公布过境流量通过其网络时将经历的最大延迟。一个网络中可以有多个边到边路径,服务提供商可以选择发布延迟列表(每个延迟对应于特定的边到边路径),也可以发布单个最大值。后一种方法简化了测量信息的发布,对于某些目的可能已经足够。类似的操作可以提供给其他度量,例如,“最大边到边分组丢失”等。

We suggest that space aggregation is generally useful to obtain a summary view of the behaviour of large network portions, or of coarser aggregates in general. The metric collection time instant, i.e., the metric collection time window of measured metrics, is not considered in space aggregation. We assume that either it is consistent for all the composed metrics, e.g., compose a set of average delays all referring to the same time window, or the time window of each composed metric does not affect the aggregated metric.

我们认为,空间聚合通常有助于获得大网络部分或更粗聚合行为的摘要视图。度量收集时间瞬间,即度量度量的度量收集时间窗口,在空间聚合中不被考虑。我们假设它对于所有组合度量都是一致的,例如,组合一组平均延迟,所有延迟都引用相同的时间窗口,或者每个组合度量的时间窗口不影响聚合度量。

5.3. Spatial Composition Description
5.3. 空间构成描述

Concatenation in space is defined as the composition of metrics of same type with (ideally) different spatial scope, so that the resulting metric is representative of what the metric would be if obtained with a direct measurement over the sequence of the several spatial scopes. An example is the sum of mean OWDs of adjacent edge-to-edge networks, where the intermediate edge points are close to each other or happen to be the same. In this way, we can for example estimate OWD_AC starting from the knowledge of OWD_AB and OWD_BC. Note that there may be small gaps in measurement coverage; likewise, there may be small overlaps (e.g., the link where test equipment connects to the network).

空间中的串联被定义为具有(理想情况下)不同空间范围的相同类型的度量的组合,因此产生的度量代表了如果在多个空间范围的序列上通过直接度量获得的度量。例如,相邻边到边网络的平均OWD之和,其中中间边点彼此接近或恰好相同。通过这种方式,我们可以从OWD_AB和OWD_BC的知识开始估算OWD_AC。注意,测量覆盖范围可能存在小差距;同样,可能存在小重叠(例如,测试设备连接到网络的链路)。

One key difference from examples of aggregation in space is that all sub-paths contribute equally to the composed metric, independent of the traffic load present.

与空间聚合示例的一个关键区别是,所有子路径对合成度量的贡献相等,与存在的流量负载无关。

5.4. Help Metrics
5.4. 帮助度量

In practice, there is often the need to compute a new metric using one or more metrics with the same spatial and time scope. For example, the metric rtt_sample_variance may be computed from two different metrics: the help metrics rtt_square_sum and the rtt_sum.

在实践中,通常需要使用具有相同空间和时间范围的一个或多个度量来计算新度量。例如,度量rtt_sample_方差可以从两个不同的度量计算:帮助度量rtt_square_sum和rtt_sum。

The process of using help metrics is a simple calculation and not an aggregation or a concatenation, and will not be investigated further in this memo.

使用帮助度量的过程是一个简单的计算,而不是聚合或串联,本备忘录将不再进一步研究。

5.5. Higher-Order Composition
5.5. 高阶合成

Composed metrics might themselves be subject to further steps of composition or aggregation. An example would be the delay of a maximal path obtained through the spatial composition of several composed delays for each complete path in the maximal path (obtained through spatial composition). All requirements for first-order composition metrics apply to higher-order composition.

组合度量本身可能要经过进一步的组合或聚合步骤。例如,通过最大路径中每个完整路径(通过空间合成获得)的多个合成延迟的空间合成获得的最大路径的延迟。一阶合成度量的所有要求都适用于高阶合成。

An example using temporal aggregation: twelve 5-minute metrics are aggregated to estimate the performance over an hour. The second step of aggregation would take 24 hourly metrics and estimate the performance over a day.

使用时间聚合的示例:聚合12个5分钟指标以估计一小时内的性能。聚合的第二步需要24小时的度量,并估计一天内的性能。

6. Requirements for Composed Metrics
6. 组合度量的需求

The definitions for all composed metrics MUST include sections to treat the following topics.

所有组合指标的定义必须包括处理以下主题的部分。

The description of each metric will clearly state:

每个指标的说明将明确说明:

1. the definition (and statistic, where appropriate);

1. 定义(和统计,如适用);

2. the composition or aggregation relationship;

2. 组成或聚集关系;

3. the specific conjecture on which the relationship is based and assumptions of the statistical model of the process being measured, if any (see [RFC2330], Section 12);

3. 关系所基于的具体推测和被测量过程的统计模型假设(如有)(见[RFC2330],第12节);

4. a justification of practical utility or usefulness for analysis using the A-frame concepts;

4. 使用a框架概念进行分析的实用性或有用性的证明;

5. one or more examples of how the conjecture could be incorrect and lead to inaccuracy;

5. 一个或多个示例,说明猜测可能不正确并导致不准确;

6. the information to be reported.

6. 要报告的信息。

For each metric, the applicable circumstances will be defined, in terms of whether the composition or aggregation:

对于每个指标,将根据构成或合计来定义适用情况:

o Requires homogeneity of measurement methodologies, or can allow a degree of flexibility (e.g., active or passive methods produce the "same" metric). Also, the applicable sending streams will be specified, such as Poisson, Periodic, or both.

o 要求测量方法的同质性,或允许一定程度的灵活性(例如,主动或被动方法产生“相同”的度量)。此外,将指定适用的发送流,例如泊松流、周期流或两者。

o Needs information or access that will only be available within an operator's network, or is applicable to inter-network composition.

o 需要仅在运营商网络内可用或适用于网络间组合的信息或访问。

o Requires precisely synchronized measurement time intervals in all component metrics, or perhaps only loosely synchronized time intervals, or has no timing requirements at all.

o 在所有组件度量中需要精确同步的测量时间间隔,或者可能只是松散同步的时间间隔,或者根本没有时间要求。

o Requires assumption of component metric independence with regard to the metric being defined/composed, or other assumptions.

o 要求假设组件度量独立于定义/组成的度量或其他假设。

o Has known sources of inaccuracy/error and identifies the sources.

o 已知不准确/错误的来源,并确定来源。

6.1. Note on Intellectual Property Rights (IPR)
6.1. 关于知识产权的说明

If one or more components of the composition process are encumbered by Intellectual Property Rights (IPR), then the resulting composed metrics may be encumbered as well. See BCP 79 [RFC3979] for IETF policies on IPR disclosure.

如果合成过程的一个或多个组成部分受到知识产权(IPR)的阻碍,那么产生的合成度量也可能受到阻碍。关于知识产权披露的IETF政策,请参见BCP 79[RFC3979]。

7. Guidelines for Defining Composed Metrics
7. 定义组合度量的指南
7.1. Ground Truth: Comparison with Other IPPM Metrics
7.1. 基本事实:与其他IPPM指标的比较

Figure 1 illustrates the process to derive a metric using spatial composition, and compares the composed metric to other IPPM metrics.

图1说明了使用空间组合导出度量的过程,并将组合的度量与其他IPPM度量进行了比较。

Metrics <M1, M2, M3> describe the performance of sub-paths between the Source and Destination of interest during time interval <T, Tf>. These metrics are the inputs for a composition function that produces a composed metric.

度量<M1,M2,M3>描述了在时间间隔<T,Tf>期间感兴趣的源和目标之间的子路径的性能。这些度量是生成合成度量的合成函数的输入。

                          Sub-Path Metrics
                 ++  M1   ++ ++  M2   ++ ++  M3   ++
             Src ||.......|| ||.......|| ||.......|| Dst
                 ++   `.  ++ ++   |   ++ ++  .'   ++
                        `.        |       .-'
                          `-.     |     .'
                             `._..|.._.'
                           ,-'         `-.
                         ,'               `.
                         |   Composition   |
                         \     Function    '
                          `._           _,'
                             `--.....--'
                                  |
                 ++               |               ++
             Src ||...............................|| Dst
                 ++        Composed Metric        ++
        
                          Sub-Path Metrics
                 ++  M1   ++ ++  M2   ++ ++  M3   ++
             Src ||.......|| ||.......|| ||.......|| Dst
                 ++   `.  ++ ++   |   ++ ++  .'   ++
                        `.        |       .-'
                          `-.     |     .'
                             `._..|.._.'
                           ,-'         `-.
                         ,'               `.
                         |   Composition   |
                         \     Function    '
                          `._           _,'
                             `--.....--'
                                  |
                 ++               |               ++
             Src ||...............................|| Dst
                 ++        Composed Metric        ++
        
                 ++      Complete Path Metric     ++
             Src ||...............................|| Dst
                 ++                               ++
                           Spatial Metric
                 ++   S1   ++   S2    ++    S3    ++
             Src ||........||.........||..........|| Dst
                 ++        ++         ++          ++
        
                 ++      Complete Path Metric     ++
             Src ||...............................|| Dst
                 ++                               ++
                           Spatial Metric
                 ++   S1   ++   S2    ++    S3    ++
             Src ||........||.........||..........|| Dst
                 ++        ++         ++          ++
        

Figure 1: Comparison with Other IPPM Metrics

图1:与其他IPPM指标的比较

The composed metric is an estimate of an actual metric collected over the complete Source-to-Destination path. We say that the complete path metric represents the ground truth for the composed metric. In other words, composed metrics seek to minimize error with regard to the complete path metric.

合成度量是在完整的源到目标路径上收集的实际度量的估计。我们说完整路径度量表示合成度量的基本真理。换句话说,组合度量寻求最小化关于完整路径度量的错误。

Further, we observe that a spatial metric [RFC5644] collected for packets traveling over the same set of sub-paths provides a basis for the ground truth of the individual sub-path metrics. We note that mathematical operations may be necessary to isolate the performance of each sub-path.

此外,我们观察到,为在同一组子路径上传输的数据包收集的空间度量[RFC5644]为各个子路径度量的基本事实提供了基础。我们注意到,可能需要进行数学运算来隔离每个子路径的性能。

Next, we consider multiparty metrics (as defined in [RFC5644]) and their spatial composition. Measurements to each of the receivers produce an element of the one-to-group metric. These elements can be composed from sub-path metrics, and the composed metrics can be combined to create a composed one-to-group metric. Figure 2 illustrates this process.

接下来,我们考虑多方度量(如[RCF564]中定义的)及其空间组成。对每个接收机的测量产生一对一度量的元素。这些元素可以由子路径度量组成,并且可以组合这些组合的度量来创建一个组合到组的度量。图2说明了这个过程。

                             Sub-Path Metrics
                    ++  M1   ++ ++  M2   ++ ++  M3   ++
                Src ||.......|| ||.......|| ||.......||Rcvr1
                    ++       ++ ++`.     ++ ++       ++
                                    `-.
                                     M4`.++ ++  M5   ++
                                         || ||.......||Rcvr2
                                         ++ ++`.     ++
                                                `-.
                                                 M6`.++
                                                     ||Rcvr3
                                                     ++
        
                             Sub-Path Metrics
                    ++  M1   ++ ++  M2   ++ ++  M3   ++
                Src ||.......|| ||.......|| ||.......||Rcvr1
                    ++       ++ ++`.     ++ ++       ++
                                    `-.
                                     M4`.++ ++  M5   ++
                                         || ||.......||Rcvr2
                                         ++ ++`.     ++
                                                `-.
                                                 M6`.++
                                                     ||Rcvr3
                                                     ++
        
                            One-to-Group Metric
                    ++        ++         ++          ++
                Src ||........||.........||..........||Rcvr1
                    ++        ++.        ++          ++
                                 `-.
                                    `-.  ++          ++
                                       `-||..........||Rcvr2
                                         ++.         ++
                                            `-.
                                               `-.   ++
                                                  `-.||Rcvr3
                                                     ++
        
                            One-to-Group Metric
                    ++        ++         ++          ++
                Src ||........||.........||..........||Rcvr1
                    ++        ++.        ++          ++
                                 `-.
                                    `-.  ++          ++
                                       `-||..........||Rcvr2
                                         ++.         ++
                                            `-.
                                               `-.   ++
                                                  `-.||Rcvr3
                                                     ++
        

Figure 2: Composition of One-to-Group Metrics

图2:一对一组指标的组成

Here, sub-path metrics M1, M2, and M3 are combined using a relationship to compose the metric applicable to the Src-Rcvr1 path. Similarly, M1, M4, and M5 are used to compose the Src-Rcvr2 metric and M1, M4, and M6 compose the Src-Rcvr3 metric.

这里,使用关系组合子路径度量M1、M2和M3,以构成适用于Src-Rcvr1路径的度量。类似地,M1、M4和M5用于构成Src-Rcvr2度量,M1、M4和M6用于构成Src-Rcvr3度量。

The composed one-to-group metric would list the Src-Rcvr metrics for each receiver in the group:

组合的一到组度量将列出组中每个接收器的Src Rcvr度量:

(Composed-Rcvr1, Composed-Rcvr2, Composed-Rcvr3)

(组合式Rcvr1、组合式Rcvr2、组合式Rcvr3)

The ground truth for this composed metric is of course an actual one-to-group metric, where a single Source packet has been measured after traversing the complete paths to the various receivers.

这个合成度量的基本事实当然是实际的一对一度量,其中在遍历到各个接收器的完整路径之后测量了单个源数据包。

7.1.1. Ground Truth for Temporal Aggregation
7.1.1. 时间聚合的基本真理

Temporal aggregation involves measurements made over sub-intervals of the complete time interval between the same Source and Destination. Therefore, the ground truth is the metric measured over the desired interval.

时间聚合涉及在相同源和目标之间的完整时间间隔的子间隔上进行的测量。因此,基本真实值是在所需间隔内测量的度量。

7.1.2. Ground Truth for Spatial Aggregation
7.1.2. 空间聚集的基本真理

Spatial aggregation combines many measurements using a weighting function to provide the same emphasis as though the measurements were based on actual traffic, with inherent weights. Therefore, the ground truth is the metric measured on the actual traffic instead of the active streams that sample the performance.

空间聚合使用加权函数将许多度量组合在一起,以提供相同的强调,就好像这些度量基于具有固有权重的实际流量一样。因此,基本事实是在实际流量上测量的度量,而不是对性能进行采样的活动流。

7.2. Deviation from the Ground Truth
7.2. 偏离事实真相

A metric composition can deviate from the ground truth for several reasons. Two main aspects are:

一个度量组合可能由于几个原因而偏离基本事实。两个主要方面是:

o The propagation of the inaccuracies of the underlying measurements when composing the metric. As part of the composition function, errors of measurements might propagate. Where possible, this analysis should be made and included with the description of each metric.

o 构成度量时,基础测量不准确度的传播。作为合成函数的一部分,测量误差可能会传播。在可能的情况下,应进行分析,并将其包括在每个指标的描述中。

o A difference in scope. When concatenating many active measurement results (from two or more sub-paths) to obtain the complete path metric, the actual measured path will not be identical to the complete path. It is in general difficult to quantify this deviation with exactness, but a metric definition might identify guidelines for keeping the deviation as small as possible.

o 范围上的差异。当连接多个活动测量结果(来自两个或多个子路径)以获得完整路径度量时,实际测量的路径将与完整路径不同。通常很难准确地量化这种偏差,但度量定义可能会确定使偏差尽可能小的准则。

The description of the metric composition MUST include a section identifying the deviation from the ground truth.

度量组合的描述必须包括一个部分,确定与地面真实值的偏差。

7.3. Incomplete Information
7.3. 不完全信息

In practice, when measurements cannot be initiated on a sub-path or during a particular measurement interval (and perhaps the measurement system gives up during the test interval), then there will not be a value for the sub-path reported, and the result SHOULD be recorded as "undefined".

实际上,如果无法在子路径上或在特定测量间隔期间启动测量(并且可能在测试间隔期间测量系统放弃),则不会报告子路径的值,并且应将结果记录为“未定义”。

7.4. Time-Varying Metrics
7.4. 时变度量

The measured values of many metrics depend on time-variant factors, such as the level of network traffic on the Source-to-Destination path. Traffic levels often exhibit diurnal (or daily) variation, but a 24-hour measurement interval would obscure it. Temporal aggregation of hourly results to estimate the daily metric would have the same effect, and so the same cautions are warranted.

许多度量的测量值取决于时变因素,例如源到目标路径上的网络流量级别。交通水平通常表现出日变化(或日变化),但24小时的测量间隔将使其变得模糊。对每小时结果进行时间汇总以估计每日指标也会产生同样的效果,因此需要采取同样的注意事项。

Some metrics are predominantly* time-invariant, such as the actual minimum one-way delay of a fixed path, and therefore temporal aggregation does not obscure the results as long as the path is stable. However, paths do vary, and sometimes on less predictable time intervals than traffic variations. (* Note: It is recognized that propagation delay on transmission facilities may have diurnal, seasonal, and even longer-term variations.)

一些度量主要是*时不变的,例如固定路径的实际最小单向延迟,因此,只要路径稳定,时间聚集就不会掩盖结果。然而,路径确实是不同的,有时比流量变化的时间间隔更不可预测。(*注:传输设施上的传播延迟可能具有日变化、季节变化甚至长期变化。)

8. Security Considerations
8. 安全考虑

The security considerations that apply to any active measurement of live networks are relevant here as well. See [RFC4656] and [RFC5357].

适用于实时网络的任何主动测量的安全注意事项也与此相关。参见[RFC4656]和[RFC5357]。

The exchange of sub-path measurements among network providers may be a source of concern, and the information should be sufficiently anonymized to avoid revealing unnecessary operational details (e.g., the network addresses of measurement devices). However, the schema for measurement exchange is beyond the scope of this memo and likely to be covered by bilateral agreements for some time to come.

网络提供商之间的子路径测量的交换可能是一个关注的来源,并且信息应充分匿名,以避免透露不必要的操作细节(例如,测量设备的网络地址)。然而,计量交换方案超出了本备忘录的范围,在未来一段时间内可能会被双边协议涵盖。

9. Acknowledgements
9. 致谢

The authors would like to thank Maurizio Molina, Andy Van Maele, Andreas Haneman, Igor Velimirovic, Andreas Solberg, Athanassios Liakopulos, David Schitz, Nicolas Simar, and the Geant2 Project. We also acknowledge comments and suggestions from Phil Chimento, Emile Stephan, Lei Liang, Stephen Wolff, Reza Fardid, Loki Jorgenson, and Alan Clark.

作者要感谢毛里齐奥·莫利纳、安迪·范梅勒、安德烈亚斯·哈尼曼、伊戈尔·维利米罗维奇、安德烈亚斯·索尔伯格、阿萨纳西奥斯·利亚科普洛斯、大卫·希茨、尼古拉斯·西马尔和Geant2项目。我们还感谢Phil Chimento、Emile Stephan、Lei Liang、Stephen Wolff、Reza Fardid、Loki Jorgenson和Alan Clark的评论和建议。

10. References
10. 工具书类
10.1. Normative References
10.1. 规范性引用文件

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2119]Bradner,S.,“RFC中用于表示需求水平的关键词”,BCP 14,RFC 2119,1997年3月。

[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998.

[RFC2330]Paxson,V.,Almes,G.,Mahdavi,J.,和M.Mathis,“IP性能度量框架”,RFC 2330,1998年5月。

[RFC3979] Bradner, S., Ed., "Intellectual Property Rights in IETF Technology", BCP 79, RFC 3979, March 2005.

[RFC3979]Bradner,S.,Ed.,“IETF技术中的知识产权”,BCP 79,RFC 3979,2005年3月。

[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, September 2006.

[RFC4656]Shalunov,S.,Teitelbaum,B.,Karp,A.,Boote,J.,和M.Zekauskas,“单向主动测量协议(OWAMP)”,RFC 46562006年9月。

[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, October 2008.

[RFC5357]Hedayat,K.,Krzanowski,R.,Morton,A.,Yum,K.,和J.Babiarz,“双向主动测量协议(TWAMP)”,RFC 5357,2008年10月。

10.2. Informative References
10.2. 资料性引用

[RFC2679] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, September 1999.

[RFC2679]Almes,G.,Kalidini,S.,和M.Zekauskas,“IPPM的单向延迟度量”,RFC 2679,1999年9月。

[RFC2680] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, September 1999.

[RFC2680]Almes,G.,Kalidini,S.,和M.Zekauskas,“IPPM的单向数据包丢失度量”,RFC 2680,1999年9月。

[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, September 1999.

[RFC2681]Almes,G.,Kalidini,S.,和M.Zekauskas,“IPPM的往返延迟度量”,RFC 2681,1999年9月。

[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002.

[RFC3393]Demichelis,C.和P.Chimento,“IP性能度量的IP数据包延迟变化度量(IPPM)”,RFC 3393,2002年11月。

[RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., and J. Perser, "Packet Reordering Metrics", RFC 4737, November 2006.

[RFC4737]Morton,A.,Ciavattone,L.,Ramachandran,G.,Shalunov,S.,和J.Perser,“数据包重新排序度量”,RFC 4737,2006年11月。

[RFC5481] Morton, A. and B. Claise, "Packet Delay Variation Applicability Statement", RFC 5481, March 2009.

[RFC5481]Morton,A.和B.Claise,“数据包延迟变化适用性声明”,RFC 54812009年3月。

[RFC5560] Uijterwaal, H., "A One-Way Packet Duplication Metric", RFC 5560, May 2009.

[RFC5560]Uijterwaal,H.,“单向数据包复制度量”,RFC 5560,2009年5月。

[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance Metrics (IPPM): Spatial and Multicast", RFC 5644, October 2009.

[RFC5644]Stephan,E.,Liang,L.,和A.Morton,“IP性能度量(IPPM):空间和多播”,RFC 56442009年10月。

[Y.1540] ITU-T Recommendation Y.1540, "Internet protocol data communication service - IP packet transfer and availability performance parameters", November 2007.

[Y.1540]ITU-T建议Y.1540,“互联网协议数据通信服务-IP数据包传输和可用性性能参数”,2007年11月。

Authors' Addresses

作者地址

Al Morton (editor) AT&T Labs 200 Laurel Avenue South Middletown, NJ 07748 USA

美国新泽西州劳雷尔大道南米德尔顿200号AT&T实验室Al Morton(编辑)07748

   Phone: +1 732 420 1571
   Fax:   +1 732 368 1192
   EMail: acmorton@att.com
   URI:   http://home.comcast.net/~acmacm/
        
   Phone: +1 732 420 1571
   Fax:   +1 732 368 1192
   EMail: acmorton@att.com
   URI:   http://home.comcast.net/~acmacm/
        

Steven Van den Berghe (editor) Alcatel-Lucent Copernicuslaan 50 Antwerp 2018 Belgium

史蒂文·范登伯格(编辑)阿尔卡特-朗讯哥白尼公司2018年安特卫普50号比利时

   Phone: +32 3 240 3983
   EMail: steven.van_den_berghe@alcatel-lucent.com
        
   Phone: +32 3 240 3983
   EMail: steven.van_den_berghe@alcatel-lucent.com