Internet Engineering Task Force (IETF)                        M. Linsner
Request for Comments: 7536                                 Cisco Systems
Category: Informational                                       P. Eardley
ISSN: 2070-1721                                             T. Burbridge
                                                                      BT
                                                             F. Sorensen
                                                                    Nkom
                                                                May 2015
        
Internet Engineering Task Force (IETF)                        M. Linsner
Request for Comments: 7536                                 Cisco Systems
Category: Informational                                       P. Eardley
ISSN: 2070-1721                                             T. Burbridge
                                                                      BT
                                                             F. Sorensen
                                                                    Nkom
                                                                May 2015
        

Large-Scale Broadband Measurement Use Cases

大规模宽带测量用例

Abstract

摘要

Measuring broadband performance on a large scale is important for network diagnostics by providers and users, as well as for public policy. Understanding the various scenarios and users of measuring broadband performance is essential to development of the Large-scale Measurement of Broadband Performance (LMAP) framework, information model, and protocol. This document details two use cases that can assist in developing that framework. The details of the measurement metrics themselves are beyond the scope of this document.

大规模测量宽带性能对于提供商和用户的网络诊断以及公共政策都很重要。了解宽带性能测量的各种场景和用户对于开发大规模宽带性能测量(LMAP)框架、信息模型和协议至关重要。本文档详细介绍了两个可以帮助开发该框架的用例。度量指标本身的详细信息超出了本文档的范围。

Status of This Memo

关于下段备忘

This document is not an Internet Standards Track specification; it is published for informational purposes.

本文件不是互联网标准跟踪规范;它是为了提供信息而发布的。

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Not all documents approved by the IESG are a candidate for any level of Internet Standard; see Section 2 of RFC 5741.

本文件是互联网工程任务组(IETF)的产品。它代表了IETF社区的共识。它已经接受了公众审查,并已被互联网工程指导小组(IESG)批准出版。并非IESG批准的所有文件都适用于任何级别的互联网标准;见RFC 5741第2节。

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc7536.

有关本文件当前状态、任何勘误表以及如何提供反馈的信息,请访问http://www.rfc-editor.org/info/rfc7536.

Copyright Notice

版权公告

Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved.

版权所有(c)2015 IETF信托基金和确定为文件作者的人员。版权所有。

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

本文件受BCP 78和IETF信托有关IETF文件的法律规定的约束(http://trustee.ietf.org/license-info)自本文件出版之日起生效。请仔细阅读这些文件,因为它们描述了您对本文件的权利和限制。从本文件中提取的代码组件必须包括信托法律条款第4.e节中所述的简化BSD许可证文本,并提供简化BSD许可证中所述的无担保。

Table of Contents

目录

   1. Introduction ....................................................3
   2. Use Cases .......................................................3
      2.1. Internet Service Provider (ISP) Use Case ...................3
      2.2. Regulator Use Case .........................................4
   3. Details of ISP Use Case .........................................5
      3.1. Understanding the Quality Experienced by Customers .........5
      3.2. Understanding the Impact and Operation of New Devices
           and Technology .............................................6
      3.3. Design and Planning ........................................6
      3.4. Monitoring Service Level Agreements ........................7
      3.5. Identifying, Isolating, and Fixing Network Problems ........7
   4. Details of Regulator Use Case ...................................8
      4.1. Providing Transparent Performance Information ..............8
      4.2. Measuring Broadband Deployment .............................9
      4.3. Monitoring Traffic Management Practices ...................10
   5. Implementation Options .........................................10
   6. Conclusions ....................................................12
   7. Security Considerations ........................................13
   8. Informative References .........................................15
   Contributors ......................................................17
   Authors' Addresses ................................................17
        
   1. Introduction ....................................................3
   2. Use Cases .......................................................3
      2.1. Internet Service Provider (ISP) Use Case ...................3
      2.2. Regulator Use Case .........................................4
   3. Details of ISP Use Case .........................................5
      3.1. Understanding the Quality Experienced by Customers .........5
      3.2. Understanding the Impact and Operation of New Devices
           and Technology .............................................6
      3.3. Design and Planning ........................................6
      3.4. Monitoring Service Level Agreements ........................7
      3.5. Identifying, Isolating, and Fixing Network Problems ........7
   4. Details of Regulator Use Case ...................................8
      4.1. Providing Transparent Performance Information ..............8
      4.2. Measuring Broadband Deployment .............................9
      4.3. Monitoring Traffic Management Practices ...................10
   5. Implementation Options .........................................10
   6. Conclusions ....................................................12
   7. Security Considerations ........................................13
   8. Informative References .........................................15
   Contributors ......................................................17
   Authors' Addresses ................................................17
        
1. Introduction
1. 介绍

This document describes two use cases for the Large-scale Measurement of Broadband Performance (LMAP). The use cases contained in this document are (1) the Internet Service Provider Use Case and (2) the Regulator Use Case. In the first, a network operator wants to understand the performance of the network and the quality experienced by customers, while in the second, a regulator wants to provide information on the performance of the ISPs in their jurisdiction. There are other use cases that are not the focus of the initial LMAP work (for example, end users would like to use measurements to help identify problems in their home network and to monitor the performance of their broadband provider); it is expected that the same mechanisms are applicable.

本文档描述了宽带性能(LMAP)大规模测量的两个用例。本文件中包含的用例为(1)互联网服务提供商用例和(2)监管机构用例。在第一种情况下,网络运营商希望了解网络性能和客户体验到的质量,而在第二种情况下,监管机构希望提供有关其管辖范围内ISP性能的信息。还有其他用例不是最初LMAP工作的重点(例如,最终用户希望使用测量来帮助识别其家庭网络中的问题并监控其宽带提供商的性能);预计同样的机制也适用。

Large-scale measurements raise several security concerns, including privacy issues. These are summarized in Section 7 and considered in further detail in [Framework].

大规模测量会引起一些安全问题,包括隐私问题。这些内容在第7节中进行了总结,并在[框架]中进行了更详细的讨论。

2. Use Cases
2. 用例

From the LMAP perspective, there is no difference between fixed service and mobile (cellular) service used for Internet access. Hence, like measurements will take place on both fixed and mobile networks. Fixed services include technologies like Digital Subscriber Line (DSL), Cable, and Carrier Ethernet. Mobile services include all those advertised as 2G, 3G, 4G, and Long Term Evolution (LTE). A metric defined to measure end-to-end services will execute similarly on all access technologies. Other metrics may be access technology specific. The LMAP architecture covers both IPv4 and IPv6 networks.

从LMAP的角度来看,用于互联网接入的固定服务和移动(蜂窝)服务之间没有区别。因此,类似的测量将在固定和移动网络上进行。固定服务包括数字用户线(DSL)、电缆和载波以太网等技术。移动服务包括所有广告称为2G、3G、4G和长期演进(LTE)的服务。定义用于测量端到端服务的度量将在所有访问技术上执行类似的操作。其他指标可能是特定于访问技术的。LMAP体系结构涵盖IPv4和IPv6网络。

2.1. Internet Service Provider (ISP) Use Case
2.1. Internet服务提供商(ISP)用例

A network operator needs to understand the performance of their networks, the performance of the suppliers (downstream and upstream networks), the performance of Internet access services, and the impact that such performance has on the experience of their customers. Largely, the processes that ISPs operate (which are based on network measurement) include:

网络运营商需要了解其网络的性能、供应商(下游和上游网络)的性能、互联网接入服务的性能以及此类性能对其客户体验的影响。在很大程度上,ISP操作的过程(基于网络测量)包括:

o Identifying, isolating, and fixing problems, which may be in the network, with the service provider, or in the end-user equipment. Such problems may be common to a point in the network topology (e.g., a single exchange), common to a vendor or equipment type (e.g., line card or home gateway), or unique to a single user line (e.g., copper access). Part of this process may also be helping

o 识别、隔离和修复网络、服务提供商或最终用户设备中可能存在的问题。此类问题可能是网络拓扑中某一点(例如,单个交换机)的常见问题,也可能是供应商或设备类型(例如,线路卡或家庭网关)的常见问题,也可能是单个用户线路的独有问题(例如,铜线接入)。这一过程的一部分可能也会有所帮助

users understand whether the problem exists in their home network or with a third-party application service instead of with their broadband (BB) product.

用户了解问题是否存在于他们的家庭网络中,或者使用第三方应用程序服务而不是宽带(BB)产品。

o Design and planning. Through monitoring the end-user experience, the ISP can design and plan their network to ensure specified levels of user experience. Services may be moved closer to end users, services upgraded, the impact of QoS assessed, or more capacity deployed at certain locations. Service Level Agreements (SLAs) may be defined at network or product boundaries.

o 设计和规划。通过监控最终用户体验,ISP可以设计和规划其网络,以确保特定级别的用户体验。服务可能会更靠近最终用户,服务可能会升级,服务质量的影响可能会评估,或者在某些位置部署更多容量。服务级别协议(SLA)可以在网络或产品边界上定义。

o Understanding the quality experienced by customers. The network operator would like to gain better insight into the end-to-end performance experienced by its customers. "End-to-end" could, for instance, incorporate home and enterprise networks, and the impact of peering, caching, and Content Delivery Networks (CDNs).

o 了解客户体验到的质量。网络运营商希望更好地了解其客户体验的端到端性能。例如,“端到端”可以结合家庭和企业网络,以及对等、缓存和内容交付网络(CDN)的影响。

o Understanding the impact and operation of new devices and technology. As a new product is deployed, or a new technology introduced into the network, it is essential that its operation and its impact are measured. This also helps to quantify the advantage that the new technology is bringing and support the business case for larger roll-out.

o 了解新设备和技术的影响和操作。在部署新产品或将新技术引入网络时,衡量其运行及其影响至关重要。这也有助于量化新技术带来的优势,并为更大规模的推广提供支持。

2.2. Regulator Use Case
2.2. 调节器用例

A regulator may want to evaluate the performance of the Internet access services offered by operators.

监管机构可能希望评估运营商提供的互联网接入服务的性能。

While each jurisdiction responds to distinct consumer, industry, and regulatory concerns, much commonality exists in the need to produce datasets that can be used to compare multiple Internet access service providers, diverse technical solutions, geographic and regional distributions, and marketed and provisioned levels and combinations of broadband Internet access services.

虽然每个司法管辖区都对不同的消费者、行业和监管问题做出了回应,但在生成可用于比较多个互联网接入服务提供商、不同技术解决方案、地理和区域分布的数据集的需求方面存在许多共性,以及销售和提供宽带互联网接入服务的级别和组合。

Regulators may want to publish performance measures of different ISPs as background information for end users. They may also want to track the growth of high-speed broadband deployment, or to monitor the traffic management practices of Internet providers.

监管机构可能希望公布不同ISP的绩效指标,作为最终用户的背景信息。他们还可能希望跟踪高速宽带部署的增长,或者监控互联网提供商的流量管理实践。

A regulator's role in the development and enforcement of broadband Internet access service policies requires that the measurement approaches meet a high level of verifiability, accuracy, and provider-independence to support valid and meaningful comparisons of Internet access service performance. Standards can help regulators'

监管机构在宽带互联网接入服务政策制定和实施中的作用要求测量方法满足高水平的可验证性、准确性和供应商独立性,以支持互联网接入服务性能的有效和有意义的比较。标准有助于监管机构的监管

shared needs for scalable, cost-effective, scientifically robust solutions to the measurement and collection of broadband Internet access service performance information.

对测量和收集宽带互联网接入服务性能信息的可扩展、经济高效、科学可靠的解决方案的共同需求。

3. Details of ISP Use Case
3. ISP用例的详细信息
3.1. Understanding the Quality Experienced by Customers
3.1. 了解客户体验到的质量

Operators want to understand the quality of experience (QoE) of their broadband customers. The understanding can be gained through a "panel", i.e., measurement probes deployed to several customers. A probe is a device or piece of software that makes measurements and reports the results, under the control of the measurement system. Implementation options are discussed in Section 5. The panel needs to include a representative sample of the operator's technologies and broadband speeds. For instance, it might encompass speeds ranging from below 8 Mbps to over 100 Mbps. The operator would like the end-to-end view of the service, rather than just the access portion. This involves relating the pure network parameters to something like a 'mean opinion score' [MOS], which will be service dependent (for instance, web-browsing QoE is largely determined by latency above a few Mbps).

运营商希望了解其宽带客户的体验质量(QoE)。可以通过“面板”获得理解,即部署到多个客户的测量探头。探针是在测量系统的控制下进行测量并报告结果的设备或软件。第5节讨论了实施方案。该小组需要包括运营商技术和宽带速度的代表性样本。例如,它可能包含从8 Mbps以下到100 Mbps以上的速度。运营商希望看到服务的端到端视图,而不仅仅是访问部分。这涉及到将纯网络参数与“平均意见分数”[MOS]之类的东西相关联,这将取决于服务(例如,web浏览QoE在很大程度上取决于几Mbps以上的延迟)。

An operator will also want compound metrics such as "reliability", which might involve packet loss, DNS failures, retraining of the line, video streaming under-runs, etc.

运营商还需要复合指标,如“可靠性”,可能涉及数据包丢失、DNS故障、线路再培训、视频流不足等。

The operator really wants to understand the end-to-end service experience. However, the home network (Ethernet, Wi-Fi, powerline) is highly variable and outside its control. To date, operators (and regulators) have instead measured performance from the home gateway. However, mobile operators clearly must include the wireless link in the measurement.

运营商很想了解端到端的服务体验。然而,家庭网络(以太网、Wi-Fi、电力线)是高度可变的,不受其控制。到目前为止,运营商(和监管机构)已经从家庭网关上衡量了性能。然而,移动运营商显然必须在测量中包括无线链路。

Active measurements are the most obvious approach, i.e., special measurement traffic is sent by -- and to -- the probe. In order not to degrade the service of the customer, the measurement data should only be sent when the user is silent, and it shouldn't reduce the customer's data allowance. The other approach is passive measurements on the customer's ordinary traffic; the advantage is that it measures what the customer actually does, but it creates extra variability (different traffic mixes give different results) and, in particular, it raises privacy concerns. [RFC6973] discusses privacy considerations for Internet protocols in general, while [Framework] discusses them specifically for large-scale measurement systems.

主动测量是最明显的方法,也就是说,特殊测量流量由探测器发送,并发送到探测器。为了不降低客户的服务质量,测量数据只能在用户沉默时发送,不应减少客户的数据余量。另一种方法是对客户的普通流量进行被动测量;其优点是,它可以测量客户实际做了什么,但会产生额外的可变性(不同的流量混合会产生不同的结果),特别是,它会引起隐私问题。[RFC6973]一般讨论了互联网协议的隐私注意事项,而[Framework]专门讨论了大规模测量系统的隐私注意事项。

From an operator's viewpoint, understanding customer experience enables it to offer better services. Also, simple metrics can be more easily understood by senior managers who make investment decisions and by sales and marketing.

从运营商的角度来看,了解客户体验使其能够提供更好的服务。此外,做出投资决策的高级管理人员以及销售和营销人员更容易理解简单的指标。

3.2. Understanding the Impact and Operation of New Devices and Technology

3.2. 了解新设备和技术的影响和操作

Another type of measurement is to test new capabilities before they are rolled out. For example, the operator may want to:

另一种测量方法是在新功能推出之前对其进行测试。例如,操作员可能希望:

o Check whether a customer can be upgraded to a new broadband option.

o 检查客户是否可以升级到新的宽带选项。

o Understand the impact of IPv6 before it is made available to customers. Questions such as these could be assessed: Will v6 packets get through? What will the latency be to major websites? What transition mechanisms will be most appropriate?

o 在向客户提供IPv6之前,了解其影响。可以评估这样的问题:v6数据包能通过吗?主要网站的延迟是什么?什么样的过渡机制最合适?

o Check whether a new capability can be signaled using TCP options (how often it will be blocked by a middlebox -- along the lines of the experiments described in [Extend-TCP]).

o 检查是否可以使用TCP选项向新功能发送信号(它被中间盒阻塞的频率——按照[Extend TCP]中描述的实验路线)。

o Investigate a QoS mechanism (e.g., checking whether Diffserv markings are respected on some path).

o 调查QoS机制(例如,检查在某些路径上是否遵守区分服务标记)。

3.3. Design and Planning
3.3. 设计和规划

Operators can use large-scale measurements to help with their network planning -- proactive activities to improve the network.

运营商可以使用大规模的测量来帮助他们进行网络规划,这就是改进网络的主动活动。

For example, by probing from several different vantage points the operator can see that a particular group of customers has performance below that expected during peak hours, which should help with capacity planning. Naturally, operators already have tools to help with this -- a network element reports its individual utilization (and perhaps other parameters). However, making measurements across a path rather than at a point may make it easier to understand the network. There may also be parameters like bufferbloat that aren't currently reported by equipment and/or that are intrinsically path metrics.

例如,通过从几个不同的有利位置进行调查,运营商可以看到特定的客户群在高峰时段的绩效低于预期,这应有助于容量规划。当然,运营商已经有了工具来帮助实现这一点——一个网元报告其单独的利用率(也许还有其他参数)。然而,在一条路径上而不是在一个点上进行测量可能更容易理解网络。也可能存在设备当前未报告的bufferbloat等参数和/或本质上是路径度量。

With information gained from measurement results, capacity planning and network design can be more effective. Such planning typically uses simulations to emulate the measured performance of the current network and understand the likely impact of new capacity and potential changes to the topology. Simulations, informed by data

利用从测量结果中获得的信息,容量规划和网络设计可以更加有效。此类规划通常使用模拟来模拟当前网络的测量性能,并了解新容量和拓扑结构潜在变化的可能影响。模拟,由数据提供信息

from a limited panel of probes, can help quantify the advantage that a new technology brings and support the business case for larger roll-out.

从有限的调查小组中,可以帮助量化新技术带来的优势,并支持更大规模的业务推广。

It may also be possible to use probes to run stress tests for risk analysis. For example, an operator could run a carefully controlled and limited experiment in which probing is used to assess the potential impact if some new application becomes popular.

也可以使用探针进行压力测试以进行风险分析。例如,操作员可以运行一个精心控制的有限实验,在该实验中,如果某个新应用程序变得流行,则使用探测来评估潜在影响。

3.4. Monitoring Service Level Agreements
3.4. 监控服务级别协议

Another example is that the operator may want to monitor performance where there is a Service Level Agreement (SLA). This could be with its own customers; in particular, enterprises may have an SLA. The operator can proactively spot when the service is degrading near the point of the SLA limit and get information that will enable more informed conversations with the customer at contract renewal.

另一个例子是,运营商可能希望在存在服务水平协议(SLA)的情况下监控性能。这可能与它自己的客户有关;特别是,企业可能有一个SLA。运营商可以主动发现服务何时在SLA限制点附近降级,并获取信息,以便在合同续签时与客户进行更明智的对话。

An operator may also want to monitor the performance of its suppliers, to check whether they meet their SLA or to compare two suppliers if it is dual-sourcing. This could include its transit operator, CDNs, peering, video source, or local network provider for a global operator in countries where it doesn't have its own network. A virtual operator may monitor the whole underlying network.

运营商还可能希望监控其供应商的绩效,检查他们是否符合SLA,或者如果是双重采购,则比较两个供应商。这可能包括其公交运营商、CDN、对等网络、视频源,或在没有自己网络的国家为全球运营商提供本地网络提供商。虚拟运营商可以监控整个底层网络。

Through a better understanding of its own network and its suppliers, the operator should be able to focus investment more effectively -- in the right place at the right time with the right technology.

通过更好地了解自己的网络和供应商,运营商应该能够更有效地集中投资——在合适的时间、合适的地点、合适的技术。

3.5. Identifying, Isolating, and Fixing Network Problems
3.5. 识别、隔离和修复网络问题

Operators can use large-scale measurements to help identify a fault more rapidly and decide how to solve it.

操作员可以使用大规模测量来帮助更快地识别故障并决定如何解决。

Operators already have Test and Diagnostic tools, where a network element reports some problem or failure to a management system. However, many issues are not caused by a point failure but something wider and so will trigger too many alarms, while other issues will cause degradation rather than failure and so not trigger any alarm. Large-scale measurements can help provide a more nuanced view that helps network management to identify and fix problems more rapidly and accurately. The network management tools may use simulations to emulate the network and so help identify a fault and assess possible solutions.

运营商已经有了测试和诊断工具,其中网元会向管理系统报告一些问题或故障。但是,许多问题不是由点故障引起的,而是更广泛的问题,因此会触发太多警报,而其他问题会导致降级而不是故障,因此不会触发任何警报。大规模测量有助于提供更细致的视图,帮助网络管理更快、更准确地识别和解决问题。网络管理工具可以使用仿真来模拟网络,从而帮助识别故障并评估可能的解决方案。

An operator can obtain useful information without measuring the performance on every broadband line. By measuring a subset, the operator can identify problems that affect a group of customers. For example, the issue could be at a shared point in the network topology (such as an exchange), or common to a vendor, or equipment type; for instance, [IETF85-Plenary] describes a case where a particular home gateway upgrade had caused a (mistaken!) drop in line rate.

运营商可以获得有用的信息,而无需测量每条宽带线路的性能。通过测量子集,操作员可以识别影响一组客户的问题。例如,问题可能出现在网络拓扑中的共享点(如交换机),或供应商或设备类型的公共点;例如,[IETF85全体会议]描述了一个特定家庭网关升级导致(错误!)线路率下降的情况。

A more extensive deployment of the measurement capability to every broadband line would enable an operator to identify issues unique to a single customer. Overall, large-scale measurements can help an operator fix the fault more rapidly and/or allow the affected customers to be informed of what's happening. More accurate information enables the operator to reassure customers and take more rapid and effective action to cure the problem.

在每条宽带线路上更广泛地部署测量能力将使运营商能够识别单一客户特有的问题。总的来说,大规模测量可以帮助操作员更快地修复故障和/或让受影响的客户了解发生的情况。更准确的信息使运营商能够让客户放心,并采取更快速有效的措施解决问题。

Often, customers experience poor broadband due to problems in the home network -- the ISP's network is fine. For example, they may have moved too far away from their wireless access point. Anecdotally, a large fraction of customer calls about fixed BB problems are due to in-home wireless issues. These issues are expensive and frustrating for an operator, as they are extremely hard to diagnose and solve. The operator would like to narrow down whether the problem is in the home (a problem with the home network, edge device, or home gateway), in the operator's network, or with an application service. The operator would like two capabilities: firstly, self-help tools that customers use to improve their own service or understand its performance better -- for example, to reposition their devices for better Wi-Fi coverage; and secondly, on-demand tests that the operator can run instantly, so that the call center person answering the phone (or e-chat) could trigger a test and get the result while the customer is still in an online session.

通常,由于家庭网络中的问题,客户会体验到较差的宽带——ISP的网络很好。例如,他们可能离无线接入点太远了。有趣的是,大部分关于固定BB问题的客户电话都是由家庭无线问题引起的。这些问题对运营商来说代价高昂且令人沮丧,因为它们极难诊断和解决。运营商希望缩小问题是在家庭中(家庭网络、边缘设备或家庭网关的问题)、在运营商的网络中还是在应用程序服务中。运营商希望有两种功能:第一,客户使用自助工具来改进自己的服务或更好地了解其性能——例如,重新定位设备以实现更好的Wi-Fi覆盖;其次,运营商可以即时运行的按需测试,以便接听电话(或电子聊天)的呼叫中心人员可以在客户仍在在线会话时触发测试并获得结果。

4. Details of Regulator Use Case
4. 调节器用例的详细信息
4.1. Providing Transparent Performance Information
4.1. 提供透明的性能信息

Some regulators publish information about the quality of the various Internet access services provided in their national market. Quality information about service offers could include speed, delay, and jitter. Such information can be published to facilitate end users' choice of service provider and offer. Regulators may check the accuracy of the marketing claims of Internet service providers and may also encourage ISPs to all use the same metrics in their service level contracts. The goal of these transparency mechanisms is to promote competition for end users and potentially also help content, application, service, and device providers develop their Internet offerings.

一些监管机构公布了其国内市场提供的各种互联网接入服务的质量信息。有关服务提供的质量信息可能包括速度、延迟和抖动。可发布此类信息,以方便最终用户选择服务提供商和提供服务。监管机构可能会检查互联网服务提供商营销声明的准确性,也可能会鼓励ISP在其服务水平合同中使用相同的标准。这些透明机制的目标是促进最终用户的竞争,并有可能帮助内容、应用程序、服务和设备提供商开发其互联网产品。

The published information needs to be:

发布的信息需要:

o Accurate - the measurement results must be correct and not influenced by errors or side effects. The results should be reproducible and consistent over time.

o 准确-测量结果必须正确且不受误差或副作用的影响。随着时间的推移,结果应是可重复和一致的。

o Comparable - common metrics should be used across different ISPs and service offerings, and over time, so that measurement results can be compared.

o 可比性-应在不同的ISP和服务产品中使用通用的度量标准,并随着时间的推移使用,以便可以比较度量结果。

o Meaningful - the metrics used for measurements need to reflect what end users value about their broadband Internet access service.

o 有意义-用于测量的指标需要反映最终用户对其宽带互联网接入服务的价值。

o Reliable - the number and distribution of measurement agents, and the statistical processing of the raw measurement data, need to be appropriate.

o 可靠-测量代理的数量和分布以及原始测量数据的统计处理需要适当。

In practical terms, the regulators may measure network performance from users towards multiple content and application providers, including dedicated test measurement servers. Measurement probes are distributed to a 'panel' of selected end users. The panel covers all the operators and packages in the market, spread over urban, suburban, and rural areas, and often includes both fixed and mobile Internet access. Periodic tests running on the probes can, for example, measure actual speed at peak and off-peak hours, but can also measure other detailed quality metrics like delay and jitter. Collected data goes afterwards through statistical analysis, deriving estimates for the whole population. Summary information, such as a service quality index, is published regularly, perhaps alongside more detailed information.

实际上,监管机构可以从用户到多个内容和应用程序提供商(包括专用测试测量服务器)测量网络性能。测量探头分配到选定最终用户的“面板”。该小组涵盖市场上的所有运营商和软件包,分布在城市、郊区和农村地区,通常包括固定和移动互联网接入。例如,在探测器上运行的定期测试可以测量高峰和非高峰时间的实际速度,但也可以测量其他详细的质量指标,如延迟和抖动。收集的数据随后经过统计分析,得出整个人口的估计数。摘要信息,如服务质量指数,会定期发布,也许会与更详细的信息一起发布。

The regulator can also facilitate end users to monitor the performance of their own broadband Internet access service. They might use this information to check that the performance meets that specified in their contract or to understand whether their current subscription is the most appropriate.

监管机构还可以帮助最终用户监控其自身宽带互联网接入服务的性能。他们可能会使用这些信息来检查性能是否符合合同中规定的性能,或者了解他们当前的订阅是否最合适。

4.2. Measuring Broadband Deployment
4.2. 衡量宽带部署

Regulators may also want to monitor the improvement over time of actual broadband Internet access performance in a specific country or a region. The motivation is often to evaluate the effect of the stimulated growth over time, when government has set a strategic goal for high-speed broadband deployment, whether in absolute terms or benchmarked against other countries. An example of such an initiative is [DAE]. The actual measurements can be made in the same way as described in Section 4.1.

监管机构还可能希望监控特定国家或地区实际宽带互联网接入性能随时间的改善情况。当政府为高速宽带的部署制定了战略目标时,无论是以绝对值还是以其他国家为基准,其动机往往是评估刺激增长的效果。[DAE]就是此类倡议的一个例子。可按照第4.1节所述的相同方式进行实际测量。

4.3. Monitoring Traffic Management Practices
4.3. 监察交通管理措施

A regulator may want to monitor traffic management practices or compare the performance of Internet access service with specialized services offered in parallel to, but separate from, Internet access service (for example, IPTV). A regulator could monitor for departures from application agnosticism such as blocking or throttling of traffic from specific applications, or preferential treatment of specific applications. A measurement system could send, or passively monitor, application-specific traffic and then measure in detail the transfer of the different packets. While it is relatively easy to measure port blocking, how to detect other types of differentiated treatment is a research topic in itself. The "Glasnost: Enabling End Users to Detect Traffic Differentiation" paper [M-Labs_NSDI-2010] and follow-on tool "Glasnost" [Glasnost] provide an example of work in this area.

监管机构可能希望监控流量管理实践,或将互联网接入服务的性能与互联网接入服务(例如,IPTV)并行但独立提供的专门服务进行比较。监管者可以监控偏离应用不可知论的情况,如特定应用的流量阻塞或节流,或特定应用的优惠待遇。测量系统可以发送或被动监控特定于应用程序的流量,然后详细测量不同数据包的传输。虽然测量端口阻塞相对容易,但如何检测其他类型的差异化处理本身就是一个研究课题。“Glasnost:使最终用户能够检测流量差异化”论文[M-Labs_NSDI-2010]和后续工具“Glasnost”[Glasnost]提供了这一领域的工作示例。

A regulator could also monitor the performance of the broadband service over time, to try and detect if the specialized service is provided at the expense of the Internet access service. Comparison between ISPs or between different countries may also be relevant for this kind of evaluation.

监管机构还可以监控宽带服务的长期性能,尝试检测是否以牺牲互联网接入服务为代价提供了专门服务。ISP之间或不同国家之间的比较也可能与此类评估相关。

The motivation for a regulator monitoring such traffic management practices is that regulatory approaches related to net neutrality and the open Internet have been introduced in some jurisdictions. Examples of such efforts are the Internet policy as outlined by the Body of European Regulators for Electronic Communications guidelines for quality of service [BEREC-Guidelines] and the US FCC's "Preserving the Open Internet" Report and Order [FCC-R&O]. Although legal challenges can change the status of policy, the take-away for LMAP purposes is that policy-makers are looking for measurement solutions to assist them in discovering biased treatment of traffic flows. The exact definitions and requirements vary from one jurisdiction to another.

监管机构监控此类流量管理实践的动机是,一些司法管辖区引入了与网络中立性和开放互联网相关的监管方法。这些努力的例子包括欧洲电子通信服务质量监管机构机构所概述的互联网政策[BEREC指南]和美国FCC的“保护开放互联网”报告和命令[FCC-R&O]。尽管法律挑战可以改变政策的状态,但出于LMAP的目的,政策制定者正在寻找测量解决方案,以帮助他们发现对交通流的偏见处理。各个司法管辖区的确切定义和要求各不相同。

5. Implementation Options
5. 实施方案

There are several ways of implementing a measurement system. The choice may be influenced by the details of the particular use case and what the most important criteria are for the regulator, ISP, or third party operating the measurement system.

有几种实现测量系统的方法。选择可能受特定用例的细节以及调节器、ISP或第三方操作测量系统的最重要标准的影响。

One type of probe is a special hardware device that is connected directly to the home gateway. The devices are deployed to a carefully selected panel of end users, and they perform measurements according to a defined schedule. The schedule can run throughout the day, to allow continuous assessment of the network. Careful design

一种类型的探头是直接连接到家庭网关的特殊硬件设备。这些设备被部署到精心挑选的最终用户面板上,并根据定义的时间表执行测量。该计划可以全天运行,以允许对网络进行持续评估。精心设计

ensures that measurements do not detrimentally impact the home user experience or corrupt the results by testing when the user is also using the broadband line. The system is therefore tightly controlled by the operator of the measurement system. One advantage of this approach is that it is possible to get reliable benchmarks for the performance of a network with only a few devices. One disadvantage is that it would be expensive to deploy hardware devices on a mass scale sufficient to understand the performance of the network at the granularity of a single broadband user.

当用户同时使用宽带线路时,通过测试确保测量不会对家庭用户体验造成不利影响或损坏结果。因此,系统由测量系统的操作员严格控制。这种方法的一个优点是,可以为只有少数设备的网络获得可靠的性能基准。一个缺点是,大规模部署硬件设备的成本很高,足以在单个宽带用户的粒度上了解网络的性能。

Another type of probe involves implementing the measurement capability as a webpage or an "app" that end users are encouraged to download onto their mobile phone or computing device. Measurements are triggered by the end user; for example, the user interface may have a button to "test my broadband now." One advantage of this approach is that the performance is measured to the end user, rather than to the home gateway, and so includes the home network. Another difference is that the system is much more loosely controlled, as the panel of end users and the schedule of tests are determined by the end users themselves rather than the measurement system. While this approach makes it easier to make measurements on a large scale, it is harder to get comparable benchmarks, as the measurements are affected by the home network; also, the population is self-selecting and so potentially biased towards those who think they have a problem. This could be alleviated by encouraging widespread downloading of the app and careful post-processing of the results to reduce biases.

另一种类型的探测涉及将测量功能实现为网页或“应用程序”,鼓励最终用户下载到他们的手机或计算设备上。测量由最终用户触发;例如,用户界面可能有一个“立即测试我的宽带”按钮。这种方法的一个优点是,性能是针对最终用户而不是家庭网关进行测量的,因此包括家庭网络。另一个区别是系统的控制要松散得多,因为最终用户小组和测试时间表由最终用户自己决定,而不是由测量系统决定。虽然这种方法更容易进行大规模测量,但由于测量受到家庭网络的影响,更难获得可比较的基准;此外,人口是自我选择的,因此可能偏向那些认为自己有问题的人。这可以通过鼓励广泛下载应用程序和仔细地对结果进行后处理以减少偏差来缓解。

There are several other possibilities. For example, as a variant on the first approach, the measurement capability could be implemented as software embedded in the home gateway, which would make it more viable to have the capability on every user line. As a variant on the second approach, the end user could initiate measurements in response to a request from the measurement system.

还有其他几种可能性。例如,作为第一种方法的一种变体,测量能力可以作为嵌入在家庭网关中的软件来实现,这将使在每一条用户线上都具有这种能力变得更加可行。作为第二种方法的变体,最终用户可以响应测量系统的请求启动测量。

The operator of the measurement system should be careful to ensure that measurements do not detrimentally impact users. Potential issues include the following:

测量系统的操作员应小心确保测量不会对用户造成不利影响。潜在问题包括:

* Measurement traffic generated on a particular user's line may impact that end user's quality of experience. The danger is greater for measurements that generate a lot of traffic over a lengthy period.

* 特定用户线路上产生的测量流量可能会影响最终用户的体验质量。对于在长时间内产生大量流量的测量,危险更大。

* The measurement traffic may impact that particular user's bill or traffic cap.

* 测量流量可能会影响特定用户的账单或流量上限。

* The measurement traffic from several end users may, in combination, congest a shared link.

* 来自多个终端用户的测量流量组合起来可能会阻塞共享链路。

* The traffic associated with the control and reporting of measurements may overload the network. The danger is greater where the traffic associated with many end users is synchronized.

* 与测量控制和报告相关的流量可能会使网络过载。如果与多个最终用户关联的通信量是同步的,则危险性更大。

6. Conclusions
6. 结论

Large-scale measurements of broadband performance are useful for both network operators and regulators. Network operators would like to use measurements to help them better understand the quality experienced by their customers, identify problems in the network, and design network improvements. Regulators would like to use measurements to help promote competition between network operators, stimulate the growth of broadband access, and monitor 'net neutrality'. There are other use cases that are not the focus of the initial LMAP charter (although it is expected that the mechanisms developed would be readily applied); for example, end users would like to use measurements to help identify problems in their home network and to monitor the performance of their broadband provider.

宽带性能的大规模测量对网络运营商和监管机构都很有用。网络运营商希望通过测量来帮助他们更好地了解客户体验的质量,识别网络中的问题,并设计网络改进方案。监管者希望通过测量来促进网络运营商之间的竞争,刺激宽带接入的增长,并监控“网络中立性”。还有其他用例不是初始LMAP章程的重点(尽管预计开发的机制将很容易应用);例如,最终用户希望使用测量来帮助识别家庭网络中的问题,并监控宽带提供商的性能。

From consideration of the various use cases, several common themes emerge, while there are also some detailed differences. These characteristics guide the development of LMAP's framework, information model, and protocol.

从各种用例的考虑中,出现了几个共同的主题,同时也存在一些详细的差异。这些特性指导了LMAP框架、信息模型和协议的开发。

A measurement capability is needed across a wide number of heterogeneous environments. Tests may be needed in the home network, in the ISP's network, or beyond; they may be measuring a fixed or wireless network; they may measure just the access network or across several networks.

跨大量异构环境需要测量能力。可能需要在家庭网络、ISP网络或其他网络中进行测试;它们可以测量固定或无线网络;它们可以只测量接入网络,也可以跨多个网络进行测量。

There is a role for both standardized and non-standardized measurements. For example, a regulator would like to publish standardized performance metrics for all network operators, while an ISP may need their own tests to understand some feature special to their network. Most use cases need active measurements, which create and measure specific test traffic, but some need passive measurements of the end user's traffic.

标准化和非标准化测量都有作用。例如,监管机构希望为所有网络运营商发布标准化的性能指标,而ISP可能需要自己的测试来了解其网络的某些特殊功能。大多数用例需要主动测量,它创建并测量特定的测试流量,但有些用例需要被动测量最终用户的流量。

Regardless of the tests being operated, there needs to be a way to demand or schedule the tests. Most use cases need a regular schedule of measurements, but sometimes ad hoc testing is needed -- for example, for troubleshooting. It needs to be ensured that measurements do not affect the user experience and are not affected by user traffic (unless desired). In addition, there needs to be a

无论运行何种测试,都需要有一种方法来要求或安排测试。大多数用例都需要定期的测量计划,但有时需要特别的测试——例如,用于故障排除。需要确保测量不会影响用户体验,也不会受到用户流量的影响(除非需要)。此外,还需要一个

common way to collect the results. Standardization of this control and reporting functionality allows the operator of a measurement system to buy the various components from different vendors.

收集结果的常用方法。此控制和报告功能的标准化允许测量系统的操作员从不同的供应商处购买各种组件。

After the measurement results are collected, they need to be understood and analyzed. Often, it is sufficient to measure only a small subset of end users, but per-line fault diagnosis requires the ability to test every individual line. Analysis requires accurate definition and understanding of where the test points are, as well as contextual information about the topology, line, product, and the subscriber's contract. The actual analysis of results is beyond the scope of LMAP, as is the key challenge of how to integrate the measurement system into a network operator's existing tools for diagnostics and network planning.

收集测量结果后,需要对其进行理解和分析。通常,仅测量终端用户的一小部分就足够了,但每条线路的故障诊断需要测试每条线路的能力。分析需要准确定义和理解测试点的位置,以及有关拓扑、线路、产品和订户合同的上下文信息。结果的实际分析超出了LMAP的范围,如何将测量系统集成到网络运营商现有的诊断和网络规划工具中也是一个关键挑战。

Finally, the test data, along with any associated network, product, or subscriber contract data, is commercial or private information and needs to be protected.

最后,测试数据以及任何相关的网络、产品或订户合同数据都是商业或私人信息,需要加以保护。

7. Security Considerations
7. 安全考虑

Large-scale measurements raise several potential security, privacy (data protection) [RFC6973], and business sensitivity issues:

大规模测量带来了几个潜在的安全、隐私(数据保护)[RFC6973]和业务敏感性问题:

1. A malicious party may try to gain control of probes to launch DoS (Denial of Service) attacks at a target. A DoS attack could be targeted at a particular end user or set of end users, a certain network, or a specific service provider.

1. 恶意方可能试图控制探测器,以在目标上发起DoS(拒绝服务)攻击。DoS攻击的目标可能是特定的最终用户或一组最终用户、特定的网络或特定的服务提供商。

2. A malicious party may try to gain control of probes to create a platform for pervasive monitoring [RFC7258] or for more targeted monitoring. [RFC7258] summarizes the threats as follows: "An attack may change the content of the communication, record the content or external characteristics of the communication, or through correlation with other communication events, reveal information the parties did not intend to be revealed." For example, a malicious party could distribute to the probes a new measurement test that recorded (and later reported) information of maleficent interest. Similar concerns also arise if the measurement results are intercepted or corrupted.

2. 恶意方可能会试图控制探测,以创建一个用于普适监测[RFC7258]或更具针对性监测的平台。[RFC7258]将威胁总结如下:“攻击可能会改变通信内容,记录通信内容或外部特征,或通过与其他通信事件的关联,泄露双方不打算透露的信息。”例如,恶意方可以向探测器分发新的测量测试,该测试记录(并随后报告)恶意信息。如果测量结果被截获或损坏,也会出现类似的问题。

* From the end user's perspective, the concerns include a malicious party monitoring the traffic they send and receive, who they communicate with, the websites they visit, and such information about their behavior as when they are at home and the location of their devices. Some of the concerns may be greater when the probe is on the end user's device rather than on their home gateway.

* 从最终用户的角度来看,问题包括恶意方监控他们发送和接收的流量、他们与谁通信、他们访问的网站,以及他们在家时的行为和设备位置等信息。当探测器在最终用户的设备上而不是在他们的家庭网关上时,一些问题可能会更严重。

* From the network operator's perspective, the concerns include the leakage of commercially sensitive information about the design and operation of their network, their customers, and suppliers. Some threats are indirect; for example, the attacker could reconnoiter potential weaknesses, such as open ports and paths through the network, which enabled it to launch an attack later.

* 从网络运营商的角度来看,问题包括泄露有关其网络、客户和供应商的设计和运营的商业敏感信息。有些威胁是间接的;例如,攻击者可以侦察潜在的弱点,例如开放的端口和通过网络的路径,从而使其能够在以后发起攻击。

* From the regulator's perspective, the concerns include distortion of the measurement tests or alteration of the measurement results. Also, a malicious network operator could try to identify the broadband lines that the regulator was measuring and prioritize that traffic ("game the system").

* 从监管机构的角度来看,关注点包括测量测试的失真或测量结果的变更。此外,恶意网络运营商可能试图识别监管机构正在测量的宽带线路,并对流量进行优先级排序(“游戏系统”)。

3. Another potential issue is a measurement system that does not obtain the end user's informed consent, fails to specify a specific purpose in the consent, or uses the collected information for secondary uses beyond those specified.

3. 另一个潜在问题是测量系统未获得最终用户的知情同意,未在同意书中指定特定用途,或将收集的信息用于指定用途以外的次要用途。

4. Another potential issue is a measurement system that does not indicate who is responsible for the collection and processing of personal data and who is responsible for fulfilling the rights of users. The responsible party (often termed the "data controller") should, as good practice, consider such issues as defining:

4. 另一个潜在问题是一个计量系统,该系统没有指明谁负责收集和处理个人数据,谁负责实现用户的权利。负责方(通常被称为“数据控制器”)应作为良好的实践,考虑定义如下的问题:

o the purpose for which the data is collected and used,

o 收集和使用数据的目的,

o how the data is stored, accessed, and processed,

o 数据的存储、访问和处理方式,

o how long the data is retained, and

o 数据保留的时间,以及

o how the end user can view, update, and even delete their personal data.

o 最终用户如何查看、更新甚至删除其个人数据。

If anonymized personal data is shared with a third party, the data controller should consider the possibility that the third party can de-anonymize it by combining it with other information.

如果匿名的个人数据与第三方共享,则数据控制器应该考虑第三方可以通过将其与其他信息结合来解除匿名的可能性。

These security and privacy issues will need to be considered carefully by any measurement system. In the context of LMAP, [Framework] considers them further, along with some potential mitigations. Other LMAP documents will specify one or more protocols that enable the measurement system to instruct a probe about what measurements to make and that enable the probe to report the measurement results. Those documents will need to discuss solutions to the security and privacy issues. However, the protocol documents

任何测量系统都需要仔细考虑这些安全和隐私问题。在LMAP的背景下,[框架]进一步考虑了它们,以及一些潜在的缓解措施。其他LMAP文件将指定一个或多个协议,使测量系统能够指示探头进行哪些测量,并使探头能够报告测量结果。这些文件需要讨论安全和隐私问题的解决方案。然而,该议定书载有一些文件

will not consider the actual usage of the measurement information. Many use cases can be envisaged, and earlier in this document we described some likely ones for the network operator and regulator.

不会考虑测量信息的实际使用。可以设想许多用例,在本文档前面,我们为网络运营商和监管机构描述了一些可能的用例。

8. Informative References
8. 资料性引用

[IETF85-Plenary] Crawford, S., "Large-Scale Active Measurement of Broadband Networks", 'example' from slide 18, November 2012, <http://www.ietf.org/proceedings/85/slides/ slides-85-iesg-opsandtech-7.pdf>.

[IETF85全体会议]Crawford,S.,“宽带网络的大规模主动测量”,幻灯片18中的“示例”,2012年11月<http://www.ietf.org/proceedings/85/slides/ 幻灯片-85-iesg-opsandtech-7.pdf>。

[Extend-TCP] Honda, M., Nishida, Y., Raiciu, C., Greenhalgh, A., Handley, M., and H. Tokuda, "Is it Still Possible to Extend TCP?", Proceedings of IETF 82, November 2011, <http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf>.

[扩展TCP]本田,M.,西田,Y.,雷丘,C.,格林哈勒,A.,汉德利,M.,和H.德田,“仍然有可能扩展TCP吗?”,IETF 82会议记录,2011年11月<http://www.ietf.org/proceedings/82/slides/IRTF-1.pdf>.

[Framework] Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., Aitken, P., and A. Akhter, "A framework for Large-Scale Measurement of Broadband Performance (LMAP)", Work in Progress, draft-ietf-lmap-framework-14, April 2015.

[框架]Eardley,P.,Morton,A.,Bagnulo,M.,Burbridge,T.,Aitken,P.,和A.Akhter,“宽带性能大规模测量框架(LMAP)”,正在进行的工作,草案-ietf-LMAP-Framework-14,2015年4月。

[RFC6973] Cooper, A., Tschofenig, H., Aboba, B., Peterson, J., Morris, J., Hansen, M., and R. Smith, "Privacy Considerations for Internet Protocols", RFC 6973, July 2013, <http://www.rfc-editor.org/info/rfc6973>.

[RFC6973]Cooper,A.,Tschofenig,H.,Aboba,B.,Peterson,J.,Morris,J.,Hansen,M.,和R.Smith,“互联网协议的隐私考虑”,RFC 69732013年7月<http://www.rfc-editor.org/info/rfc6973>.

[RFC7258] Farrell, S. and H. Tschofenig, "Pervasive Monitoring Is an Attack", BCP 188, RFC 7258, May 2014, <http://www.rfc-editor.org/info/rfc7258>.

[RFC7258]Farrell,S.和H.Tschofenig,“普遍监控是一种攻击”,BCP 188,RFC 7258,2014年5月<http://www.rfc-editor.org/info/rfc7258>.

[FCC-R&O] United States Federal Communications Commission, "Preserving the Open Internet; Broadband Industries Practices: Report and Order", FCC 10-201, December 2010, <http://hraunfoss.fcc.gov/edocs_public/attachmatch/ FCC-10-201A1.pdf>.

[FCC-R&O]美国联邦通信委员会,“保护开放互联网;宽带行业实践:报告和命令”,FCC 10-201,2010年12月<http://hraunfoss.fcc.gov/edocs_public/attachmatch/ FCC-10-201A1.pdf>。

[BEREC-Guidelines] Body of European Regulators for Electronic Communications, "BEREC Guidelines for quality of service in the scope of net neutrality", <http://berec.europa.eu/eng/ document_register/subject_matter/berec/download/0/ 1101-berec-guidelines-for-quality-of-service-_0.pdf>.

[BEREC指南]欧洲电子通信监管机构,“网络中立范围内的BEREC服务质量指南”<http://berec.europa.eu/eng/ 文件\u register/subject\u matter/berec/download/0/1101 berec服务质量指南-\u 0.pdf>。

   [M-Labs_NSDI-2010]
              M-Lab, "Glasnost: Enabling End Users to Detect Traffic
              Differentiation", <http://www.measurementlab.net/
              download/AMIfv945ljiJXzG-fgUrZSTu2hs1xRl5Oh-
              rpGQMWL305BNQh-BSq5oBoYU4a7zqXOvrztpJhK9gwk5unOe-
              fOzj4X-vOQz_HRrnYU-aFd0rv332RDReRfOYkJuagysstN3GZ__lQHTS8_
              UHJTWkrwyqIUjffVeDxQ/>.
        
   [M-Labs_NSDI-2010]
              M-Lab, "Glasnost: Enabling End Users to Detect Traffic
              Differentiation", <http://www.measurementlab.net/
              download/AMIfv945ljiJXzG-fgUrZSTu2hs1xRl5Oh-
              rpGQMWL305BNQh-BSq5oBoYU4a7zqXOvrztpJhK9gwk5unOe-
              fOzj4X-vOQz_HRrnYU-aFd0rv332RDReRfOYkJuagysstN3GZ__lQHTS8_
              UHJTWkrwyqIUjffVeDxQ/>.
        

[Glasnost] M-Lab tool "Glasnost", <http://mlab-live.appspot.com/ tools/glasnost>.

[Glasnost]M-Lab工具“Glasnost”<http://mlab-live.appspot.com/ 工具/glasnost>。

[MOS] Wikipedia, "Mean Opinion Score", January 2015, <http://en.wikipedia.org/w/index.php? title=Mean_opinion_score&oldid=644494161>.

[MOS]维基百科,“平均意见分数”,2015年1月<http://en.wikipedia.org/w/index.php? title=Mean\u opinion\u score&oldid=644494161>。

[DAE] Digital Agenda for Europe, COM(2010)245 final, "Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions", <http://eur-lex.europa.eu/legal-content/EN/TXT/ PDF/?uri=CELEX:52010DC0245&from=EN>.

[DAE]欧洲数字议程,COM(2010)245最终版,“委员会向欧洲议会、理事会、欧洲经济社会委员会和区域委员会的通报”<http://eur-lex.europa.eu/legal-content/EN/TXT/ PDF/?uri=CELEX:52010DC0245&from=EN>。

Contributors

贡献者

The information in this document is partially derived from text written by the following contributors:

本文件中的信息部分来源于以下撰稿人编写的文本:

James Miller jamesmilleresquire@gmail.com

詹姆斯·米勒jamesmilleresquire@gmail.com

Rachel Huang rachel.huang@huawei.com

瑞秋黄瑞秋。huang@huawei.com

Authors' Addresses

作者地址

Marc Linsner Cisco Systems, Inc. Marco Island, FL United States

Marc Linsner思科系统公司,美国佛罗里达州马可岛

   EMail: mlinsner@cisco.com
        
   EMail: mlinsner@cisco.com
        

Philip Eardley BT B54 Room 77, Adastral Park, Martlesham Ipswich, IP5 3RE United Kingdom

Philip Eardley BT B54英国IP5 IP5伊普斯威奇市Martlesham Ipswich Adastral公园77室

   EMail: philip.eardley@bt.com
        
   EMail: philip.eardley@bt.com
        

Trevor Burbridge BT B54 Room 70, Adastral Park, Martlesham Ipswich, IP5 3RE United Kingdom

Trevor Burbridge BT B54英国IP5区伊普斯威奇市马特勒沙姆阿达斯特拉尔公园70室

   EMail: trevor.burbridge@bt.com
        
   EMail: trevor.burbridge@bt.com
        

Frode Sorensen Norwegian Communications Authority (Nkom) Lillesand Norway

弗罗德·索伦森挪威通信管理局(Nkom)挪威利勒山

   EMail: frode.sorensen@nkom.no
        
   EMail: frode.sorensen@nkom.no