CN117882432A

CN117882432A - Improvement in quality of experience measurement

Info

Publication number: CN117882432A
Application number: CN202180101836.9A
Authority: CN
Inventors: 贺敬; 平静
Original assignee: Nokia Shanghai Bell Co Ltd; Nokia Solutions and Networks Oy
Current assignee: Nokia Shanghai Bell Co Ltd; Nokia Solutions and Networks Oy
Priority date: 2021-08-27
Filing date: 2021-08-27
Publication date: 2024-04-12
Also published as: WO2023024105A1; EP4393197A1

Abstract

Improved methods, devices, and computer readable storage media for QoE measurement. In particular, in the event that the first device (210) determines that there is at least one overlapping metric for the second device (220), the first device (210) may generate at least one measurement configuration according to some example embodiments of the present disclosure, wherein a measurement configuration of the at least one measurement configuration comprises the at least one overlapping metric. Alternatively, in case the second device (220) determines that at least one overlapping metric is configured to the second device (220), in some example embodiments of the present disclosure, the second device (220) generates at least one measurement report when reporting the measurement result, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to the at least one overlapping metric. In this way unnecessary and redundant transmissions in the air interface can be avoided.

Description

Improvement in quality of experience measurement

Technical Field

Embodiments of the present disclosure relate generally to the field of telecommunications, and more particularly, to improved devices, methods, and computer-readable storage media for quality of experience (QoE) measurements.

Background

The QoE measurement indicates an objective experience of the user for Application (APP) level communication service satisfaction. Therefore, it is generally used as one of key indexes for evaluating and evaluating the quality of communication service. Further, the service provider or operator can obtain the quality of the communication service by collecting measurement results from the terminal device of the user, and improve the quality of the corresponding communication service according to the collected measurement results.

In wireless communication, a Management System (MS) may send a QoE measurement configuration (e.g., qoE measurement collection QMC) via an access network device to activate QoE measurements at a terminal device. Further, the QoE measurement configuration may indicate at least one measurement metric. Upon receiving the QoE measurement configuration, the APP level of the terminal device will perform QoE measurements and communicate the measurement results to the access level/access layer of the terminal device. The measurement results will be processed and sent to the access network device (e.g. the gNB) and the access network device will forward the measurement results to the corresponding QoE data consumer (e.g. the message collecting entity MCE).

Recently, it has been proposed to have more than one QMC for a particular service type configured for one terminal device. In this case, metrics included in different QMCs may overlap. In other words, different QMCs may comprise one or more common metrics. The terminal device will thus report redundant and repeated measurements, resulting in unexpected and unwanted waste of resources in the air interface. It is therefore desirable to be able to improve and optimize the QoE procedure, thereby avoiding redundant and repetitive data transmissions (e.g. measurements).

Disclosure of Invention

In general, example embodiments of the present disclosure provide solutions for improving QoE measurements.

In a first aspect, a first device is provided. The first device includes at least one processor; and at least one memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the first device to: obtaining, from a first data collector, a first measurement requirement of a QoE measurement at a second device, the first measurement requirement comprising at least one metric; obtaining, from a second data collector, a second measurement requirement of QoE measurements at a second device, the second measurement requirement comprising at least one metric; if at least one metric included in the first measurement requirement at least partially overlaps with at least one metric included in the second measurement requirement, at least one measurement configuration is generated based on the first measurement requirement and the second measurement requirement, the measurement configuration of the at least one measurement configuration including the at least one overlapping metric.

In a second aspect, a third device is provided. The third device includes at least one processor; and at least one memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to: receiving a first message from a first device, the first message for activating a measurement configuration for QoE measurements at a second device, the first message indicating: a measurement configuration, session identifications allocated for measurement sessions corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration, creating correspondence between identifications used to identify the session identifications and the information of the plurality of data collectors; and transmitting a configuration message to the second device based on the received first message.

In a third aspect, a second device is provided. The second device includes at least one processor; and at least one memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the second device to: receiving, from the third device, a fourth measurement configuration for QoE measurements at the second device, the fourth measurement requirement indicating at least one metric and corresponding to a first identification for identifying the first session identification; receiving, from the third device, a fifth measurement configuration for QoE measurements at the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identification for identifying a second session identification; and if the at least one metric of the fourth measurement configuration indication at least partially overlaps with the at least one metric of the fifth measurement configuration indication, generating at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration, wherein a measurement report of the at least one measurement report includes a measurement report equivalent to the at least one overlapping metric.

In a fourth aspect, a third device is provided. The third device includes at least one processor; and at least one memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to: receiving, from the second device, a measurement report message for QoE measurements at the second device, the measurement report message comprising: a measurement report and a plurality of identifications identifying session identifications; and transmitting a measurement report to the respective data collectors based on correspondence between a plurality of identifications for identifying the plurality of session identifications and a plurality of identifications of the respective data collectors.

In a fifth aspect, a method is provided. The method comprises the following steps: at the first device, obtaining, from a first data collector, a first measurement requirement for QoE measurements at the second device, the first measurement requirement comprising at least one metric; obtaining, from a second data collector, a second measurement requirement for QoE measurements at a second device, the second measurement requirement comprising at least one metric; if at least one metric included in the first measurement requirement at least partially overlaps with at least one metric included in the second measurement requirement, at least one measurement configuration is generated based on the first measurement requirement and the second measurement requirement, the measurement configuration in the at least one measurement configuration including the at least one overlapping metric.

In a sixth aspect, a method is provided. The method comprises the following steps: receiving, at a third device, a first message from a first device, the first message for activating a measurement configuration for QoE measurements at a second device, the first message indicating: a measurement configuration for creating a correspondence between an identification used to identify the session identification and information of the plurality of data collectors corresponding to the measurement configuration, and the session identification allocated for the measurement session corresponding to the measurement configuration, and the information of the plurality of data collectors; and transmitting a configuration message to the second device based on the received first message.

In a seventh aspect, a method is provided. The method comprises the following steps: receiving, at the second device, a fourth measurement configuration for QoE measurements at the second device from the third device, the fourth measurement requirement indicating the at least one metric and corresponding to a first identification for identifying the first session identification; receiving, from the third device, a fifth measurement configuration for QoE measurements at the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identification for identifying a second session identification; if the at least one metric indicated by the fourth measurement configuration at least partially overlaps with the at least one metric indicated by the fifth measurement configuration, generating at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to the at least one overlapping metric.

In an eighth aspect, a method is provided. The method comprises the following steps: receiving, at the third device, a measurement report message of QoE measurements at the second device from the second device, the measurement report message comprising: a measurement report and a plurality of identifications identifying session identifications; and transmitting a measurement report to the respective data collectors based on correspondence between the plurality of identifications for identifying the session identifications and the plurality of identifications of the respective data collectors.

In a ninth aspect, a first apparatus is provided. The first device comprises: means for obtaining, at a first device, a first measurement requirement for QoE measurements at a second device from a first data collector, the first measurement requirement comprising at least one metric; means for obtaining, from a second data collector, a second measurement requirement for QoE measurements at a second device, the second measurement requirement comprising at least one metric; and means for generating at least one measurement configuration based on the first measurement requirement and the second measurement requirement if the at least one metric included in the first measurement requirement at least partially overlaps with the at least one metric included in the second measurement requirement, the measurement configuration in the at least one measurement configuration including the at least one overlapping metric.

In a tenth aspect, a third apparatus is provided. The third device includes: means for receiving, at a third device, a first message from a first device, the first message for activating a measurement configuration for QoE measurement at a second device, the first message indicating: a measurement configuration, a session identification allocated for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration; means for creating correspondence between an identification used to identify the session identification and information of the plurality of data collectors; and means for sending a configuration message to the second device based on the received first message.

In an eleventh aspect, a second apparatus is provided. The second device includes: means for receiving, at the second apparatus, a fourth measurement configuration for QoE measurements at the second apparatus from the third apparatus, the fourth measurement requirement indicating at least one metric and corresponding to a first identification for identifying the first session identification; means for receiving a fifth measurement configuration for QoE measurements at the second device from the third device, the fifth measurement requirement indicating the at least one metric and corresponding to a first identification for identifying the first session identification; and means for generating at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration if the at least one metric of the fourth measurement configuration indication at least partially overlaps with the at least one metric of the fifth measurement configuration indication, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to the at least one overlapping metric.

In a twelfth aspect, a third apparatus is provided. The third apparatus includes means for receiving, at the third apparatus, a measurement report message for QoE measurements at the second apparatus from the second apparatus, the measurement report message including: a measurement report and a plurality of identifications identifying session identifications; and means for sending measurement reports to the respective data collectors based on correspondence between a plurality of identifications identifying the plurality of session identifications and a plurality of identifications of the respective data collectors.

In a thirteenth aspect, a computer-readable medium is provided. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the fifth aspect.

In a fourteenth aspect, a computer readable medium is provided. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the sixth aspect.

In a fifteenth aspect, a computer readable medium is provided. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the seventh aspect.

In a sixteenth aspect, a computer readable medium is provided. The computer readable medium comprises program instructions for causing an apparatus to perform at least the method according to the eighth aspect.

It should be understood that the summary is not intended to identify key or essential features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the description that follows.

Drawings

Some example embodiments will now be described with reference to the accompanying drawings, in which:

fig. 1A illustrates a conventional signaling flow for QMC activation and reporting procedures in LTE;

FIG. 1B illustrates a conventional format of a QMC configuration file;

FIG. 2 illustrates an example communication network in which example embodiments of the present disclosure may be implemented;

fig. 3 illustrates a signaling diagram of an example process of making QoE measurements, according to some embodiments of the present disclosure;

fig. 4 illustrates a signaling diagram of another example process of making QoE measurements, according to some embodiments of the present disclosure;

FIG. 5 illustrates a flowchart of an example method performed at a first device according to some embodiments of the present disclosure;

FIG. 6 illustrates a flowchart of an example method performed at a third device according to some embodiments of the present disclosure;

FIG. 7 illustrates a flowchart of an example method performed at a second device according to some embodiments of the present disclosure;

FIG. 8 illustrates a flowchart of an example method performed at a third device according to some embodiments of the present disclosure;

FIG. 9 illustrates a simplified block diagram of an apparatus suitable for implementing exemplary embodiments of the present disclosure; and

fig. 10 illustrates a block diagram of an example computer-readable medium, according to an example embodiment of the present disclosure.

The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements.

Detailed Description

Principles of the present disclosure will now be described with reference to some example embodiments. It should be understood that these embodiments are described merely for the purpose of illustrating and helping those skilled in the art to understand and practice the present disclosure and do not imply any limitation on the scope of the present disclosure. The present disclosure may be implemented in various other ways besides those described below.

In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

References in the present disclosure to "one embodiment," "an example embodiment," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.

It will be understood that, although the terms "first" and "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element could be termed a second element, and, likewise, a second element could be termed a first element, without departing from the scope of example embodiments. As used herein, the term "and/or" includes any and all combinations of one or more of the listed terms.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises," "comprising," "has," "having," "includes" and/or "including" when used herein, specify the presence of stated features, elements, and/or components, but do not preclude the presence or addition of one or more other features, elements, components, and/or groups thereof.

As used herein, the term "determine/decide" (and grammatical variants thereof) may include, inter alia: calculation, operation, processing, derivation, measurement, investigation, lookup (e.g., in a table, database, or other data structure), determination, etc. Further, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), obtaining, etc. Further, "determining/deciding" may include parsing, selecting, choosing, establishing, etc.

As used herein, the term "circuitry" may refer to one or more or all of the following:

(a) Only hardware circuitry (e.g., only in analog and/or digital circuitry), and (b) a combination of hardware circuitry and software, e.g., as applicable:

(i) Combination of analog and/or digital hardware circuitry and software/firmware

(ii) Any portion of a hardware processor (including digital signal processors), software, and memory having software that co-operate to cause a device, such as a mobile phone or server, to perform various functions, and (c) hardware circuitry and/or a portion of a processor, such as a microprocessor or microprocessor, requires software (such as firmware) to operate, but software may not be present when software is not required to operate.

This definition of circuit applies to all uses of this term in this application, including in any claims. As another example, as used in this application, the term circuitry also encompasses an implementation of only hardware circuitry or processor (or multiple processors) or a portion of hardware circuitry or processor and its (or their) accompanying software and/or firmware. For example, if applicable to particular claim elements, the term "circuitry" also encompasses baseband integrated circuits or processor integrated circuits of a mobile device or server, a cellular network device, or similar integrated circuits in other computing or network devices.

As used herein, the term "communication network" refers to a network that conforms to any suitable communication standard, such as Long Term Evolution (LTE), LTE-advanced (LTE-a), wideband Code Division Multiple Access (WCDMA), high Speed Packet Access (HSPA), narrowband internet of things (NB-IoT), and the like. Furthermore, the communication between the terminal device and the network device in the communication network may be performed according to any suitable generation communication protocol, including, but not limited to, a first generation (1G), a second generation (2G), 2.5G, 2.75G, a third generation (3G), a fourth generation (4G), 4.5G, a future fifth generation (5G) communication protocol, and/or any other protocol now known or developed in the future. Embodiments of the present disclosure may be applied to various communication systems. In view of the rapid development of communications, there will of course be future types of communication technologies and systems that may embody the present disclosure. The scope of the present disclosure should not be considered limited to the systems described above.

As used herein, the term "access network device" refers to a node in a communication network via which a terminal device accesses the network and receives services therefrom. A network device may refer to a Base Station (BS) or an Access Point (AP), e.g., a node B (NodeB or NB), an evolved NodeB (eNodeB or eNB), an NR NB (also known as a gNB), a Remote Radio Unit (RRU), a Radio Head (RH), a Remote Radio Head (RRH), a repeater, a low power node (e.g., femto, pico, etc.), depending on the terminology and technology of the application.

The term "terminal device" refers to any end device capable of wireless communication. By way of example and not limitation, a terminal device may also be referred to as a communication device, a User Equipment (UE), a Subscriber Station (SS), a portable subscriber station, a Mobile Station (MS), or an Access Terminal (AT). Terminal devices may include, but are not limited to, mobile phones, cellular phones, smart phones, voice over IP (VoIP) phones, wireless local loop phones, tablet computers, wearable terminal devices, personal Digital Assistants (PDAs), laptops, desktops, image capture terminal devices (e.g., digital cameras), gaming terminal devices, music storage and playback devices, in-vehicle wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded devices (LEEs), laptop-installed devices (LMEs), USB dongles, smart devices, wireless customer premise devices (CPE), internet of things (loT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., tele-surgery), industrial devices and applications (e.g., robots and/or other wireless devices operating in an industrial and/or automated processing chain environment), consumer electronic devices, devices operating on a commercial and/or industrial wireless network, and the like. In the following description, the terms "terminal device", "communication device", "terminal", "user equipment" and "UE" are used interchangeably.

The term "MS device" refers to any device or entity that provides management functions in the Core Network (CN). By way of example and not limitation, the MS device may be a device providing operation, administration, and maintenance functions (also referred to as an "OAM device"), a Home Subscriber Server (HSS), a Mobility Management Entity (MME), an access and mobility management function (AMF), a Session Management Function (SMF), a User Plane Function (UPF), and the like. In other embodiments, the MS device may be any other suitable device or entity.

Although in various example embodiments, the functionality described herein may be performed in fixed and/or wireless network nodes, in other example embodiments, the functionality may be implemented in a user equipment device (such as a cell phone or tablet or laptop or desktop or mobile IoT device or fixed IoT device). For example, the user equipment device may be equipped with the respective functions described in connection with the fixed and/or wireless network nodes as appropriate. The user equipment device may be a user equipment and/or a control device, such as a chipset or processor, which when installed therein is configured to control the user equipment. Examples of such functions include a bootstrapping server function and/or a home user server, which may be implemented in a user equipment device by providing the user equipment device with software configured to cause the user equipment device to perform from the perspective of these functions/nodes.

Work Item (WI) was developed in the third generation partnership project (3 GPP) for QoE measurements in New Radios (NR). Specific targets for WI are as follows:

specify support for QoE measurement collection in NR independent mode.

As defined in the 3GPP standard of TR 38.890, the configuration, activation, and deactivation procedures are specified for signaling-based and management-based QoE measurement collection and reporting, referenced to the LTE QoE solution.

Configuration and reporting for multiple simultaneous QoE measurements at the UE is specified.

QoE measurement processing in the event of RAN overload is specified, including suspension and resumption of QoE measurement reporting.

QoE measurement processing at rrc_inactive is specified, i.e., upon transition from rrc_inactive to rrc_connected, qoE measurement configuration is maintained without measurement and the same configuration is reused.

Support QoE measurement collection and reporting continuity in intra-RAT mobile scenarios within signaling-based QoE systems.

To support RAN visible QoE, a set of initially related RAN visible QoE parameters is evaluated and specified, followed by configuration and reporting.

Specify support for QoE measurements per slice.

Specify the necessary mechanisms to support the consistency of radio related measurements (i.e. minimization of drive tests, MDT) and QoE measurements.

Referring to fig. 1A, a conventional signaling flow of QMC activation and reporting procedure 100 in LTE is illustrated. The process 100 includes the following operations:

1. the Network Manager (NM) transmits activatearea qmcjob to the domain management/element management (DM/EM) of the controlling eNB, and the activatearea qmcjob includes the following parameters: service type, area scope, qoE CE address, PLMN target, qoE reference, and QMC profile.

dm/EM forwards activatearea qmcjob to eNB and activatearea qmcjob includes the following parameters: service type, area coverage, qoE CE address, PLMN target, qoE reference, and QMC profile.

The eNB finds a UE matching the criteria.

The enb sends an RRCConnectionReconfiguration message to the UE, the RRCConnectionReconfiguration message comprising: service type, qoE reference, and QMC profile.

The access layer in the ue sends an AT command + CAPPLEVMC, AT command + CAPPLEVMC to the application layer including the following: service type, qoE reference, and QMC profile.

6. An application at the UE is started and QoE measurement collection is initiated.

The APP layer of the UE sends an AT command + CAPPLEVMR, AT command + CAPPLEVMR to the access layer of the UE including a record session indication.

The ue sends a message MeasurementReport including a record session indication to the eNB.

The enb sends a notification to the NM.

10. When QoE measurement collection is complete, the application layer sends an AT command + CAPPLEVMR including QoE reference and reporting to the access layer. The report includes QoE reference and a recording session indication.

The ue sends a message MeasurementReport to the eNB, which includes QoE references and reports.

The enb sends a report to the MCE associated with the QoE reference. The report includes QoE reference and a recording session indication.

According to conventional solutions, the NM may inform/indicate the object to be measured through a QMC profile (e.g., activatearea qmcjob). Furthermore, according to conventional solutions, each activatearea qmcjob may include only one QoE CE address and one QoE reference when configuring/activating QoE for the terminal device, which means that one QoE reference may correspond to only one MCE. Furthermore, in reporting the measurement results, a MeasurementReport may only include a QoE reference (associated with an MCE), which means that a MeasurementReport may only be sent to an MCE.

In release 17 of 3GPP, in 5G, there will be analysis, assurance and other automation functions using QMC mechanisms, furthermore, these automation functions may need to collect different QoE measurements from the same UE. It is therefore proposed to have more than one QMC for one specific service type configured for one terminal device.

According to the conventional solution illustrated in fig. 1A, when different MCEs (e.g. different auto functions) require multiple QMCs, measurement results need to be sent to the different MCEs. Specifically, multiple QMCs of one service type may be configured to one gNB and then to a certain UE in a cell. All configured QMCs are performed in the application level of the UE, and the measurement result of each configured QMC is reported to the gNB via the QoE reporting container together with the corresponding QoE reference ID and service type. The gNB will forward each received QoE reporting container to one of the different MCEs according to the QoE reference ID attached to the container.

Furthermore, to support multiple MCEs served by one or more operators, an operator's MS may configure multiple QMCs for one UE for one service type to support different QoE metric requirements for different MCEs. Referring to fig. 1B, an example conventional format 150 of a QMC profile is illustrated.

As illustrated in fig. 1B, when multiple QMCs are configured, these QMCs may have overlapping measurement metrics/common measurement metrics for a certain service type, according to the conventional format 150 of the QMC profile.

A specific example is described by referring to fig. 1B. In a specific example, for Multimedia Telephony Service (MTSI) service types of IMS, the UE is configured to measure QoE metrics of "voice" and "video" (i.e., indicated in QoE profile 1), and the measurement result (i.e., qoE reporting container 1) is configured to be reported to the first MCE (i.e., MCE 1). Furthermore, the UE is also configured to measure metrics of "video" and "text" (i.e. indicated in QoE profile 2), the measurement result (i.e. QoE reporting container 2) being configured to be reported to the second MCE (i.e. MCE 2). Thus, the UE is configured to measure the "video" metric twice. According to conventional solutions, the measurement results of the "video" will be reported twice to the gNB by means of two reporting containers, which will result in unexpected and unwanted waste of resources in the air interface.

The specific examples described above are discussed for illustrative purposes only. In an actual communication scenario, other metrics may be configured for the UE in addition to "text", "video", and "voice". In addition, as illustrated in FIG. 1B, more than one metric may be further configured for any of "text," video, "and" voice. Thus, when multiple MCEs are enabled, a common scenario is that metrics indicated by some MCEs overlap at least in part with metrics indicated by other MCEs. In addition, qoE measurement report data is sent to the gNB over low priority SRBs, compared to normal RRC signaling. The QoE measurement data in the uplink should therefore be optimized to avoid unnecessary and repeated data transmissions in the air interface, in order to save resources as much as possible.

In view of the above, improvements and optimizations to the process of configuring QoE measurements and reporting QoE measurement results are desired.

According to some example embodiments of the present disclosure, an improved solution for QoE measurement is provided. In this solution, avoiding unnecessary data redundancy transmission of measurement results in the air interface may be implemented at the first device (e.g. MS device) or the second device (e.g. terminal device). In particular, in the event that the first device determines that there is at least one overlapping metric/common metric, the first device may generate at least one measurement configuration according to some example embodiments of the present disclosure, wherein a measurement configuration of the at least one measurement configuration comprises the at least one overlapping metric. In this way, overlapping metrics/common metrics are separately indicated, thereby avoiding unnecessary and repetitive data transmissions. Alternatively, in the event that the second device determines that there is at least one overlapping metric/common metric, the second device generates at least one measurement report in accordance with some example embodiments of the present disclosure when reporting the measurement results, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to the at least one overlapping metric. In this way, the measurement results corresponding to the overlapping metrics can be sent once, avoiding unnecessary and repetitive data transmissions. The procedure of the above proposal will be discussed in detail below.

Fig. 2 illustrates an example communication network 200 in which example embodiments of the present disclosure may be implemented.

The communication network 200 includes a first device 210, a second device 220, a third device 230, and data collectors 240-1 and 240-2 (hereinafter collectively referred to as data collectors 240 or individually referred to as data collectors 240). Furthermore, in the following text, data collector 240-1 is also referred to as a first data collector 240-1, and data collector 240-2 is also referred to as a second data collector 240-2.

In the communication environment 200, the first device 210 may communicate with the second device 220 and the third device 230 via physical communication channels or links.

In the example of fig. 2, the first device 210 is illustrated as an MS device, the second device 220 is illustrated as a terminal device, and the third device 230 is illustrated as an access network device serving the second device 220.

In the specific example of fig. 2, the first data collector 240-1 or the second data collector 240-2 may send a measurement requirement to the first device 210, wherein the measurement requirement includes at least one metric. In one example, the first device 210 provides a common interface and either the first data collector 240-1 or the second data collector 240-2 invokes the common interface to create the corresponding measurement requirements. In another example, the first data collector 240-1 or the second data collector 240-2 sends a message with a predefined structure to the first device 210 to indicate the corresponding measurement requirements.

In some example embodiments, each measurement requirement may include information for the data collector 240 in addition to the metrics. One example of information for the data collector 240 is an identifier for the data collector 240. Another example of information for the data collector 240 is an Internet Protocol (IP) address or a Media Access Control (MAC) address of the data collector 240. In other examples, the information of the data collector 240 may be any identification that may be used to identify the data collector 240.

In some example embodiments, the first device 210 may generate a measurement configuration based on the measurement requirements and then send the generated measurement configuration to the third device 230.

The third device 230 may forward the measurement configuration to the second device 220. The second device 220 (i.e., the APP level of the second device 220) then performs the corresponding QoE measurements. The QoE measurements are sent to the third device 230 and then forwarded to the corresponding data collector 240.

It should also be understood that the number of first devices, second devices, third devices, and data collectors are for illustration purposes only and are not meant to be limiting in any way. The communication environment 200 may include any suitable number of first devices, second devices, third devices, and data collectors suitable for implementing embodiments of the present disclosure.

The principles and implementations of the present disclosure will be described in detail below with reference to fig. 3 and 4.

In the following text, an example procedure will be discussed for two measurement requirements, namely a first measurement requirement and a second measurement requirement. It should be understood that the specific number of measurement requirements (i.e., two) is for illustration purposes only and does not imply any limitation. Indeed, the example processes discussed herein may be adapted to scenes with any suitable number of measurement requirements through adaptive tuning.

Similarly, the example procedure discussed for the two measurement configurations (i.e., the fourth measurement configuration and the fifth measurement configuration) is also for illustration purposes and does not imply any limitation. Indeed, the example processes discussed herein may be adapted to a scenario with any suitable number of measurement configurations through adaptive tuning.

As discussed above, unnecessary and redundant measurement result transmissions in the air interface may be avoided at the first device 210 (e.g., MS device) by an improved configuration procedure or at the second device 220 (e.g., terminal device) by an improved reporting procedure. Both of these are discussed in detail separately in the text below.

It should be understood that the IEs/parameters described below are for illustration purposes only and are not limiting. In some other embodiments, when any Information Element (IE)/parameter is preconfigured or otherwise available, the IE/parameter may not exist, and some additional parameters may be added due to other specific scenarios or requirements.

It should be understood that the specific values of the IEs/parameters (e.g., type of service, metrics, session identification, measurement configuration, measurement results, etc.) illustrated in the following specific examples are for illustration purposes only and do not imply any limitation. In other example embodiments, the (IE)/parameter may be any suitable value.

In some example embodiments, when the third device 230 receives the first message including information such as the measurement configuration, the data collector 240, and the session identification, the third device 230 will generate a configuration message based on the first message and send the configuration message to the second device 220. In some example embodiments, the third device 230 indicates information associated with the session identification to the second device 220. In one example, the third device 230 sends the session identification directly to the second device 220. In another example, the third device 230 will configure one or more identities/identities (e.g., RRC IDs, etc.) corresponding to the session identity/data collector 240 and send the one or more identities to the second device 220 instead of the session identity. Thus, the second device 220 attaches one or more identities to the measurement report instead of the session identity. It should be appreciated that the identification may be used to identify the session and may be represented in any suitable manner.

In addition, the third device 230 may also create a correspondence or association between one or more identities and the data collectors 240/session identities, such that upon receiving the measurement report message, the third device 230 may send a measurement report to the respective data collectors 240 based on the created correspondence.

Thus, for communication between the second device 220 and the third device 230, the session identification and the one or more identities are identical to each other. In the following text, the example in which the third device 230 sends the session identification in the configuration message is for illustration purposes only. It should be appreciated that the session identification communicated between the second device 220 and the third device 230 may be equivalently replaced with one or more identifications. The same or similar descriptions are omitted herein for brevity.

Example procedure for improved configuration procedure

The principles and implementations of the improved configuration procedure of the present disclosure will be described in detail below with reference to fig. 3, fig. 3 illustrating a signaling diagram of an example procedure 300 for QoE measurement according to some embodiments of the present disclosure.

For discussion purposes, the example process 300 will be described with reference to FIG. 2. The example process 300 may involve a first device 210, a second device 220, a third device 230, a first data collector 240-1, and a second data collector 240-2.

In operation, the first device 210 may obtain 305 a first measurement requirement of QoE at the second device 220 from the first data collector 240-1, wherein the first measurement requirement comprises at least one metric. Furthermore, as illustrated in fig. 3, the first device 210 further obtains 320 a second measurement requirement of QoE at the second device 220 from the second data collector 240-2, wherein the second measurement requirement comprises at least one metric. In one example, the first device 210 obtains the first measurement requirement and the second measurement requirement by receiving respective messages from the first data collector 240-1 and the second data collector 240-2. In another example, the first device 210 provides a common interface and the first data collector 240-1 and the second data collector 240-2 call the common interface to create the corresponding measurement requirements.

In addition to the metrics, some IEs/parameters may be included in the first and second measurement requirements. One example of an IE/parameter is a service type (e.g., dynamic adaptive streaming over hypertext transfer protocol (HTTP), DASH). Another example of an IE/parameter is information of the data collector 240 (e.g., qoE CE address). As discussed, the information of the data collector 240 may be an identifier, an IP address, a MAC address, and any suitable identification that may be used to identify the data collector 240.

Two specific examples of a part of the first measurement requirement and the second measurement requirement are explained below.

Specific examples of a portion of the first measurement requirement

QoE data request

-Service Type(DASH)

-QoE CE Address(MCE2)

-QoE metrics(Video,Text)

Specific examples of a portion of the second measurement requirement

QoE data request

-Service Type(DASH)

-QoE CE Address(MCE2)

-QoE metrics(Video,Text)

After obtaining the first measurement requirement and the second measurement requirement, the first device 210 may determine whether at least one metric included in the first measurement requirement at least partially overlaps (including fully overlaps or partially overlaps) with at least one metric included in the second measurement requirement. In other words, the first device 210 may determine whether a common metric is included in the first measurement requirement and the second measurement requirement. Further, in some embodiments, the first device 210 performs the determining operation only if the first measurement requirement and the second measurement requirement comprise the same service type value. In this way, since different service types generally correspond to different metrics, overlapping metrics/common metrics can be quickly determined without introducing a significant amount of additional computation.

In some example embodiments, if there is no overlapping metric or common metric, the first device 210 generates two measurement configurations corresponding to the first measurement requirement and the second measurement requirement, respectively.

In some example embodiments, the first device 210 generates 325 at least one measurement configuration based on the first measurement requirement and the second measurement requirement if at least one metric included in the first measurement requirement at least partially overlaps with at least one metric included in the second measurement requirement. In particular, a measurement configuration of the at least one measurement configuration comprises at least one overlapping metric.

In some example embodiments, three measurement configurations may be generated for the first measurement requirement and the second measurement requirement. In particular, the first device 210 may generate a first measurement configuration indicating metrics that are included in the first measurement requirement but not in the second measurement requirement; a second measurement configuration indicating metrics included in the second measurement requirement but not in the first measurement requirement; and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

Unlike conventional solutions where the first device 210 generates two measurement configurations corresponding to the first and second measurement requirements, respectively, the first device 210 reconstructs (restructures) the first and second measurement requirements according to the present disclosure such that each metric is included/indicated only once.

Furthermore, the overlapping relationship of the metrics may be varied, such that some of the above-described first, second and third measurement configurations are not generated. In particular, in some scenarios, the first measurement configuration or the second measurement configuration is not generated. Some example embodiments of such a scenario are discussed below.

In some example embodiments, the first device 210 needs to generate the second measurement configuration and the third measurement configuration if the at least one metric included in the first measurement requirement is a subset of the at least one metric included in the second measurement requirement.

Alternatively, in some example embodiments, the first device 210 needs to generate the first measurement configuration and the third measurement configuration if the at least one metric included in the second measurement requirement is a subset of the at least one metric included in the first measurement requirement.

Alternatively, in some example embodiments, if at least one metric included in the first measurement requirement is the same as at least one metric included in the second measurement requirement, the first device 210 need only generate a third configuration, the third configuration indicating that the metrics are included in both the first measurement requirement and the second measurement requirement.

Further, in some example embodiments, each measurement configuration may correspond to a respective measurement session. During a measurement session, a measurement configuration may be configured and measurement results associated with the measurement configuration may be reported.

In some example embodiments, the session identification may be determined/assigned by the first device 210. In the specific example described above, the first session identification (denoted "ID 3-1") is determined by the first device 210 to be assigned to the measurement session corresponding to the first measurement configuration. Similarly, a second session identification (denoted "ID 3-2") is determined by the first device 210 to be assigned to the measurement session corresponding to the second measurement configuration, and a third session identification (denoted "ID 3-c") is determined by the first device 210 to be assigned to the measurement session corresponding to the third measurement configuration.

The first device 210 may then send at least one first message to activate the respective generated measurement configuration (e.g., the first configuration, the second configuration, and the third configuration).

In some example embodiments, the generated measurement configuration is sent via a container included in the first message. That is, the measurement configuration is transparent to the third device 230.

In some example embodiments, the first message includes some other IEs/parameters in addition to the measurement configuration. Examples of other IEs/parameters include, but are not limited to, information of the service type and at least one data collector 240 corresponding to the corresponding measurement configuration, which is obtained/derived from the measurement requirements.

Further, in some example embodiments, the first message further includes a session identification allocated for the measurement session corresponding to the respective measurement configuration.

In some example embodiments, the first message for activating the third measurement configuration indicates: the first measurement configuration, information of the first collector 240-1, and the first session identification. Similarly, the first message for activating the second measurement configuration indicates: the second measurement configuration, information of the second collector 240-2, and the second session identification, and the first message for activating the third measurement configuration indicates: the third measurement configuration, information of both the first data collector 240-1 and the second collector 240-2, and a third session identification.

Three specific examples of a portion of the first message corresponding to the first measurement configuration, the second measurement configuration, and the third measurement configuration are described below.

Specific example of a part of the first message corresponding to the first measurement requirement activateAreaQMCJob

-Service Type(DASH)

-QoE CE Address(MCE1)

-QoEreference ID(ID3-1)

-Container of QoE Config (specific) specific example of a part of the first message corresponding to the second measurement requirement

-Service Type(DASH)

-QoE CE Address(MCE1)

-QoE reference ID(ID3-2)

-Container of QoE Config(Text)

Specific example of a part of the first message corresponding to the third measurement requirement

-Service Type(DASH)

-QoE CE Address(MCE1,MCE2)

-QoE reference ID(ID3-c)

-Container of QoE Config(Text)

The generated first message is then sent to the third device 230. In the specific example of fig. 3, the first device 210 sends 340-3 a first message to the third device 230 corresponding to the third measurement configuration. Further, if a first measurement configuration is generated, the first device 210 will send 340-1 a first message corresponding to the first measurement configuration to the third device 230, and if a second measurement configuration is generated, the first device 210 will send 340-2 a first message corresponding to the second measurement configuration to the third device 230.

Furthermore, in an actual communication scenario, the first measurement requirement and the second measurement requirement are typically obtained at different points in time. That is, the first device 210 may have processed the first measurement requirement when the second measurement requirement is obtained. For example, the first device 210 may have generated a corresponding first message to activate the first measurement requirement. As a specific example, the first device 210 has sent 310 a first message (e.g., activateAreaQMCJob) to the third device 230, the first message including the following parameters: containers for type of service (DASH), qoE CE address (MCE 1), qoE reference (ID 1) and QMC profiles (video, voice). In addition, the third device 230 may have processed the first message, e.g., a rrcrecon configuration message has been sent 315 to the second device 220, and the rrcrecon configuration message includes the following: containers for service type (DASH), qoE reference (ID 1) and QMC profiles (video, voice).

In case the first device 210 has activated the first measurement requirement, some additional operations need to be performed. Specifically, the first device 210 needs to send 330 a second message to deactivate the first measurement requirement.

As a specific example, the first device 210 sends a second message (e.g., dectivantearea qmcjob) to the third device 230, the second message including the following parameters: type of service (DASH) and QoE reference (ID 1). Similarly, in the case where the third device 230 has configured the first measurement requirement (e.g., has sent the rrcrecon configuration message described above) to the second device 220, the third device 230 also needs to perform some additional operations. Specifically, the third device 230 needs to send 335 a rrcreconditiona message to the second device 220, the rrcreconditiona message comprising: type of service (DASH), qoE reference (ID 1), and other configurations (measconfigapplayer=discard). When the second device 220 receives the rrcrecon configuration, the second device 220 may terminate the QoE measurement accordingly.

In this way, an improved solution for handling overlapping metrics/common metrics may continue to be performed even if the first device 210 has activated the first measurement requirement.

With continued reference to fig. 3, upon receipt of the corresponding first message, the third device 230 stores the session identification (e.g., qoE reference ID), information of the data collector 240, and optionally other visible QMC parameters. Further, the third device 230 may establish a correspondence or association between the session identification and the information of the data collector 240. The third device 230 then forwards the configuration message to the second device 220 based on the received first message. For example, the third device 230 sends a rrcrecon configuration message to the second device 220, the rrcrecon configuration message including: a container of service type, qoE reference ID and QoE configuration, which are included in the first message.

In addition, the third device 230 may also configure one or more identities corresponding to the session identification/data collector 240. In this case, the third device 230 may send the identity instead of the session identity to the second device 220. As a specific example, the third device 230 sends RRC signaling for QoE measurement in rrc_connected, wherein an identity (e.g. RRC ID or MeasConfigAppLayerId) is used to identify the QoE configuration between the second device 220 and the third device 230.

More specifically, the third device 230 configures a first identity/identifier corresponding to the first session identifier, configures a second identity/identifier corresponding to the second session identifier, and simultaneously configures two identities (referred to as a third identity and a fourth identity)/identifiers corresponding to the third session identifier.

In the event that the third device 230 receives the first message corresponding to the first measurement configuration, the third device may create 345-1 a correspondence between the first session identification (or first identity) and the information of the first data collector 240-1. Further, the third device 230 sends 350-1 RRCRECONfigure message to the second device 220, the RRCRECONfigure message comprising: type of service (DASH), qoE reference ID (ID 3-1) (or first identity), and container (voice) including QoE configuration.

Similarly, in the event that the third device 230 receives the first message corresponding to the second measurement configuration, the third device 230 may create 345-2 a correspondence between the second session identification (or second identity) and information of the second data collector 240-3. Further, the third device 230 sends 350-2RRCeconfiguration message to the second device 220, the RRCeconfiguration message comprising: type of service (DASH), qoE reference ID (ID 3-2) (or second identity), and container (text) including QoE configuration.

Finally, in the event that the third device 230 receives the first message corresponding to the third measurement configuration, the third device 230 may create 345-3 a correspondence between the third session identification and the information of the first data collector 240-1 and the second data collector 240-2. In addition, the third device 230 sends 350-3rrcrecon configuration message to the second device 220, the rrceconconfiguration message including: service type (DASH), qoE reference ID (ID 3-c), and container (video) including QoE configuration.

Alternatively, when identities are used for communication between the second device 220 and the third device 230, there may be two identities (i.e. a third identity and a fourth identity) associated with the third session identity. The third device 230 may create a correspondence between the third and fourth identities and the information of the first and second data collectors 240. For example, a correspondence is created between the third identity and the information of the first data collector 240-1, and a correspondence is created between the fourth identity and the information of the second data collector 240-2. Thereafter, the third device 230 sends 350-3rrcrecon configuration message to the second device 220, the rrcrecon configuration message including the following: type of service (DASH), qoE reference ID (ID 3-c) (or third and fourth identities) and container (video) including QoE configuration.

Thus, one session identification may correspond to more than one data collector.

Upon receiving the corresponding rrcrecon configuration message, the second device 220 may perform a corresponding QoE measurement and obtain a measurement result. The measurement results (i.e., measurement reports) may be sent via the container to the third device 230. In particular, the second device 220 may generate a measurement report message comprising a session identification (e.g., qoE reference ID) and a measurement report (e.g., a container of reports).

Then, upon receiving the measurement report message including the session identification, the third device 230 may transmit a measurement report to the corresponding data collector based on the correspondence between the created session identification and the information of the data collector 240.

As illustrated in fig. 3, if a measurement report message (e.g., measReportAppLayer, including a first session identification (e.g., qoE reference ID (ID 3-1)) (or a first identification identifying the first session identification)) and a measurement report (e.g., a container (voice) of the report)) are sent 355-1 to the third device 230, the third device 230 determines 360-1 that the first session identification (or the first identification identifying the first session identification) corresponds to the first data collector 240-1 and forwards 365-1 the measurement report to the first data collector 240-1 accordingly.

Furthermore, if a measurement report message (e.g., measreportapplyer) including a second session identification (e.g., qoE reference ID (ID 3-2)) (or a second identification identifying the second session identification) and a measurement report (e.g., a container (text) of the report) are sent 355-2 to the third device 230, the third device 230 determines 360-2 that the second session identification (or the second identification identifying the second session identification) corresponds to the second data collector 240-2 and forwards 365-2 the measurement report to the second data collector 240-2 accordingly.

Furthermore, if a measurement report message (e.g., measreportapplyer) including a third session identification (e.g., qoE reference ID (ID 3-c)) or an identification list corresponding to the third session identification (e.g., qoE reference IDs (ID 3-1 and ID 3-2), or identifications corresponding to the first and second identifications) and a measurement report (e.g., a container (video) of the report) is sent 355-3 to the third device 230, the third device 230 determines 360-3 that the third session identification corresponds to both the first and second data collectors 240-1, 240-2, and forwards 365-3-2 the measurement report to the first and second data collectors 240-1, 240-2, respectively.

In this way, the measurement results (corresponding to the overlapping metrics/common metrics) are sent only once and air interface resources can be saved. Further, by optimizing the QoE profile at the first device 210 (e.g., MS), additional enhancements to the third device 230 (e.g., network access device) may be minimized. In particular, no additional enhancements are required for the second device 220 (e.g., a terminal device).

Example procedure for improved reporting procedure

The principles and implementation of the improved reporting procedure of the present disclosure will be described in detail below with reference to fig. 4, which illustrates a signaling diagram of an example procedure 400 for QoE measurement according to some embodiments of the present disclosure.

For discussion purposes, the example process 400 will be described with reference to fig. 2. The example process 400 may involve the first device 210, the second device 220, the third device 230, the first data collector 240-1, and the second data collector 240-2.

It should be appreciated that the process illustrated in fig. 4 (e.g., obtaining measurement requirements, generating a first message corresponding to a respective measurement configuration, sending the first message, creating a correspondence or association between session identification and data collector 240-2, sending a configuration message) is similar to the process discussed with respect to fig. 3. The same or similar descriptions are omitted herein for brevity.

In operation, the second device 220 receives 420 a fourth measurement configuration indicating at least one metric for QoE at the second device 220. Further, the second device 220 receives 440 a fifth measurement configuration indicative of the at least one metric. In the specific example of fig. 4, the fourth measurement configuration and the fifth measurement configuration are configured for one and the same service type and correspond to different session identifications. In the following text, the session identifier corresponding to the fourth configuration is referred to as a first session identifier (denoted as "ID 1"), and the session identifier corresponding to the fifth configuration is referred to as a second session identifier (denoted as "ID 2").

As a specific example, the fourth measurement configuration includes metrics for speech and video, while the fourth measurement configuration includes metrics for text and video.

In some embodiments, the fourth measurement configuration and the fifth measurement configuration may be included in different configuration messages (e.g., rrcrecon configuration messages) as IEs/parameters (e.g., containers of QoE configurations) sent from the third device 230.

Two specific examples of a part of the configuration message corresponding to the fourth measurement configuration and the fifth measurement configuration are explained as follows.

Specific example of a portion of the configuration message corresponding to the fourth measurement configuration

RRCReconfiguration

-Service Type(DASH)

-QoEreference ID(ID1)

-Container of QoE Config(Video,Speech)

Specific example of a part of the configuration message corresponding to the fifth measurement configuration

RRCReconfiguration

-Service Type(DASH)

-QoEreference ID(ID1)

-Container of QoE Config(Video,Text)

Further, the fourth measurement configuration and the fifth measurement configuration may result from two measurement requirements from different data collectors 240. Specifically, as illustrated in FIG. 4, the first device 210 obtains 405 a first measurement requirement from the first data collector 240-1 and further obtains 425 a second measurement requirement from the second data collector 240-2. The first device 210 then generates two measurement configurations (i.e., a fourth measurement configuration and a fifth measurement configuration) based on the first measurement requirement and the second measurement requirement, respectively.

Further, the first device 210 sends 410 a first message to the third device 230 for activating the fourth measurement configuration, wherein the first message indicates: the fourth measurement configuration, information of the first collector 240-1, and the first session identification. The third device 230 creates 415 a correspondence between the first session identification and the information of the first collector 240-1.

Similarly, the first device 210 sends 430 a first message to the third device 230 for activating the fifth measurement configuration, wherein the first message indicates: the fifth measurement configuration, information of the second collector 240-2, and the second session identification. The third device 230 creates 435 a correspondence between the second session identification and the information of the second collector 240-2.

After obtaining the fourth measurement configuration and the fifth measurement configuration, the second device 220 (e.g., an application level of the second device 220) may determine whether at least one metric included in the at least one metric indicated by the fourth measurement configuration at least partially overlaps (including fully overlaps and partially overlaps) with the at least one metric indicated by the fifth measurement configuration. In other words, the second device 220 may determine whether there is a common metric indicated by both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if there is no overlapping metric/common metric, the second device 220 generates QoE reporting containers according to the fourth and fifth measurement configurations, respectively, and the QoE reporting containers have corresponding session identifications, respectively. Specifically, the QoE reporting container corresponding to the fourth device is accompanied by a first session identification, and the QoE reporting container corresponding to the fifth device is accompanied by a second session identification.

In some example embodiments, the third device 230 generates at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration if at least one metric included in the at least one metric indicated by the fourth measurement configuration at least partially overlaps with at least one metric indicated by the fifth measurement configuration. In particular, the measurement reports of the at least one measurement report comprise measurement reports corresponding to the at least one overlapping metric/common metric.

In some example embodiments, three measurement reports may be generated for the fourth measurement configuration and the fifth measurement configuration. In particular, the third device 230 may generate 450 a first measurement report comprising measurement results corresponding to the metrics comprised in the fourth measurement configuration and not to the metrics comprised in the fifth measurement configuration; a second measurement report including measurement results corresponding to metrics included in the fifth measurement configuration but not to metrics included in the fourth configuration requirement; and a third measurement report including measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

Furthermore, the overlapping relationship of the metrics may be varied, which may allow for some of the above-described first, second, and third measurement reports to not be generated, as discussed below. In particular, in some scenarios, the first measurement report or the second measurement report is not generated. Some example embodiments for such a scenario are discussed below.

In some example embodiments, if the at least one metric of the fourth measurement configuration indication is a subset of the at least one metric of the fifth measurement configuration indication, the second device 220 performs the measurement based on the fifth measurement configuration and further generates a second measurement report and a third measurement report.

Alternatively, in some example embodiments, if the at least one metric of the fifth measurement configuration indication is a subset of the at least one metric of the fourth measurement configuration indication, the second device 220 performs the measurement based on the fourth measurement configuration and further generates the first measurement report and the third measurement report.

Alternatively, in some example embodiments, if the at least one metric indicated by the fifth measurement configuration is the same as the at least one metric indicated by the fifth measurement configuration, the second device 220 performs the measurement based on the fourth measurement configuration or the fifth measurement configuration and further generates only the third measurement report.

In some example embodiments, the generated measurement report may be sent via a container included in the measurement report message. That is, the measurement report is transparent to the third device 230.

In some example embodiments, the measurement report message further comprises at least one identification in addition to the measurement reports, wherein the at least one session identification corresponds to each measurement report and is used to identify the at least one corresponding session identification. More specifically, the first measurement report corresponds to a first session identity, the second measurement report corresponds to a second session identity, and the third measurement report corresponds to both the first and second session identities.

Three specific examples of a part of the measurement report message corresponding to the first measurement report, the second measurement report, and the third measurement report are explained as follows.

Specific example of a portion of a measurement report message corresponding to a first measurement report

MeasReportAppLayer

-QoEreference ID(ID1)

-Container of Report(Speech)

Specific example of a part of the measurement report message corresponding to the second measurement report

MeasReportAppLayer

-QoEreference ID(ID2)

-Container of Report(Text)

Specific example of a part of the measurement report message corresponding to the third measurement report

MeasReportAppLayer

-QoEreference ID(ID1,ID2)

-Container of Report(Video)

The generated measurement report message is then transmitted to the third device 230. In the specific example of fig. 4, the second device 220 transmits 455-3 a measurement report message corresponding to the third measurement report to the third device 230. Further, if the first measurement report is generated, the second device 220 transmits 455-2 a measurement report message corresponding to the first measurement report to the third device 230, and if the second measurement report is generated, the first device 210 transmits 455-3 a measurement report message corresponding to the third measurement report to the third device 230.

Upon receiving the measurement report message, the third device 230 may send a measurement report to the data collector based on the correspondence between the created session identification and the information of the data collector 240.

As illustrated in fig. 4, if a measurement report message, such as MeasReportAppLayer, including a first session identification (e.g., qoE reference ID (ID 1)) and a first measurement report (e.g., container of report (voice)), is sent 455-1 to the third device 230, the third device 230 determines 460-1 that the first session identification corresponds to the first data collector 240-1 and forwards 465-1 the first measurement report to the first data collector 240-1 accordingly.

Furthermore, if a measurement report message (e.g., measReportAppLayer including a second session identity (e.g., qoE reference ID (ID 2)) and a second measurement report (e.g., container (text) of the report)) is sent 455-2 to the third device 230, the third device 230 determines 460-2 that the second session identity corresponds to the second data collector 240-2 and forwards 465-2 the second measurement report to the second data collector 240-2 accordingly.

Furthermore, if a measurement report message (e.g., measreportapplyer including the first and second session identifications (e.g., qoE reference IDs (ID 1, ID 2)) and a third measurement report (e.g., measreportapplyer of the container (video) of the report) are sent 455-3 to the third device 230, the third device 230 determines 460-3 that the third measurement report corresponds to both the first data collector 240-1 and the second data collector 240-2, and forwards 465-3-1, 465-3-2 the third measurement report to the first data collector 240-1 and the second data collector 240-2, respectively.

In this way, the measurement results (corresponding to the overlapping metrics/common metrics) are sent only once and air interface resources can be saved.

Furthermore, by filtering QoE measurement data at the second device 220 (e.g., a terminal device), additional enhancements to the third device 230 (e.g., a network access device) may be minimized. In particular, no additional enhancements are required for the first device 210 (e.g., MS).

Fig. 5 shows a flowchart of an example method 500 implemented at the first device 210, according to some example embodiments of the present disclosure. For discussion purposes, the method 500 will be described from the perspective of the first device 210 of fig. 2. It should be understood that method 500 may include additional blocks not shown and/or may omit some of the blocks shown, the scope of the present disclosure being not limited in this respect.

At block 510, the first device 210 obtains a first measurement requirement for QoE measurements at the second device 220 from the first data collector 240-1, the first measurement requirement comprising at least one metric.

At block 530, the first device 210 obtains a second measurement requirement for QoE measurements at the second device 220 from the second data collector 240-2, the second measurement requirement comprising at least one metric.

At block 530, if at least one metric included in the first measurement requirement at least partially overlaps at least one metric included in the second measurement requirement, the first device 210 obtains a generation of at least one measurement configuration based on the first measurement requirement and the second measurement requirement, the measurement configuration in the at least one measurement configuration including the at least one overlapping metric.

In some example embodiments, the first device 210 sends at least one first message to the third device 230 serving the second device 220, each first message for activating a respective measurement configuration of the at least one measurement configuration, and each first message comprises: a respective measurement configuration, a session identification allocated for a measurement session corresponding to the respective measurement configuration, and information of at least one data collector corresponding to the respective measurement configuration.

In some example embodiments, if at least one metric included in the first measurement requirement partially overlaps at least one metric included in the second measurement requirement, the first device 210 generates at least one measurement configuration by generating: a first measurement configuration indicating metrics included in the first measurement requirement but not in the second measurement requirement, a second measurement configuration indicating metrics included in the second measurement requirement but not in the first measurement requirement, and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, if the at least one metric included in the first measurement requirement is a subset of the at least one metric included in the second measurement requirement, the first device 210 generates the at least one measurement configuration by generating: a second measurement configuration indicating metrics included in the second measurement requirement but not in the first measurement requirement, and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, if the at least one metric included in the second measurement requirement is a subset of the at least one metric included in the first measurement requirement, the first device 210 generates the at least one measurement configuration by generating: a first measurement configuration indicating metrics included in the first measurement requirement but not in the second measurement requirement, and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, if at least one metric included in the first measurement requirement is the same as at least one metric included in the second measurement requirement, the first device 210 generates at least one measurement configuration by generating: a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, the first measurement configuration corresponds to the first data collector 240-1, the second measurement configuration corresponds to the second data collector 240-2, and the third measurement configuration corresponds to both the first data collector 240-1 and the second data collector 240-1.

In some example embodiments, the first measurement requirement is to be activated or has been activated by the first device and the second measurement requirement is to be activated by the first device.

In some example embodiments, if the first measurement requirement has been activated by the first device, the first device 210 sends a second message to deactivate the first measurement requirement.

In some example embodiments, the first device 210 is a management system device and the second device 220 is a terminal device.

The first device capable of performing the method 500 may include means for performing the operations of the method 500. The components may be implemented in any suitable form. For example, components may be implemented in circuits or software modules. The first means may be implemented as the first device 210 or comprised in the first device 210.

In some example embodiments, a first apparatus includes means for obtaining, at the first apparatus, a first measurement requirement for QoE measurements at a second apparatus from a first data collector, the first measurement requirement including at least one metric; means for obtaining, from a second data collector, a second measurement requirement for QoE measurements at a second device, the second measurement requirement comprising at least one metric; and means for generating at least one measurement configuration based on the first measurement requirement and the second measurement requirement if the at least one metric included in the first measurement requirement at least partially overlaps with the at least one metric included in the second measurement requirement, the measurement configuration in the at least one measurement configuration including the at least one overlapping metric.

In some example embodiments, the first apparatus further comprises: means for sending at least one first message to a third device serving the second device, each first message for activating a respective measurement configuration of the at least one measurement configuration, and each first message comprising: a respective measurement configuration, a session identification allocated for a measurement session corresponding to the respective measurement configuration, and information of at least one data collector corresponding to the respective measurement configuration.

In some example embodiments, if at least one metric included in the first measurement requirement partially overlaps at least one metric included in the second measurement requirement, the means for generating at least one measurement configuration comprises: means for generating: a first measurement configuration indicating metrics included in the first measurement requirement but not in the second measurement requirement; a second measurement configuration indicating metrics included in the second measurement requirement but not in the first measurement requirement; and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, if the at least one metric included in the first measurement requirement is a subset of the at least one metric included in the second measurement requirement, the means for generating the at least one measurement configuration comprises: the following are generated: a second measurement configuration indicating metrics included in the second measurement requirement but not in the first measurement requirement; and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, if the at least one metric included in the second measurement requirement is a subset of the at least one metric included in the first measurement requirement, the means for generating the at least one measurement configuration comprises: the following are generated: a first measurement configuration indicating metrics included in the first measurement requirement but not in the second measurement requirement; and a third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, if the at least one metric included in the first measurement requirement is the same as the at least one metric included in the second measurement requirement, the means for generating the at least one measurement configuration comprises: a third measurement configuration is generated indicating metrics included in both the first measurement requirement and the second measurement requirement.

In some example embodiments, the first measurement requirement is to be activated or has been activated by the first apparatus and the second measurement requirement is to be activated by the first apparatus.

In some example embodiments, the first apparatus further comprises: means for sending a second message to deactivate the first measurement requirement.

In some example embodiments, the first device is a management system device and the second device is a terminal device.

Fig. 6 shows a flowchart of an example method 600 implemented at the third device 230 according to some example embodiments of the disclosure. For discussion purposes, the method 600 will be described from the perspective of the third device 230 with respect to fig. 2. It should be understood that method 600 may include additional blocks not shown and/or omit some of the blocks shown, the scope of the present disclosure being not limited in this respect.

At block 610, the third device 230 receives a first message from the first device 210, the first message for activating a measurement configuration for QoE measurements at the second device 220, the first message indicating: a measurement configuration, a session identification assigned for a measurement session corresponding to the measurement configuration, and information of the plurality of data collectors 240 corresponding to the measurement configuration.

At block 620, the third device 230 creates a correspondence or association between the session identification and the information of the plurality of data collectors 240.

At block 630, the third device 230 sends a configuration message to the second device 220 based on the received first message.

In some example embodiments, the third device 230 receives a measurement report message from the second device 220, the message comprising: the measurement report and the identification identifying the measurement session, and the measurement report is sent to the plurality of data collectors 240, respectively, based on the correspondence.

In some example embodiments, the first device 210 is a management system device, the second device 220 is a terminal device, and the third device 230 is an access network device.

A third apparatus capable of performing the method 600 may include means for performing the operations of the method 600. These components may be implemented in any suitable form. For example, these components may be implemented in a circuit or software module. The third means may be implemented as the third device 230 or comprised in the third device 230.

In some example embodiments, the third apparatus includes means for receiving, at the third apparatus, a first message from the first apparatus, the first message for activating a measurement configuration for QoE measurements at the second apparatus, the first message indicating: a measurement configuration, a session identification allocated for a measurement session corresponding to the measurement configuration, and information of a plurality of data collectors corresponding to the measurement configuration; means for creating a correspondence between the session identification and information of the plurality of data collectors; and means for sending a configuration message to the second device based on the received first message.

In some example embodiments, the third apparatus further comprises: means for receiving a measurement report message from a second apparatus, the measurement report message comprising: a measurement report, an identification for identifying a measurement session; and means for transmitting measurement reports to the plurality of data collectors, respectively, based on the correspondence.

In some example embodiments, the first device is a management system device, the second device is a terminal device, and the third device is an access network device.

Fig. 7 shows a flowchart of an example method 700 implemented at the second device 220, according to some example embodiments of the present disclosure. For discussion purposes, the method 700 will be described from the perspective of the second device 220 with respect to fig. 2. It should be understood that method 700 may include additional blocks not shown and/or may omit some of the blocks shown, the scope of the present disclosure being not limited in this respect.

At block 710, the second device 220 receives a fourth measurement configuration for QoE measurements at the second device 220 from the third device 230, the fourth measurement requirement indicating at least one metric and corresponding to a first identification for identifying a first session identification;

at block 720, the second device 220 receives a fifth measurement configuration for QoE measurements at the second device 220 from the third device 230, the fifth measurement requirement indicating at least one metric and corresponding to a second identification for identifying a second session identification.

At block 730, if the at least one metric of the fourth measurement configuration indication at least partially overlaps with the at least one metric of the fifth measurement configuration indication, the second device 220 generates at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to the at least one overlapping metric.

In some example embodiments, the second device 220 transmits at least one measurement report message to the third device 230, each measurement report message corresponding to a respective measurement report of the at least one measurement report, and includes: a respective measurement report and at least one identity, wherein the at least one identity corresponds to the respective measurement report and is used to identify the at least one session identity.

In some example embodiments, if the at least one metric of the fourth measurement configuration indication partially overlaps with the at least one metric of the fifth measurement configuration indication, the second device 220 generates at least one measurement report by: performing measurements based on the fourth measurement configuration and the fifth measurement configuration; the following is generated: a first measurement report including measurement results corresponding to metrics included in the fourth measurement configuration but not in the fifth measurement configuration; a second measurement report including measurement results corresponding to metrics included in the fifth measurement configuration but not included in the fourth configuration requirement; and a third measurement report including measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if the at least one metric of the fourth measurement configuration indication is a subset of the at least one metric of the fifth measurement configuration indication, the second device 220 generates at least one measurement report by: performing measurements based on the fifth measurement configuration; the following is generated: the second measurement report includes measurement results corresponding to the metrics included in the fifth measurement configuration but not in the fourth measurement configuration, and the third measurement report includes measurement results corresponding to the metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if the at least one metric of the fifth measurement configuration indication is a subset of the at least one metric of the fourth measurement configuration indication, the at least one memory and the computer program code are configured to, with the at least one processor, cause the second device 220 to generate the at least one measurement report by: performing measurements based on the fourth measurement configuration; the following is generated: a first measurement report including measurement results corresponding to metrics included in the fourth measurement configuration but not included in the second measurement configuration; and a third measurement report including measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if the at least one metric of the fifth measurement configuration indication is the same as the at least one metric of the fifth measurement configuration indication, the second device 220 generates at least one measurement report by: performing measurements based on the fourth measurement configuration or the fifth measurement configuration; and generating a third measurement report comprising measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, the second device 220 sends a measurement report message to the third device 230, the message comprising: a third measurement report, a first identification for identifying the first session identification, and a second identification for identifying the second session identification.

In some example embodiments, the second device 220 is a terminal device and the third device 230 is an access network device.

A second device capable of performing the method 700 may include means for performing the operations of the method 700. The components may be implemented in any suitable form. For example, the components may be implemented in circuitry or software modules. The second means may be implemented as the second device 220 or comprised in the second device 220.

In some example embodiments, the second apparatus includes means for receiving, at the second apparatus, a fourth measurement configuration for QoE measurements at the second apparatus from the third apparatus, the fourth measurement requirement indicating the at least one metric and corresponding to a first identification for identifying the first session identification; means for receiving a fifth measurement configuration for QoE measurements at the second device from the third device, the fifth measurement requirement indicating the at least one metric and corresponding to a second identification for identifying a second session identification; and means for generating at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration if the at least one metric of the fourth measurement configuration indication at least partially overlaps with the at least one metric of the fifth measurement configuration indication, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to the at least one overlapping metric.

In some example embodiments, the second apparatus further comprises: means for transmitting at least one measurement report message to a third apparatus, each measurement report message corresponding to a respective measurement report of the at least one measurement report, and comprising: a respective measurement report and at least one identity, wherein the at least one identity corresponds to the respective measurement report and is used to identify the at least one session identity.

In some example embodiments, if the at least one metric of the fourth measurement configuration indication overlaps in part with the at least one metric of the fifth measurement configuration indication, the means for generating the at least one measurement report comprises: means for performing measurements based on the fourth measurement configuration and the fifth measurement configuration; means for generating: a first measurement report including measurement results corresponding to metrics included in the fourth measurement configuration but not in the fifth measurement configuration; a second measurement report including measurement results corresponding to metrics included in the fifth measurement configuration but not included in the fourth configuration requirement; and a third measurement report including measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if the at least one metric of the fourth measurement configuration indication is a subset of the at least one metric of the fifth measurement configuration indication, the means for generating the at least one measurement report comprises: means for performing a measurement based on the fifth measurement configuration; means for generating: a second measurement report including measurement results corresponding to metrics included in the fifth measurement configuration but not in the fourth measurement configuration; and a third measurement report including measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if the at least one metric of the fifth measurement configuration indication is a subset of the at least one metric of the fourth measurement configuration indication, the means for generating at least one measurement report comprises: means for performing a measurement based on the fourth measurement configuration; means for generating: a first measurement report including measurement results corresponding to metrics included in the fourth measurement configuration but not included in the second measurement configuration; and a third measurement report including measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, if the at least one metric of the fifth measurement configuration indication is the same as the at least one metric of the fifth measurement configuration indication, the means for generating the at least one measurement report comprises: means for performing measurements based on the fourth measurement configuration or the fifth measurement configuration; and means for generating a third measurement report comprising measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

In some example embodiments, the second apparatus further comprises: means for sending a measurement report message to a third apparatus, the measurement report message comprising: a third measurement report, a first identification for identifying the first session identification, and a second identification for identifying the second session identification.

In some example embodiments, the second device is a terminal device and the third device is an access network device.

Fig. 8 shows a flowchart of an example method 800 implemented at the third device 230 according to some example embodiments of the disclosure. For discussion purposes, the method 800 will be described from the perspective of the third device 230 with respect to fig. 2. It should be understood that method 800 may include additional blocks not shown and/or may omit some of the blocks shown, the scope of the present disclosure being not limited in this respect.

At block 810, the third device 230 receives, from the second device 220, a measurement report message for QoE measurements at the second device 220, the measurement report message comprising: measurement reports and a plurality of identities for identifying session identities.

At block 820, the third device 230 sends measurement reports to the respective data collectors 240 based on the correspondence or association between the plurality of identifications identifying the plurality of session identifications and the plurality of identifications of the respective data collectors 240.

A third apparatus capable of performing the method 800 may include means for performing the operations of the method 800. The components may be implemented in any suitable form. For example, the components may be implemented in circuitry or software modules. The third means may be implemented as the third device 230 or comprised in the third device 230.

In some example embodiments, the third apparatus includes means for receiving, at the third apparatus, a measurement report message for QoE measurements at the second apparatus from the second apparatus, the measurement report message including: a measurement report and a plurality of session identifications; and means for sending measurement reports to respective data collectors based on correspondence or association between a plurality of identities identifying a plurality of session identities and a plurality of identities of respective data collectors.

Fig. 9 is a simplified block diagram of an apparatus 900 suitable for use in implementing embodiments of the present disclosure. The device 900 may be provided for implementing communication devices, such as the first device 210, the second device 220, and the third device 230 shown in fig. 2. As shown, the device 900 includes one or more processors 910, one or more memories 920 coupled with the processors 910, and one or more communication modules 940 (e.g., transmitters and/or receivers) coupled with the processors 910.

The communication module 940 is used for two-way communication. The communication module 940 has at least one antenna to facilitate communication. The communication interface may represent any interface required to communicate with other network elements.

The processor 910 may be of any type suitable to the local technical network and may include one or more of the following: general purpose computers, special purpose computers, microprocessors, digital Signal Processors (DSPs) and processors based on a multi-core processor architecture, as non-limiting examples. The device 900 may have multiple processors, such as application specific integrated circuit chips that are temporally slaved to a clock that is synchronized to the master processor.

Memory 920 may include one or more non-volatile memories and one or more volatile memories. Examples of non-volatile memory include, but are not limited to, read Only Memory (ROM) 924, electrically Programmable Read Only Memory (EPROM), flash memory, hard disk, compact Disk (CD), digital Video Disk (DVD), and other magnetic and/or optical storage devices. Examples of volatile memory include, but are not limited to, random Access Memory (RAM) 922 and other volatile memory that does not last for the duration of the power loss.

The computer program 930 includes computer-executable instructions that are executed by the associated processor 910. Program 930 may be stored in ROM 920. Processor 910 may perform any suitable actions and processes by loading program 930 into RAM 920.

Embodiments of the present disclosure may be implemented by the program 930 such that the device 900 may perform any of the processes of the present disclosure, as discussed with reference to fig. 3-8. Embodiments of the present disclosure may also be implemented in hardware or a combination of software and hardware.

In some embodiments, program 930 may be tangibly embodied in a computer-readable medium, which may be embodied in device 900 (e.g., in memory 920) or in another storage device accessible to device 900. Device 900 may load program 930 from a computer-readable medium into RAM 922 for execution. The computer readable medium may include any type of tangible, non-volatile storage, such as ROM, EPROM, flash memory, hard disk, CD, DVD, etc. Fig. 10 shows an example of a computer readable medium 1000 in the form of a CD or DVD. The computer readable medium has a program 930 stored thereon.

In general, the various embodiments of the disclosure may be implemented in hardware or special purpose circuits, software, logic or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device. While various aspects of the embodiments of the disclosure have been illustrated and described in terms of block diagrams, flow charts, or using some other pictorial representation, it is well understood that the blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product comprises computer executable instructions, such as instructions contained in program modules, which are executed in a device on a target physical or virtual processor to perform the methods 500 to 800 described above with reference to fig. 5-8. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. In various embodiments, the functionality of the program modules may be combined or split between program modules as desired. Machine-executable instructions of program modules may be executed within local or distributed devices. In distributed devices, program modules may be located in both local and remote memory storage media.

Program code for carrying out the methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, results in the implementation of the functions/operations specified in the flowchart and/or block diagram. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine, partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, computer program code or related data may be carried by any suitable carrier to enable an apparatus, device, or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, and the like.

The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a computer-readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Moreover, although operations are described in a particular order, this should not be construed as requiring that such operations be performed in the particular order or sequential order shown or that all illustrated operations be performed in order to achieve desirable results. In some cases, multitasking and parallel processing may be advantageous. Also, while the above discussion contains specific implementation details, these should not be construed as limitations on the scope of the disclosure, but rather as descriptions of features that may be specific to particular embodiments. Certain features that are described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the disclosure has been described in language specific to structural features and/or methodological acts, it is to be understood that the disclosure defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A first device, comprising:

At least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the first device to:

obtaining, from a first data collector, a first measurement requirement for a quality of experience measurement at a second device, the first measurement requirement comprising at least one metric;

obtaining, from a second data collector, a second measurement requirement for a quality of experience measurement at the second device, the second measurement requirement comprising at least one metric; and

if the at least one metric included in the first measurement requirement at least partially overlaps with the at least one metric included in the second measurement requirement, at least one measurement configuration is generated based on the first measurement requirement and the second measurement requirement, the measurement configuration of the at least one measurement configuration including at least one overlapping metric.

2. A first device of claim 1, wherein the first device is further caused to:

transmitting at least one first message to a third device serving the second device, each first message being used to activate a respective measurement configuration of the at least one measurement configuration, and each first message comprising:

The corresponding measurement configuration is described as such,

a session identification allocated for a measurement session corresponding to the respective measurement configuration, and

information of at least one data collector corresponding to the respective measurement configuration.

3. The first device of claim 1, wherein if the at least one metric included in the first measurement requirement partially overlaps the at least one metric included in the second measurement requirement, the first device is caused to generate at least one measurement configuration by:

the following were generated:

a first measurement configuration indicating metrics included in the first measurement requirement but not in the second measurement requirement,

a second measurement configuration indicating metrics included in the second measurement requirement but not in the first measurement requirement, an

A third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

4. The first device of claim 1, wherein if the at least one metric included in the first measurement requirement is a subset of the at least one metric included in the second measurement requirement, the first device is caused to generate at least one measurement configuration by:

The following were generated:

5. The first device of claim 1, wherein if the at least one metric included in the second measurement requirement is a subset of the at least one metric included in the first measurement requirement, then causing the first device to generate at least one measurement configuration by:

the following were generated:

a first measurement configuration indicating metrics included in the first measurement requirement but not in the second measurement requirement, an

6. The first device of claim 1, wherein if the at least one metric included in the first measurement requirement is the same as the at least one metric included in the second measurement requirement, then causing the first device to generate at least one measurement configuration by:

A third measurement configuration is generated, the third measurement configuration indicating metrics included in both the first measurement requirement and the second measurement requirement.

7. The first device of any of claims 3 to 6, wherein,

the first measurement configuration corresponds to the first data collector;

the second measurement configuration corresponds to the second data collector; and

the third measurement configuration corresponds to both the first data collector and the second data collector.

8. The first device of any of claims 1 to 7, wherein the first measurement requirement is to be activated by the first device, or has been activated by the first device, and the second measurement requirement is to be activated by the first device.

9. The first device of claim 8, wherein if the first measurement requirement has been activated by the first device, further causing the first device to:

a second message is sent to deactivate the first measurement requirement.

10. The first device of any of claims 1 to 9, wherein the first device is a management system device and the second device is a terminal device.

11. A third device, comprising:

at least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the third device to:

receiving a first message from a first device, the first message for activating a measurement configuration for quality of experience measurements at a second device, the first message indicating:

the measurement configuration is described in terms of,

a session identification allocated for a measurement session corresponding to the measurement configuration, and

information of a plurality of data collectors corresponding to the measurement configuration;

creating a correspondence between an identification used to identify the session identification and the information of the plurality of data collectors; and

and sending a configuration message to the second device based on the received first message.

12. A third device according to claim 11, wherein the third device is further caused to:

receiving a measurement report message from the second device, the measurement report message comprising:

measurement reporting

An identification for identifying the measurement session; and

And respectively sending the measurement reports to the plurality of data collectors based on the corresponding relation.

13. The third device according to claim 11 or 12, wherein the first device is a management system device, the second device is a terminal device, and the third device is an access network device.

14. A second device, comprising:

at least one processor; and

at least one memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the second device to:

receiving, from a third device, a fourth measurement configuration for quality of experience measurements at the second device, the fourth measurement requirement indicating at least one metric and corresponding to a first identification for identifying a first session identification;

receiving, from the third device, a fifth measurement configuration for quality of experience measurements at the second device, the fifth measurement requirement indicating at least one metric and corresponding to a second identification for identifying a second session identification; and

if the at least one metric indicated by the fourth measurement configuration at least partially overlaps with the at least one metric indicated by the fifth measurement configuration, generating at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to at least one overlapping metric.

15. A second device of claim 14, wherein the second device is further caused to:

transmitting at least one measurement report message to the third device, each measurement report message corresponding to a respective measurement report of the at least one measurement report, and comprising:

the corresponding measurement report, and

at least one identification corresponding to the respective measurement report and used to identify at least one respective session identification.

16. The second device of claim 14, wherein if the at least one metric of the fourth measurement configuration indication partially overlaps the at least one metric of the fifth measurement configuration indication, then causing the second device to generate at least one measurement report by:

performing measurements based on the fourth measurement configuration and the fifth measurement configuration; and

the following were generated:

a first measurement report comprising measurement results corresponding to metrics included in the fourth measurement configuration but not in the fifth measurement configuration,

a second measurement report including measurement results corresponding to metrics included in the fifth measurement configuration but not in the fourth configuration requirement, an

A third measurement report comprising measurement results corresponding to metrics comprised in both the fourth measurement configuration and the fifth measurement configuration.

17. The second device of claim 14, wherein if the at least one metric of the fourth measurement configuration indication is a subset of the at least one metric of the fifth measurement configuration indication, then causing the second device to generate at least one measurement report by:

performing measurements based on the fifth measurement configuration; and

the following were generated:

a second measurement report including measurement results corresponding to metrics included in the fifth measurement configuration but not included in the fourth measurement configuration, and

18. The second device of claim 14, wherein if the at least one metric of the fifth measurement configuration indication is a subset of the at least one metric of the fourth measurement configuration indication, further causing the second device to generate at least one measurement report by:

performing measurements based on the fourth measurement configuration; and

The following were generated:

a first measurement report including measurement results corresponding to metrics included in the fourth measurement configuration but not included in the second measurement configuration, an

19. The second device of claim 14, wherein if the at least one metric of the fifth measurement configuration indication is the same as the at least one metric of the fifth measurement configuration indication, then causing the second device to generate at least one measurement report by:

performing measurements based on the fourth measurement configuration or the fifth measurement configuration; and

a third measurement report is generated, the third measurement report comprising measurement results corresponding to metrics included in both the fourth measurement configuration and the fifth measurement configuration.

20. A second device according to any of claims 16 to 19, wherein the second device is further caused to:

transmitting a measurement report message including:

the third measurement report is a report of the first measurement,

the first identifier for identifying the first session identifier, and

The second identifier for identifying the second session identifier.

21. The second device of any of claims 14 to 20, wherein the second device is a terminal device and the third device is an access network device.

22. A third device, comprising:

at least one processor; and

at least one memory including computer program code;

receiving, from a second device, a measurement report message for quality of experience measurements at the second device, the measurement report message comprising:

measurement reporting

A plurality of identifications for identifying session identifications; and

the measurement report is sent to a respective data collector based on correspondence between a plurality of identifications identifying a plurality of session identifications and a plurality of identifications of the respective data collector.

23. The third device of claim 22, wherein the second device is a terminal device and the third device is an access network device.

24. A method, comprising:

at a first device, obtaining a first measurement requirement for a quality of experience measurement at a second device from a first data collector, the first measurement requirement comprising at least one metric;

25. The method of claim 24, further comprising:

the corresponding measurement configuration is described as such,

26. The method of claim 24, wherein generating the at least one measurement configuration if the at least one metric included in the first measurement requirement partially overlaps the at least one metric included in the second measurement requirement comprises:

The following were generated:

a second measurement configuration indicating inclusion in the second measurement requirement, but not inclusion

Metrics in the first measurement requirement

27. The method of claim 24, wherein generating the at least one measurement configuration if the at least one metric included in the first measurement requirement is a subset of the at least one metric included in the second measurement requirement comprises:

the following were generated:

28. The method of claim 24, wherein generating the at least one measurement configuration if the at least one metric included in the second measurement requirement is a subset of the at least one metric included in the first measurement requirement comprises:

The following were generated:

A third measurement configuration indicating metrics included in the first measurement requirement and the second measurement requirement.

29. The method of claim 24, wherein generating the at least one measurement configuration if the at least one metric included in the first measurement requirement is the same as the at least one metric included in the second measurement requirement comprises:

30. The method according to any one of claims 26 to 29, wherein,

the first measurement configuration corresponds to the first data collector;

31. The method of any of claims 24 to 30, wherein the first measurement requirement is to be activated by the first device, or has been activated by the first device, and the second measurement requirement is to be activated by the first device.

32. The method of claim 31, wherein if the first measurement requirement has been activated by the first device, the method further comprises:

a second message is sent to deactivate the first measurement requirement.

33. The method of any of claims 24 to 32, wherein the first device is a management system device and the second device is a terminal device.

34. A method, comprising:

at a third device, receiving a first message from a first device, the first message for activating a measurement configuration for quality of experience measurements at a second device, the first message indicating:

the measurement configuration is described in terms of,

35. The method as in claim 34, further comprising:

Measurement reporting

An identification for identifying the measurement session; and

36. The method of claim 34 or 35, wherein the first device is a management system device, the second device is a terminal device, and the third device is an access network device.

37. A method, comprising:

at the second device, receiving a fourth measurement configuration for quality of experience measurements at the second device from a third device, the fourth measurement requirement indicating at least one metric and corresponding to a first identification for identifying a first session identification;

38. The method of claim 37, further comprising:

the corresponding measurement report, and

39. The method of claim 37, wherein generating the at least one measurement report if the at least one metric of the fourth measurement configuration indication partially overlaps with the at least one metric of the fifth measurement configuration indication comprises:

the following were generated:

40. The method of claim 37, wherein generating the at least one measurement report if the at least one metric of the fourth measurement configuration indication is a subset of the at least one metric of the fifth measurement configuration indication comprises:

performing measurements based on the fifth measurement configuration; and

the following were generated:

41. The method of claim 37, wherein generating the at least one measurement report if the at least one metric of the fifth measurement configuration indication is a subset of the at least one metric of the fourth measurement configuration indication comprises:

performing measurements based on the fourth measurement configuration; and

the following were generated:

a first measurement report including measurement results corresponding to metrics included in the fourth measurement configuration but not included in the second measurement configuration; and

42. The method of claim 37, wherein generating the at least one measurement report if the at least one metric of the fifth measurement configuration indication is the same as the at least one metric of the fifth measurement configuration indication comprises:

43. The method of any one of claims 39 to 42, further comprising:

transmitting a measurement report message including:

the third measurement report is a report of the first measurement,

the first identifier for identifying the first session identifier, and

the second identifier for identifying the second session identifier.

44. The method of any of claims 37 to 43, wherein the second device is a terminal device and the third device is an access network device.

45. A method, comprising:

at a third device, receiving a measurement report message of quality of experience measurements at a second device from the second device, the measurement report message comprising:

measurement reporting

A plurality of identifications for identifying session identifications; and

46. The method of claim 45, wherein the second device is a terminal device and the third device is an access network device.

47. A first apparatus, comprising:

means for obtaining, at a first device, a first measurement requirement for a quality of experience measurement at a second device from a first data collector, the first measurement requirement comprising at least one metric;

means for obtaining a second measurement requirement for a quality of experience measurement at the second device from a second data collector, the second measurement requirement comprising at least one metric; and

means for generating at least one measurement configuration based on the first measurement requirement and the second measurement requirement if the at least one metric included in the first measurement requirement at least partially overlaps with the at least one metric included in the second measurement requirement, the measurement configuration in the at least one measurement configuration comprising at least one overlapping metric.

48. A third apparatus, comprising:

means for receiving, at a third device, a first message from a first device, the first message for activating a measurement configuration for quality of experience measurements at a second device, the first message indicating:

the measurement configuration is described in terms of,

means for creating correspondence between an identification used to identify the session identification and the information of the plurality of data collectors; and

means for sending a configuration message to the second device based on the received first message.

49. A second apparatus, comprising:

means for receiving, at a second apparatus, a fourth measurement configuration for quality of experience measurements at the second apparatus from a third apparatus, the fourth measurement requirement indicating at least one metric and corresponding to a first identification for identifying a first session identification;

means for receiving, from the third apparatus, a fifth measurement configuration for quality of experience measurements at the second apparatus, the fifth measurement requirement indicating at least one metric and corresponding to a second identification for identifying a second session identification; and

Means for generating at least one measurement report based on the fourth measurement configuration and the fifth measurement configuration if the at least one metric indicated by the fourth measurement configuration at least partially overlaps with the at least one metric indicated by the fifth measurement configuration, wherein a measurement report of the at least one measurement report comprises a measurement report corresponding to at least one overlapping metric.

50. A third apparatus, comprising:

means for receiving, at a third device, a measurement report message for a quality of experience measurement at a second device from the second device, the measurement report message comprising:

measurement reporting

A plurality of identifications for identifying session identifications; and

means for sending the measurement report to a respective data collector based on correspondence between a plurality of identifications identifying a plurality of session identifications and a plurality of identifications of the respective data collector.

51. A computer readable medium having embodied thereon a computer program which, when executed, causes an apparatus to perform the method according to any of claims 1-10.

52. A computer readable medium having embodied thereon a computer program which, when executed, causes an apparatus to perform the method according to any of claims 11-13.

53. A computer readable medium having embodied thereon a computer program which, when executed, causes an apparatus to perform the method of any of claims 14-21.

54. A computer readable medium having embodied thereon a computer program which, when executed, causes an apparatus to perform the method of claim 22 or 23.