JP7501662B2 - Objective evaluation device, objective evaluation method, and objective evaluation program - Google Patents

Description

The present invention relates to an objective evaluation device, an objective evaluation method, and an objective evaluation program.

In order to provide users with comfortable video communication services, it is necessary to efficiently evaluate the quality of experience (QoE) experienced by users using the service.

For example, when multiple students in a classroom use wireless LAN devices to simultaneously stream video from a server on the Internet, it becomes necessary to evaluate whether each student is able to play the video at a specified quality.

Methods for evaluating the quality of video media (communication quality) include subjective evaluation, in which a person watches the video and makes a subjective judgment, and objective evaluation, in which the subjective quality of the video media is estimated using video information (data) (see non-patent document 1).

For example, in subjective evaluation, students who watch a video are asked to rate the quality of the video on a 5-point scale (5: very good, 4: good, 3: normal, 2: bad, 1: very bad), and the evaluation is based on that value. In conventional objective evaluation, the evaluation is based on information obtained from higher layers such as the media layer and packet layer.

Subjective evaluation can provide more accurate evaluation than objective evaluation. However, conducting subjective evaluation requires a great deal of time and effort. Therefore, there is a demand for an efficient objective evaluation method to estimate subjective quality.

Okamoto, et al., "Latest Trends in Video Media Quality Assessment Technology," Institute of Electronics, Information and Communication Engineers, April 2013, Fundamentals Review Vol.6 No.4, pp.276-284

However, in the past, it was not possible to estimate subjective quality without using information from higher layers such as the media layer and packet layer in video communication services.

The present invention aims to provide an objective evaluation device, an objective evaluation method, and an objective evaluation program that can easily estimate the subjective quality of video media without using information from higher layers.

An objective evaluation device according to one aspect of the present invention is characterized in that it has a capture unit that captures wireless frames of video data transmitted to a plurality of terminals over a specified wireless channel, a calculation unit that calculates an index indicating fairness per unit time of communication time for each of the terminals based on the wireless frames captured by the capture unit, and an output unit that outputs the index calculated by the calculation unit.

In addition, an objective evaluation method according to one aspect of the present invention is characterized in that it includes a capture step of capturing wireless frames of video data transmitted to a plurality of terminals over a specified wireless channel, a calculation step of calculating an index indicating the fairness of communication time per unit time of each of the terminals based on the captured wireless frames, and an output step of outputting the calculated index.

According to the present invention, the subjective quality of video media can be easily estimated without using information from higher layers.

1 is a diagram illustrating a wireless communication system that provides video communication services and functions of an objective evaluation device that evaluates the quality of video media. 11 is a flowchart illustrating a first example of processing performed by a calculation unit. 13 is a flowchart illustrating a second example of processing performed by the calculation unit. 2 is a diagram illustrating wireless frames of a plurality of terminals evaluated by the objective evaluation device shown in FIG. 1 . FIG. 1 is a diagram illustrating an example of a hardware configuration of an objective evaluation device according to an embodiment.

Below, an embodiment of an objective evaluation device for evaluating the quality of video media in a video communication service is described with reference to the drawings. Figure 1 is a diagram illustrating a wireless communication system 10 that provides a video communication service and the functions of an objective evaluation device 5 that evaluates the quality of the video media.

The wireless communication system 10 is configured such that, for example, terminals 1 to 3 (n units: 1≦i≦n=3) communicate with an access point 4 as a wireless LAN. For example, terminals 1 to 3 are used for learning by each student in a classroom. As a specific example, terminals 1 to 3 simultaneously stream learning videos stored, for example, on a server (not shown) on the Internet via the access point 4.

The objective evaluation device 5 is a device that estimates the subjective quality and performs an objective evaluation of the video played by each of the terminals 1 to 3. For example, the objective evaluation device 5 has an acquisition unit 51, a setting unit 52, a capture unit 53, a calculation unit 54, and an output unit 55.

The acquisition unit 51 acquires information identifying the MAC addresses and wireless channels of multiple terminals to be evaluated by the objective evaluation device 5, and outputs the acquired information to the setting unit 52. For example, the acquisition unit 51 acquires information from an administrator (maintenance person) of the wireless communication system 10 via the input unit 500 or the communication unit 520 described later with reference to FIG. 5.

The setting unit 52 performs settings for the capture unit 53 and the calculation unit 54 based on the MAC address and wireless channel of the terminal input from the acquisition unit 51. Specifically, the setting unit 52 performs settings for the capture unit 53 to specify the wireless channel from which wireless frames should be captured. The setting unit 52 also performs settings for the calculation unit 54 to specify MAC addresses that specify multiple target terminals (e.g., terminals 1 to 3) so as to specify multiple video data to be used to calculate an index, which will be described later.

The capture unit 53 captures each wireless frame transmitted through the access point 4 on the wireless channel set by the setting unit 52, and outputs the captured data to the calculation unit 54. For example, the capture unit 53 receives and imports all wireless frames transmitted on the wireless channel 36ch, and temporarily stores them.

The calculation unit 54 calculates an index indicating the fairness of the communication time per unit time for each of the multiple target terminals (e.g., terminals 1 to 3) based on, for example, the wireless frames captured by the capture unit 53, and outputs the calculated index to the output unit 55. For example, the calculation unit 54 extracts wireless frames for terminals (e.g., terminals 1 to 3) with MAC addresses set by the setting unit 52 from the wireless frames captured by the capture unit 53 on the 36ch wireless channel. Then, the calculation unit 54 calculates an index indicating the fairness of the communication time per unit time for each of the multiple target terminals (e.g., terminals 1 to 3) based on the frame length and PHY data rate (physical layer data rate) in the wireless layer (physical layer) of the extracted wireless frames.

Here, the unit of the frame length is, for example, bits. The unit of the PHY data rate (hereinafter, PHY rate) is, for example, bit/s. The calculation unit 54 then calculates an index (for example, a Fairness Index, described later) indicating fairness based on a value in units of time (for example, Airtime _i , described later).

At this time, the calculation unit 54 checks whether the MAC address of the wireless frame input from the capture unit 53 is included in the multiple MAC addresses set by the setting unit 52, extracts the wireless frame necessary for calculating the index, and calculates the index. That is, the calculation unit 54 checks the correspondence between the MAC address set by the setting unit 52 and the MAC address of the wireless frame input from the capture unit 53, extracts the wireless frame necessary for calculating the index, and calculates the index.

The output unit 55 outputs the indicator calculated by the calculation unit 54 to an administrator (maintenance person) of the wireless communication system 10, for example by displaying the indicator.

Next, a specific example of the processing performed by the calculation unit 54 will be described with reference to Figures 2 and 3. For example, the calculation unit 54 performs the first processing illustrated in Figure 2 and the second processing illustrated in Figure 3.

First, the calculation unit 54 obtains the MAC address, frame length, and PHY rate of the wireless frame from the capture unit 53 as shown in FIG. 2 (S100).

Next, the calculation unit 54 determines whether or not the MAC address of the wireless frame input from the capture unit 53 is included in the multiple MAC addresses set by the setting unit 52. For example, the calculation unit 54 determines whether or not the MAC address of the wireless frame input from the capture unit 53 is the MAC address of terminal i (1≦i≦n=3) (S102).

Then, if the MAC address of the wireless frame input from the capture unit 53 is the MAC address of terminal i (S102: Yes), the calculation unit 54 proceeds to processing of S104, and otherwise (S102: No), terminates the processing.

In the process of S104, the calculation unit 54 performs calculation to update the airtime (Airtime _i ) in units of time shown in the following formula (1) for each of the terminals 1 to 3.

3, the calculation unit 54 calculates the airtime ratio (AirtimeRatio _i ) shown in the following formula (2) for each of the terminals 1 to 3 to be evaluated at each predetermined period T (S200). After calculating the airtime ratio, the calculation unit 54 sets Airtime _i = 0.

Then, the calculation unit 54 calculates an index (Fairness Index) serving as a criterion for judging fairness, shown in the following formula (3), using the AirtimeRatio _i of each of the terminals 1 to 3 (S202).

The Fairness Index is also described in the non-patent literature R. Jain, D. Chiu, and W. Hawe, “A Quantitative Measure of Fairness and Discrimination for Resource Allocation in Shared Computer Systems”, DEC Research Report TR-301, September 26, 1984.

The Fairness Index takes a value between 0 and 1, and the closer it is to 1, the fairer it is. Here, we explain the reason for calculating the Fairness Index.

For example, when students in a classroom use wireless LAN terminals to simultaneously stream video from a server on the Internet, whether or not each student is able to play the video fairly depends on whether or not the wireless LAN terminals used by each student are consuming wireless (air) resources fairly.

In the objective evaluation device 5, when the calculation unit 54 calculates the Fairness Index of the airtime rate consumed by the terminals 1 to 3 and the output unit 55 displays it, the administrator of the wireless communication system 10 can know whether the terminals 1 to 3 used by each student are consuming wireless (air) fairly. In other words, the administrator of the wireless communication system 10 can know whether each student is playing back the video fairly.

In addition, the standard specifications for implementing the GIGA School Initiative (Ministry of Education, Culture, Sports, Science and Technology, March 3, 2020) lists the following as a guideline for bandwidth usage per device for each learning activity:

Distance learning (video conference): 2.0Mbps
NHK For School 0.7Mbps
YouTube (registered trademark: HD720p image quality) 2.5Mbps

The Fairness Index tends to be larger as the estimated bandwidth usage per terminal is smaller. In other words, the smaller the estimated bandwidth usage per terminal, the easier it is for terminals 1 to 3 used by each student to consume wireless (air) fairly.

Next, an example of the evaluation performed by the objective evaluation device 5 will be described. Figure 4 is a diagram illustrating wireless frames of terminals 1 to 3 evaluated by the objective evaluation device 5 shown in Figure 1.

As shown in Figure 1, for example, assume that one access point 4 is installed in a classroom and access point 4 is using channel 36.

The MAC addresses of terminals 1 to 3 are assumed to be AA:AA:AA:11:11:11, AA:AA:AA:22:22:22, and AA:AA:AA:33:33:33, respectively. Terminals 1 to 3 then each stream video from a server (not shown) on the Internet via access point 4.

The acquisition unit 51 acquires the MAC addresses of terminals 1 to 3 (AA:AA:AA:11:11:11, AA:AA:AA:22:22:22, AA:AA:AA:33:33:33) and information indicating the wireless channel (36ch) that the capture unit 53 should capture from the administrator (maintenance person) of the wireless communication system 10.

The setting unit 52 sets the MAC addresses of terminals 1 to 3 (AA:AA:AA:11:11:11, AA:AA:AA:22:22:22, AA:AA:AA:33:33:33) to the calculation unit 54 and sets the wireless channel (36ch) to the capture unit 53.

The capture unit 53 starts capturing wireless frames transmitted on the set wireless channel (36ch). Here, as shown in FIG. 4, the capture unit 53 captures wireless frames from terminal 1, terminal 2, terminal 1, and terminal 3 at times t=0.2 seconds, 0.4 seconds, 0.6 seconds, and 0.8 seconds, respectively.

At time t=0.2 seconds, the capture unit 53 outputs the following information to the calculation unit 54:
・MAC address: AA:AA:AA:11:11:11
Frame length: 120000 (bits)
・PHY rate: 150Mbps

Since the MAC address of the wireless frame input from the capture unit 53 is included among the multiple MAC addresses set by the setting unit 52, the calculation unit 54 performs the calculation shown in the following formula (4) to calculate Airtime ₁ (communication time).

At time t=0.4 seconds, the capture unit 53 outputs the following information to the calculation unit 54:
・MAC address: AA:AA:AA:22:22:22
Frame length: 90,000 (bits)
・PHY rate: 150Mbps

Since the MAC address of the wireless frame input from the capture unit 53 is included in the multiple MAC addresses set by the setting unit 52, the calculation unit 54 performs the calculation shown in the following formula (5) to calculate Airtime ₂ (communication time).

At time t=0.6 seconds, the capture unit 53 outputs the following information to the calculation unit 54:
・MAC address: AA:AA:AA:11:11:11
Frame length: 90,000 (bits)
・PHY rate: 150Mbps

Since the MAC address of the wireless frame input from the capture unit 53 is included among the multiple MAC addresses set by the setting unit 52, the calculation unit 54 performs a calculation to update Airtime ₁ shown in the following equation (6).

At time t=0.8 seconds, the capture unit 53 outputs the following information to the calculation unit 54:
・MAC address: AA:AA:AA:33:33:33
Frame length: 120000 (bits)
・PHY rate: 150Mbps

Since the MAC address of the wireless frame input from the capture unit 53 is included in the multiple MAC addresses set by the setting unit 52, the calculation unit 54 performs the calculation shown in the following formula (7) to calculate Airtime ₃ (communication time).

Here, if the above-mentioned specified period T is 1 second, the airtime rates of terminals 1 to 3 at time t = 1 second will be the values shown in the following equations (8) to (10).

Then, the calculation unit 54 calculates the Fairness Index shown in the following formula (11) and outputs the calculated Fairness Index value (0.88) to the output unit 55.

The output unit 55 outputs the Fairness Index value (0.88) input from the calculation unit 54, for example by displaying it. Therefore, the administrator of the wireless communication system 10 can understand that the terminals 1 to 3 used by each student are consuming wireless (air) fairly, since the Fairness Index value is close to 1. In other words, it can be said that each student is able to play the video fairly.

In this way, the objective evaluation device 5 in one embodiment has a calculation unit 54 that calculates an index indicating the fairness of the communication time occupied by each wireless frame captured by the capture unit 53 per unit time, and an output unit 55 that outputs the index calculated by the calculation unit 54, so that the subjective quality of the video media can be easily estimated without using information from higher layers.

In addition, each function of the objective evaluation device 5 may be configured in part or in whole by hardware such as a PLD (Programmable Logic Device) or an FPGA (Field Programmable Gate Array), or may be configured as a program executed by a processor such as a CPU.

In addition, the objective evaluation device 5 of the present invention can be realized using a computer and a program, and the program can be recorded on a storage medium or provided via a network.

Figure 5 is a diagram showing an example of the hardware configuration of an objective evaluation device 5 according to one embodiment. As shown in Figure 5, for example, the objective evaluation device 5 has an input unit 500, a display unit 510, a communication unit 520, a CPU 530, a memory 540 and a HDD 550 connected via a bus 560, and has the functions of a computer. In addition, the objective evaluation device 5 is configured to be able to input and output data between it and a computer-readable storage medium 570.

The input unit 500 is, for example, a keyboard and a mouse. The display unit 510 is, for example, a display. The communication unit 520 is, for example, a communication interface, and is also included in, for example, the capture unit 53 described above. The communication unit 520 may also be included in the output unit 55 described above.

The CPU 530 controls each component of the objective evaluation device 5 and performs predetermined processing, etc. The memory 540 and HDD 550 store data, etc.

The storage medium 570 is capable of storing programs and the like that execute the functions of the objective evaluation device 5. Note that the architecture constituting the objective evaluation device 5 is not limited to the example shown in FIG.

Although an embodiment of the present invention has been described above with reference to the drawings, it is clear that the above-mentioned embodiment is merely an example of the present invention and the present invention is not limited to the above-mentioned embodiment. Therefore, addition, omission, substitution, and other modifications of components may be made without departing from the technical concept and scope of the present invention.

1, 2, 3... Terminal, 4... Access point, 5... Objective evaluation device, 10... Wireless communication system, 51... Acquisition unit, 52... Setting unit, 53... Capture unit, 54... Calculation unit, 55... Output unit, 500... Input unit, 510... Display unit, 520... Communication unit, 530... CPU, 540... Memory, 550... HDD, 560... Bus, 570... Storage medium

Claims (5)

a capture unit that captures wireless frames of video data transmitted to a plurality of terminals via a predetermined wireless channel;
a calculation unit that calculates an index indicating fairness of communication time per unit time of each of the terminals based on the wireless frames captured by the capture unit;
and an output unit that outputs the index calculated by the calculation unit. The calculation unit is
2. The objective evaluation device according to claim 1, wherein the index is calculated based on a frame length and a data rate of a wireless frame for each of the terminals. a capture step of capturing wireless frames of video data transmitted to a plurality of terminals via a predetermined wireless channel;
a calculation step of calculating an index indicating fairness per unit time of communication time of each of the terminals based on the captured wireless frames;
and an output step of outputting the calculated index. In the calculation step,
4. The objective evaluation method according to claim 3, wherein the index is calculated based on a frame length and a data rate of a radio frame for each of the terminals. An objective evaluation program for causing a computer to function as each part of the objective evaluation device described in claim 1 or 2.

Images