EP3939330A1

EP3939330A1 - Method for managing the download of images associated with image jumps capable of being carried out during accelerated reading of multimedia content which is continuously broadcast

Info

Publication number: EP3939330A1
Application number: EP20710215.3A
Authority: EP
Inventors: Hervé Marchand; Mathieu Rivoalen
Original assignee: Orange SA
Current assignee: Orange SA
Priority date: 2019-03-12
Filing date: 2020-02-13
Publication date: 2022-01-19
Also published as: FR3093885A1; US12081826B2; US20220159333A1; WO2020183080A1

Abstract

The invention relates to a method for managing the download of images (S16,S32,S48) associated with image jumps which are capable of being carried out during accelerated reading of content (VDO). The content can be downloaded from a communications network. The invention is characterised in that at least one image which is associated with a jump is downloaded in advance.

Description

DESCRIPTION

TITLE:

PROCESS FOR MANAGING THE DOWNLOADING OF IMAGES ASSOCIATED WITH IMAGE BREAKS LIKELY TO BE CARRIED OUT DURING ACCELERATED PLAYBACK OF MULTIMEDIA CONTENT BROADCAST IN CONTINUOUSLY

Technical area

The field of the invention is that of digital multimedia content, namely digital audio and / or video content, also called audiovisual content. The invention relates very particularly to a method for managing the display of images associated with image jumps that may be performed during accelerated playback of multimedia content.

In the present text, acceleration of reading aims at both fast rewinding and fast forwarding making it possible to perform picture jumps in multimedia content.

A content is for example television on demand content, video on demand content, etc.

State of the art

Today there are many audiovisual content restitution terminals allowing a user to access multimedia content such as video on demand. These rendering terminals are suitable for receiving this digital content in the form of multimedia data and for rendering it to the user. This restitution consists in providing, at the level of the terminal, the digital content in a form accessible to the user. For example, received data corresponding to a video is generally decoded and then restored at the terminal in the form of a display of the corresponding video with its associated soundtrack. The display can be done on a screen of the rendering terminal itself or on a remote screen associated with it. Such multimedia content restitution terminals are most often controllable remotely by the user by means of remote control, and in particular remote controls. A remote control offers several commands that can be executed by the playback terminal. The purpose of commands, for example, is fast forward (") or fast reverse ("). These commands make it possible to act on the restitution of the digital content and in particular to accelerate the restitution of the content on the screen. When accelerating playback, images (also called thumbnails) of the content are displayed on the screen. The thumbnail thus displayed allows a user in particular to locate himself in the content during the acceleration of reading.

Note that the activation of the fast forward / rewind function by the user can be done other than by means of a remote control. Activation of the fast forward / rewind function can also be done via an application dedicated to managing the multimedia stream player when it is associated with a client terminal such as a tablet or a smart phone, for example. The functions can also be present on the rendering device by means of keys present on the rendering device.

The accelerations traditionally used today multiply by two (x2), by four (x4), by sixteen (x16), by thirty-two (x32) or even sixty-four (x64), the reading speed. Thus, when a user watches for example a video, he has the possibility of going quickly to a precise place of it by speeding up its playback by activating the fast forward / rewind function.

These accelerations cause a number of frame jumps in the content which depends on the chosen acceleration rate. The chosen speed determines new playback moments following acceleration, and the respective images extracted from the content to be displayed on the screen.

One problem is that the time for downloading the image, the image associated with the new reading instant resulting from a jump, can be long. This download time is all the longer when the image is included in an image segment and the downloading of the image requires the download of the associated segment. Such a segment is used for example when multimedia content is received in adaptive streaming mode. Indeed, the MPEG-DASH standard (for English “Dynamic Adaptive Streaming over HTTP ”, in French“ dynamic adaptive streaming over HTTP ”) is a standard for audiovisual broadcasting format on the Internet; this standard is based on the preparation of the content in different presentations of variable quality and bit rate, broken up into short segments (of the order of a few seconds), also called “chunks” by those skilled in the art. Each of these segments is made available individually by means of an exchange protocol between the rendering terminal and the server providing multimedia content. The mainly targeted protocol is HTTP, but other protocols (eg FTP) can also be used. The organization of the segments and the associated parameters are published in a manifest in XML format.

Ultimately, the display of an image representative of a jump includes at least the following durations:

- a segment loading time,

- a period linked to the extraction of an image of the segment,

- and a duration for the restitution of this segment.

The addition of these times can unfortunately be greater than the time between two hops, especially when the throughput on the network used is low; this is often the case with an ADSL type network. It follows that the chosen acceleration, for example acceleration (x64), is intentionally reduced so as to render each frame correctly. Furthermore, this decrease in acceleration speed occurs without the user's knowledge. This solution is not acceptable in terms of quality of service.

The invention offers a solution which does not have the drawbacks of the state of the art.

The invention

To this end, according to a functional aspect, the object of the invention is a method for managing the display of images associated with image jumps that may be produced during accelerated playback of multimedia content on a rendering device, the content being downloadable from a communication network, characterized in that at least one image associated with a jump is downloaded in advance. According to the invention, an image associated with a jump (also called a thumbnail in the present application) is downloaded before the execution of a command to read this image. During accelerated reading, the method comprises displaying successive images, loaded in advance, representative of the state of progress of accelerated reading in the content.

It will be seen below that if several images are downloaded in advance, these images can be downloaded in a single message or downloaded one after the other. The choice can depend on the number of images to download.

The early download of an image associated with a jump allows images to be provisioned before and / or during the content playback phase and can therefore be used later during accelerated content playback without requiring a download. The time associated with reading a thumbnail is then reduced to the reading time of the image stored in the memory of the rendering device, the latter having been downloaded in advance and therefore stored in memory ready to be read.

The proposed solution aims to anticipate the loading of images that will be displayed during an acceleration of multimedia content playback. We will see in the following that the image can be included in a segment of images. Early loading allows you to anticipate the loading of the segment and the extraction of the image of the segment; The display time is then reduced to the time associated with the playback of the image stored in a memory of the playback terminal. In this way, even if the throughput on the communication network is low, the invention ensures that the acceleration applied to playing the content will be the selected acceleration.

According to a first embodiment, the reading step is preceded by a step of receiving the content from a communication network; in this configuration, the anticipated download is performed when the speed on the network is lower than a given speed. This first mode avoids implementing the method of the invention when it is not necessary.

According to a second embodiment, which can be implemented alternately or cumulatively with the previous one, a stored image in advance is stored in memory, and in that it is erased from memory when content playback is stopped. This second mode has the advantage of optimizing the memory space in the rendering terminal. The stop in question is indifferently aimed at a final stop, a pause, or any other function making it possible to stop or end a reading of multimedia content.

According to a third embodiment, which can be implemented alternately or cumulatively with the previous ones, an image stored in advance is stored in a memory; and is erased from memory after a given number of reading of this image. Like the first mode, this second mode has the advantage of optimizing the memory space in the rendering terminal by erasing the thumbnail when it has been read a number N (N is an integer) of times, for example a single time. This third mode also avoids re-downloading of images in advance when the content is destined to be read several times. In this configuration, N is strictly greater than 1. For example, if N is set to 2, the pre-downloaded image can be read twice in memory.

According to a fourth embodiment, which can be implemented alternately or cumulatively with the previous ones, the images stored in advance are stored in a memory; and are then kept in memory for a given time. This third mode has the advantage, in particular if the content is destined to be replayed, of having the images loaded in advance at a later date and therefore avoids having to reload the images in advance during a later access to this same content.

According to a fifth embodiment, which can be implemented alternately or cumulatively with the previous ones, in the case where the image to be displayed is included in a segment of images, the image chosen in the segment is an intra image.

According to a sixth embodiment, which can be implemented alternately or cumulatively with the previous ones, the anticipated loading step is carried out after a waiting period. This sixth mode takes into account a right of repentance of the user who can decide not to not view the content for example after a few seconds of playback. This mode avoids downloading segments in advance when the user decides to stop the restitution of the content shortly after the start of the restitution.

According to a seventh embodiment, which can be implemented alternately or cumulatively with the previous ones, when reading is scheduled for a given date, the early loading step is carried out before the given date. This mode avoids downloading images in advance while the content is playing. This mode optimizes the use of bandwidth. More generally, the advance downloading in the background of the thumbnails will preferably be such that it does not affect the quality of the current reading of the content. The download of the thumbnails (actually the segments that contain the images) will be done at a controlled rate that does not interfere with the retrieval of the segments of the stream being played.

According to a hardware aspect, the invention relates to an entity for managing the display of images associated with image jumps capable of being produced during accelerated reading of the content, the content being downloadable from a network. communication, characterized in that it comprises a download module capable of downloading in advance at least one image associated with a jump. According to another material aspect, the invention relates to a decoder comprising a management entity as defined above.

According to another hardware aspect, the invention relates to a computer program capable of being implemented on an entity as defined above, the program comprising code instructions which, when it is executed by a processor , performs the steps of the selection process defined above.

Finally, according to another hardware aspect, the invention relates to a data medium on which has been stored at least one series of program code instructions for the execution of a management method as defined above. . Such a recording medium can be any entity or device capable of storing the program. For example, the medium may comprise a storage means, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or else a magnetic recording means, for example a USB key or a hard disk.

On the other hand, such a recording medium can be a transmissible medium such as an electrical or optical signal, which can be conveyed via an electrical or optical cable, by radio or by other means, so that the program computer it contains can be executed remotely. The program according to the invention can in particular be downloaded over a network, for example the Internet network.

Alternatively, the recording medium can be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the aforementioned display control method. The invention will be better understood on reading the following description, given by way of example and made with reference to the appended drawings in which:

[Fig. 1] Figure 1 shows a computer system on which is illustrated an exemplary embodiment of the invention.

[Fig. 2] FIG. 2 illustrates a division of multimedia content into segments according to one embodiment of the invention.

[Fig. 3] FIG. 3 illustrates an exchange of messages between a rendering device and a server providing multimedia content.

[Fig. 4] Figure 4 illustrates a variant of the embodiment of the invention described with reference to Figure 2.

[Fig. 5] FIG. 5 is a simplified block diagram of the hardware structure of the ordered reproduction device.

Detailed description of embodiments of the invention

With reference to FIG. 1, a general block diagram of all the equipment items implemented according to one embodiment of the invention is now presented. In this example, it is assumed that a television set, equipped with a multimedia content display screen ECR, is connected to a content restitution terminal STB, for example an HDMI key referenced STB. Such an STB key for example allows the user to access digital content in progressive adaptive download mode for real-time (or Live) television content, but also to access video-on-demand content, or to display on the TV's ECR screen personal content stored in a home network, such as a video of the last vacation or the last family event. This is the case, for example, with the Chromecast® device developed by Google®, or the CléTV® from Orange®.

Such STB keys are conventionally plugged into the HDMI port of an ECR television set and communicate, by Wi-Fi® connection, with another device of the home communication network connected to an extended communication network such as the Internet (residential gateway, computer , smart phone such as smartphone, tablet, etc.), in order to reproduce, on the ECR television set, the multimedia content received by a compatible software application.

It will be noted that the technique of the invention is of course not limited to such an HDMI key, which is described here by way of simple illustrative example, but that it can be applied to all the audiovisual reproduction equipment controllable from a connected remote control, such as a decoder (also called a set-top box by those skilled in the art) or a connected television set for example. These devices will be referred to below under the generic designation of STB decoders.

It is assumed that the STB decoder is able to enter into communication with a content server (not shown in the figure) to receive one or more content, for example a video on demand that the user wishes to view, of which it then controls the reproduction on the ECR television screen.

The STB decoder is also configured to be able to communicate with a TCD remote control device, illustrated by means of a remote control, for example according to the Bluetooth® communication protocol. The exemplary embodiment is based on wireless communication between the remote control and the set-top box; however, the invention can also be implemented with wired communication. The TCD remote control includes control buttons. The keys are either physical keys or virtual keys displayed on a Man-Machine interface, present on the TCD remote control.

The remote control is equipped with communication means so as to transmit commands to the STB decoder to be executed there. To illustrate the invention, we are interested in the control keys for speeding up the playback of content, namely the fast forward (") and rewind (") keys. The invention is obviously not limited to the two aforementioned commands but extends on the contrary to other similar commands.

These keys offer the possibility of speeding up the playback of multimedia content. Several playback speeds allow you to reach a scene of multimedia content more or less quickly.

The multimedia content storage location is arbitrary. In our example, the content is a video of the video on demand type stored on an SRV server.

In our example, this video is downloaded in streaming according to the MPEG Dash standard. According to this standard, the multimedia content comprises several time segments which are downloaded. Recall that a segment conventionally comprises several types of images, namely Intra images (I), and one or more images which can be predicted from this Intra image by motion estimation / compensation, namely Predictive images (P) , and Bidirectional images (B).

In our example, when the content is accelerated, an image from a segment is displayed on the screen. The image chosen is generally an intra image of the segment, this image being of better quality than the other two because it is fully encoded unlike the other two types of image mentioned above. Of course, the invention is not limited to this choice; another type of image that can be chosen instead of an intra image

As seen previously, the loading of a segment linked to a jump can take a certain time which can alter the realization of the jumps according to the chosen speed, in particular when this speed is high, for example the speed (x64) which makes it possible to accelerate the reading speed 64 times faster than normal reading.

According to the invention, the images associated with the frame jumps are downloaded in advance. In our example embodiment, since the image is part of a segment, several segments related to the frame jumps are loaded in advance. The thumbnails extracted from the segments, preferably the Intra images, will be displayed on the screen during an acceleration of the reading.

More precisely, the method comprises, according to one embodiment, the following steps:

- A transmission of a content access command,

- Receipt and storage of images of the content in advance;

- Receipt of a fast forward command,

- An execution of an accelerated reading from the current moment,

- During accelerated playback, a display of successive, pre-loaded images representative of the status of fast-forwarding through the content.

The invention makes it possible, during an accelerated reading phase, to show the progress of reading by restoring successive images that have been loaded in advance. A user viewing the screen during accelerated playback and wishing to access a particular point in time of the content, for example a given scene of the content, can interrupt the accelerated playback and go to the desired scene

Consider, with reference to Figure 2, a VDO video having segments S1 -Sn (n is an integer). An arrow referenced RD indicates the current playback time of the VDO video.

In our example, S16-S32-S48 segments are associated with jumps made during playback acceleration. If the chosen acceleration is x16, the segments S16xn (n is an integer) will be displayed successively during accelerated playback at each jump of 16 segments.

Figure 3 illustrates a first embodiment of the invention.

We will distinguish in the following two types of segments - the segments, called in this text segments to be read, which correspond to the segments of the video stream in normal playback mode,

- and segments, called advance segments, received in advance which are intended to be used when the reading of the content is accelerated.

With reference to FIG. 3, the chosen embodiment comprises the following steps:

During a first step ET1, the STB decoder requests the playback of VDO video content. For this, the decoder STB transmits several access requests REQn (Sn) (n = 1 to 3 in Figure 3) to respective Sn segments and receives in return the requested segments S1 -S3.

The successive segments S1 -S3 received at this stage correspond to the segments to be read. In FIG. 3, a first segment S1 is received, a second segment S2 is then received.

In our example, following the reception of segments to be read S1 and S2, the decoder STB receives segments S16 / S32 / S48 in advance which will be used during a possible accelerated playback of the VDO video. In this configuration, during an acceleration of the reading, the jump made corresponds to the segment S16, the image associated with this segment S16 is found in the memory of the STB decoder and displayed on the screen ECR; These steps are performed for the jumps performed. The time to start the anticipated segment downloads may vary. The start time of the anticipatory load can take place before the video is playing, or it can take place during playback.

A start before video playback can be implemented when video playback is scheduled for a given date. This is because the user can plan to watch a video on a given date with friends. The segments are loaded in advance in this case before rendering.

If the startup takes place during playback, it can take place as soon as the content is played, or later after a DR wait time.

Starting the download in advance, as soon as the content is read, does not take into account a possible withdrawal by the user. In this case, segments may be pre-downloaded unnecessarily if the user stops playing prematurely. A DR wait time can be useful in this case. Indeed, such a waiting time DR, chosen judiciously, for example 2 minutes, has the advantage of confirming a high probability that the user will continue playing the video VDO. The storage of segments in advance is therefore not done unnecessarily. The start time of the waiting time is arbitrary; this instant may correspond to the reception of the request REQ1 (S1) at step ET1 as illustrated in FIG. 3 This instant could also have corresponded to the instant of pressing the fast forward button on the remote control .

Also, the early loading of images may end at different times. For example, the anticipated loading is not carried out on the end credits of a film because it is rare that the restitution of a content is carried out until the end, in fact, the user often stops the restitution at the beginning. of the end credits. In our example, we consider that the start-up begins after a DR wait time. In Figure 3, segments S16 / S32 / S48 are downloaded in advance at the expiration of this duration DR, just after the reception of the second segment S2.

In our example, only images associated with x16 fast forward are pre-loaded with a pitch of sixteen segments. Thus, if fast forward x2 is requested, the segments relating to the jumps are downloaded, for example the segments S2 / S4 / S6 / etc., Except for the segments downloaded in advance, for example S16 / S32 / etc. which are read directly from the memory of the rendering device.

Note that the invention is obviously not limited to this example, the images corresponding to the speeds x2 and x4 could also have been downloaded in advance.

In our example, with reference to FIG. 4 which will be described below, the segments downloaded in advance with the speed x16 are the three segments S16, S32, S48. These three segments can be downloaded at the same time, as explained above with reference to FIG. 3, or successively at different times. The number of segments downloaded in advance can obviously be lower or higher than three. In our example, once the three segments S16, S32, S48 have been downloaded in advance have been received, the intra images included in the segments are preferably extracted from the segments received and are stored in a buffer memory of the decoder STB in association with instants of the video. At this point, the pre-downloaded images are ready to play in case of accelerated video playback.

In a second step ET2, following the storage of the three segments S16, S32, S48 in advance, it is assumed that a user of the remote control TCD wishes to speed up the playback of the VDO video. To do this, this user presses the acceleration button "on the remote control and selects for example the speed (x16). The acceleration command is transmitted to the STB decoder which receives it. The STB decoder then executes the command received and performs segment jumps in the video, for example with a pitch of sixteen segments. At each jump made, the decoder searches its buffer memory for the pre-loaded image that corresponds to the jump made and displays it on the ECR screen.

Subsequently, several actions are possible. The user can interrupt the accelerated playback because they are ready to watch the rest of the video at normal speed. The user restarts the normal video playback mode, for example via a button on the remote control or via the dedicated application. The multimedia stream player then resumes processing the video stream from the moment desired by the user and the images of the video which are displayed on the client terminal are those corresponding to the images of the video from the moment to restart the video playback.

The embodiment described above may be subject to variants described below.

The preview loading of thumbnails is a process that is performed in the background preferably without impacting the quality of the video played in normal mode. If the available bandwidth is not sufficient, a MOD management module present in the decoder reduces the number of segments to be downloaded in advance.

Figure 4 shows schematically segments of a VDO video. In the example chosen, all segments multiple of 16 are downloaded early. In this FIG. 4, a pointer RD represents the current instant of reading. Arrows F16 / F32 / F48 / etc. show downloads for S16 / S32 / S48 / etc. which take place in advance in a memory MR1 of the decoder STB. In our example, this download takes place at the instant of the RD read.

According to one variant, out of a number NB of segments that can be downloaded in advance, only a subset are downloaded, for example the first.

According to another variant, the number of segments downloaded in advance is determined as a function of the size of the buffer memory.

In relation to FIG. 5, we now consider the simplified structure of the STB decoder according to an exemplary embodiment of the invention. Such an STB decoder is suitable for implementing the method of managing the display of images during the acceleration of the playback of multimedia content.

The STB decoder comprises physical and / or software resources, namely memories M associated with a processor CPU. The memories can be of the ROM type (from the English "Read Only Memory") or RAM (from the English "Random Access Memory") or even Flash. They allow in particular the storage of a content selection manager according to one embodiment of the invention.

The decoder further comprises a processing circuit CT1 for implementing the management method according to various embodiments of the invention, the processing circuit CT1 containing a processor PROC1 driven by a computer program PG1 representative of the management method , stored in ROM. On initialization, the code instructions of the computer program PG1 are for example loaded into a RAM memory, denoted MR1, before being executed by the processing circuit CT1.

The decoder mainly includes:

a communication interface COM1 1 which is adapted to receive commands from the remote control TCD;

- a COM22 communication interface which is suitable for exchanging data with a remote server so as to receive segments of multimedia content as described above.

The communication interfaces COM10, COM1 1, as well as the management entity MNG, are controlled by the processor PROC1 of the processing circuit CT1.

FIG. 5 illustrates only one particular way, among several possible, of implementing the STB decoder, so that it carries out the steps of the method detailed above, in relation to FIGS. 2 to 4 (in any one of the different modes embodiment, or in a combination of these embodiments). In fact, these steps can be carried out either on a reprogrammable computing machine (a PC computer, a DSP processor or a microcontroller) executing a program comprising a sequence of instructions, or on a dedicated computing machine (for example a set of logic gates such as an FPGA or ASIC, or any other hardware module).

Finally, it should be noted here that the term module or the term entity can correspond both to a software component and to a hardware component or a set of hardware and software components, a software component itself corresponding to one or more programs or subroutines. computer or more generally to any element of a program capable of implementing a function or a set of functions as described for the modules concerned. In the same way, a hardware component corresponds to any element of a hardware set (or hardware) capable of implementing a function or a set of functions for the module concerned (integrated circuit, smart card, memory card, etc. .).

Claims

1. A method of managing the display of images associated with image jumps that may be produced during accelerated playback of multimedia content (CNT) on a reproduction device, the content being downloadable from a communication network, characterized in that at least one image associated with a jump is downloaded in advance.

2. Management method according to claim 1, characterized in that the content reading step (CNT) is preceded by a content receiving step from a communication network, and in that the anticipated download is performed when the throughput on the network is less than a given throughput.

3. Management method according to claim 1, characterized in that an image stored in advance is stored in a memory, and in that it is erased from the memory when the reading of the content is stopped.

4. Management method according to claim 1, characterized in that an image stored in advance is stored in a memory, and in that it is erased from the memory after a given number of reading of this image.

5. Management method according to claim 1, characterized in that the images stored in advance are stored in a memory, and in that they are kept in the memory for a given period.

6. Management method according to claim 1, characterized in that the early loading step takes place after a waiting period (DR).

7. Management method according to claim 1, characterized in that when the reading is scheduled for a given date, the early loading step is performed before the given date.

8. Management entity (MNG) for the display of images associated with image jumps likely to be produced during accelerated reading of the content, the content being downloadable from a network of communication, characterized in that it comprises a download module capable of downloading in advance at least one image associated with a jump.

9. Decoder comprising a management entity as defined in claim 8.

10. Computer program capable of being implemented on a management entity (MNG) as defined in claim 8, the program comprising code instructions which, when executed by a processor, carries out the steps of process defined in claim 1.

11. Data medium on which has been stored at least a series of program code instructions for the execution of a method according to claim 1.