WO2004010270A2

WO2004010270A2 - Regulating content usage in a device

Info

Publication number: WO2004010270A2
Application number: PCT/IB2003/002919
Authority: WO
Inventors: Willem Jonker; Adrianus J. M. Denissen
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2002-07-22
Filing date: 2003-06-23
Publication date: 2004-01-29
Also published as: WO2004010270A3; AU2003242951A1

Abstract

A method of regulating usage of a content item (120) by a user, comprising registering usage information for the content item (120) upon usage of the content item (120), and after the registering, billing the user in accordance with the registered usage information for the content item (120). Registration preferably takes places in an end-user device (100) arranged for usage of the content item (120). The usage information could for instance comprise the number of times a content item has been played back, or the duration of such playback.

Description

Regulating content usage in a device

The invention relates to a method of regulating usage of a content item by a user in a device, and to a device for usage such as playback or copying, of a content item.

In recent years, the copying and distribution of content items, such as music or movies, in digital form has become a common practice. The majority of such copying and distribution takes place without authorization of the right holders for those content items. To put an end to this practice, various so-called Digital Rights Management (DRM) systems have been introduced. In their most basic form, the systems simply try to prevent copying of content. Such systems are sometimes also referred to as copy protection systems.

More extensive DRM systems provide elaborate rights management to facilitate various different business models. For example, a user can purchase a right to play back a content item ten times, or a right to make a recording of the content item. However, all these business models are based on the concept that copying is to be restricted, so that the content providers and the rights holders can earn money by selling access to protected specimens of the content.

Most users at present are accustomed to freely using the content they purchase on compact discs, DVDs and so on. They can play their content anywhere, make copies for playback at work or in their cars, and even give their copies to friends. This suggests that the new DRM-based business models might not be well received by these users. However, if unlimited copying and distribution in digital form is permitted without any form of copyright protection, the content industry will be seriously harmed. A fair balance between the interests of the rights holders and the desires of the users needs to be struck.

It is an object of the present invention to provide a method according to the preamble, which balances the interests of the rights holders and the desires of the users.

This object is achieved according to the present invention in a method comprising registering usage information for the content item upon usage of the content item in the device, and after the registering, billing the user for a certain amount in accordance with the registered usage information for the content item. By permitting unlimited usage of the content item, the method does not hamper the interests of the user. Usage usually means playback of the content item, but also refers to copying and/or distribution of the content item. The registered usage information for instance comprises the number of times the content item has been played back, how long the content item has been played back each time, the number of copies and their associated quality levels, the method of distribution and so on.

Any such usage of the content item is registered or metered, typically in the device in which such usage takes place. The registered usage information can then be supplied to the (copy)rights holder for the content item, or to a rights clearinghouse or to another third party. The receiving party can then bill the user for his usage of the content item in accordance with his actual usage. The rights holder thus now receives a fair compensation for the usage of his content.

There is now no longer a need to protect against unauthorized copying: the more copies are made, the more copies are played back, and the more money the rights holders makes.

In an embodiment the content item comprises an advertisement, and the method further comprises determining, on the basis of the registered usage information, whether the advertisement has been played back substantially completely by the user, and if so, crediting the bill with a first sum. Content items such as television programs sometimes contain embedded advertisements. The producer of the television program receives money from the advertisers for presenting these advertisements. It then seems fair that the user shouldn't have to pay the full amount for usage of the television program.

In a further embodiment the method further comprises recording user profile information for the user, and crediting the bill with a second sum upon receipt of the recorded user profile information together with the registered usage information. User profile information, for example indicating which television program the user watches or what music he is interested in, can be very valuable information, especially to advertisers. To entice users in supplying this information, their usage bill is credited with a certain amount if they are willing to share their profiling information.

In a further embodiment the method comprises identifying a manufacturer of the device, and sharing a portion of the certain amount with the manufacturer. This encourages manufacturers to build devices that contain the necessary functionality. It is a further object of the invention to provide a device according to the preamble, which enables balancing the interests of the rights holders and the desires of the users.

This object is achieved according to the present invention in a device as claimed in claim 5. Using the identification and tracking means, it becomes possible to register or meter the usage of the content item in the device. The registered usage information can then be transmitted to a third party for afterwards billing puφoses. Such a device does not inhibit the usage of the content item. The user can use the content exactly as he wishes. On the other hand, the registered usage information allows a third party to send a bill so as to collect royalties for the usage of its content. This way, a fair balance between the interests of the rights holders and the desires of the users is struck.

In an embodiment the identifying means comprise a fingeφrint calculator arranged to obtain the identifier by computing a fingeφrint for at least a portion of the content item. This has the advantage that the identifier can be obtained for any type of content item, even when associated metadata may have been lost because of some type of conversion or copying.

In another embodiment the identifying means are arranged to obtain the identifier from metadata associated with the content item, preferably using a watermark detector arranged to detect a watermark in the content item and to extract the identifier from the metadata encoded using the watermark.

The usage information being registered for the content item preferably comprises a number of times the content item is being used, or an indication of how long the content item has been used. The predetermined criterion preferably comprises a predetermined number of times the content item has been used.

In a further embodiment the device is further arranged to inhibit at least some usage of the content item in response to the reporting means failing to transmit the recorded data to the third party. This provides a simple but effective penalty to users who try to prevent transmission of the recorded data in order to prevent being billed for their usage of the content.

In a further embodiment the device further comprises user profile maintenance means for maintaining a user profile, the reporting means being arranged to additionally transmit at least a portion of the user profile to the third party. This has the advantage that it allows the third party to credit the user on his bill for permitting the transmission of user profile data. Such data is valuable to entities like the third party, and the credit on the bill provides an incentive to the user to permit transmission of such data.

These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments shown in the drawing, in which:

Fig. 1 schematically shows a device arranged for usage of a content item; Fig. 2 schematically shows an alternative embodiment of the device; and Fig. 3 schematically shows a server and a fingeφrint database in more detail.

Throughout the figures, same reference numerals indicate similar or corresponding features. Some of the features indicated in the drawings are typically implemented in software, and as such represent software entities, such as software modules or objects.

Fig. 1 schematically shows a device 100 arranged for usage of a content item 120. In this embodiment, the device 100 is shown as an audio playback apparatus such as a radio or compact disc player. The content item 120, which in this embodiment represents audio material, can be obtained in a variety of ways, as is well known in the art. For example, the content item 120 could have been recorded on a compact disc which is inserted in a receptacle in the device 100, or the content item 120 could be part of a radio transmission or be downloaded from the Internet.

The term "usage" when referring to usage of the content item 120 is of course very broad. The most common types of usage are rendering or playback of the content item 120, recording the content item 120 on a storage medium like a hard disk, a CD-R(W) disc or a DVD+RW disc, or transmitting a copy of the content item 120 to another device, e.g. over an Internet connection.

For the sake of brevity in the embodiment of Fig. 1 the only type of usage discussed is playback or rendering of the content item 120. To this end, a playback module 101 is provided in the device 100 which is arranged to render the content item 120. The device 100 further comprises an identification module 102, a tracking module 103, a recording module 104 and a reporting module 105, whose workings will become apparent below. The exact way in which a content item is rendered depends on the type of device and the type of content. For instance, in a radio receiver, rendering comprises generating audio signals and feeding them to loudspeakers 110. For a television receiver, rendering generally comprises generating audio and video signals and feeding those to a display screen and loudspeakers. For other types of content a similar appropriate action must be taken. Rendering may also include operations such as decrypting or descrambling a received signal, synchronizing audio and video signals and so on.

In accordance with the present invention, the device 100 obtains an identifier for the content item 120 and registers usage information for the content item 120 upon usage of the content item 120. The usage information is then supplied to a third party 130. The third party 130 subsequently bills the user in accordance with the registered usage information for the content item. The third party 130 could for example be a copyright clearinghouse such as the RIAA or the Dutch BUMA/Stemra.

The third party 130 could also be a copyright holder for the content item 120, although this makes the system slightly more complex. The device 100 then has to have some way to find out which copyright holder into contact for a particular content item. This could for instance be realized by making an Internet URL part of the identifier for the content item 120. Assuming the device 100 is Internet-enabled, it can then supply the usage information by submitting the information to the URL in question.

When the content item 120 is being used, for instance when the playback module 101 is used to render the content item 120 on the loudspeakers 110, the identifying module 102 is activated. The identifying module 102 then obtains an identifier for the content item upon usage of the content item, and feeds this identifier to the tracking module 103.

In a preferred embodiment the identification module 102 comprises a watermark detector arranged to detect a watermark in the content item and to extract the identifier from the watermark. Watermarking, the process of inserting extra information in a signal such as an audio or video signal, is an important and well-known technique to mark or protect those signals.

Watermarking an image is essentially a process of altering the pixel values of an image in a manner that ensures that a viewer of the image does not notice any perceptual change between the original and the watermarked image. Altering a large number of pixel values in an arbitrary manner will result in noticeable artifacts. Every pixel value of an image can be altered only to a certain limit without making perceptible differences to the image quality. For audio, the audio signal is modified in a way that a person listening to the resulting audio signal does not notice any perceptual change between the original and the watermarked signal. Technologies for watermarking audio and/or video, and for reliably detecting such watermarks are well known in the art and will not be elaborated upon further.

The alterations in the audio or video signal are typically used to hold some extra information. A watermark detector can extract this extra information by looking at the specific alterations. For instance, a simple watermarking technique manipulates the least significant bit (LSB) of every data word representing the signal. If a bit of the extra information represents a zero, the corresponding LSB is also set to zero. Similarly, if a bit of the extra information represents a one, the corresponding LSB is also set to one.

In this embodiment the extra information represents the identifier for the content item 120. The identifier could be simply an (alpha)numerical string which uniquely identifies the content item 120. For instance, if the content item 120 comprises an electronic book, its ISBN could be embedded using a watermark. The ISBN uniquely identifies the book.

Of course more extensive identifiers could also be used. The only limitation is how much information can be embedded using the chosen watermark technology. If technology (and the size of the content item 120) permits it, one could for instance embed the full title, author, publisher and so on in the content item 120.

In another preferred embodiment the identification module 102 comprises a fingeφrint calculator arranged to obtain the identifier by computing a fingeφrint for at least a portion of the content item. A fingeφrint of a content item is a representation of the most relevant perceptual features of the item in question. Such fmgeφrints are sometimes also known as "(robust) hashes". The fingeφrints of a large number of multimedia objects along with their associated respective metadata, such as the title, artist, genre and so on, are stored in a database. The metadata of a content item is retrieved by computing its fingeφrint and performing a lookup or query in the database using the computed fingeφrint as a lookup key or query parameter. The lookup then returns the metadata associated with the fingeφrint.

An example of a method of computing such a fingeφrint is described in European patent application number 01200505.4 (attorney docket PHNL010110), as well as in Jaap Haitsma, Ton Kalker and Job Oostveen, "Robust Audio Hashing For Content Identification", International Workshop on Content-Based Multimedia Indexing, Brescia, September 2001.

European patent application 01200505.4 describes a method that generates robust fingeφrints for multimedia objects such as, for example, audio clips. The audio clip is divided in successive (preferably overlapping) time intervals. For each time interval, the frequency spectrum is divided in bands. A robust property of each band (e.g. energy) is computed and represented by a respective fingeφrint bit.

A content item is thus represented by a fingeφrint comprising a concatenation of binary values, one for each time interval. The fϊngeφrint does not need to be computed over the whole content item, but can be computed when a portion of a certain length, typically about three seconds, has been received. There can thus be plural fingeφrints for one content item, depending on which portion is used to compute the fingeφrint over. For reasons of clarity, the term "the fingeφrint" will be used even in cases when multiple fingeφrints for one content item can exist.

The fingeφrint for the content item 120 can be considered to be an identifier for the content item 120 if the method used to calculate the fingeφrint is robust enough. When reporting usage information the fingeφrint can be supplied to the third party 130 as well, allowing the third party 130 to properly identify the content item 120. However, since there is always a slight chance that a particular fingeφrint is unreliable, it is recommended that after computing a fingeφrint, a database lookup is performed to obtain metadata comprising a proper identifier. Such a lookup typically takes only a few seconds. If the lookup fails, there is still enough time to compute a new fingeφrint for the content item 120 and to perform a database lookup using the new fingeφrint.

Usually, the database lookup is handled by a central server. This way, the device 100 does not need to maintain the rather large database necessary to identify content items by their fingeφrints. The workings of such a central server are explained below with reference to Fig. 3.

After the identification module 102 has properly obtained at identifier for the content item 120, it transmits the identifier to the tracking module 103. The tracking module 103 in turn registers usage information for the content item 120 and feeds the registered usage information together with the identifier for the content item 120 to the recording module 104 for recording of this data. The recording module 104 could for example be a small hard disk in the device 100.

The registered usage information serves as the basis for afterwards billing. This means that the tracking module 103 has to be programmed with advance knowledge of the billing model that will be used. For example, if billing is done on a pay-per-view or pay- per-play basis, the tracking module 103 only needs to keep track of the number of times a particular content item has been played. If the duration of the playback matters, this duration also should be recorded. The tracking module 103 monitors the operations performed by the playback module 101 to obtain the necessary usage information.

When the recorded data meets a predetermined criterion, the reporting module 105 transmits the recorded data to the third party 130 to allow afterwards billing for usage of the content item 120 in accordance with the registered usage information for the content item 120. An important issue is when the device 100 should submit the information, i.e. what predetermined criterion should be used to determine whether the registered usage information should be transmitted to the third party. Various possibilities exist. Probably the most straightforward one is a fixed period of time, such as a week or a month, after the last time the usage information was transmitted.

Alternatively, the predetermined criterion could comprise a predetermined number of identifiers being recorded. This way, users who use a lot of content items are billed more often than people who only occasionally listen to a song. Further, it is now no longer necessary to send out bills for trivial amounts of money, which would be the case with periodic billing for people who only occasionally use content items.

Instead of only keeping track of the number of content items, additionally also the amount of time that a particular content item was used can be recorded. For example, the device could record that a particular song was played for only 10 seconds, and another song was played for the full 4 minutes 30 seconds. In this case, the predetermined criterion could be chosen on the basis of the recorded amounts of time, for example as a predetermined total amount of time being recorded.

A budget-based approach is also possible. This requires that the tracking module 103 has at least some knowledge of the costs associated with particular types of usage. For example, the user of the device 100 may be provided with a budget of 20 Euros. Upon playback of the content item 120, the tracking module 103 determines the costs associated with such playback and subtracts it from the budget. The predetermined criterion then represents the case that the budget has reached zero, or is within a certain distance from zero. A similar effect can be achieved by choosing as the predetermined criterion a maximum amount of money and instead of subtracting from the budget, adding up the costs until the predetermined maximum has been reached.

It may be desirable to give the user of the device a choice between the various possible predetermined criteria. Some users prefer periodic billing, and others would rather have usage-based bills. The actual values used in the predetermined criteria could also be user defined. The device 100 may be programmed to contain a unique identifier, or at least an identifier of its manufacturer 130. This identifier can be used for a variety of pmrposes. In one application, the identifier is transmitted to the third party 130 together with the registered usage information. From this information the third party 130 can derive which manufacturer made it possible to bill the user of device 100 for a particular amount. The third party 130 can then share a portion of the billed amount with the manufacturer 140.

Fig. 2 schematically illustrates another embodiment of the device 100. Various enhancements are possible to improve the workings of the device 100. In the embodiment of Fig. 2, the device 100 further comprises a user profile maintenance module 301 which maintains a user profile for the user. Such a profile comprises information regarding the user's browsing habits, lifestyle, interests, favorite search keywords and other information that can be gathered by observing the user's browsing behavior. This allows, among other things, the device 100 to recommend content items that may be of interest to the user, or to filter out content items that are less likely to be of interest.

It is also possible to use such user profile information for targeted marketing or advertising. See e.g. international patent application PCT/ LB02/00073 (attorney docket PHNL010072) by the same applicant as the present application. It is thus desirable from a marketing point of view to gain access to user profile information maintained by the module 301. To provide an incentive to the user to supply his user profile information, he could be credited on the bill which he needs to pay for the usage of the content items. The reporting module 105 is now arranged to additionally transmit at least a portion of the user profile to the third party 130.

Additionally, some penalties can be provided in case the reporting module 105 fails to transmit the recorded data to the third party 130. If this happens more than once or twice, one could reasonably assume that the user is trying to prevent the transmission of the recorded data to prevent being billed. In response, the reporting module 105 could cause the playback module 101 to inhibit at least some usage of the content item 120. For example, playback or recording could be refused, the quality of the playback could be reduced or the number of content items that may be played in a row can be reduced.

Fig. 3 schematically shows a server 200 and a fingeφrint database 210 in more detail. The server 200 here comprises an input module 201, an optional fingeφrinting module 202, a Database Management System (DBMS) backend module 203, and a response module 204. The input module 201 receives a fingeφrint from the device 100 and supplies the fingeφrint to the DBMS backend module 203. In an alternative embodiment, the input module 201 receives a plurality of audio clips from the device 100 instead of a number of fingeφrints. These audio clips are then fed to the fingeφrinting module 202. The fingeφrinting module 202 computes a fingeφrint from the received audio clip. As mentioned above, one method for computing a robust fingeφrint is described in European patent application 01200505.4 (attorney docket PHNL010110), although of course any method for computing a robust fingeφrint can be used. The fingeφrinting module 202 then supplies the computed fingeφrint to the DBMS backend module 203.

The DBMS backend module 203 performs a query on the database 210 to retrieve a set of metadata associated with the received fingeφrints from the database 210. As shown in Fig. 3, the database 210 comprises fingeφrints FP1, FP2, FP3, FP4 and FP5 and respective associated sets of metadata MDSl, MDS2, MDS3, MDS4 and MDS5. The above- mentioned European patent application 01200505.4 (attorney docket PHNL010110) describes various matching strategies for matching fingeφrints computed for an audio clip with fingeφrints stored in a database.

European patent application 01202720.7 (attorney docket PHNL010510) describes an efficient method of matching a fingeφrint representing an unknown information signal with a plurality of fingeφrints of identified information signals stored in a database to identify the unknown signal. This method uses reliability information of the extracted fingeφrint bits. The fmgeφrint bits are determined by computing features of an information signal and thresholding said features to obtain the fingeφrint bits. If a feature has a value very close to the threshold, a small change in the signal may lead to a fingeφrint bit with opposite value. The absolute value of the difference between feature value and threshold is used to mark each fϊngeφrint bit as reliable or unreliable. The reliabilities are subsequently used to improve the actual matching procedure.

The database 210 can be organized in various ways to optimize query time and/or data organization. The output of the fingeφrinting module 113 (or fingeφrinting module 202) should be taken into account when designing the tables in the database 210. In the embodiment shown in Fig. 3, the database 210 comprises a single table with entries (records) comprising respective fingeφrints and sets of metadata.

Another way to realize the database 210 is to set up several tables. A first table comprises a plurality of unique identifiers (primary keys) each associated with respective sets of metadata. Such tables can be obtained from various music identification sources. The combination of artist, title and year of release could be combined to form a unique identifier, although this is not guaranteed to be unique, so preferably a really globally unique value is used.

A second table is then set up with entries comprising for each multimedia object the fingeφrints and the unique identifiers from the first table. This way, multiple fingeφrints can be associated with one set of metadata without having to duplicate the metadata. If multiple fingeφrints are possible for one multimedia object, all these fingeφrints are stored in the second table, all associated with the one unique identifier for that multimedia object.

The DBMS backend module 203 then matches the received fingeφrints against the fingeφrints in the second table, obtains an identifier and matches the identifier against the first table to obtain the metadata. If the database 210 is an SQL database, the two tables could be joined on the identifier. The DBMS backend module 203 feeds the results of the query to the response module 204, which transmits the metadata found back to the device 100.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. For example, the database 210 and some or all of the modules 201-204 could be installed in the device 100, if enough storage space and processing capacity is available. This way, no network connection is necessary.

In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.

The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A method of regulating usage of a content item by a user in a device, comprising registering usage information for the content item upon usage of the content item in the device, and after the registering, billing the user for a certain amount in accordance with the registered usage information for the content item.

2. The method of claim 1, in which the content item comprises an advertisement, the method further comprising determining, on the basis of the registered usage information, whether the advertisement has been played back substantially completely by the user, and if so, crediting the bill with a first sum.

3. The method of claim 1 or 2, further comprising recording user profile information for the user, and crediting the bill with a second sum upon receipt of the recorded user profile information together with the registered usage information.

4. The method of claim 1, further comprising identifying a manufacturer of the device, and sharing a portion of the certain amount with the manufacturer.

5. A device arranged for usage of a content item, comprising

- identifying means for obtaining an identifier for the content item upon usage of the content item by a playback module,

- tracking means for registering usage information for the content item,

- recording means for recording data comprising the identifier for the content item and the registered usage information for the content item, and

- reporting means for, when the recorded data meets a predetermined criterion, transmitting the recorded data to a third party to allow afterwards billing for usage of the content item in accordance with the registered usage information for the content item.

6. The device of claim 5, in which the identifying means are arranged to obtain the identifier from metadata associated with the content item.

7. The device of claim 6, in which the identifying means comprise a watermark detector arranged to detect a watermark in the content item and to extract the identifier from the metadata encoded using the watermark.

8. The device of claim 5, in which the identifying means comprise a fingeφrint calculator arranged to obtain the identifier by computing a fingeφrint for at least a portion of the content item.

9. The device of claim 5, in which the usage information being registered for the content item comprises at least one of: a number of times the content item is being used, and an indication of how long the content item has been used.

10. The device of claim 5, in which the predetermined criterion comprises a predetermined number of times the content item has been used.

11. The device of claim 5, in which the predetermined criterion comprises a predetermined amount of time after the last time the usage information has been transmitted previously.

12. The device of claim 5, in which the predetermined criterion comprises a budget associated with usage of content items reaching a particular limit.

13. The device of claim 5, being arranged to inhibit at least some usage of the content item in response to the reporting means failing to transmit the recorded data to the third party.

14. The device of claim 5, further comprising user profile maintenance means for maintaining a user profile, the reporting means being arranged to additionally transmit at least a portion of the user profile to the third party.