JP2007035121A - Reproduction controller and method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce simultaneously two data by selecting data to be reproduced and other data obtained at the same time and applying an effect in accordance with the data to be reproduced. <P>SOLUTION: A management part 31 stores captured voice data and picture data in a DB32. A voice analyzing part 33 reads out voice data selected by a user as a data to be reproduced from the DB 32 and analyzes it. A reproduction data selecting part 34 reads out voice data and a time code added to the voice data from the DB32, while reads out the time code and picture data of the time code from the DB32. A reproducing part 35 outputs the voice data supplied from the reproduction data selecting part 34 from a loudspeaker, while applying the effect adjusted to a sound pattern obtained by the voice analyzing part 33 and displays it on a display. This invention can be applied to a personal computer. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a playback control apparatus and method, and a program, and more particularly to a playback control apparatus and method that can easily reproduce certain data by applying an effect according to other data related to the data. , As well as programs.

  2. Description of the Related Art Conventionally, there is an application that displays on a monitor an image that matches the atmosphere of music to be played when playing music captured in a personal computer or the like.

  Some portable audio players with built-in flash memory and HDD (Hard Disk Drive) that have become popular in recent years are equipped with a display unit such as an LCD (Liquid Crystal Display). The display function is also adopted in an audio player equipped with such a display unit.

  Patent Document 1 discloses a technique for detecting the tempo, the rising position, and the like of a song to be played, and playing a moving image that matches the song together with the song based on the detection result.

Patent Document 2 discloses a technique for realizing a so-called visualizer that gives a visual effect during music reproduction while reducing the processing load.
JP 2004-127019 A JP 2005-122664 A

  By the way, instead of the images prepared for the applications as described above, moving images and still images shot by the user with a digital video camera or the like are displayed in accordance with the sound (music, voice) recorded by another device. In such a case, there is a problem that the user needs to perform an operation for selecting which image and which sound is to be reproduced at which timing using an application or the like for editing a moving image. An image that is played simultaneously with a certain sound is preferably related to the sound, but it is troublesome even to find such a related image.

  Also, during such editing, an effect that is synchronized with the sound that is reproduced with the image (a special effect that matches the atmosphere of the sound) may be applied to the image to be displayed. Since it is necessary to check the contents of the sound and select the contents of the effect in accordance with the confirmation, it is troublesome to apply the effect synchronized with the sound to each image.

  The present invention has been made in view of such a situation, and selects other data acquired at the same time as the data to be reproduced, applies the effect according to the data to be reproduced, and reproduces simultaneously. Is to be able to.

  One aspect of the present invention is a management means for managing first and second data to which a time code representing an acquisition time is added, and the second data acquired at the same time as the first data to be reproduced. And a playback control unit that plays back the first data together with the first data by applying an effect corresponding to the characteristics of the first data.

  The reproduction control means includes the management means in the second directory acquired at the same time as the first data to be reproduced and in the same directory as the first data to be reproduced. The second data managed by the method can be reproduced together with the first data.

  Analyzing means for analyzing the first data can be further provided, and the reproduction control means causes the second data to be subjected to an effect corresponding to the characteristic obtained from the analysis result by the analyzing means. can do.

  The first data may be audio data, and the second data may be image data.

  One aspect of the present invention is a playback control method for performing playback control processing for managing and playing back first and second data to which a time code at the time of acquisition is added, or a program for causing a computer to execute the playback control processing. The second data acquired at the same time as the first data to be reproduced is selected based on the added time code, and an effect corresponding to the characteristics of the first data is applied. A reproduction control method or program including a step of reproducing together with the first data.

  In one aspect of the present invention, the second data acquired at the same time as the first data to be reproduced is selected based on the added time code, and according to the characteristics of the first data The effect is applied and reproduced together with the first data.

  According to one aspect of the present invention, other data acquired at the same time as the data to be reproduced can be selected and reproduced simultaneously. In addition, according to one aspect of the present invention, it is possible to simultaneously reproduce other data acquired at the same time as the data to be reproduced by applying an effect corresponding to the data to be reproduced.

  Embodiments of the present invention will be described below. Correspondences between the configuration requirements of the present invention and the embodiments described in the detailed description of the present invention are exemplified as follows. This description is to confirm that the embodiments supporting the present invention are described in the detailed description of the invention. Accordingly, although there are embodiments that are described in the detailed description of the invention but are not described here as embodiments corresponding to the constituent elements of the present invention, It does not mean that the embodiment does not correspond to the configuration requirements. On the contrary, even if an embodiment is described herein as corresponding to the invention, this does not mean that the embodiment does not correspond to other than the configuration requirements. .

  The reproduction control apparatus according to one aspect of the present invention (for example, the information processing apparatus 1 in FIG. 1) is a management unit (for example, in FIG. 3) that manages the first and second data to which the time code indicating the acquisition time is added. The management unit 31) and the second data (for example, image data) acquired at the same time as the first data (for example, audio data) to be reproduced are based on the added time code. Reproduction control means (for example, the reproduction unit 35 in FIG. 3) that selects and reproduces the first data together with an effect according to the characteristics of the first data.

  The reproduction control apparatus may further include an analysis unit (for example, the voice analysis unit 33 in FIG. 3) that analyzes the first data.

  A reproduction control method or program according to one aspect of the present invention is a reproduction control method of a reproduction control process for managing and reproducing the first and second data to which time codes at the time of acquisition are added, or the reproduction control process. In a program to be executed by a computer, the second data (for example, image data) acquired at the same time as the first data (for example, audio data) to be reproduced is based on an added time code. Selecting, reproducing with the first data by applying an effect according to the characteristics of the first data (for example, step S8 in FIG. 8).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing an information processing apparatus 1 according to an embodiment of the present invention, and an IC recorder 2 and a digital camera 3 connected to the information processing apparatus 1 via a cable.

  The information processing apparatus 1 has a function of taking sound recorded by the IC recorder 2 into a built-in HDD (Hard Disk Drive) 20 (FIG. 2) via a USB (Universal Serial Bus) cable and outputting it to a speaker or the like. Have Further, the information processing apparatus 1 has a function of taking an image taken by the digital camera 3 into the built-in HDD 20 via a USB cable or the like and displaying it on a display or the like.

  As will be described in detail later, when the reproduction of audio data stored in the HDD 20 is instructed, the information processing apparatus 1 adds the same time code as the time code added to the audio data instructed to reproduce. The acquired image data is acquired and the audio data is output to the speaker, and the acquired image data is displayed on the display with an effect according to the sound pattern of the audio data.

  As a result, the user can simultaneously reproduce the audio data and the image data acquired by different devices, and further eliminates the time-consuming editing work of applying an effect synchronized with the sound.

  FIG. 2 is a block diagram illustrating a configuration example of the information processing apparatus 1 in FIG.

  A CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, and a RAM (Random Access Memory) 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14.

  The input / output interface 15 is connected to a recording / playback unit 17, an input unit 18, an output unit 19, an HDD 20, a communication unit 21, a memory card I / F 22, a drive 24, and a USB port 26.

  The CPU 11 executes various processes according to a program stored in the ROM 12 or a program loaded from the HDD 20 to the RAM 13. The RAM 13 also appropriately stores data necessary for the CPU 11 to execute various processes.

  For example, the CPU 11 controls the USB port 26 to capture the audio data recorded by the IC recorder 2 connected to the information processing apparatus 1 via the USB cable, and stores the acquired audio data in the HDD 20. To the audio data captured from the IC recorder 2, a time code indicating the recording date, the recording start time, and the end time is added.

  Further, the CPU 11 takes in the image data captured by the digital camera 3 connected to the information processing apparatus 1 via the USB cable by controlling the USB port 26, and stores the taken-in image data in the HDD 20. Information (time code) indicating the shooting date and shooting time is added to the image data captured from the digital camera 3 according to Exif (Exchangeable image file format).

  Further, when the audio data to be reproduced is selected by the user, for example, the CPU 11 acquires a time code added to the audio data and manages the time added to the audio data managed in the designated directory. Search for image data with the same time code as the code. Then, the CPU 11 acquires image data to which the same time code as the audio data to be reproduced is added (image data taken between the recording start time and the recording end time of the audio data), and outputs the audio data to the speaker. At the same time, the acquired image data is displayed on the display with an effect according to the sound pattern of the audio data.

  The recording / playback unit 17 compresses the television program data acquired from the signal supplied from the antenna 16 by, for example, MPEG (Moving Picture Experts Group) 2 system, and the data (video content) obtained by the compression is compressed. The data is supplied to and stored in the HDD 20 via the input / output interface 15. In addition, the recording / playback unit 17 expands the video content stored in the HDD 20 and causes the output unit 19 to output image data of the obtained television program to the display. That is, the information processing apparatus 1 has a function of recording and playing back a television program.

  The input unit 18 includes, for example, an infrared light receiving unit. The input unit 18 receives a signal from a remote controller (not shown) and outputs information representing the operation contents of the user to the CPU 11.

  The output unit 19 converts the image data supplied via the input / output interface 15 into an analog signal, and outputs the obtained image to a display. The output unit 19 converts audio data supplied via the input / output interface 15 into an analog signal, and outputs the obtained audio to a speaker.

  The HDD 20 stores video content obtained by the recording / playback unit 17, still images captured from the memory card 23 by the memory card I / F 22, audio or digital captured from the IC recorder 2 via the USB port 26 and the USB cable. A still image captured from the camera 3 and music data (audio content) captured from the optical disc 25 by the drive 24 and compressed by the MP3 (MPEG Audio Layer-3) method or the like are stored.

  The communication unit 21 performs communication processing via a network.

  The memory card I / F 22 reads data stored in a memory card 23 installed in a memory card slot formed in the housing of the information processing apparatus 1 and stores the read data in the HDD 20 or the like. For example, still image data or the like is taken into the information processing apparatus 1 via the memory card 23.

  The drive 24 drives the mounted optical disk 25 to read data stored on the optical disk 25 and write data to the optical disk 25. The optical disk 25 is a CD (Compact Disk), a DVD (Digital Versatile Disc), or the like. Data such as still images, audio contents, and video contents are also taken into the information processing apparatus 1 by the optical disk 25.

  The information processing apparatus 1 also has a function as a game machine. An image of a game (program) read from the optical disc 25 by the drive 24 is also supplied to the output unit 19 via the input / output interface 15 and output to the display.

  The USB port 26 communicates with external devices such as the IC recorder 2 and the digital camera 3 via a USB cable, and stores the captured audio and still images in the HDD 20.

  FIG. 3 is a block diagram illustrating a functional configuration example of the information processing apparatus 1. At least a part of the functional units shown in FIG. 3 is realized by executing a predetermined program by the CPU 11 of FIG.

  As shown in FIG. 3, in the information processing apparatus 1, a management unit 31, a database (DB) 32, a voice analysis unit 33, a reproduction data selection unit 34, and a reproduction unit 35 are realized.

  The management unit 31 controls the input / output interface 15 to acquire audio data and image data, and stores the acquired data in a predetermined directory created in the DB 32 for management.

  The voice analysis unit 33 reads, for example, voice data to be reproduced according to the selection by the user from the DB 32, and analyzes the read voice data. By the analysis by the sound analysis unit 33, for example, a sound pattern such as a volume, a frequency, and a tempo of sound to be reproduced is acquired as an analysis result and output to the reproduction unit 35.

  The tempo is calculated based on BPM (Beats per minute) analysis technology, for example, by detecting the rising or falling of a drastic change (attack sound) in the characteristics of the acoustic signal. The tempo of the music can be calculated from the period of the acoustic signal extracted characteristically like the attack sound and the time code of the data. In addition, frequency information having characteristic signal characteristics such as a baseline or a cymbal is extracted as an attack sound.

  The reproduction data selection unit 34 reads out the audio data selected by the user as data to be reproduced from the DB 32 and outputs the read out audio data to the reproduction unit 35.

  Further, the reproduction data selection unit 34 reads the time code added to the audio data selected by the user as the data to be reproduced from the DB 32 and acquires the recording date and time represented by the read time code. As described above, a time code representing the date and time recorded by the IC recorder 2 is added to the audio data.

  When the reproduction data selection unit 34 obtains the recording date and time of the audio data, the image data taken at the same date and time as the audio data is added to each image data from the image data stored in the DB 32. Search based on the current time code. As described above, a time code representing the date and time when the image was taken by the digital camera 3 is added to the image data. The reproduction data selection unit 34 reads out the image data captured at the same date and time as the recording date and time of the audio data from the DB 32 and outputs the image data to the reproduction unit 35.

  The reproduction unit 35 reproduces the audio data supplied from the reproduction data selection unit 34 and outputs the audio data from the speaker, and the sound data obtained by the audio analysis unit 33 using the image data supplied from the reproduction data selection unit 34. Apply effects according to the pattern and display them on the display. That is, the playback unit 35 has a correspondence (table) between the sound pattern and the content of the effect applied to the image data in advance, and determines the content of the effect applied to the image data based on the correspondence.

  FIG. 4 is a diagram illustrating an example of a table included in the reproduction unit 35.

  As shown in FIG. 4, in the table, a sound pattern of audio data and an effect pattern (effect contents) applied to image data are associated with each other. In the example of FIG. 4, sound patterns A1 to A4 and effect patterns E1 to E4 are associated with each other.

  Here, a sound pattern of audio data will be described with reference to FIG.

  As shown in FIG. 5A, a sound pattern in which the sound change amount is smaller than the first threshold (the sound change amount is small) based on the level of the sound change amount (for example, the change amount of the volume) in which the sound data appears in each part The sound change amount is larger than the first threshold value and smaller than the second threshold value (middle sound change amount), and the sound pattern whose sound change amount is larger than the second threshold value (large sound change amount). being classified.

  Further, as shown in FIG. 5B, the sound pattern in which the sound change amount increases smoothly (fade in) based on the temporal increase / decrease of the sound change amount that appears in each part of the sound data, and the sound change amount Is categorized as a sound pattern that fades out smoothly.

  Furthermore, as shown in FIG. 5C, the audio data is classified into sound pattern portions corresponding to changes that appear periodically (for example, attack sounds).

  Note that the sound patterns to be classified are not limited to the sound patterns in FIGS. 5A to 5C, such as a sound pattern in which the sound change amount periodically increases and attenuates.

  Next, effect patterns applied to image data will be described with reference to FIGS.

  As shown in FIG. 6A, an effect pattern for enlarging the image to a predetermined size, as shown in FIG. 6B, an effect pattern for reducing the image to a predetermined size, and as shown in FIG. 6C, the image is displayed at the top of the screen. For example, an effect pattern for moving the image from the right side to the left side as shown in FIG. 6D, and an effect pattern for rotating the image counterclockwise as shown in FIG. 6E are prepared. The

  In addition to the pattern shown in FIG. 6A, for example, the effect pattern for enlarging the image to a predetermined size is predetermined from the state in which the image is displayed on the entire screen display frame (display area) as shown in FIG. 7A. The display range of the image may be changed by enlarging to the size (so-called partially enlarged display). Similarly, the effect pattern for reducing the image to a predetermined size is, for example, as shown in FIG. 7B, in addition to the pattern of FIG. The state may be reduced so as to fit in the entire screen display frame.

  In addition to the pattern shown in FIG. 6C, for example, as shown in FIG. 7C, the effect pattern for moving the image from the upper part to the lower part of the screen is such that an area surrounded by a solid line traces a part of the image. The image inside the area surrounded by the solid line is displayed and the image outside the area surrounded by the solid line (the part indicated by the dotted line in the figure) is covered. Good (may be hidden). Similarly, in the effect pattern for moving the image from the right side to the left side of the screen, in addition to the pattern in FIG. 6D, for example, as shown in FIG. 7D, a region surrounded by a solid line in the drawing traces a part of the image. As described above, the image may be moved from the left side to the right side, and the image in the area surrounded by the solid line may be displayed, and the image outside the area surrounded by the solid line may be covered.

  Furthermore, as the effect pattern for rotating the image counterclockwise, in addition to the pattern of FIG. 6E, for example, as shown in FIG. 7E, an image that does not fit in the screen display frame (partially enlarged state) is displayed. You may make it rotate counterclockwise.

  In addition to the effect patterns shown in FIGS. 6 and 7, although not shown, an effect pattern for moving the image from the bottom to the top of the screen, an effect pattern for moving the image from the left to the right of the screen, and For example, an effect pattern for rotating the image clockwise is prepared.

That is, the prepared effect patterns are not limited to the effect patterns shown in FIGS. 6A to 6E and FIGS. 7A to 7E. For example, an effect pattern for enlarging an image (FIG. 6A) and an image from the top of the screen are displayed. It goes without saying that various patterns are prepared, such as a combination of effect patterns (FIG. 6C) for moving to the bottom, an effect pattern for rotating an image (FIG. 7E), and an effect pattern for reducing an image (FIG. 7B). Nor.

  In addition to the effect pattern as described above, a special effect called a transition can be applied to the image when switching between the previous image and the next image. This is another effect. When there are multiple image data shot at the same date and time as the audio data, the audio data is played and the multiple images are displayed continuously with the effect applied. This transition is appropriately applied to the joint portion.

  Next, transition patterns applied to image data will be described with reference to FIG.

  As shown in FIG. 8A, a transition pattern for simply switching from the previous image to the next image, as shown in FIG. 8B, fades out from the previous image to the black image, and then fades in from the black image to the next image. 8C, a transition pattern in which the previous image fades out as shown in FIG. 8C, and a transition pattern in which the previous image fades out after the previous image fades out, as shown in FIG. 8D. For example, a pattern is prepared.

  A table associating the sound pattern and the effect pattern (including the transition pattern) as described above is prepared in advance and is held by the playback unit 35.

  Next, the reproduction process executed by the information processing apparatus 1 will be described with reference to the flowchart of FIG. This process is started when audio data is selected as reproduction target data by the user and reproduction is instructed.

  In step S <b> 1, the reproduction data selection unit 34 reads out the audio data instructed to be reproduced by the user from the DB 32 and outputs the read out audio data to the reproduction unit 35. The reproduction unit 35 reproduces the audio data supplied from the reproduction data selection unit 34 and outputs it from the speaker.

  In step S2, the sound analysis unit 33 reads out from the DB 32 the sound data instructed to be reproduced by the user, acquires the volume, frequency, tempo, and the like of the read sound data, and analyzes the sound pattern.

  In step S3, the reproduction data selection unit 34 reads the time code added to the audio data instructed to be reproduced by the user from the DB 32, and acquires the recording date and time represented by the read time code.

  In step S4, the reproduction data selection unit 34 stores, in the DB 32, the image data of the same time code as the audio data to be reproduced (image data photographed at the same date) in the DB 32 based on the recording date and time acquired in the process of step S3. Search from the image data in the specified directory stored. Here, the designated directory is a directory in which audio data to be reproduced is stored, but may be a directory arbitrarily designated by the user.

  In step S5, the reproduction data selection unit 34 determines whether or not image data having the same time code as that of the audio data to be reproduced has been searched by the process of step S4, and if it is determined that the image data has been searched, Proceed to S6.

  In step S <b> 6, the reproduction data selection unit 34 acquires image data having the same time code as the searched reproduction target audio data from the DB 32 and outputs the image data to the reproduction unit 35.

  In step S7, the reproducing unit 35 selects an effect pattern corresponding to the sound pattern analyzed in the process of step S2 from the table (FIG. 4). In step S8, the playback unit 35 applies an effect to the image data supplied from the playback data selection unit 34 based on the effect pattern selected in the process of step S7, and displays the effect on the display.

  On the other hand, if it is determined in step S5 that image data having the same time code as the audio data to be reproduced has not been searched, the processes in steps S6 to S8 described above are skipped (that is, no new image display is performed). The process proceeds to step S9.

  In step S9, the playback unit 35 determines whether or not the playback of the audio data has ended. If it is determined that the playback has not ended, the playback unit 35 returns to step S2 and repeats the subsequent processing.

  If it is determined in step S9 that the reproduction of the audio data has been completed, the reproduction process is terminated, and the display of the currently displayed image data is also terminated at substantially the same timing.

  As described above, even without an IC recorder that can capture an image or a composite device such as a digital camera that can record sound, an image related to the sound can be easily reproduced while the sound is reproduced. Can be displayed. In addition, while audio is being played, image data with the same time code as the time code of the audio data (that is, image data related to the audio data) is repeatedly searched, and an effect is applied to the searched image data. Thus, a slide show that sequentially displays images in accordance with the sound is easily performed (without user operation).

  In the above description, image data having the same time code (recording date and time) as the time code (recording date and time) of the audio data is searched. For example, a time code close to the time code of the audio data (for example, the time of the audio data Image data for 30 seconds before and after the code may be searched.

  Further, when a plurality of image data are searched, the plurality of searched images may be displayed simultaneously (all), or may be sequentially displayed one by one.

  Further, in the above description, while the sound is being reproduced, the image data having the same time code as the time code of the sound data is searched. However, the present invention is not limited to this. Audio data having the same time code as that of the image data (that is, audio data related to the image data) is searched and played back while the image data (including moving images and still images) are displayed. Good.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, the program is installed in a general-purpose personal computer from the program storage medium.

  As shown in FIG. 2, a program storage medium for storing a program that is installed in a computer and can be executed by the computer is a magnetic disk (including a floppy disk), an optical disk (CD-ROM (Compact Disk-Read Only). Memory), DVD (including Digital Versatile Disk)), magneto-optical disk (including MD (Mini-Disk)), or removable media such as semiconductor memory, or ROM 12 in which programs are temporarily or permanently stored Or a hard disk included in the HDD 20. The program is stored in the program storage medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 21 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the step of describing the program stored in the program storage medium is not limited to the processing performed in time series according to the described order, but is not necessarily performed in time series. Or the process performed separately is also included.

1 is a diagram illustrating an information processing apparatus, an IC recorder, and a digital camera according to an embodiment of the present invention. It is a block diagram which shows the structural example of the information processing apparatus of FIG. It is a block diagram which shows the function structural example of information processing apparatus. It is a figure which shows the example of a table. It is a figure explaining a sound pattern. It is a figure explaining an effect pattern. It is a figure explaining another effect pattern. It is a figure explaining a transition pattern. It is a flowchart explaining a reproduction | regeneration process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Information processing apparatus, 2 IC recorder, 3 Digital camera, 11 CPU, 19 Output part, 20 HDD, 26 USB port, 31 Management part, 32 DB, 33 Voice analysis part, 34 Playback data selection part, 35 Playback part

Claims (6)

Management means for managing the first and second data to which the time code indicating the acquisition time is added;
The second data acquired at the same time as the first data to be reproduced is selected based on an added time code, and an effect is applied according to the characteristics of the first data. A reproduction control device comprising: reproduction control means for reproducing together with the first data. The reproduction control means is the second data acquired at the same time as the first data to be reproduced, and is further stored in the same directory as the first data to be reproduced by the management means. The reproduction control apparatus according to claim 1, wherein the second data being managed is reproduced together with the first data. An analysis means for analyzing the first data;
The reproduction control device according to claim 1, wherein the reproduction control unit applies an effect to the second data according to a characteristic obtained as a result of analysis by the analysis unit. The first data is audio data;
The reproduction control apparatus according to claim 1, wherein the second data is image data. In the reproduction control method of the reproduction control apparatus for managing and reproducing the first and second data to which the time code at the time of acquisition is added,
The second data acquired at the same time as the first data to be reproduced is selected based on an added time code, and an effect is applied according to the characteristics of the first data. A reproduction control method including a step of reproducing together with first data. In a program for managing a first and second data to which a time code at the time of acquisition is added and causing a computer to execute a reproduction control process for reproduction,
The second data acquired at the same time as the first data to be reproduced is selected based on the added time code, and an effect corresponding to the characteristics of the first data is applied to the first data. A program including a step of reproducing together with one data.

Images