JP6421470B2 - Virtual machine migration program, virtual machine migration system, and virtual machine migration method - Google Patents

Description

  The present invention relates to a virtual machine migration program, a virtual machine migration system, and a virtual machine migration method.

  Currently, there are information processing services that hold computer resources such as processors, memories, and hard disks, and allow users to use the resources via a network. Such an information processing service is sometimes called a cloud service.

  In a facility that provides a cloud service such as a data center, a plurality of virtual computers (virtual machines) may be formed on one physical computer (physical machine) using a virtualization technique. Control software that controls a virtual machine such as a hypervisor allocates resources to each of a plurality of virtual machines. The cloud service facility, for example, secures resources in response to a request from a user and generates a virtual machine for the user. The user can execute desired software such as an operating system (OS) and application software on the generated virtual machine. That is, the user can use the virtual machine resource for the user. A cloud service that provides a user with an information infrastructure (infrastructure) itself, such as a virtual machine, may be referred to as IaaS (Infrastructure as a Service).

  In some cases, a virtual machine can be migrated from one physical machine to another under the control of control software such as a hypervisor. In particular, it is possible to copy memory data, processor status data, etc. to the migration destination physical machine without shutting down the virtual machine OS on the migration source physical machine and take over the virtual machine processing on the migration destination physical machine. There are cases where you can Such a migration method is sometimes called live migration. The migration of virtual machines may be performed in the same cloud service facility when a shortage of resources occurs in some physical machines or when maintenance work is performed with some physical machines stopped.

  When migrating a virtual machine from one physical machine to another, migrating a virtual machine that preferentially sends memory data of a frequently executed program among the programs executed on the virtual machine between physical machines A system has been proposed. In addition, a distributed processing system that selectively executes process migration and virtual machine migration according to a migration destination physical machine when processing is migrated between physical machines has been proposed. Migration of memory data and register data is performed in process units in process migration, and in virtual machine migration in virtual machine units.

  An information processing system for migrating a guest OS between physical machines has been proposed. In this information processing system, image data of the same guest OS is stored in the storage referenced by the migration source physical machine and the storage referenced by the migration destination physical machine. Further, the migration source physical machine prohibits writing of the guest OS into the image data and caches the write data in the memory. When migrating the guest OS, the migration source physical machine copies the memory data to the migration destination physical machine, and the migration destination physical machine uses the image data stored in the migration destination storage and the received memory data to Start the OS.

  Also, a virtualization system has been proposed that reduces the amount of data written to the memory per unit time in the migration source physical machine so as to be smaller than the data transmission bandwidth between the physical machines during live migration. In this virtualization system, interrupts issued when reporting to the program that data has been written to the memory are temporarily suspended and delayed to reduce the writing speed for continuous data writing to the memory. To do.

International Publication No. 2009/0669573 International Publication No. 2010/035480 JP 2011-210151 A JP 2013-250950 A

  By the way, a user may use a plurality of information processing facilities (for example, a plurality of data centers owned by different service providers) separated from each other. At this time, it is conceivable that a virtual machine placed in a certain information processing facility is to be transferred to another information processing facility. For example, when the load on a certain virtual machine becomes high, it may be desired to move the virtual machine to an information processing facility with abundant resources. In addition, for example, it may be possible to move a virtual machine to an information processing facility with less charge or an information processing facility with high security.

  On the other hand, the information processing facility has a management network used for controlling the virtual machine, separately from the user network used for data communication between the user's virtual machines or data communication between the user's virtual machine and an external network. May be provided. In this case, a management network is used for migration within the same information processing facility.

  However, it is not easy to interconnect a management network between a plurality of information processing facilities (making each other's management network accessible) from the viewpoint of security and the like. In particular, it is difficult to interconnect management networks between data centers of different service providers. For this reason, there is a problem that it is difficult to apply the migration method in the same information processing facility to migration between different information processing facilities. As a result, it has been difficult to smoothly migrate virtual machines between different information processing facilities.

  In one aspect, an object of the present invention is to provide a virtual machine migration program, a virtual machine migration system, and a virtual machine migration method that allow a virtual machine to be smoothly migrated between different information processing facilities.

  In one aspect, a virtual machine migration program that causes a computer to execute the following processing is provided. Data of the first operating system using the second operating system included in the second virtual machine started on the computer from another computer in which the first virtual machine including the first operating system is arranged To get. The boot information indicating the boot method of the second virtual machine is rewritten so that the first operating system is executed based on the data of the first operating system acquired when the second virtual machine is rebooted. Reboot the second virtual machine.

In one aspect, a virtual machine migration system having a first information processing apparatus and a second information processing apparatus is provided.
In one aspect, a virtual machine migration method executed by a system having a first information processing apparatus and a second information processing apparatus is provided.

  In one aspect, virtual machines can be smoothly migrated between different information processing facilities.

It is a figure which shows the virtual machine migration system of 1st Embodiment. It is a figure which shows the information processing system of 2nd Embodiment. It is a block diagram which shows the hardware example of a host server. It is a block diagram which shows the example of arrangement | positioning of a virtual machine and a hypervisor. It is a 1st figure which shows the example of migration of a virtual machine. It is a 2nd figure which shows the example of migration of a virtual machine. FIG. 13 is a third diagram illustrating an example of migration of a virtual machine. FIG. 14 is a fourth diagram illustrating an example of migration of a virtual machine. It is a block diagram which shows the function example of a virtual machine. It is a figure which shows the example of a state table. It is a figure which shows the example of disk update information and memory update information. It is a figure which shows the example of a process table. It is a figure which shows the example of a resource restriction table and an important process list. It is a figure which shows the example of a virtual machine generation request. It is a flowchart which shows the example of a procedure of a migration start. It is a flowchart which shows the example of a procedure of disk data transmission. It is a flowchart which shows the example of a procedure of disk monitoring. It is a flowchart which shows the example of a procedure of memory data transmission. It is a flowchart which shows the example of a procedure of memory monitoring. It is a flowchart which shows the example of a procedure of process monitoring. It is a flowchart (continuation) which shows the example of a procedure of process monitoring. It is a flowchart which shows the example of a procedure of the completion of migration. It is a flowchart (continuation) which shows the example of a procedure of a migration completion.

Hereinafter, the present embodiment will be described with reference to the drawings.
[First Embodiment]
FIG. 1 is a diagram illustrating the virtual machine migration system according to the first embodiment.

  The virtual machine migration system according to the first embodiment includes information processing apparatuses 1 and 2. The information processing apparatuses 1 and 2 are computers on which one or two or more virtual machines can be arranged, for example, server computers that provide cloud services such as IaaS to users. The information processing apparatuses 1 and 2 are installed in different information processing facilities (for example, different data centers). The information processing facility to which the information processing device 1 belongs and the information processing facility to which the information processing device 2 belongs may be managed by different service providers. The information processing apparatuses 1 and 2 can communicate with each other via a wide area network such as the Internet.

  A virtual machine 1 a is arranged in the information processing apparatus 1. Information processing resources such as a processor and a memory included in the information processing apparatus 1 are allocated to the virtual machine 1a. The virtual machine 1a includes an operating system 1b. The operating system 1b is executed using resources allocated to the virtual machine 1a.

  The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The processor may include an electronic circuit for a specific purpose such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The processor executes, for example, a program stored in the memory. A set of multiple processors (multiprocessor) may be referred to as a “processor”. The memory is, for example, a semiconductor memory such as a RAM (Random Access Memory).

  The information processing apparatus 1 includes a storage unit 1c. The storage unit 1c is a non-volatile storage device such as an HDD (Hard Disk Drive) or a flash memory. The storage unit 1c includes a storage area 1d. The storage area 1d is a resource assigned to the virtual machine 1a and stores data of the operating system 1b. The data of the operating system 1b may include data other than programs such as programs that define the processing of the operating system 1b and setting files. The information processing apparatus 2 includes a storage unit 2c. The storage unit 2c is a non-volatile storage device such as an HDD or a flash memory.

  Here, consider the migration of the virtual machine 1 a from the information processing apparatus 1 to the information processing apparatus 2. In migration within the same information processing facility, operating system data may be transferred via a management network provided separately from the user network. The user network is a network used for data communication between user virtual machines or between a user virtual machine and the outside of an information processing facility. The management network is a network used for controlling virtual machines.

  However, it is assumed that direct access (interconnection) to the management network is restricted between different information processing facilities from the viewpoint of security and the like. For this reason, it may be difficult to perform migration between the information processing apparatuses 1 and 2 using the management network. Therefore, in the first embodiment, the virtual machine 1a can be migrated without using the management network (for example, using a user network).

  In the migration of the virtual machine 1a, the information processing apparatus 2 activates the virtual machine 2a. The virtual machine 2a includes an operating system 2b. The operating system 2b is software that is temporarily used during migration, and is executed using the processor and memory of the information processing apparatus 2 assigned to the virtual machine 2a.

  Storage areas 2d, 2e, and 2f included in the storage unit 2c are allocated to the virtual machine 2a. The storage area 2d is an empty storage area. The storage area 2e stores data of the operating system 2b. The storage area 2f stores activation information. The activation information indicates a method for activating the virtual machine 2a. For example, the activation information when the virtual machine 2a is activated for the first time indicates that the data stored in the storage area 2e is read at the time of activation. As a result, the operating system 2b is executed when the virtual machine 2a is activated (S1).

  When the virtual machine 2a is activated, the data of the operating system 1b is copied from the information processing apparatus 1 to the information processing apparatus 2. In data transmission, for example, a wide area network and a user network in each information processing facility are used, and a management network in each information processing facility is bypassed. The information processing apparatus 1 transmits the data stored in the storage area 1d to the information processing apparatus 2. The information processing apparatus 2 acquires the data of the operating system 1b from the information processing apparatus 1 using the operating system 2b of the virtual machine 2a. The acquired data of the operating system 1b is written in the storage area 2d (S2).

  When the copy of the data of the operating system 1b is completed, the information processing apparatus 2 rewrites the activation information stored in the storage area 2f using the operating system 2b (S3). The rewritten start information indicates that the data stored in the storage area 2d is read instead of the data stored in the storage area 2e when the virtual machine 2a is restarted. As a result, when the virtual machine 2a is restarted, the operating system 1b is executed instead of the operating system 2b.

  When the activation information is rewritten, the information processing apparatus 2 restarts the virtual machine 2a. For example, the information processing apparatus 2 shuts down the operating system 2b of the virtual machine 2a. The information processing device 2 reads the activation information stored in the storage area 2f, and reads the data stored in the storage area 2d according to the read activation information. As a result, the operating system 1b is executed in the virtual machine 2a (S4). At this time, the virtual machine 1a may be stopped and the virtual machine 1a may be deleted from the information processing apparatus 1. Further, the data of the operating system 2b stored in the storage area 2e may be deleted. The processing of the migration source virtual machine 1a is taken over by the migration destination virtual machine 2a. Therefore, the virtual machine 1a has substantially shifted from the information processing apparatus 1 to the information processing apparatus 2.

  According to the virtual machine migration system of the first embodiment, the virtual machine 2a is activated in the migration destination information processing apparatus 2, and the operating system 2b is temporarily executed on the virtual machine 2a. The data of the operating system 1b can be copied as communication between the virtual machine 1a and the virtual machine 2a. Therefore, even if direct access to the management network from the outside of the information processing facility is restricted, the data of the operating system 1b can be copied using the user network bypassing the management network.

  When the virtual machine 2a is restarted by rewriting the startup information, the operating system 1b is executed on the virtual machine 2a instead of the operating system 2b. Thereby, the migration of the virtual machine 1a can be performed smoothly even between different information processing facilities (for example, between data centers of different service providers). The above migration control can be implemented as a program module incorporated in the operating systems 1b and 2b, for example.

[Second Embodiment]
FIG. 2 illustrates an information processing system according to the second embodiment.
The information processing system according to the second embodiment includes data centers 10 and 20, the Internet 30, and a client 31. The data centers 10 and 20 are information processing facilities that provide cloud services. The data center 10 and the data center 20 are managed by different service providers. The user of the client 31 can use the cloud services of both the data centers 10 and 20 by contract. The client 31 accesses the data centers 10 and 20 via the Internet 30 which is a wide area network.

  The data center 10 includes a user network 11, a management network 12, a management server 13, and host servers 100 and 100a. The data center 20 also has a user network 21, a management network 22, a management server 23, and host servers 200 and 200a so as to correspond to the data center 10. Hereinafter, the data center 10 will be described as a representative, and description of the data center 20 will be omitted.

  The user network 11 is a local network used for data communication by a user virtual machine arranged in the host servers 100 and 100a. The user network 11 is a wired communication network including a communication device such as a switch, for example. The user network 11 is connected to the Internet 30 outside the data center 10. Computers and clients 31 belonging to the data center 20 can communicate with a user virtual machine via the Internet 30 and the user network 11.

  The management network 12 is a local network used by the management virtual machine and the management server 13 arranged on the host servers 100 and 100a to control the user virtual machine. The management network 12 is a wired communication network including a communication device such as a switch, for example. Control of the virtual machine includes generating a new virtual machine on the host servers 100 and 100a, migrating the virtual machine between the host servers 100 and 100a in the data center 10, and the like. For example, the management server 13 instructs the host server 100 to generate a new virtual machine via the management network 12. Computers and clients 31 belonging to the data center 20 do not directly access the management network 12.

  The management server 13 is a server computer that provides a cloud service management interface to the outside of the data center 10. The management server 13 is connected to the Internet 30. Computers and clients 31 in the data center 20 can access the management server 13 via the Internet 30. The management server 13 may receive commands such as creating, starting, stopping, and deleting virtual machines.

  For example, when the generation of a virtual machine is requested, the management server 13 selects a host server having free resources equal to or larger than the requested amount from among the host servers in the data center 10, and uses the selected host server resource as a new virtual server. Assign to a machine. When the activation of the virtual machine is requested, the management server 13 activates the designated virtual machine and causes the operating system to execute on the virtual machine. When the stop of the virtual machine is requested, the management server 13 shuts down the operating system on the designated virtual machine and stops the virtual machine. When deletion of a virtual machine is requested, the management server 13 releases resources allocated to the designated virtual machine.

  The host servers 100 and 100a are server computers having a virtual environment for operating a plurality of virtual machines. Each of the host servers 100 and 100a has one management virtual machine and one or more user virtual machines. The management virtual machine executes a host OS, and the user virtual machine executes a guest OS.

  Each of the host servers 100 and 100a executes a hypervisor that allocates resources such as a processor and a memory included in the host server to a plurality of virtual machines. The hypervisor performs creation / deletion of virtual machines and migration of virtual machines in the data center 10 in accordance with instructions from the management server 13. Further, the hypervisor uses the user network 11 and the management network 12 properly for virtual machine communication. The management network 12 is mainly used for communication of the management virtual machine. The user network 11 is mainly used for communication of a user virtual machine.

  The client 31 is a client computer as a terminal device operated by a user. The client 31 accesses the management server 13 via the Internet 30 when using the cloud service of the data center 10. For example, when a new virtual machine is arranged in the data center 10, the client 31 requests the management server 13 to generate a virtual machine. The generated virtual machine can be accessed via the Internet 30 and the user network 11. Similarly, the client 31 accesses the management server 23 when using the cloud service of the data center 20.

  In the second embodiment, a virtual machine can be migrated between the data center 10 and the data center 20. Data transfer in migration between different data centers is performed as data communication between virtual machines using the user networks 11 and 21 without using the management networks 12 and 22. When migrating a virtual machine from the data center 10 to the data center 20, the client 31 instructs the migration source virtual machine to migrate via the user network 11. When migrating a virtual machine from the data center 20 to the data center 10, the client 31 instructs the migration source virtual machine to migrate via the user network 21.

FIG. 3 is a block diagram illustrating a hardware example of the host server.
The host server 100 includes a CPU 101, a RAM 102, an HDD 103, an image signal processing unit 104, an input signal processing unit 105, a medium reader 106, and communication interfaces 107 and 108. Each of the above units is connected to the bus 109. The HDD 103 is an example of the storage unit 1c according to the first embodiment.

  The CPU 101 is a processor including an arithmetic circuit that executes program instructions. The CPU 101 loads at least a part of the program and data stored in the HDD 103 into the RAM 102 and executes the program. The CPU 101 may include a plurality of processor cores, the host server 100 may include a plurality of processors, and the processes described below may be executed in parallel using a plurality of processors or processor cores. A set of processors (multiprocessor) may be called a “processor”.

  The RAM 102 is a volatile semiconductor memory that temporarily stores programs executed by the CPU 101 and data used by the CPU 101 for calculations. Note that the host server 100 may include a type of memory other than the RAM, or may include a plurality of memories.

  The HDD 103 is a non-volatile storage device that stores software programs such as an OS, middleware, and application software, and data. The program includes a program that controls migration of the virtual machine. The host server 100 may include other types of storage devices such as a flash memory and an SSD (Solid State Drive), and may include a plurality of nonvolatile storage devices.

  The image signal processing unit 104 outputs an image to the display 111 connected to the host server 100 in accordance with a command from the CPU 101. As the display 111, a CRT (Cathode Ray Tube) display, a liquid crystal display (LCD), a plasma display (PDP), an organic electro-luminescence (OEL) display, or the like can be used. .

  The input signal processing unit 105 acquires an input signal from the input device 112 connected to the host server 100 and outputs it to the CPU 101. As the input device 112, a mouse, a touch panel, a touch pad, a pointing device such as a trackball, a keyboard, a remote controller, a button switch, or the like can be used. A plurality of types of input devices may be connected to the host server 100.

  The medium reader 106 is a reading device that reads programs and data recorded on the recording medium 113. Examples of the recording medium 113 include a magnetic disk such as a flexible disk (FD) and an HDD, an optical disk such as a CD (Compact Disc) and a DVD (Digital Versatile Disc), a magneto-optical disk (MO), A semiconductor memory or the like can be used. The medium reader 106 stores, for example, a program or data read from the recording medium 113 in the RAM 102 or the HDD 103.

  The communication interface 107 is connected to the user network 11 and communicates with computers inside and outside the data center 10 via the user network 11. The communication interface 108 is connected to the management network 12 and communicates with computers in the data center 10 via the management network 12. The communication interface 107 may be a wired communication interface connected to a communication device such as a switch by a cable, or may be a wireless communication interface connected to a base station or an access point by a wireless link.

  The host server 100 may not include the medium reader 106, and may not include the image signal processing unit 104 and the input signal processing unit 105 when control is possible from a terminal device operated by the user. . Further, the display 111 and the input device 112 may be formed integrally with the housing of the host server 100. The management servers 13 and 23, the client 31, and the host servers 100a, 200, and 200a can also be realized by the same hardware configuration as the host server 100.

FIG. 4 is a block diagram illustrating an arrangement example of virtual machines and hypervisors.
The host server 100 includes virtual machines 121, 123, and 125. The virtual machines 121 and 123 are user virtual machines generated in response to a request from the user of the client 31. The virtual machine 121 executes a guest OS 122. The virtual machine 123 executes the guest OS 124. On the guest OSs 122 and 124, application software instructed by the user is executed. The virtual machine 125 is a management virtual machine used for managing the host server 100. The virtual machine 125 executes the host OS 126. Management software is executed on the host OS 126.

  In addition, the host server 100 executes the hypervisor 127. The hypervisor 127 allocates resources such as the processing time of the CPU 101 and the storage area of the RAM 102 to the virtual machines 121, 123, and 125, and controls the operations of the virtual machines 121, 123, and 125. The hypervisor 127 has virtual switches 128 and 129.

  The virtual switch 128 manages the communication band of the communication interface 108 and realizes communication between the virtual machine 125 and the management network 12. For example, the virtual switch 128 delivers a packet received from the communication interface 108 to the host OS 126 and transmits a packet output from the host OS 126 from the communication interface 108.

  The virtual switch 129 manages the communication band of the communication interface 107 and realizes communication between the virtual machines 121 and 123 and the user network 11 and communication between the virtual machines 121 and 123. For example, the virtual switch 129 distributes the packet received from the communication interface 107 to the guest OSs 122 and 124 according to the destination address. Further, the virtual switch 129 transmits the packet output from the guest OSs 122 and 124 from the communication interface 107 according to the destination address or delivers the packet to another guest OS.

Next, migration of virtual machines between the data centers 10 and 20 will be described.
FIG. 5 is a first diagram illustrating an example of virtual machine migration.
In the following description of the second embodiment, migration from the host server 100 belonging to the data center 10 to the host server 200 belonging to the data center 20 is considered.

  As described above, the host server 100 has resources such as the CPU 101, the RAM 102, and the HDD 103. Using these resources, the virtual machine 121 including the guest OS 122 operates on the host server 100. Data relating to the virtual machine 121 can be stored in the register 101 a of the CPU 101. Data related to the virtual machine 121 is temporarily stored in the storage area allocated to the virtual machine 121 in the RAM 102. This allocated storage area can also be called a virtual memory.

  Of the storage area in the HDD 103, partitions 131 and 134 are allocated to the virtual machine 121. The partition 131 stores system data related to the guest OS 122. The system data includes a program of the guest OS 122 and a setting file. The partition 134 stores data other than system data that is used by the virtual machine 121. The partition 134 can be used by a user to store arbitrary data. The partitions 131 and 134 can also be called virtual disks for the virtual machine 121.

  The partition 131 includes a startup information area 132 and a save area 133. The activation information area 132 stores activation information. The activation information indicates a partition in which operating system data executed when the virtual machine 121 is activated is stored. Here, the activation information points to the partition 131. Therefore, system data is loaded from the partition 131 to the RAM 102 when the virtual machine 121 is activated.

  The save area 133 is a storage area for saving data related to the virtual machine 121 stored in the register 101 a or the RAM 102. For example, when the virtual machine 121 suspends processing such as transitioning to a suspended state, an interruption file including data of the register 101a, the RAM 102, and other peripheral devices at the time of interruption is written in the save area 133. When the virtual machine 121 resumes the suspended processing, the state is reproduced using the suspended file.

  The host server 200 has resources such as a CPU 201, a RAM 202, and an HDD 203. The virtual machine 221 including the guest OS 222 operates on the host server 200 using these resources. The virtual machine 221 is a migration destination virtual machine that takes over the processing of the virtual machine 121. The guest OS 222 is an operating system that is temporarily executed for migration of the virtual machine 121.

  Data relating to the virtual machine 221 can be stored in the register 201 a of the CPU 201. Data related to the virtual machine 221 is temporarily stored in the storage area allocated to the virtual machine 221 in the RAM 202.

  Of the storage area in the HDD 203, partitions 231, 234, and 235 are allocated to the virtual machine 221. The partition 231 is a storage area prepared for storing system data related to the guest OS 122 migrated from the host server 100. The partition 234 is a storage area prepared for storing data other than system data migrated from the host server 100. The partition 231 corresponds to the partition 131, and the partition 234 corresponds to the partition 134. The partition 235 stores system data related to the guest OS 222.

  The partition 231 includes a startup information area 232 and a save area 233. Similarly to the activation information area 132, the activation information area 232 stores activation information. However, the activation information in the activation information area 232 at the time when the virtual machine 221 is generated points to the partition 235. Therefore, when the virtual machine 221 is activated for the first time, the system data of the guest OS 222 is loaded from the partition 235.

  When starting migration, a virtual machine 221 is generated on the host server 200. Partitions 231, 234, and 235 of the HDD 203 are allocated to the virtual machine 221. The size of the partition 231 may be the same as that of the partition 131, and the size of the partition 234 may be the same as that of the partition 134. An area other than the activation information area 232 in the partition 231 and the partition 234 may be empty.

  The host server 200 stores the system data of the guest OS 222 in the partition 235. The system data of the guest OS 222 is prepared in, for example, a predetermined storage device provided in the data center 10. The host server 200 copies system data from a predetermined storage device to the partition 235. The predetermined storage device may be a nonvolatile storage device included in any of the host servers, or may be a storage device connected to the management network 12. In addition, the host server 200 writes the boot information specifying the partition 235 as the boot partition in the boot information area 232.

  Then, the host server 200 activates the virtual machine 221. At this time, the host server 200 loads the activation information stored in the activation information area 232 into the RAM 202, and loads system data from the partition 235 to the RAM 202 according to the activation information. As a result, the guest OS 222 is executed on the virtual machine 221.

FIG. 6 is a second diagram illustrating an example of virtual machine migration.
When the virtual machine 221 is activated on the host server 200, the guest OS 122 of the virtual machine 121 transmits the system data stored in the partition 131 and the data stored in the partition 134 to the host server 200. The guest OS 222 of the virtual machine 221 stores the received system data of the partition 131 in the partition 231 and stores the received data of the partition 134 in the partition 234.

  In addition, the guest OS 122 of the virtual machine 121 transmits the data of the virtual machine 121 stored in the RAM 102 to the host server 200. The guest OS 222 of the virtual machine 221 stores the received data of the RAM 102 in the save area 233 of the HDD 203. Transmission of data in the partitions 131 and 134 and the RAM 102 is performed as communication between the virtual machine 121 and the virtual machine 221. Therefore, the data transmission is performed using the user networks 11 and 21 without using the management networks 12 and 22.

  Thereafter, the guest OS 122 of the virtual machine 121 monitors whether the data transmitted to the host server 200 has been updated in the host server 100. When updated, the guest OS 122 of the virtual machine 121 transmits the difference data from the previous transmission to the host server 200. The guest OS 222 of the virtual machine 221 overwrites and saves the received difference data in the corresponding storage area in the partitions 231 and 234. The transmission of the difference data is continued until the data update amount from the previous difference data transmission in the host server 100 decreases to a threshold value or until the decrease in the data update amount reaches a limit.

FIG. 7 is a third diagram illustrating an example of virtual machine migration.
When the data update amount per unit time in the host server 100 decreases, the guest OS 122 of the virtual machine 121 transmits final data before stopping the virtual machine 121 to the host server 200. In addition to the update data of the partitions 131 and 134 and the update data of the RAM 102, the final data includes state data of the CPU 101 stored in the register 101a, device data indicating the state of peripheral devices, and the like. The data in the register 101a can be extracted from the interrupt file stored in the save area 133 when the guest OS 122 temporarily transitions to the suspended state.

  The guest OS 222 of the virtual machine 221 overwrites and saves the update data of the partitions 131 and 134 included in the received final data in the partitions 231 and 234. Further, the guest OS 222 of the virtual machine 221 overwrites and saves the update data of the RAM 102 included in the received final data in the save area 233, and stores the state data of the CPU 101 included in the received final data in the save area 233.

  Then, the guest OS 222 of the virtual machine 221 generates an interruption file for reproducing the state of the virtual machine 121 from the latest data of the RAM 102 and the state data of the CPU 101 and stores it in the save area 233. Also, the guest OS 222 of the virtual machine 221 rewrites the boot information stored in the boot information area 232 so that the system data of the partition 231 is loaded instead of the system data of the partition 235 at the time of booting. At this time, the guest OS 122 of the virtual machine 121 may be shut down.

FIG. 8 is a fourth diagram illustrating an example of migration of a virtual machine.
When the interrupt file is generated in the host server 200 and the startup information is rewritten, the guest OS 222 of the virtual machine 221 issues a command for restarting the virtual machine 221. Then, the guest OS 222 is shut down, and the virtual machine 221 loads the activation information stored in the activation information area 232. Since the startup information points to the partition 231, the virtual machine 221 loads system data from the partition 231 to the RAM 202. The loaded system data is the system data of the guest OS 122.

  When the guest OS 122 is activated on the virtual machine 221, the guest OS 122 of the virtual machine 221 uses the interruption file stored in the save area 233 to reproduce the state of the virtual machine 121 when the virtual machine 121 is stopped. The guest OS 122 of the virtual machine 221 loads the state data of the CPU 101 included in the interruption file into the register 201a of the CPU 201. In addition, the guest OS 122 of the virtual machine 221 loads the data in the RAM 102 included in the interruption file into the RAM 202. Thereby, the virtual machine 221 can take over the processing of the virtual machine 121 when the virtual machine 121 is stopped.

Next, the migration function that the virtual machines 121 and 221 have will be described.
FIG. 9 is a block diagram illustrating an example of functions of a virtual machine.
The guest OS 122 executed in the virtual machine 121 includes a migration control unit 141, an update monitoring unit 142, and a kernel 143. For example, the migration control unit 141 and the update monitoring unit 142 can be implemented as program modules incorporated in the guest OS 122. The above program module may be incorporated in advance by the data center 10 or may be incorporated by the user of the virtual machine 121.

  For example, when the virtual machine 121 is created in the data center 10, the guest OS 122 having the migration control unit 141 and the update monitoring unit 142 may be automatically used. Further, the user of the virtual machine 121 may add the above program module to the operating system prepared by the data center 10. The user of the virtual machine 121 may change the operating system executed by the virtual machine 121 to the guest OS 122 after the virtual machine 121 is generated.

  The migration control unit 141 controls migration of the virtual machine 121. The migration control unit 141 receives an instruction to start migration from the client 31 via the user network 11. Then, the migration control unit 141 accesses the management server 23 of the data center 20 via the user network 11 and causes the data center 20 to generate the migration destination virtual machine 221. Then, the migration control unit 141 transmits the data of the HDD 103 and the data of the RAM 102 related to the virtual machine 121 to the virtual machine 221 via the user networks 11 and 21.

  Further, the migration control unit 141 monitors a process being executed in the virtual machine 121 while continuing to transmit data to the virtual machine 221. When the data update frequency by the process of the virtual machine 121 is high, the migration control unit 141 limits the CPU usage rate of each process so that the data update frequency decreases. When the data transmission is completed and the migration destination virtual machine 221 is restarted, the guest OS 122 is executed on the virtual machine 221. In this case, the migration control unit 141 executes post-migration processing (for example, deletion of the migration source virtual machine 121) in the virtual machine 221.

  The update monitoring unit 142 monitors the update of the data in the HDD 103 or the data in the RAM 102 in accordance with an instruction from the migration control unit 141. The writing of data to the HDD 103 or the RAM 102 can be detected using the kernel 143, for example. The update monitoring unit 142 provides information indicating the updated data to the migration control unit 141.

  The kernel 143 is a program module that implements a core function of the guest OS 122. The kernel 143 controls access to hardware such as the CPU 101 and the RAM 102, and provides an interface related to hardware access to program modules other than the kernel 143 in the guest OS 122. For example, the kernel 143 collects process information and information on access to the RAM 102 and the HDD 103. Further, the kernel 143 assigns the processing time of the CPU 101 to each process.

  The virtual machine 121 has a control information storage unit 144. The control information storage unit 144 can be realized as, for example, a storage area of the RAM 102 allocated to the virtual machine 121 or a storage area of the HDD 103. The control information storage unit 144 stores control information used by the migration control unit 141 and the update monitoring unit 142 for the migration process.

  The guest OS 222 executed on the virtual machine 221 has a migration control unit 241. For example, the migration control unit 241 can be implemented as a program module incorporated in the guest OS 222. This program module may be incorporated in advance by the data center 20. For example, the guest OS 222 having the migration control unit 241 may be automatically selected by the data center 20 when the virtual machine 221 is generated as a migration destination.

  The migration control unit 241 controls the migration of the virtual machine 121 in cooperation with the migration source migration control unit 141. The migration control unit 241 receives data related to the virtual machine 121 from the migration control unit 141 and stores the received data in the HDD 203. When all the data has been received, the migration control unit 241 rewrites the startup information of the virtual machine 221 so that when the virtual machine 221 is restarted, the guest OS 122 is executed based on the received data.

  Also, the migration control unit 241 accesses the management server 13 of the data center 10 via the user network 21 and stops the virtual machine 121. Then, the migration control unit 241 restarts the virtual machine 221. In the virtual machine 221 after the restart, the guest OS 122 is executed instead of the guest OS 222. Therefore, the migration control unit 141 takes over from the migration control unit 241 for the migration process in the migration destination virtual machine 221.

  The virtual machine 221 has a control information storage unit 242. The control information storage unit 242 can be realized, for example, as a storage area of the RAM 202 allocated to the virtual machine 221 or a storage area of the HDD 203. The control information storage unit 242 stores control information that the migration control unit 241 uses for migration processing. When the system data of the guest OS 122 is transferred from the virtual machine 121 to the virtual machine 221, the control information stored in the control information storage unit 144 is also transferred from the virtual machine 121 to the virtual machine 221.

FIG. 10 is a diagram illustrating an example of a state table.
The control information storage unit 144 stores a state table 151. The state table 151 has items of parameter name and flag. In the parameter name item, “migrated”, “disk update drop”, and “memory update drop” are registered as parameter names. In the flag item, “1” (True) or “0” (False) is registered.

  The migrated parameter = 1 indicates that the data of the virtual machine 121 has been copied to the virtual machine 221, and the migrated parameter = 0 indicates that the data has not been copied. The disk update decrease parameter = 1 indicates that the amount of difference data of the untransmitted HDD 103 is equal to or less than the threshold, and the disk update decrease parameter = 0 indicates that it is greater than the threshold. The memory update decrease parameter = 1 indicates that the amount of difference data in the untransmitted RAM 102 is equal to or less than the threshold, and the memory update decrease parameter = 0 indicates that it is greater than the threshold. The initial value of the flag for all parameters is “0”.

FIG. 11 is a diagram illustrating an example of disk update information and memory update information.
The control information storage unit 144 stores disk update information 152 and memory update information 153. The disk update information 152 is used when the update monitoring unit 142 monitors the update of data in the HDD 103. The memory update information 153 is used when the update monitoring unit 142 monitors the update of data in the RAM 102.

  The disk update information 152 is a bitmap that represents the presence or absence of writing to the storage area of the HDD 103 allocated to the virtual machine 121 in units of sectors. The sector is, for example, a fixed-length area obtained by subdividing the storage area of the HDD 103. When data is written to a certain sector and the data has not been transmitted to the virtual machine 221, the bit corresponding to the sector is set to “1”. The other bits are set to “0”.

  The memory update information 153 is a bitmap representing whether or not data is written to the storage area of the RAM 102 allocated to the virtual machine 121 in units of pages. The page is a fixed-length area obtained by subdividing the storage area of the RAM 102, for example. When data is written to a certain page and the data has not been transmitted to the virtual machine 221, the bit corresponding to the page is set to “1”. The other bits are set to “0”.

FIG. 12 is a diagram illustrating an example of a process table.
The control information storage unit 144 stores a process table 154. The process table 154 is used when the migration control unit 141 limits the CPU usage rate of each process. The process table 154 includes items of process ID, process name, start state, state, CPU usage rate, disk update amount, and memory update amount.

  In the process ID item, the process identification number assigned by the kernel 143 is registered. In the process name item, the name of the process (for example, the name of the program that defines the processing of the process) is registered. In the start state item, the state of each process at the start of the migration of the virtual machine 121 is registered. Examples of the start state include “running” or “pause”. In the status item, the current status of each process is registered. Examples of the state include “running”, “pause”, and “terminated”.

  In the item of CPU usage rate, an index value of CPU time (resource amount) currently allocated to each process by the kernel 143 is registered. The CPU time can be expressed, for example, as a percentage when the processing capability of one processor or processor core is 100. In the disk update amount item, the write amount to the HDD 103 per unit time by each process is registered. The amount of writing to the RAM 102 per unit time by each process is registered in the item of memory update amount. Information registered in the process table 154 can be acquired using, for example, an interface of the kernel 143.

FIG. 13 is a diagram illustrating an example of a resource limit table and an important process list.
The control information storage unit 144 stores a resource limit table 155 and an important process list 156. The resource limit table 155 and the important process list 156 are used when the migration control unit 141 limits the CPU usage rate of each process. The resource restriction table 155 and the important process list 156 are created by the user of the virtual machine 121 and stored in the control information storage unit 144, for example.

  The resource limit table 155 has items of process name and resource upper limit. In the process name item, a process name similar to the process table 154 is registered. The upper limit of the CPU usage rate when the CPU usage rate is most severely limited for each process is registered in the resource upper limit item. For example, in the example of FIG. 13, the resource upper limit of the process B is set to 20%, while in the example of FIG. 12, the current CPU usage rate of the process B is 25%. Therefore, when the CPU usage rate is limited, the CPU usage rate of the process B can be reduced to 20% or less by reducing the allocation of the CPU time to the process B.

  One or more process names are registered in the important process list 156. The process indicated by the important process list 156 is an important process for the user of the virtual machine 121, and indicates a process that does not reduce the CPU time allocation (excludes the CPU usage rate from being restricted). For example, the processes A and C among the processes A, B, C, D, and E are registered in the important process list 156. In this case, the CPU usage rates of the processes B, D, and E may be limited, while the CPU usage rates of the processes A and C are not limited.

FIG. 14 is a diagram illustrating an example of a virtual machine generation request.
The virtual machine generation request 157 is transmitted from the migration control unit 141 of the virtual machine 121 to the management server 23 of the data center 20. The virtual machine generation request 157 indicates a request for generating the migration destination virtual machine 221. The virtual machine generation request 157 includes a plurality of parameter name and setting value pairs. The parameters include the number of CPUs, memory capacity, system disk capacity, other data disk capacity, network settings, and initial OS.

  “Number of CPUs” indicates the number of CPUs assigned to the virtual machine 221. Usually, the number of CPUs may be the same as that of the virtual machine 121. “Memory capacity” indicates the size of the storage area of the RAM 202 allocated to the virtual machine 221. Usually, the memory capacity may be the same as that of the virtual machine 121. “System disk capacity” indicates the size of the partitions 231 and 235 of the HDD 203 allocated to the virtual machine 221. Normally, the system disk capacity is the size of the partition 131 of the virtual machine 121 plus the size of the partition 235 that stores the system data of the guest OS 222. The format of the system disk may be the same as that of the partition 131 of the virtual machine 121.

  “Other data disk capacity” indicates the size of the partition 234 of the HDD 203 allocated to the virtual machine 221. Usually, the other data disk capacity may be the same as the partition 134 of the virtual machine 121. The format of the other data disk may be the same as the partition 134 of the virtual machine 121. “Network setting” indicates a network that the virtual machine 221 can use. Usually, the network setting may be the same as that of the virtual machine 121. However, since the virtual machine 221 communicates with the virtual machine 121, if the Internet 30 is not included in the network used by the virtual machine 121, the Internet 30 is additionally registered (in this case, the setting of the virtual machine 121 is also changed). .

  “Initial OS” indicates the type of operating system executed when the virtual machine 221 is first started. When the migration OS is designated as the initial OS, the system data of the guest OS 222 having the migration control unit 241 is stored in the partition 235. When the virtual machine 221 is activated, the guest OS 222 is executed.

Next, migration processing performed by the virtual machines 121 and 221 will be described.
FIG. 15 is a flowchart illustrating an exemplary procedure for starting migration.
The client 31 transmits a migration start instruction to the virtual machine 121 via the Internet 30 and the user network 11 (S110).

  When the migration start instruction is received, the migration control unit 141 on the virtual machine 121 transmits a virtual machine generation request 157 to the management server 23 of the data center 20 via the user network 11 and the Internet 30 (S111). At this time, the migration control unit 141 calculates the setting value of each parameter of the virtual machine generation request 157 from the resource allocation status of the virtual machine 121. In principle, the configuration of the virtual machine 221 may be the same as that of the virtual machine 121 except for the partition 235.

  Upon receiving the virtual machine generation request 157, the management server 23 allocates the amount of resources specified by the virtual machine generation request 157 to the virtual machine 221, thereby generating the virtual machine 221 on the host server 200 (S112). The management server 23 notifies the virtual machine 121 of the address (for example, IP (Internet Protocol) address) given to the virtual machine 221 via the Internet 30 and the user network 11 (S113). The migration control unit 141 generates the state table 151 in the control information storage unit 144, and initializes all the flags to “0” (S114).

  The migration control unit 141 transmits a virtual machine activation request designating the virtual machine generated in step S112 to the management server 23 via the user network 11 and the Internet 30 (S115). When receiving the virtual machine activation request, the management server 23 specifies the host server 200 on which the designated virtual machine 221 is arranged, and instructs the hypervisor of the host server 200 to initiate a virtual machine via the management network 12. Is transmitted (S116). The hypervisor of the host server 200 activates the virtual machine 221. At this time, in accordance with the boot information stored in the boot information area 232, the system data stored in the partition 235 is loaded and the guest OS 222 is executed. Then, the migration control unit 241 is activated and enters a connection waiting state (S117).

  The migration control unit 141 transmits a connection request to the virtual machine 221 via the user network 11 and the Internet 30 using the address notified in step S113 (S118). The migration control unit 241 performs a connection process and returns a connection completion notification to the virtual machine 121 (S119). The migration control unit 141 activates three processes of disk data transmission, memory data transmission, and process monitoring described below (S120). These three processes are executed in parallel.

FIG. 16 is a flowchart illustrating an example of a procedure for transmitting disk data.
This disk data transmission process is started in step S120.
The migration control unit 141 instructs the update monitoring unit 142 to initialize the disk update information 152 (S210). The update monitoring unit 142 initializes all bits of the disk update information 152 stored in the control information storage unit 144 to “0” (S211). The update monitoring unit 142 notifies the migration control unit 141 of completion of bitmap initialization. The migration control unit 141 notifies the update monitoring unit 142 of a disk write monitoring instruction (S212). The update monitoring unit 142 starts the disk monitoring process by starting the disk monitoring process (S213). The disk monitoring process is executed in parallel with other processes such as the process of FIG. 16 (S214). The disk monitoring process will be described later.

  The update monitoring unit 142 notifies the migration control unit 141 of the start of disk monitoring. The migration control unit 141 transmits the data stored in the partitions 131 and 134 of the HDD 103 to the virtual machine 221 on a sector basis via the user network 11, the Internet 30, and the user network 21 (S215). At this time, the migration control unit 141 may not transmit the data of the sector in which the bit of the disk update information 152 is “1”. At the time of data transmission, a sector address is added. The migration control unit 241 stores the received data in the storage area indicated by the sector address in the partitions 231 and 234 (S216).

  The migration control unit 241 returns a data reception completion to the virtual machine 121. When the initial data transmission is completed for the entire partitions 131 and 134, the migration control unit 141 inquires of the update monitoring unit 142 about disk update (S217). The update monitoring unit 142 identifies the sector in which the bit of the disk update information 152 is “1”, that is, the sector in which the data has been written, and notifies the migration control unit 141 of the sector address of the identified sector (S218). ).

  The migration control unit 141 determines whether the number of sectors in which data has been written or the amount of data (disk update amount) corresponding to the number of sectors is equal to or less than a threshold (S219). If the disk update amount is less than or equal to the threshold, the disk update drop flag in the status table 151 is set to “1”, and the process proceeds to step S226. If the disk update amount is larger than the threshold, the disk update decrease flag is set to “0”, and the process proceeds to step S220.

  If the disk update amount is larger than the threshold value, the migration control unit 141 instructs the update monitoring unit 142 to initialize the disk update information 152 (S220). The update monitoring unit 142 initializes all bits of the disk update information 152 to “0” (S221). The update monitoring unit 142 notifies the migration control unit 141 of completion of bitmap initialization.

  The migration control unit 141 reads data (updated data) indicated by the sector address notified in step S218 or step S225 described later from the partitions 131 and 134 as disk differential data. The migration control unit 141 adds a sector address to the disk difference data, and transmits it to the virtual machine 221 via the user network 11, the Internet 30, and the user network 21 (S222). The migration control unit 241 overwrites and saves the received data in the storage area indicated by the sector address in the partitions 231 and 234 (S223).

  The migration control unit 241 returns a data reception completion to the virtual machine 121. The migration control unit 141 inquires of the update monitoring unit 142 about disk update (S224). The update monitoring unit 142 identifies the sector in which the bit of the disk update information 152 is “1”, that is, the sector in which data has been written since the previous confirmation, and the sector address of the identified sector is designated as the migration control unit 141. (S225).

  The migration control unit 141 sets the disk update decrease flag of the status table 151 to “1” when the disk update amount is equal to or less than the threshold value, and sets the disk update decrease flag to “0” when the disk update amount is greater than the threshold value. Set. Then, the migration control unit 141 determines whether both the disk update decrease flag and the memory update decrease flag in the state table 151 are “1” (S226). When the two flags are “1”, the disk data transmission process ends, and a migration completion process described later is started. If at least one of the flags is “0”, the process proceeds to step S220.

FIG. 17 is a flowchart illustrating an example of a procedure for disk monitoring.
This disk monitoring process is started in step S214.
The update monitoring unit 142 performs hook setting using the interface of the kernel 143 so that the writing is detected when writing to the HDD 103 occurs (S230). When the writing to the HDD 103 is detected, the update monitoring unit 142 specifies a writing destination sector (S231). The update monitoring unit 142 sets the bit corresponding to the sector specified in step S231 in the disk update information 152 to “1” (S232).

  The update monitoring unit 142 determines whether both the disk update decrease flag and the memory update decrease flag in the status table 151 are “1” (S233). If the two flags are “1”, the disk monitoring process ends. If at least one of the flags is “0”, the process proceeds to step S231 and waits for the next writing to the HDD 103 to be detected.

FIG. 18 is a flowchart illustrating a procedure example of memory data transmission.
This memory data transmission process is started in step S120 described above.
The migration control unit 141 instructs the update monitoring unit 142 to initialize the memory update information 153 (S240). The update monitoring unit 142 initializes all bits of the memory update information 153 stored in the control information storage unit 144 to “0” (S241). The update monitoring unit 142 notifies the migration control unit 141 of completion of bitmap initialization. The migration control unit 141 notifies the update monitoring unit 142 of a memory write monitoring instruction (S242). The update monitoring unit 142 starts monitoring memory writing by starting a memory monitoring process (S243). The memory monitoring process is executed in parallel with other processes such as the process of FIG. 18 (S244). The memory monitoring process will be described later.

  The update monitoring unit 142 notifies the migration control unit 141 of the start of memory monitoring. The migration control unit 141 transmits the data stored in the storage area of the RAM 102 allocated to the virtual machine 121 to the virtual machine 221 in units of pages via the user network 11, the Internet 30, and the user network 21 (S245). . At this time, the migration control unit 141 may not transmit the data of the page in which the bit of the memory update information 153 is “1”. A page address is added when data is transmitted. The migration control unit 241 stores the received data in the storage area indicated by the page address in the RAM 202 (S246).

  The migration control unit 241 returns a data reception completion to the virtual machine 121. When the initial data transmission is completed for the entire storage area of the RAM 102 allocated to the virtual machine 121, the migration control unit 141 inquires of the update monitoring unit 142 about memory update (S247). The update monitoring unit 142 identifies a page in which the bit of the memory update information 153 is “1”, that is, a page where data has been written, and notifies the migration control unit 141 of the page address of the identified page (S248). ).

  The migration control unit 141 determines whether the number of pages in which data has been written or the amount of data corresponding to the number of pages (memory update amount) is equal to or less than a threshold (S249). If the memory update amount is less than or equal to the threshold, the memory update decrease flag in the state table 151 is set to “1”, and the process proceeds to step S256. If the memory update amount is larger than the threshold, the memory update decrease flag is set to “0”, and the process proceeds to step S250.

  When the memory update amount is larger than the threshold value, the migration control unit 141 instructs the update monitoring unit 142 to initialize the memory update information 153 (S250). The update monitoring unit 142 initializes all bits of the memory update information 153 to “0” (S251). The update monitoring unit 142 notifies the migration control unit 141 of completion of bitmap initialization.

  The migration control unit 141 reads data (updated data) indicated by the memory address notified in step S248 or step S255 described later from the RAM 102 as memory difference data. The migration control unit 141 adds a page address to the memory difference data, and transmits it to the virtual machine 221 via the user network 11, the Internet 30, and the user network 21 (S252). The migration control unit 241 overwrites and saves the received data in the storage area indicated by the page address in the RAM 202 (S253).

  The migration control unit 241 returns a data reception completion to the virtual machine 121. The migration control unit 141 inquires of the update monitoring unit 142 about memory update (S254). The update monitoring unit 142 identifies a page in which the bit of the memory update information 153 is “1”, that is, a page in which data has been written since the previous check, and the page address of the identified page is designated as the migration control unit 141. (S255).

  The migration control unit 141 sets the memory update decrease flag of the state table 151 to “1” when the memory update amount is equal to or less than the threshold value, and sets the memory update decrease flag to “0” when the memory update amount is greater than the threshold value. Set. Then, the migration control unit 141 determines whether both the disk update decrease flag and the memory update decrease flag in the state table 151 are “1” (S256). When the two flags are “1”, the memory data transmission process ends, and a migration completion process described later is started. If at least one of the flags is “0”, the process proceeds to step S250.

FIG. 19 is a flowchart illustrating an example of a memory monitoring procedure.
This memory monitoring process is started in step S244 described above.
The update monitoring unit 142 performs hook setting using the interface of the kernel 143 so that the writing is detected when writing to the RAM 102 occurs (S260). When the update monitoring unit 142 detects writing in the RAM 102, the update monitoring unit 142 specifies a page to be written (S261). The update monitoring unit 142 sets the bit corresponding to the page specified in step S261 in the memory update information 153 to “1” (S262).

  The update monitoring unit 142 determines whether both the disk update decrease flag and the memory update decrease flag in the state table 151 are “1” (S263). If the two flags are “1”, the memory monitoring process ends. If at least one of the flags is “0”, the process proceeds to step S261 and waits for the next writing to the RAM 102 to be detected.

FIG. 20 is a flowchart illustrating an example of a process monitoring procedure.
This process monitoring process is started in step S120 described above.
The migration control unit 141 acquires a list of processes existing on the guest OS 122 using the interface of the kernel 143 (S270). The process list includes a process ID, a process name, and a status (running or paused). The migration control unit 141 registers the acquired process ID and process name in the process table 154, and registers the acquired state in the process table 154 as a start state.

  The migration control unit 141 acquires the current state, CPU usage rate, disk update amount, and memory update amount for each process using the interface of the kernel 143 (S271). The migration control unit 141 registers the acquired state, CPU usage rate, disk update amount, and memory update amount in the process table 154.

  The migration control unit 141 selects at most one process that satisfies the following conditions regarding disk update from among the processes registered in the process table 154 (S272). Processes that satisfy the conditions are not registered in the important process list 156, and among the processes in which the CPU usage rate exceeds the resource upper limit registered in the resource limit table 155, the disk update amount per unit time is the largest. Is.

  The migration control unit 141 selects at most one process that satisfies the following conditions regarding the memory update from among the processes registered in the process table 154 (S273). Processes that satisfy the conditions are not registered in the important process list 156, and among processes whose CPU usage rate exceeds the resource upper limit registered in the resource limit table 155, the memory update amount per unit time is the largest. Is.

  If there is a process selected in steps S272 and S273, the migration control unit 141 uses the interface of the kernel 143 to set the upper limit of the CPU usage rate of the process to the resource upper limit registered in the resource limit table 155 ( S274). As a result, it can be expected that the allocation of the CPU time to the selected process is reduced and the disk update amount or the memory update amount per unit time is reduced.

  The migration control unit 141 acquires the current state, CPU usage rate, disk update amount, and memory update amount for each process using the interface of the kernel 143 (S275). The migration control unit 141 overwrites and saves the acquired state, CPU usage rate, disk update amount, and memory update amount in the process table 154. The migration control unit 141 acquires the disk update amount and memory update amount per unit time of the entire guest OS 122 using the interface of the kernel 143 (S276). However, the migration control unit 141 may calculate the value of the entire guest OS 122 from the disk update amount and the memory update amount for each process acquired in step S275.

FIG. 21 is a flowchart (continuation) illustrating an example of a process monitoring procedure.
The migration control unit 141 determines whether the total disk update amount acquired in step S276 is equal to or less than a threshold value (S277). If the total disk update amount is less than or equal to the threshold, the process proceeds to step S278, and if greater than the threshold, the process proceeds to step S279. When the total disk update amount is equal to or smaller than the threshold, the migration control unit 141 sets the disk update decrease flag of the status table 151 to “1” (S278). Then, the process proceeds to step S280. When the total disk update amount is larger than the threshold, the migration control unit 141 sets the disk update decrease flag of the status table 151 to “0” (S279).

  The migration control unit 141 determines whether the total memory update amount acquired in step S276 is equal to or less than a threshold value (S280). If the total memory update amount is equal to or smaller than the threshold value, the process proceeds to step S281, and if larger than the threshold value, the process proceeds to step S282. When the total memory update amount is equal to or smaller than the threshold, the migration control unit 141 sets the memory update decrease flag of the state table 151 to “1” (S281). Then, the process proceeds to step S283. When the total memory update amount is larger than the threshold value, the migration control unit 141 sets the memory update decrease flag of the state table 151 to “0” (S282).

  Note that the threshold value in step S277 is defined in advance by, for example, a user or an administrator of the data center 10. The threshold value in step S277 may be the same as the threshold value used in steps S219 and S226 described above. Further, the threshold value in step S280 is defined in advance by, for example, a user or an administrator of the data center 10. The threshold value in step S280 may be the same as the threshold value used in steps S249 and S256 described above.

  The migration control unit 141 further determines whether there is a process that can limit the CPU usage rate (S283). Furthermore, processes that can limit the CPU usage rate are those that are not registered in the important process list 156 and whose current CPU usage rate exceeds the resource upper limit registered in the resource limit table 155. If the corresponding process exists, the process proceeds to step S285, and if not, the process proceeds to step S284.

  The migration control unit 141 sets both the disk update decrease flag and the memory update decrease flag of the state table 151 to “1” (S284). This means that although the disk update amount or memory update amount per unit time is still large, the transmission of the difference data is stopped and the migration completion process described later is started.

  The migration control unit 141 determines whether both the disk update decrease flag and the memory update decrease flag are “1” (S285). When the two flags are “1”, the process monitoring process ends, and when at least one of the flags is “0”, the process proceeds to step S272. When the two flags are “1”, the determinations in steps S226 and S256 are “YES”, and the disk data transmission and memory data transmission processes are also terminated. Also, a migration completion process described later is started.

FIG. 22 is a flowchart illustrating an exemplary procedure for completing migration.
The migration control unit 141 instructs the kernel 143 to transition to the suspended state (S121). In the suspended state, the guest OS 122 does not shut down, and the execution of the program of the virtual machine 121 by the CPU 101 stops.

  In response to the suspend instruction, the kernel 143 generates an interrupt file indicating the current state of the virtual machine 121 and stores it in the save area 133 of the HDD 103 (S122). The interrupt file includes status data stored in the register 101a of the CPU 101, memory data stored in the RAM 102, device data indicating the status of other peripheral devices, and the like. Further, the guest OS 122 refers to the disk update information 152 and creates a snapshot of a storage area including difference data that has not been transmitted to the virtual machine 221. In addition, the kernel 143 is set to cancel the suspended state immediately.

  After transitioning to the suspended state, the kernel 143 releases the suspended state according to the setting in step S122 (S123). At this time, the state of the CPU 101 immediately before the suspension is restored using the interruption file created in step S122. Thereby, the processing of the migration control unit 141 in the guest OS 122 is resumed.

  The migration control unit 141 stops the three monitoring processes (S124). The three monitoring processes include disk monitoring by the update monitoring unit 142, memory monitoring by the update monitoring unit 142, and process monitoring by the migration control unit 141.

  The migration control unit 141 inquires of the update monitoring unit 142 about memory update, and acquires the page address of the updated page. Then, the migration control unit 141 extracts the memory data of the updated page from the interruption file stored in the save area 133. Further, the migration control unit 141 extracts the state data of the CPU 101, device data of peripheral devices, and the like from the interruption file. Also, the migration control unit 141 extracts the updated sector disk data from the snapshot created in step S122. The migration control unit 141 transmits final data including the extracted data to the virtual machine 221 via the user network 11, the Internet 30, and the user network 21 (S125).

  The migration control unit 141 sets the migrated flag of the state table 151 to “1” (S126). The migration control unit 241 saves various data included in the final data received from the virtual machine 121 in an appropriate storage area of the HDD 203 (S127). The disk data included in the final data is overwritten and saved in a storage area in the HDD 203 indicated by the sector address. Memory data included in the final data, state data of the CPU 101, device data of peripheral devices, and the like are stored in the save area 233.

  The migration control unit 241 generates an interrupt file for reproducing the state of the virtual machine 121 based on the data stored in the save area 233 (S128). The migration control unit 241 generates a status table (similar to the status table 151) in the control information storage unit 242, and sets the migrated flag to “1” (S129).

  The migration control unit 241 transmits a stop request for the virtual machine 121 to the management server 13 of the data center 10 via the user network 21 and the Internet 30 (S130). When receiving the stop request, the management server 13 transmits a stop instruction to the virtual machine 121 via the management network 12 (S131). The virtual machine 121 shuts down the guest OS 122 and stops the virtual machine 121 (S132).

  The migration control unit 241 edits the activation information stored in the activation information area 232 (S133). The edited activation information indicates that system data is loaded from the partition 231 when the virtual machine 221 is activated. In addition, the migration control unit 241 sets so that the process is started from an intermediate state using the interruption file stored in the save area 233. The migration control unit 241 instructs the kernel of the guest OS 222 to restart (S134). When the guest OS 222 is shut down and the virtual machine 221 is restarted, system data is loaded from the partition 231 according to the startup information. As a result, the guest OS 122 is executed on the virtual machine 221. Further, based on the interrupt file stored in the save area 233, the state of the migration source CPU 101 or RAM 102 is reproduced on the CPU 201 or RAM 202.

FIG. 23 is a flowchart (continued) showing an example of the procedure for completing the migration.
The migration control unit 141 activated on the virtual machine 221 refers to the state table stored in the control information storage unit 242 and determines whether the migrated flag is “1” (S135). If the migrated flag is “1”, the process proceeds to step S136. If the migrated flag is “0”, the process of the migration control unit 141 ends.

  When the migrated flag is “1”, the migration control unit 141 uses the interface of the kernel 143 to cancel the upper limit of the CPU usage rate set in step S274 described above (S136). The migration control unit 141 deletes control information such as the state table 151 stored in the control information storage unit 144 (S137). The migration control unit 141 releases the partition 235 in which the system data of the guest OS 222 is stored so that it is not recognized by the guest OS 122 (S138).

  The migration control unit 141 transmits a deletion request for the virtual machine 121 to the management server 13 of the data center 10 via the user network 21 and the Internet 30 (S139). When receiving the deletion request, the management server 13 deletes the virtual machine 121 from the data center 10 by releasing the allocation of the resource of the host server 100 to the virtual machine 121 (S140). As a result, the partitions 131 and 134 are released and other virtual machines can be used.

  According to the information processing system of the second embodiment, the virtual machine 221 is activated on the migration destination host server 200, and the guest OS 222 is temporarily executed on the virtual machine 221. Data regarding the virtual machine 121 can be copied as communication between the virtual machine 121 and the virtual machine 221. Therefore, even if direct access to the management network 12 from the outside of the data center 10 and direct access to the management network 22 from the outside of the data center 20 are restricted, the management networks 12 and 22 are bypassed and the user networks 11 and 21 are bypassed. Data related to the virtual machine 121 can be copied by using this.

  Then, when the virtual machine 221 is restarted by rewriting the startup information of the virtual machine 221, the guest OS 122 is executed on the virtual machine 221 instead of the guest OS 222. Thereby, the migration of the virtual machine 121 can be smoothly performed between the data centers 10 and 20 managed by different service providers.

  In addition, when the update frequency of the memory data and the disk data is high in the migration source virtual machine 121, the update frequency of the memory data and the disk data is lowered by limiting the CPU usage rate of at least some processes. Thereby, the amount of difference data to be transmitted can be reduced, and the time required for migration of the virtual machine 121 can be shortened even when the communication delay of the Internet 30 is large. Also, by defining an important process that does not limit the CPU usage rate, it is possible to reduce the impact on business.

  As described above, the information processing according to the first embodiment can be realized by causing the information processing apparatuses 1 and 2 to execute a program. The information processing according to the second embodiment can be realized by causing the management servers 13 and 23, the client 31, the host servers 100, 100a, 200, and 200a to execute programs.

  The program can be recorded on a computer-readable recording medium (for example, the recording medium 113). As the recording medium, for example, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like can be used. Magnetic disks include FD and HDD. Optical discs include CD, CD-R (Recordable) / RW (Rewritable), DVD, and DVD-R / RW. The program may be recorded and distributed on a portable recording medium. In this case, the program may be copied from a portable recording medium to another recording medium such as an HDD (for example, the HDD 103) and executed.

DESCRIPTION OF SYMBOLS 1, 2 Information processing apparatus 1a, 2a Virtual machine 1b, 2b Operating system 1c, 2c Storage part 1d, 2d, 2e, 2f Storage area

Claims (7)

On the computer,
The second operating system included in the second virtual machine booted on the computer is executed from another computer in which the first virtual machine including the first operating system is arranged, thereby executing the second operating system. Obtaining data of the first operating system by an operating system;
An operating system executed when the second virtual machine is restarted by the second operating system by executing the second operating system is obtained from the second operating system. Rewriting boot information indicating a boot method of the second virtual machine so as to change to the first operating system executed based on data of the first operating system ;
Restart the second virtual machine;
Virtual machine migration program that executes processing. The second virtual machine has a first storage area, a second storage area for storing data of the second operating system, and a third storage area for storing the startup information, among the storage devices included in the computer. Storage space is allocated,
The data of the first operating system is written to the first storage area;
The startup information is rewritten so that data stored in the first storage area is read when the second virtual machine is restarted.
The virtual machine migration program according to claim 1. Allocation of the second storage area to the second virtual machine is released after the first operating system is executed in the second virtual machine;
The virtual machine migration program according to claim 2. A first information processing apparatus in which a first virtual machine including a first operating system is arranged and transmits data of the first operating system;
Start a second virtual machine including a second operating system;
Obtaining the data of the first operating system from the first information processing apparatus by the second operating system by executing the second operating system;
An operating system executed when the second virtual machine is restarted by the second operating system by executing the second operating system is obtained from the second operating system. Rewriting boot information indicating a boot method of the second virtual machine so as to change to the first operating system executed based on data of the first operating system ;
A second information processing apparatus for restarting the second virtual machine;
A virtual machine migration system. The first information processing apparatus extracts at least one of data stored in a memory and data stored in a register in a processor by temporarily transitioning the first operating system to a suspended state.
The virtual machine migration system according to claim 4. When the data update amount per unit time in the first information processing apparatus exceeds a threshold, the first information processing apparatus performs one or more processes executed in the first information processing apparatus. Limit processor resource allocation to
The virtual machine migration system according to claim 4 or 5. A virtual machine migration method executed by a system having a first information processing apparatus and a second information processing apparatus,
A second virtual machine including a second operating system different from the first virtual machine including the first operating system arranged in the first information processing apparatus is started on the second information processing apparatus. And
Copying the data of the first operating system from the first information processing apparatus to the second information processing apparatus by the second operating system by executing the second operating system;
An operating system executed when the second virtual machine is restarted by the second operating system by executing the second operating system is obtained from the second operating system. Rewriting boot information indicating a boot method of the second virtual machine so as to change to the first operating system executed based on data of the first operating system ;
Restart the second virtual machine;
Virtual machine migration method.

Images