JP2010165022A - Inter-processor communication device, inter-processor communication method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an inter-processor communication device, an inter-processor communication method, a program, and a recording medium that reduce communication overhead generated by communication processing using shared memory between processor nodes to enable high-speed communications. <P>SOLUTION: This inter-processor communication device includes the shared memory accessible and shared from first and second processor nodes, a writing means for writing communication data in the shared memory according to a program at the first processor node, a socket for interconnecting the first and second processor nodes, a first reporting means for transmitting write-in reporting information to the second processor node via the socket, a reading means for reading the communication data from the shared memory according to the write-in reporting information, and a second reporting means for transmitting read-out reporting information to the first processor node via the socket. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an inter-processor communication apparatus, an inter-processor communication method, a program, and a recording medium, each including a shared memory that is configured by a plurality of processor nodes and is provided so as to be accessible from each of the processor nodes. .

  Conventionally, the operation efficiency of a loosely coupled multiprocessor system, which is an example of an interprocessor communication apparatus having a plurality of processor nodes, is affected by the communication overhead generated in the communication processing between the processor nodes. Depends on how to reduce If there is a job with a heavy processor node load among the jobs executed by the multiprocessor system, even if this job is subdivided into a plurality of processes and assigned to each processor node, it is usually Since each process such as each subdivided task has some relation or dependency, each processor node receives information such as calculation results and calculation results from each other as it goes through each stage of processing. This is because the passing communication process is performed.

  In such communication processing between processor nodes, interfaces such as Ethernet (registered trademark), InifiBand, Myrinet have been conventionally used. However, in Ethernet (registered trademark), the time from when data is transmitted in the process in which the transmission source processor node is operating until the data is received in the process in which the reception side processor node is operating, that is, There are disadvantages such as long latency during communication and heavy protocol processing such as TCP / IP.

  If the latency during communication is long, when data is frequently exchanged between processor nodes, communication overhead increases and the performance and operation efficiency of the entire system decrease. In addition, if the protocol processing is heavy, valuable CPU performance is wasted in processing other than the original purpose such as original arithmetic processing and calculation processing.

  On the other hand, InifiBand and Myrinet have the advantage that the latency is short and the protocol processing is implemented in hardware, so that the burden on the CPU is light. However, these interface cards have higher functionality than Ethernet (registered trademark). Because of its high performance, it is very expensive, and it has been difficult to achieve low cost with embedded devices.

  In addition, in embedded devices used in multiprocessor systems, low cost is required and the number of CPUs is at most 2 to 4 at most. Processing was often performed using shared memory. Conventionally, a memory device referred to as a dual port memory has been used as the shared memory in such a case. However, in the function of this dual port memory, only one CPU is limited to one. There was a restriction that it could be connected.

  Recently, in addition to the dual port memory, a PCI-PCI bridge for connecting PCI buses of two processor nodes or a PCI-Express switch for connecting PCI-Express buses of two or more processor nodes by a crossbar. LSI using is used.

  In these PCI-PCI bridges and PCI-Express switches, the main memory normally occupied by one processor node can be directly accessed by other processor nodes, so the function of the shared memory can be easily performed at low cost. Can have. Also, these PCI-PCI bridges and PCI-Express switches are used in multiprocessor systems having two or more CPUs.

  In a computer system described in Japanese Patent No. 3743381 (Patent Document 1), a computer system including a plurality of hosts, a shared memory accessible from any of these hosts, and a communication path connecting the hosts. It accepts an inter-host processing synchronization request issued from a program that operates in cooperation between hosts within its own host, records information to that effect in a shared memory, and communicates that fact to other hosts via the communication path. Triggering and recognizing the establishment of inter-host processing synchronization with reference to the shared memory, triggered by the acceptance of the inter-host processing synchronization request of the local host and the acceptance notification of the inter-host processing synchronization request notified from other hosts Inter-host processing within each host and inter-host processing from other hosts so that it can cope with the loss of communication between hosts. Starts a timer in each host that issues a process synchronization confirmation request so that the inter-host process synchronization means makes a determination of establishment of inter-host process synchronization based on reference to the shared memory at regular intervals even if there is no acceptance request notification Means.

  However, the shared memory is completely different from other general communication processes in terms of software convenience. To use the shared memory, a mechanism such as synchronization or exclusion between programs of each processor node is used. There was a problem in convenience that the programmer had to do it in some way. Furthermore, if you intend to migrate software developed for a tightly coupled multiprocessor system and using socket communication to a loosely coupled multiprocessor system with shared memory, the source code in the program In particular, the description of the part using the socket communication must be rewritten to the process using the shared memory, so that there has been a problem that the burden of software development for the programmer is greatly increased.

  The present invention has been made in view of such a problem, and reduces the communication overhead generated in communication processing using a shared memory between processor nodes, and enables an interprocessor communication apparatus and processor capable of high-speed communication. It is an object to provide an intercommunication method, a program, and a recording medium.

  In order to solve the above problems, in the present invention, there is provided a shared memory that is configured by first and second processor nodes and is provided so as to be accessible from each of the first and second processor nodes. In the inter-processor communication device provided, when the writing means writes the communication data to the shared memory according to the program operating on the first processor node, the first and second processor nodes are connected with the communication data between the programs. The first notification means transmits the write notification information to the second processor node via the socket communication that allows connection and reception. Then, the reading means reads communication data from the shared memory in accordance with the write notification information transmitted by the first notification means, and the second notification means sends the read notification information to the first processor via socket communication. Send to node.

  Therefore, by sharing the same shared memory between the first and second processor nodes, this shared memory functions as a distributed shared memory. In addition, by transmitting and receiving write notification information and read notification information via a socket, the characteristics of socket communication that this shared memory can be used between a plurality of processor nodes and the characteristics of shared memory that are wideband, respectively The communication overhead can be reduced and high-speed communication can be achieved.

  According to the present invention, there are provided an inter-processor communication device, an inter-processor communication method, a program, and a recording medium capable of reducing communication overhead generated in communication processing using a shared memory between processor nodes and capable of high-speed communication. be able to.

It is explanatory drawing which shows the whole structure of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the whole structure of the multiprocessor system in 2nd Embodiment. It is explanatory drawing which shows the use condition of the storage area of the main memory of the multiprocessor system in 2nd Embodiment. It is explanatory drawing which shows the whole structure of the multiprocessor system in 3rd Embodiment. It is explanatory drawing which shows the process which CPU wrote with respect to the shared memory in another processor node from the processor node of the multiprocessor system in 1st Embodiment. FIG. 6 is an explanatory diagram illustrating processing in which a plurality of processor nodes of the multiprocessor system according to the first embodiment are connected by a socket and communication is performed using the socket 600. It is explanatory drawing which shows the process in which two processor nodes are communicating among several processor nodes of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the process in which two processor nodes are communicating among several processor nodes of the multiprocessor system in 1st Embodiment. It is a sequence diagram which shows the transmission / reception process of communication data in case the some processor nodes of the multiprocessor system in 1st Embodiment are connected by the socket. It is a sequence diagram which shows the specific example after the connection establishment of the transmission / reception process of the communication data of the multiprocessor system in 1st Embodiment. It is a sequence diagram which shows the other specific example after the connection establishment of the transmission / reception process of the communication data of the multiprocessor system in 1st Embodiment. FIG. 5 is a sequence diagram illustrating connection establishment and disconnection processing executed by a client of the multiprocessor system according to the first embodiment. It is explanatory drawing which shows the table which memorize | stored the socket instruction information for instruct | indicating the operation | movement of the socket of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the example of the process performed with respect to the shared memory according to the socket instruction information of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the other example of the process performed with respect to the shared memory according to the socket instruction information of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the data structure of the memory window allocated by the storage area in the shared memory of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the process performed using communication data for the ring buffer in the shared memory of the multiprocessor system in 1st Embodiment. It is explanatory drawing which shows the process which two processor nodes among the several processor nodes of the multiprocessor system in 1st Embodiment performed using communication data for shared memory. It is explanatory drawing which shows the other process which two processor nodes among the several processor nodes of the multiprocessor system in 1st Embodiment performed using communication data for shared memory. FIG. 3 is an explanatory diagram illustrating a hardware configuration of a processor node of the multiprocessor system according to the first embodiment.

[First Embodiment]
Hereinafter, although it demonstrates with 1st Embodiment concerning this invention, this invention is not limited to embodiment. FIG. 1 is an explanatory diagram showing an overall configuration in a first embodiment of a multiprocessor system 10 which is an example of an interprocessor communication apparatus according to the present invention. The multiprocessor system 10 is a loosely coupled multiprocessor system including, for example, a shared memory. The multiprocessor system 10 includes a plurality of processor nodes, and the shared memory is provided so as to be accessible from each of the processor nodes. .

  As shown in FIG. 1, the multiprocessor system 10 is an example of a network for transmitting and receiving data and the like between a plurality of processor nodes 100, 110, and 120 and the processor nodes 100, 110, and 120. The network includes an Ethernet (registered trademark) 130 and a shared memory 140 that is provided so as to be accessible from each of the processor nodes 100, 110, and 120.

  The processor node 100 includes a CPU 101 that executes a program for performing various processes such as an application, a memory controller 102 that accesses a main memory 103 and reads out data and the like according to the process of the CPU 101, and a process of the CPU 101. And a main memory 103 for storing various data obtained by the above.

  The CPU 101 executes various types of processing on an application, for example, by executing a program prepared in advance, and is input from the outside via the memory controller 102, a host-PCI bridge 104 described later, and an Ethernet (registered trademark) card 105. The calculated communication data is used to perform arithmetic processing and determination processing.

  The memory controller 102 accesses the main memory 103 and writes data such as calculation results and determination results obtained by processing in the CPU 101 to a shared memory formed in the main memory 103, and the memory controller 102 In order to read each data including communication data from the shared memory, and transmit the data to the other processor nodes 110 and 120 via the host-PCI bridge 104, the Ethernet (registered trademark) card 105, and the Ethernet (registered trademark) 130. Perform the process.

  The processor node 100 is connected to the host-PCI bridge 104 for inputting / outputting data between the processor node 100 and the outside, and the Ethernet (registered trademark) 130, and is connected to the outside via the Ethernet (registered trademark) 130. And an Ethernet (registered trademark) card 105 that transmits and receives data between them.

  The host-PCI bridge 104 has an internal side of the processor node 100 connected to the memory controller 102 and an external side connected to the Ethernet (registered trademark) card 105 and the shared memory 140. The host-PCI bridge 104 switches the external connection target to the Ethernet (registered trademark) card 105 or the shared memory 140, and between the processor node 100 and the Ethernet (registered trademark) 130, the processor node 100, and the other processor nodes 110. , 120 input / output data.

  The Ethernet (registered trademark) card 105 is mounted on, for example, a PCI card throttle of the processor node 100 and directly connected to the Ethernet (registered trademark) 130, and transmits / receives data to / from the outside via the Ethernet (registered trademark) 130. Do.

  The Ethernet (registered trademark) card 105 receives request information for requesting communication data requested from one of the other processor nodes 110 and 120. Further, the Ethernet (registered trademark) card 105 transmits write notification information for notifying that the communication data has been written to the shared memory 140 by the memory controller 102 to the other processor nodes 110 and 120, respectively, Read notification information for notifying that the communication data has been read by 102 is transmitted to the other processor nodes 110 and 120, and is transmitted by the Ethernet (registered trademark) card 105 of the other processor nodes 110 and 120. In accordance with the read notification information, the communication data read by the memory controller 102 is transmitted to the processor node that requested the communication data.

  The processor nodes 110 and 120 have the same configuration as the processor node 100, and the CPU 101, the memory controller 102, the main memory 103, the host-PCI bridge 104, and the CPU 111 having the same functions as the Ethernet (registered trademark) card 105, 121, memory controllers 112 and 122, main memories 113 and 123, host-PCI bridges 114 and 124, and Ethernet (registered trademark) cards 115 and 125.

  FIG. 20 is an explanatory diagram showing the hardware configuration of the processor node 100. As shown in FIG. 20, the processor node 100 has a memory controller 102 connected to the CPU 101, the main memory 103 can write and read data to the memory controller 102, and the host-PCI bridge 104 communicates. Each data such as data is connected so as to be able to transmit and receive.

  A PCI bus 401 to which a plurality of types of PCI device devices can be connected is connected to the host-PCI bridge 104, and a PCI-PCI bridge 402 connected to the shared memory 140 via the PCI bus is connected to the PCI bus 401. A plurality of types of PCI devices 403 and an Ethernet (registered trademark) card 105 connected to the Ethernet (registered trademark) 130 are connected to each other.

  When the memory controller 102 is connected to the host bus of the CPU 101 and receives a read or write request for the main memory 103, the Ethernet (registered trademark) card 105, the PCI-PCI bridge 402, and various PCI devices 403 from the CPU 101, the memory controller 102 The role of distributing requests to. When a personal computer is used as a specific example of the multiprocessor system 100, for example, an x86 CPU manufactured by Intel, for example, is generally mounted. On this occasion, a north bridge or MCH (Memory) is mounted on a main board called a motherboard. A chip set referred to as “Controller Hub” is mounted, and the memory controller 102 in FIG. 20 corresponds to these chip sets.

  On the other hand, the host-PCI bridge 104 corresponds to a chip set referred to as a south bridge or an ICH (I / O Controller Hub), and the CPU 101 to various Ethernet (registered trademark) cards 105, the PCI-PCI bridge 402, and various PCI devices. It has a function of issuing an access request for 403 or receiving a DMA request for the main memory 103 from the PCI device 403 and transmitting it to the memory controller 102.

  FIG. 5 is an explanatory diagram illustrating a process in which the CPU 101 writes data from, for example, the processor node 110 to the shared memory 140 in another processor node 100. As shown in FIG. 5, a part of the storage area of the main memory 103 of the processor node 100 is mapped to a memory window in the main memory 113 of the processor node 110. At this time, when a process operating on the processor node 110 writes to the storage area of the memory window, the written data and an address indicating the storage area are transmitted to the processor node 100 and used as a remote memory. Write processing is performed on the main memory 103 in the processor node 100. When the process operating in the processor node 100 reads out the main memory 103, communication data can be transmitted from the processor node 110 to the processor node 100.

  In FIG. 5, it is possible to transfer data by sharing the same shared memory 140 between processes operating on different processor nodes. Such a memory can be used as a distributed shared memory. Refer to as The distributed shared memory has a socket communication feature that can be used between a plurality of processor nodes, and a shared memory feature that has a wide bandwidth.

  Among the OSs provided in the multiprocessor system 10, for example, an OS having a memory protection function between processes in a program in each processor node 100, 110, 120 such as UNIX (registered trademark), between processes. Data cannot be transmitted using global variables. Such an OS includes means for realizing IPC (Inter-Process Communication) such as socket communication and shared memory as a function for performing communication such as data transmission / reception between processes. Since the socket communication is a network-transparent function, it has an advantage that it can be used not only in each processor node 100, 110, 120 but also when data is exchanged between each processor node 100, 110, 120. On the other hand, there is a disadvantage that the processing efficiency is reduced when a large amount of data is transmitted / received due to a large communication overhead. The shared memory 140 has low communication overhead and can be efficiently executed even when a large amount of data is transmitted / received (broadband). However, since the shared memory 140 can usually be used only in each of the processor nodes 100, 110, 120, a plurality of processor nodes 100, 110, 120 In a multiprocessor system composed of the above, it has not been used much.

  FIG. 6 is an explanatory diagram showing processing in which a plurality of processor nodes 100, 110, and 120 are connected by a socket 600 and a program operating on the processor node 100 performs socket communication using the socket 600, for example. It is. Here, in the socket 600, an IP address and a port number are set between each of the plurality of processor nodes 100, 110, and 120, and the processor nodes connected by the socket 600 perform socket communication with each other. Indicates the possible state.

  Among the processor nodes 100, 110, 120, between the processor nodes 100, 110, as shown in FIG. 6, for example, two programs connected by a socket 600 that passes through various networks such as Ethernet (registered trademark) 130. Communication data can be transmitted and received bi-directionally between 601 and 602.

  In the socket communication indicated by the connection with the socket 600, communication data can be transmitted and received between the programs operating on any two of the processor nodes 100, 110, and 120. . Further, the data that can be transmitted / received through the socket 600 is a byte stream. Here, as shown in FIG. 6, the byte stream is data in which byte data is arranged in a line for each data to be transmitted / received. Each byte data sent by the client reaches the receiving processor node in the same sending order.

  FIGS. 7 and 8 are explanatory diagrams illustrating processing in which, for example, programs operating on the processor nodes 100 and 110 among the plurality of processor nodes 100, 110, and 120 are communicating. As shown in FIG. 7, between the processor nodes 100 and 110, the processor nodes 100 and 110 operate by reading programs from the internal main memories 103 and 113, respectively, and request to write communication data to each other according to the processing of each program. To do.

  In a write request from the processor node 100 to the processor node 110, as shown in FIG. 8, a part of the storage area of the main memory 103 of the processor node 100 is mapped to a memory window in the main memory 113 of the processor node 110. Yes. At this time, when the process B operating in the program of the processor node 110 writes to the storage area of the memory window, the written data and the address indicating the storage area are transmitted to the processor node 100, and the remote memory Write processing is performed on the main memory 103 in the processor node 100 used as. When the process operating in the processor node 100 reads out the main memory 103, communication data can be transmitted from the processor node 110 to the processor node 100.

  On the other hand, in a write request from the processor node 110 to the processor node 100, a part of the storage area of the main memory 113 of the processor node 110 is mapped to a memory window in the main memory 103 of the processor node 100. Similarly, communication data is transmitted from the processor node 100 to the processor node 110.

  FIG. 9 is a sequence diagram showing communication data transmission / reception processing when a plurality of processor nodes 100, 110, 120 are connected by a socket 600 as shown in FIG. 6. In the transmission / reception process in this case, as shown in FIG. 9, the processor node 100 in one server 90 and the processor node 110 in the other client 91 are connected via a socket 600.

  When the server 90 receives the socket 600 use request (socket) data transmitted from the client 91 in step S901, the server 90 sends the same data and the data (bind), (listen), and (accept) in response to this. A process for generating and establishing a connection between the server 90 and the client 91 is performed.

  In step S902, the server 90 and the client 91 perform processing for initializing the socket 600, and set various data to a state where they can be transmitted and received.

  The client 91 once returns the signal (p_cond_signal) transmitted through the socket 600 in step S903 to the program operating on the CPU 111, transmits the communication data to the shared memory 140 according to this signal, and transmits the communication data to the shared memory 140. Perform the writing process.

  In step S904, the client 91 transmits write notification information for notifying that communication data has been written in the shared memory 140 in step S903 to the server 90 via the socket 600.

  On the other hand, the server 90 returns the signal (p_cond_signal) transmitted via the socket 600 to the program operating on the CPU 101 once in the same manner as the client 91 in step S903. The server 90 receives the write notification information transmitted from the client 91 in step S904, and performs a process of reading communication data from the shared memory 140 in accordance with the write notification information in step S905.

  In step S906, the server 90 transmits read notification information for notifying that the communication data has been read from the shared memory 140 in step S905 to the client 91 via the socket 600.

  The client 91 transmits a disconnection notification signal for notifying the disconnection of the connection between the server 90 and the client 91 to the server 90 via the socket 600 in step S907. On the other hand, when the server 90 receives the disconnection notification signal in step S908, the server 90 transmits a disconnection notification signal to the client 91 via the socket 600 in order to notify that it responds to disconnecting the connection accordingly.

  In step S909, the server 90 and the client 91 perform processing to end the operation of the socket 600, and set various data in a state where transmission and reception are impossible. In addition, the server 90 performs processing for disconnecting the connection between the server 90 and the client 91.

  FIG. 10 is a sequence diagram showing a specific example after establishing a connection for communication data transmission / reception processing, as shown in FIG. In the transmission / reception processing of this example, as shown in FIG. 10, the client 91 returns the signal (p_cond_signal) transmitted in step S1003 once to the program operating on the CPU 101, and in accordance with this signal, two pieces of communication data of 256 KB are shared memory. 140. The client 91 performs a process of copying and writing two pieces of communication data on the storage area of the shared memory 140 twice.

  In step S1004, the client 91 transmits write notification information for notifying that the communication data has been written to the shared memory 140 in step S1003 to the server 90.

  On the other hand, similarly to the client 91, the server 90 returns the transmitted signal (p_cond_signal) once to the program operating on the CPU 101 in step S1003, and enters a waiting state for communication data in step S1005. The server 90 receives the write notification information transmitted from the client 91 in step S1004, and copies and reads 512 KB communication data from the storage area of the shared memory 140 according to the write notification information in step S1006. Process.

  In step S1007, the server 90 transmits, to the client 91, read notification information for notifying that the communication data has been read from the shared memory 140 in step S1006.

  FIG. 11 is a sequence diagram illustrating another specific example after establishing a connection for communication data transmission / reception processing, as illustrated in FIG. 9. In the transmission / reception processing of this example, as shown in FIG. 11, the signal (p_cond_signal) transmitted by the client 91 in step S1103 is once returned to the program operating on the CPU 111, and 1.5 MB of communication data is transferred to the shared memory according to this signal. 140. The client 91 divides this communication data into three pieces of communication data of 512 KB each having the same capacity, and performs a process of copying and writing two pieces of communication data in the storage area of the shared memory 140 twice.

  In step S1105, the client 91 transmits write notification information for notifying that the communication data has been written to the shared memory 140 in step S1103 to the server 90 every two write processes. Here, for example, when the capacity of the shared memory 140 is 1 MB, the storage area of the shared memory 140 is filled and writing processing becomes impossible, and communication data is waited in step S1104.

  On the other hand, the server 90 returns the transmitted signal (p_cond_signal) to the program operating on the CPU 101 once in the same manner as the client 91 in step S1103 described above. Then, the server 90 receives the write notification information transmitted from the client 91 in step S1105, and copies the first communication data from the storage area of the shared memory 140 according to the write notification information in step S1106. And read out.

  In step S <b> 1107, the server 90 transmits read notification information for notifying that the communication data has been read from the shared memory 140 in step S <b> 1106 to the client 91.

  The client 91 receives the write notification information transmitted from the server 90 in step S1107, and releases the communication data wait state accordingly. Then, the client 91 deletes the communication data written in the storage area of the shared memory 140 in step S1108 and then copies and writes the remaining third communication data on the storage area of the shared memory 140. Do.

  In step S1109, the client 91 transmits to the server 90 write notification information for notifying that communication data has been written in the shared memory 140 in step S1108.

  FIG. 12 is a sequence diagram showing connection establishment and disconnection processing executed by the client 91 as shown in FIG. In these processes, as shown in FIG. 12, when starting the multiprocessor system 10, first, the client 91 writes communication data in the shared memory 140 via the memory controller 102 and the host-PCI bridge 104 in step S1201. A storage area is secured. Then, the client 91 transmits the socket 600 use request (socket) data from the CPU 111 to the memory controller 102 and the host-PCI bridge 104 in steps S1202 and S1203, and initializes them. In addition, a connection is established and various data can be transmitted and received.

  In step S1204, the client 91 transmits an allocation request for allocating a storage area for writing communication data in the shared memory 140 to the memory controller 102 and the host-PCI bridge 104. In step S1205, the client 91 sets the shared memory 140 to an access waiting state such as writing or reading.

  In step S1206, the client 91 accesses the shared memory 140 via the memory controller 102 and the host-PCI bridge 104, and performs processing for allocating a storage area for writing communication data. For example, the client 91 detects a free area where data has not been written or an area where data has been erased from the storage areas in the shared memory 140 and writes communication data from each of these detected areas. An area is secured and a process for assigning each address in the secured area is performed. The client 91 transmits the allocation result of allocating the storage area in step S1206 to the CPU 111 by the memory controller 102 and the host-PCI bridge 104 in step S1207.

  In step S1208, the client 91 transmits to the server 90 allocation notification information for notifying the allocation result transmitted in step S1207 and each allocated address. In step S1209, the client 91 sends the allocation notification information for notifying the allocation result allocated in the storage area in the shared memory 140 and the allocated addresses transmitted from the server 90 in step S1209. Receive.

  The client 91 once returns the signal (p_cond_signal) transmitted via the socket 600 in steps S1210 and S1211 to the program operating on the CPU 111, and transmits the communication data to the shared memory 140 according to this signal, and the communication data is transmitted to the shared memory. A process of writing to 140 is performed. Here, the communication data is written in the area allocated in step S1206 among the storage areas in the shared memory 140.

  In step S <b> 1212, the client 91 transmits write notification information for notifying that communication data has been written in the shared memory 140 in step S <b> 1211 to the server 90 via the socket 600.

  On the other hand, the server 90 once returns the signal (p_cond_signal) transmitted via the socket 600 to the program operating on the CPU 101 in the same manner as the client 91 in step S1210. Then, the server 90 receives the write notification information transmitted from the client 91 in step S1211, and performs processing for reading the communication data from the shared memory 140 according to the write notification information.

  In step S1213, the server 90 transmits read notification information for notifying that the communication data has been read from the shared memory 140 in step S1211 to the client 91 via the socket 600.

  The client 91 transmits a disconnection notification signal for notifying the disconnection of the connection between the server 90 and the client 91 to the server 90 via the socket 600 in step S1214. On the other hand, when the server 90 receives the disconnection notification signal in step S1215, the server 90 transmits a response notification signal for notifying that it responds to disconnection to the client 91 via the socket 600.

  In step S1216, the client 91 transmits a release request for releasing the storage area in the shared memory 140 to the memory controller 102 and the host-PCI bridge 104. In step S1217, the client 91 sets the shared memory 140 in an access waiting state such as writing or reading.

  In step S1218, the client 91 accesses the shared memory 140 via the memory controller 102 and the host-PCI bridge 104, and performs processing for releasing and releasing the storage area allocated in step S1206. The client 91 transmits the release result of releasing the storage area in step S1218 to the CPU 111 by the memory controller 102 and the host-PCI bridge 104 in step S1219.

  In step S1220, the client 91 performs a process of terminating the operation of the socket 600, and sets various data in a state where transmission and reception are not possible. Then, a signal (p_cond_signal) for ending the process is returned to the program operating on the CPU 101, and the execution of the program is ended according to this signal.

  FIG. 13 is an explanatory diagram showing a table 1300 that stores socket instruction information for instructing the operation of the socket 600. In this table 1300, as shown in FIG. 13, a plurality of types of socket instruction information and information on the operation content of the socket 600 indicated by each socket instruction information are stored in association with each other. For example, in the table 1300, as shown in FIG. 13, “sockfd” that is socket instruction information and “socket descriptor” that indicates the operation content of the socket 600 are stored in association with each other. Further, “state”, which is other socket instruction information, and “socket state” indicating the operation content of the socket 600 are stored in association with each other.

  FIG. 14 is an explanatory diagram illustrating an example of processing performed on the shared memory 140 in accordance with the socket instruction information. As shown in FIG. 14, the server 90 accesses the shared memory 140, secures an area for writing communication data from the storage area in the shared memory 140, and allocates each address in the secured area. Process. Then, the server 90 transmits socket instruction information “local.buff_adrs” indicating that the storage area has been allocated to the CPU 101 via the socket 600 by the host-PCI bridge 104.

  Further, as shown in FIG. 14, the server 90 accesses the shared memory 140, secures an area for reading communication data written by the client 91 from the storage area in the shared memory 140, and this secured area. The process for allocating each address in is performed. Then, the server 90 transmits socket instruction information “remote.buff_adrs” indicating that the storage area has been allocated to the CPU 101 via the socket 600 by the host-PCI bridge 104.

  On the other hand, as shown in FIG. 14, the client 91 accesses the shared memory 140, secures an area for writing communication data from a storage area in the shared memory 140, and assigns each address in the secured area. Process. Then, the client 91 transmits socket instruction information “local.buff_adrs” indicating that the storage area has been allocated to the CPU 111 via the socket 600 by the host-PCI bridge 114.

  Further, as shown in FIG. 14, the client 91 accesses the shared memory 140, secures an area for reading communication data written by the server 90 from the storage area in the shared memory 140, and this secured area. The process for allocating each address in is performed. Then, the client 91 transmits socket instruction information “remote.buff_adrs” indicating that the storage area has been allocated to the CPU 111 via the socket 600 by the host-PCI bridge 114.

  FIG. 15 is an explanatory diagram illustrating another example of processing performed on the shared memory 140 in accordance with socket instruction information. As shown in FIG. 15, for example, the server 90 or the client 91 accesses the shared memory 140 and secures a plurality of areas for writing communication data from the storage area in the shared memory 140, and each of these secured areas The process for allocating each address in is performed.

  Then, as shown in FIG. 15, socket instruction information indicating that the first (memory window 1) storage area has been allocated is transmitted to the CPU 101 via the socket 600. The socket instruction information includes “local.buff_size” indicating the capacity of the allocated area (256 MB) and “local.buff_adrs” indicating the address of the allocated area.

  Also, socket instruction information indicating that the second (memory window 2) storage area has been allocated is transmitted to the CPU 101 via the socket 600. The socket instruction information includes “local.buff_size” indicating the capacity of the allocated area (512 MB) and “local.buff_adrs” indicating the address of the allocated area.

  Also, socket instruction information indicating that the third (memory window 3) storage area has been allocated is transmitted to the CPU 101 via the socket 600. The socket instruction information includes “local.buff_size” indicating the capacity (128 MB) of the allocated area and “local.buff_adrs” indicating the address of the allocated area.

  FIG. 16 is an explanatory diagram showing the data structure of the memory window allocated in the storage area in the shared memory 140. As shown in FIG. 16A, the memory window includes magic number, buffer size, write pointer, read pointer, and ring buffer data. The magic number is identification information for identifying the head position of the memory window.

  The buffer size is, for example, data such as the number of bytes indicating the total capacity in the shared memory 140 of the memory window. The write pointer is a pointer indicating a start position when writing communication data in the ring buffer, and includes an offset address in the ring buffer. The read pointer is a pointer indicating a starting position when reading communication data in the ring buffer, and includes an offset address in the ring buffer.

  If the start position in the storage area of the ring buffer included in the memory window is empty, writing the entire contents of the communication data in the ring buffer area will result in the start of the ring buffer. The buffer can return to the position and continue the writing process, and can perform the writing process cyclically. As shown in FIG. 16 (b), the ring buffer includes a packet that is a data group divided into a plurality of groups when transmitting and receiving via the socket 600.

  Each packet includes a packet data size and packet data as shown in FIG. The packet data size is data such as the number of bytes indicating the capacity of the entire packet data, and is composed of 4 bytes. The packet data includes the contents of communication data separately for each of a plurality of packets.

  FIG. 17 is an explanatory diagram showing processing performed using communication data in the ring buffer in the shared memory 140. As shown in FIG. 17, first, in (1), for example, the server 90 or the client 91 accesses the shared memory 140 and sets the ring buffer of the shared memory 140 to the initial state. That is, processing for setting the positions of the write pointer and the read pointer to the head position in the ring buffer is performed.

  In (2), the first packet including the contents of the communication data is written to the ring buffer, and then the position of the write pointer is moved to the end position of this packet. Here, the process of writing the contents of the communication data is performed every 4 bytes. When writing data such as 5 bytes, a process of writing up to 8 bytes is performed every two 4-byte units. In (3), the second packet including the contents of the communication data is continuously written, and then the position of the write pointer is moved to the end position of this packet.

  Next, in (4), another server 90 or client 91 accesses the shared memory 140, and reads the first packet written in (1) described above from the ring buffer of the shared memory 140. Do. At this time, the read packet is erased, and then the position of the read pointer is moved to the start position of the second packet.

  In (5), the server 90, the client 91, and the like access the shared memory 140 and perform processing for writing the third packet including the contents of the communication data to the ring buffer. At this time, the contents of the communication data are written, and when all the data is written in the area of the ring buffer, the process returns to the head position in the ring buffer and the writing process is continued after the area erased in the above (4). . Thereafter, the position of the write pointer is similarly returned to the head position in the ring buffer and moved to the end position of this packet.

  In (6), the fourth packet including the contents of the communication data is written, and then the position of the write pointer is moved to the end position of this packet. Here, the contents of the communication data are written, and when there is a remaining 4 bytes in the ring buffer area, the ring buffer is full at this point, and writing is disabled.

  FIG. 18 is an explanatory diagram showing processing performed by the processor nodes 100, 110, for example, using the communication data for the shared memory 140 among the plurality of processor nodes 100, 110, 120. As shown in FIG. 18, first, the CPU 101 of the processor node 100 accesses the shared memory 140 in step S1801 and designates the head position (0xE800 1008) in the ring buffer. Then, the CPU 101 writes the first communication data in the ring buffer, and then moves the position of the write pointer to the end position (0x0000 FFF0) of the packet included at the end of the communication data.

  On the other hand, the CPU 111 of the processor node 110 accesses the shared memory 140 in step S1802, and sequentially reads out the communication data written in step S1801 from the ring buffer of the shared memory 140 for each packet. At this time, the read packet is erased, and then the position of the read pointer is moved to the start position of the packet next to the read packet.

  The CPU 111 continuously accesses the shared memory 140 in steps S1803 and S1804, and repeats the process of sequentially reading the communication data written in step S1801 from the ring buffer of the shared memory 140 for each packet. At this time, each of the read packets is erased, and then the position of the read pointer is moved to the start position of the packet next to the read packets.

  Next, the CPU 101 accesses the shared memory 140 in step S1805 and designates the head position (0xE800 1008) in the ring buffer. Then, the CPU 101 writes the second communication data in the ring buffer, and then moves the position of the write pointer to the end position (0x0001 0010) of the packet included at the end of the communication data.

  The CPU 111 sequentially accesses the shared memory 140 in steps S1806 and S1807, and sequentially reads out the communication data written in steps S1806 and S1807 from the ring buffer of the shared memory 140 for each packet. At this time, each of the read packets is erased, and then the position of the read pointer is moved to the start position of the packet next to the read packets.

  FIG. 19 is an explanatory diagram showing another process performed by the processor nodes 100, 110, for example, using the communication data in the shared memory 140 among the plurality of processor nodes 100, 110, 120. As shown in FIG. 19, first, the CPU 101 of the processor node 100 accesses the shared memory 140 in steps S1901 and S1902 and starts the head positions (0xE800 1008) and (0xE800) in the two ring buffers in the shared memory 140, respectively. 100A). Then, the CPU 101 writes two pieces of communication data in each of these ring buffers, and thereafter, the position of each write pointer is changed to the end position (0x0000 FFF0) of the packet included in the end of each piece of communication data. Move to (0x0000 0000).

  On the other hand, the CPU 111 of the processor node 110 accesses the shared memory 140 in step S1903, and sequentially reads out the communication data written in steps S1901 and S1902 from the ring buffer of the shared memory 140 for each packet. At this time, each of the read packets is erased, and then the position of the read pointer is moved to the start position of the packet next to the read packets.

  Next, the CPU 101 accesses the shared memory 140 in steps S1904 and S1906, and designates the top positions (0xE800 1008) and (0xE800 100A) in the two ring buffers in the shared memory 140, respectively. Then, the CPU 101 writes the two pieces of communication data in each of these ring buffers, and thereafter, the position of each write pointer is associated with the end position (0x0000 0010) of the packet included in the end of each piece of communication data. Move to (0x0000 0001).

  Here, during the processing in steps S1904 and S1906 described above, the CPU 111 accesses the shared memory 140 in step S1905, and the communication data written in steps S1904 and S1906 from the ring buffer of the shared memory 140 for each packet. The process of reading sequentially is performed. At this time, each of the read packets is erased, and then the position of the read pointer is moved to the start position of the packet next to the read packets.

  Further, the CPU 111 continuously accesses the shared memory 140 in step S1907, and repeats the process of sequentially reading the communication data written in steps S1904 and S1906 from the ring buffer of the shared memory 140 for each packet. At this time, each of the read packets is erased, and then the position of the read pointer is moved to the start position of the packet next to the read packets.

  As described above, in the multiprocessor system 10 according to the first embodiment, the processor nodes 100, 110, and 120 are connected to each other through the socket 600 that passes through the network, and communication data can be transmitted and received in both directions. . When performing writing or reading processing on the shared memory 140 using communication data, the CPU 111 of the client 91 transmits the communication data to the shared memory 140 according to a signal, and writes the communication data to the shared memory 140. Do. Then, the CPU 111 transmits write notification information to the server 90 via the socket 600 and notifies that the communication data has been written to the shared memory 140.

  On the other hand, the server 90 performs processing for reading communication data from the shared memory 140 in accordance with the write notification information. In addition, the server 90 transmits read notification information to the client 91 via the socket 600 and notifies that the communication data has been read from the shared memory 140.

  For this reason, the processor nodes 100, 110, and 120 are connected to each other through the socket 600, and the same shared memory 140 is shared and used among the processes on the program running on the processor nodes 100, 110, and 120. As a result, the shared memory 140 functions as a distributed shared memory. The shared memory 140 has a socket feature that can be used between a plurality of processor nodes, and a shared memory 140 feature that has a wide bandwidth, thereby reducing communication overhead caused by communication processing and enabling high-speed communication. It can be.

[Second Embodiment]
Hereinafter, a second embodiment according to the present invention will be described. FIG. 2 is an explanatory diagram showing the overall configuration of the multiprocessor system 10 as an example of the inter-processor communication apparatus according to the present invention in the second embodiment. The multiprocessor system 10 includes a plurality of processor nodes 200, 210, and 220, an Ethernet (registered trademark) 230 for transmitting and receiving data between the processor nodes 200, 210, and 220, and the processor nodes 200. , 210, and 220 are connected to a PCI bus 240 that connects and transmits data.

  The processor node 200 includes a CPU 201, a memory controller 202, a main memory 203, a host-PCI bridge 204, and an Ethernet (registered trademark) card 205. Each of these components has the same functions as the CPU 101, the memory controller 102, the main memory 103, the host-PCI bridge 104, and the Ethernet (registered trademark) card 105 included in the processor node 100 in the first embodiment. The description is omitted.

  The processor node 200 includes a PCI-PCI bridge 206 that is connected to the PCI bus 240 and transmits / receives data to / from the outside via the PCI bus 240 in addition to the above-described components. The PCI-PCI bridge 206 is directly connected between the host-PCI bridge 204 and the PCI bus 240 in the processor node 200, for example, and transmits / receives data to / from the CPU 201 via the PCI bus 240.

  The processor nodes 210 and 220 have the same configuration as the processor node 200, and the CPU 211, the memory controller 202, the main memory 203, the host-PCI bridge 204, and the CPU 211 having the same functions as the Ethernet (registered trademark) 205, 221, memory controllers 212 and 222, main memories 213 and 223, host-PCI bridges 214 and 224, and Ethernet (registered trademark) cards 215 and 225.

  Although not shown, in the second embodiment as well, in the multiprocessor system 10, the processor nodes 200, 210, and 220 are connected to each other through a socket 600 that passes through the network, and communication data can be transmitted and received in both directions. It has become.

  FIG. 3 is an explanatory diagram showing the usage state of the storage area of the main memory 203. In the second embodiment, as shown in FIG. 3, a part of the storage area in the main memories 203, 213, and 223 is used as a shared memory. By adopting such a configuration, it is possible to reduce the cost as compared with the configuration in which the single shared memory 140 is provided separately from each processor node 100, 110, 120 in the first embodiment described above. It becomes.

[Third Embodiment]
Hereinafter, a third embodiment according to the present invention will be described. FIG. 4 is an explanatory diagram showing the overall configuration of the multiprocessor system 10 as an example of the inter-processor communication apparatus according to the present invention in the third embodiment. The multiprocessor system 10 includes a plurality of processor nodes 300, 310, and 320, an Ethernet (registered trademark) 330 for transmitting and receiving data between the processor nodes 300, 310, and 320, and each processor node 300. , 310, and 320 are configured by a PCI-Express switch 340 that connects data and the like so that data can be transmitted and received.

  The processor node 300 includes a CPU 301, a memory controller 302, a main memory 303, a host-PCI bridge 304, and an Ethernet (registered trademark) card 305. Each of these components has the same functions as the CPU 101, the memory controller 102, the main memory 103, the host-PCI bridge 104, and the Ethernet (registered trademark) card 105 included in the processor node 100 in the first embodiment. The description is omitted.

  The processor nodes 310 and 320 have the same configuration as the processor node 300, and the CPU 311 has the same functions as the CPU 301, the memory controller 302, the main memory 303, the host-PCI bridge 304, and the Ethernet (registered trademark) card 305. 321, memory controllers 312 and 322, main memories 313 and 323, host-PCI bridges 314 and 324, and Ethernet (registered trademark) cards 315 and 325.

  Although not shown, in the multiprocessor system 10 also in the third embodiment, the processor nodes 300, 310, and 320 are connected to each other by a socket 600 that passes through the network, and communication data can be transmitted and received bidirectionally. It has become.

  In the third embodiment, a part of the storage area in the main memories 303, 313, and 323 is used as a shared memory. By adopting such a configuration, it is possible to reduce the cost as compared with the configuration in which the single shared memory 140 is provided separately from each processor node 100, 110, 120 in the first embodiment described above. It becomes.

[Other Embodiments]
In the above-described embodiment, the write notification information is transmitted via the socket 600 to notify that the communication data has been written to the shared memory 140. However, the data amount of the communication data written to the shared memory 140 May be notified when the amount reaches a predetermined amount.

  In this case, each of the processor nodes including the processor nodes 100, 110, and 120 includes a detection unit that detects the amount of data written in the shared memory 140 in advance, and the process of writing communication data to the shared memory 140 is started. When this is done, the amount of data written is detected accordingly. When this data amount reaches a predetermined amount set in advance, data amount notification information for notifying that the communication data has been written and the written data amount is generated and transmitted via the socket 600.

  In the above-described embodiment, each processor node including the processor nodes 100, 110, and 120 has the amount of communication data written in the shared memory 140 and the free area of the ring buffer included in the memory window. For example, when it is reduced to a predetermined amount of 4 bytes, it may be possible to arbitrarily set a condition for notification by notification information according to a user's request.

  The functions of the present invention are described in legacy programming languages such as C, C ++, Java (registered trademark), Java (registered trademark) Applet, JavaScript (registered trademark) Script, Perl, and Ruby, and object-oriented programming languages. It can be realized by a program executable by the apparatus, and can be stored in a device-readable recording medium and distributed.

  The present invention has been described with the first to third embodiments shown in FIGS. 1 to 20, but the present invention is not limited to this. Other embodiments, additions, modifications, deletions, and the like can be changed without departing from the scope of the present invention, and any aspect is included in the scope of the present invention as long as the operations and effects of the present invention are exhibited. It is what

DESCRIPTION OF SYMBOLS 10 ... Multiprocessor system, 100, 110, 120, 200, 210, 220, 300, 310, 320 ... Processor node, 130, 230, 330 ... Ethernet (registered trademark), 140 ... Shared memory, 240 ... PCI bus, 340 ... PCI-Express switch

Japanese Patent No. 3743381

Claims (10)

In an inter-processor communication device comprising a shared memory that is configured by first and second processor nodes and is provided so as to be accessible from each of the first and second processor nodes.
Writing means for writing communication data to the shared memory in accordance with a program running on the first processor node;
First notification means for transmitting write notification information for notifying that the communication data has been written to the shared memory by the writing means to the second processor node by socket communication;
Reading means for reading communication data from the shared memory in response to the write notification information transmitted by the first notification means;
An inter-processor communication device comprising: second notification means for transmitting read notification information for notifying that the communication data has been read by the reading means to the first processor node by the socket communication. The shared memory is
A ring buffer capable of cyclically writing to the communication data;
A write pointer indicating a start position when writing communication data in the ring buffer;
The interprocessor communication device according to claim 1, further comprising: a read pointer indicating a start position when reading communication data in the ring buffer.   The inter-processor communication device according to claim 1, wherein the shared memory is formed by using a part of a storage area of a main memory provided in each of the first and second processor nodes.   When the amount of the communication data written to the shared memory by the writing means reaches a predetermined amount, data amount notification information for notifying that the communication data has been written is sent via the socket communication. The inter-processor communication device according to claim 1, further comprising a data amount notification means for transmitting to the second processor node.   5. The inter-processor communication according to claim 1, wherein the first notification unit includes a condition setting unit that sets a condition for transmitting the write notification information to the second processor node by the socket communication. apparatus.   6. The allocating unit according to claim 1, further comprising an allocating unit that detects a free area in the shared memory, and allocates an area for the writing unit to write communication data from each of the detected areas. Interprocessor communication device.   Transmitting means for transmitting, to the second processor node via the socket communication, assignment notification information assigned by the assigning means for notifying an address of an area in which the writing means writes communication data; The interprocessor communication device according to claim 1, further comprising: An interprocessor communication method executed by an interprocessor communication device, wherein the interprocessor communication device includes:
Accessing from each of the first and second processor nodes in accordance with a program operating on the first processor node among the first and second processor nodes configured in the interprocessor communication device. Writing communication data to a shared memory provided to be shared,
Write notification information for notifying that the communication data has been written to the shared memory is transmitted via socket communication that connects the first and second processor nodes so that communication data can be transmitted and received between the programs. Transmitting to the second processor node;
Reading communication data from the shared memory in response to the transmitted write notification information;
And a step of transmitting read notification information for notifying that the communication data has been read to the first processor node via the socket communication. An inter-processor communication apparatus is an apparatus-executable program for performing inter-processor communication, and the program includes an inter-processor communication apparatus,
Of the first and second processor nodes configured in the inter-processor communication device, access is made from each of the first and second processor nodes according to an operation program operating on the first processor node. Writing means for writing communication data to a shared memory provided in a shared manner,
The write notification information for notifying that the communication data has been written to the shared memory by the writing means is connected to the first and second processor nodes so that the communication data can be transmitted and received between the operation programs. First notification means for transmitting to the second processor node by socket communication;
Reading means for reading communication data from the shared memory in response to the write notification information transmitted by the first notification means;
An apparatus-executable program for causing read-out notification information for notifying that the communication data has been read out by the read-out means to function as second notification means for transmitting to the first processor node via the socket communication .   An information processing apparatus-readable recording medium storing the program according to claim 9.

Images