CN112486632A - User-state virtual device driving framework facing k8s - Google Patents
User-state virtual device driving framework facing k8s Download PDFInfo
- Publication number
- CN112486632A CN112486632A CN202011413906.8A CN202011413906A CN112486632A CN 112486632 A CN112486632 A CN 112486632A CN 202011413906 A CN202011413906 A CN 202011413906A CN 112486632 A CN112486632 A CN 112486632A
- Authority
- CN
- China
- Prior art keywords
- virtual
- driver
- pod
- container
- virtual device
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000006870 function Effects 0.000 claims abstract description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000000034 method Methods 0.000 claims description 6
- 238000002955 isolation Methods 0.000 abstract description 5
- 230000005540 biological transmission Effects 0.000 abstract description 4
- 238000004891 communication Methods 0.000 abstract description 3
- 238000011161 development Methods 0.000 description 4
- 238000010586 diagram Methods 0.000 description 3
- 238000004883 computer application Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Stored Programmes (AREA)
Abstract
The invention discloses a k8 s-oriented user-mode virtual device driving framework, which adopts an SRIOV technology to virtualize IO devices into a plurality of virtual devices sharing hardware resources, maps IO addresses and interrupts of the virtual devices into a pause container of a pod, and runs a driver of the virtual devices in the pause container. And other application containers in the pod access the driver in the pause container in a memory sharing mode. The framework provides contents such as virtual equipment management, application programs, a driver communication interface and the like, and a user only needs to write the driver codes of specific equipment under the framework, so that the complexity of writing the driver of the equipment is reduced. The framework adopts a virtualization and transparent transmission mode, most functions of the device driver are transplanted into the pause container to be realized, the device driver and the application program can be delivered together, and the isolation and the deliverable performance of the driver and the reliability of a system are improved while the efficiency of the device is ensured.
Description
Technical Field
The invention belongs to the field of user mode device driving frameworks, and particularly relates to a k8 s-oriented user mode virtual device driving framework.
Background
In the Linux system, a conventional driver runs in a kernel mode and shares the same address space with other parts of the kernel. Due to the complexity of the driver itself and the fact that the developer is not completely familiar with the kernel architecture, the driver developed by the driver is inevitable to be in error. Once a driver generates an error, it will affect all applications using the driver and other kernel code referencing the driver, and may even cause a crash of the entire operating system, affecting the reliability of the entire system.
To improve the reliability of the system, moving the driver to the user mode is an effective way. The main advantages of running the driver in the user mode include: (1) the driver developer can use abundant user mode application program development tools and software libraries, so that the development efficiency is improved, and the development difficulty is reduced; (2) the whole system cannot be influenced by the user-mode driver errors, the fault domain is reduced, and the system reliability is improved.
At present, a framework mainly adopted for developing a user mode driver comprises UIO and VFIO, wherein the UIO runs a small part of a driver in a kernel space, most functions of the driver are realized in the user space, but the UIO cannot realize a DMA function, so that the UIO is only suitable for less equipment for data transmission. The VFIO is an upgraded version of the UIO, and realizes isolation of the device address space by using support of hardware such as the IOMMU, and can safely map the device address space and interrupts to the user space. Based on a VFIO framework, Baizi autumn, Huhuaxiang and the like propose a user mode driver design scheme of NVMe, and the paper is published in journal of computer application and software.
K8s as container orchestration software is just like the standard configuration of a cloud computing platform, and common cloud platforms all provide k8s services; however, the current user-mode device driver framework is mainly proposed for the traditional application mode and is not suitable for containerized application.
Disclosure of Invention
The present invention aims to solve the above problems in the prior art, and provide a k8 s-oriented virtual device driver framework, which facilitates the development of user-mode device drivers under the k8s platform.
The technical solution for realizing the purpose of the invention is as follows: a k8 s-oriented user-state virtual device driver framework, the driver framework comprising IO devices, a Linux kernel, i.e. kernel space, and a k8s pod, i.e. user space;
the IO device supports SRIOV and can be virtualized into a plurality of virtual devices (VF), and the plurality of VFs share hardware resources;
the Linux kernel maps the IO address and the interrupt of the VF into a pause container of k8s pod through VFIO and eventfd; the VF device driver runs in the pool container of the pod, and other application containers in the pod access the VF device driver in the pool container in a memory sharing manner, and in addition, a virtual device manager is deployed in one application container of the pod.
Further, the functions of the virtual device hypervisor include virtual device allocation, virtual device IO address and interrupt mapping.
Further, when a pod requests a virtual device, the virtual device manager assigns the virtual device as follows:
step 1, inquiring whether equipment supporting virtualization exists or not, and if yes, turning to step 2; otherwise, go to step 6;
step 2, whether the equipment is virtualized or not is judged, and if the equipment is virtualized, the step 4 is carried out; otherwise, go to step 3;
step 3, virtualizing a given number of virtual devices according to the virtual attribute of the devices and the virtualization method; the virtual attribute comprises a supported number of virtual devices;
step 4, whether the virtual equipment which is not distributed exists or not exists, if so, the step 5 is carried out; otherwise, go to step 6;
step 5, mapping the IO address and the interrupt of the virtual equipment to a pause container of the pod through the VFIO and the eventfd;
and 6, failing to distribute the equipment, and exiting.
Further, the application program communicates with the VF device driver through two queues to implement the application program read/write device, and the specific implementation manner includes:
providing a drive interface and an application interface, wherein the drive interface is deployed in a pool container of the pod and is connected with a VF device driver; the application interface is deployed in the application container and is connected with the application program;
meanwhile, the states of the two queues are described by an array desc.
Further, the specific process of the application program writing device includes:
step 1, adding a buffer stored with data into a write queue by an application program;
step 2, changing the state of the array desc to show that valid data exists;
step 3, informing the VF device that the driver data is updated;
and 4, checking the state of the array desc by the VF device driver, and reading out valid data.
Compared with the prior art, the invention has the following remarkable advantages: 1) a user only needs to write the drive codes of specific equipment under the framework, so that the complexity of writing the drive programs of the equipment is reduced; 2) by adopting a virtualization and transparent transmission mode, the device efficiency is ensured, and meanwhile, the isolation and the deliverable performance of the driving program and the reliability of the system are improved.
The present invention is described in further detail below with reference to the attached drawing figures.
Drawings
FIG. 1 is a component diagram of a k8 s-oriented user-mode virtual device driver framework in one embodiment.
FIG. 2 is a flow diagram of virtual device allocation in one embodiment.
FIG. 3 is a diagram illustrating application program and device driver communication in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
In one embodiment, in conjunction with fig. 1, there is provided a k8 s-oriented user-mode virtual device driver framework, the driver framework including an IO device, a Linux kernel, i.e., kernel space, and a k8s pod, i.e., user space;
the IO device supports SRIOV and can be virtualized into a plurality of virtual devices (VF), and the plurality of VFs share hardware resources;
the Linux kernel maps the IO address and the interrupt of the VF into a pause container of k8s pod through VFIO and eventfd; the VF device driver runs in a pool container of a pod, and other application containers in the pod access the VF device driver in the pool container in a shared memory manner, and in addition, a virtual device manager is deployed in one application container of the pod, and functions of the program include virtual device allocation, virtual device IO address and interrupt mapping.
The framework provides contents such as virtual equipment management programs, application programs, driver communication interfaces and the like, and a user only needs to write the driver codes of specific equipment under the framework, so that the complexity of writing the driver of the equipment is reduced. The framework adopts a virtualization and transparent transmission mode, most functions of the device driver are transplanted into the pause container to be realized, the device driver and the application program can be delivered together, and the isolation and the deliverable performance of the driver and the reliability of a system are improved while the efficiency of the device is ensured.
Further, in one embodiment, in conjunction with FIG. 2, when a pod requests a virtual device, the virtual device manager assigns the virtual device as follows:
step 1, inquiring whether equipment supporting virtualization exists or not, and if yes, turning to step 2; otherwise, go to step 6;
step 2, whether the equipment is virtualized or not is judged, and if the equipment is virtualized, the step 4 is carried out; otherwise, go to step 3;
step 3, virtualizing a given number of virtual devices according to the virtual attribute of the devices and the virtualization method; the virtual attribute comprises a supported number of virtual devices;
step 4, whether the virtual equipment which is not distributed exists or not exists, if so, the step 5 is carried out; otherwise, go to step 6;
step 5, mapping the IO address and the interrupt of the virtual equipment to a pause container of the pod through the VFIO and the eventfd;
and 6, failing to distribute the equipment, and exiting.
Further, in one embodiment, with reference to fig. 3, the application program and the VF device driver communicate through two queues to implement the application program read/write device, and the specific implementation manner includes:
providing a drive interface and an application interface, wherein the drive interface is deployed in a pool container of the pod and is connected with a VF device driver; the application interface is deployed in the application container and is connected with the application program;
meanwhile, the states of the two queues are described by an array desc.
Further, in one embodiment, the specific process of the application program writing device includes:
step 1, adding a buffer stored with data into a write queue by an application program;
step 2, changing the state of the array desc to show that valid data exists;
step 3, informing the VF device that the driver data is updated;
and 4, checking the state of the array desc by the VF device driver, and reading out valid data.
The invention provides a user mode device driving framework aiming at a Pod framework of k8s based on a VFIO framework, and supports the development of a user mode device driving program under a k8s framework. The framework encapsulates the device driver into a container, and delivers, deploys and runs together with the applications, so that the isolation among the applications is stronger, the interference degree among different applications is reduced, and the reliability of the system is improved.
The foregoing illustrates and describes the principles, general features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (5)
1. A k8 s-oriented user-mode virtual device driver framework, wherein the driver framework comprises an IO device, a Linux kernel, namely kernel space, and a k8s pod, namely user space;
the IO device supports SRIOV and can be virtualized into a plurality of virtual devices (VF), and the plurality of VFs share hardware resources;
the Linux kernel maps the IO address and the interrupt of the VF into a pause container of k8s pod through VFIO and eventfd; the VF device driver runs in the pool container of the pod, and other application containers in the pod access the VF device driver in the pool container in a memory sharing manner, and in addition, a virtual device manager is deployed in one application container of the pod.
2. The k8 s-oriented user-state virtual device driver framework of claim 1, wherein the functions of the virtual device hypervisor include virtual device allocation, virtual device IO address and interrupt mapping.
3. The k8 s-oriented user-mode virtual device driver framework of claim 2, wherein when a pod requests a virtual device, the virtual device hypervisor allocates the virtual device by:
step 1, inquiring whether equipment supporting virtualization exists or not, and if yes, turning to step 2; otherwise, go to step 6;
step 2, whether the equipment is virtualized or not is judged, and if the equipment is virtualized, the step 4 is carried out; otherwise, go to step 3;
step 3, virtualizing a given number of virtual devices according to the virtual attribute of the devices and the virtualization method; the virtual attribute comprises a supported number of virtual devices;
step 4, whether the virtual equipment which is not distributed exists or not exists, if so, the step 5 is carried out; otherwise, go to step 6;
step 5, mapping the IO address and the interrupt of the virtual equipment to a pause container of the pod through the VFIO and the eventfd;
and 6, failing to distribute the equipment, and exiting.
4. The k8 s-oriented user-state virtual device driver framework according to claim 3, wherein the application program and the VF device driver communicate with each other through two queues to implement an application program read/write device, and the specific implementation manner includes:
providing a drive interface and an application interface, wherein the drive interface is deployed in a pool container of the pod and is connected with a VF device driver; the application interface is deployed in the application container and is connected with the application program;
meanwhile, the states of the two queues are described by an array desc.
5. The k8 s-oriented user-state virtual device driver framework as claimed in claim 4, wherein the application program writes to the device by a specific process including:
step 1, adding a buffer stored with data into a write queue by an application program;
step 2, changing the state of the array desc to show that valid data exists;
step 3, informing the VF device that the driver data is updated;
and 4, checking the state of the array desc by the VF device driver, and reading out valid data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011413906.8A CN112486632B (en) | 2020-12-07 | 2020-12-07 | K8 s-oriented user mode virtual device driving frame |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011413906.8A CN112486632B (en) | 2020-12-07 | 2020-12-07 | K8 s-oriented user mode virtual device driving frame |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112486632A true CN112486632A (en) | 2021-03-12 |
| CN112486632B CN112486632B (en) | 2024-06-18 |
Family
ID=74939895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011413906.8A Active CN112486632B (en) | 2020-12-07 | 2020-12-07 | K8 s-oriented user mode virtual device driving frame |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112486632B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114640678A (en) * | 2022-03-14 | 2022-06-17 | 明阳产业技术研究院(沈阳)有限公司 | Pod management method, device and medium based on SR-IOV |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107844304A (en) * | 2016-09-18 | 2018-03-27 | 阿里巴巴集团控股有限公司 | The hot upgrade method of virtual machine and device |
| CN111651269A (en) * | 2020-05-18 | 2020-09-11 | 青岛镕铭半导体有限公司 | Method, device and computer readable storage medium for realizing equipment virtualization |
-
2020
- 2020-12-07 CN CN202011413906.8A patent/CN112486632B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107844304A (en) * | 2016-09-18 | 2018-03-27 | 阿里巴巴集团控股有限公司 | The hot upgrade method of virtual machine and device |
| CN111651269A (en) * | 2020-05-18 | 2020-09-11 | 青岛镕铭半导体有限公司 | Method, device and computer readable storage medium for realizing equipment virtualization |
Non-Patent Citations (2)
| Title |
|---|
| SPYROS CHIOTAKIS,: "vFPGAmanager: A Hardware-Software Framework for Optimal FPGA Resources Exploitation in Network Function Virtualization", 《IEEE XPLORE》, pages 2 - 4 * |
| 王伟军;: "基于Kubernetes的容器云平台建设", 电脑知识与技术, no. 36, 25 December 2019 (2019-12-25) * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114640678A (en) * | 2022-03-14 | 2022-06-17 | 明阳产业技术研究院(沈阳)有限公司 | Pod management method, device and medium based on SR-IOV |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112486632B (en) | 2024-06-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2009357325B2 (en) | Method and apparatus for handling an I/O operation in a virtualization environment | |
| CN103034524B (en) | Half virtualized virtual GPU | |
| EP2035936B1 (en) | An apparatus and method for memory address re-mapping of graphics data | |
| US10133504B2 (en) | Dynamic partitioning of processing hardware | |
| US20180336158A1 (en) | Systems and methods for data transfer with coherent and non-coherent bus topologies and attached external memory | |
| US8065441B2 (en) | Method and apparatus for supporting universal serial bus devices in a virtualized environment | |
| US11836091B2 (en) | Secure memory access in a virtualized computing environment | |
| KR20130111593A (en) | Direct sharing of smart devices through virtualization | |
| WO2007036072A1 (en) | Apparatus and method for expedited virtual machine (vm) launch in vm cluster environment | |
| US10620963B2 (en) | Providing fallback drivers for IO devices in a computing system | |
| US20180335956A1 (en) | Systems and methods for reducing data copies associated with input/output communications in a virtualized storage environment | |
| US10235195B2 (en) | Systems and methods for discovering private devices coupled to a hardware accelerator | |
| US12019560B2 (en) | Virtual partitioning a processor-in-memory (“PIM”) | |
| US10990436B2 (en) | System and method to handle I/O page faults in an I/O memory management unit | |
| CN112486632B (en) | K8 s-oriented user mode virtual device driving frame | |
| CN113626148B (en) | Terminal virtual machine generation system and method based on hybrid virtualization | |
| US8402191B2 (en) | Computing element virtualization | |
| Kristiansen | PCIe Device Lending | |
| Yin et al. | A user-space virtual device driver framework for Kubernetes |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 222001 No.18 Shenghu Road, Lianyungang City, Jiangsu Province Applicant after: The 716th Research Institute of China Shipbuilding Corp. Address before: 222001 No.18 Shenghu Road, Lianyungang City, Jiangsu Province Applicant before: 716TH RESEARCH INSTITUTE OF CHINA SHIPBUILDING INDUSTRY Corp. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |