CN112486632B - K8 s-oriented user mode virtual device driving frame - Google Patents

Abstract

The invention discloses a k8 s-oriented user mode virtual device driving framework, which adopts an SRIOV technology to virtualize IO devices into a plurality of virtual devices sharing hardware resources, maps IO addresses and interrupts of the virtual devices into a pause container of a pod, and operates a driving program of the virtual devices in the pause container. Other application containers in the pod access the driver in the pause container by sharing memory. The framework provides contents such as virtual device management, communication interfaces of application programs and drivers, and the like, and a user only needs to write the driver codes of specific devices under the framework, so that the complexity of writing the device drivers is reduced. The framework adopts a virtualization and transparent transmission mode to transfer most functions of the device driver into the pause container for realization, and can be delivered together with the application program, so that the isolation, the deliverability and the reliability of the system of the driver are improved while the efficiency of the device is ensured.

Description

K8 s-oriented user mode virtual device driving frame

Technical Field

The invention belongs to the field of user mode equipment driving frames, and particularly relates to a k8 s-oriented user mode virtual equipment driving frame.

Background

In Linux systems, a conventional driver runs in kernel mode and shares the same address space with other parts of the kernel. Because of the complexity of the driver itself and the fact that the developer is not fully familiar with the kernel architecture, it is inevitable that the driver it develops is subject to errors. Once a driver makes an error, all applications using the driver and other kernel code referencing the driver will be affected, even causing a crash of the entire operating system, affecting the reliability of the entire system.

To improve the reliability of the system, moving the driver to the user state is an efficient approach. The main advantages of the driver running in user mode include: (1) The driver developer can use rich user mode application program development tools and software libraries, so that development efficiency is improved, and development difficulty is reduced; (2) The error of the user mode driver does not affect the whole system, the fault domain is reduced, and the reliability of the system is improved.

At present, a framework mainly adopted by developing a user mode driver comprises UIO and VFIO, wherein the UIO runs a small part of the driver in a kernel space, and most of the functions of the driver are realized in a user space, but the UIO cannot realize a DMA function, so that the method is only suitable for equipment with less data transmission. VFIO is an upgrade version of uoo, which uses hardware support such as IOMMU to isolate the device address space, and can safely map the device address space and interrupt to user space. Based on VFIO framework, bai Ziqiu, hu Huaixiang et al propose a user-oriented driver design scheme for NVMe, which is published in journal of computer applications and software.

K8s serving as container arrangement software is just like a standard of a cloud computing platform, and all places share the cloud platform to provide k8s services; however, the current user mode device driving framework is mainly proposed for the traditional application mode, and is not suitable for containerized application.

Disclosure of Invention

Aiming at the problems in the prior art, the invention provides a k8 s-oriented virtual device driving framework, which is convenient for developing a user-state device driving program under a k8s platform.

The technical solution for realizing the purpose of the invention is as follows: a k8 s-oriented user mode virtual device driving frame, wherein the driving frame comprises IO devices, a Linux kernel, namely a kernel space, and a k8s pod, namely a user space;

the IO device supports the SRIOV and can be virtualized into a plurality of virtual devices VFs, and the VFs share hardware resources;

the Linux kernel maps IO addresses and interrupts of the VF to the pause container of the k8s pod through VFIO and eventfd; the VF device driver operates in the path container of the pod, other application containers in the pod access the VF device driver in the path container in a memory sharing mode, and in addition, a virtual device management program is deployed in the application container of one of the pods.

Further, the functions of the virtual device management program include virtual device allocation, virtual device IO address and interrupt mapping.

Further, when the pod requests a virtual device, the virtual device manager allocates the virtual device as follows:

Step 1, inquiring whether equipment supporting virtualization exists or not, and if so, turning to step 2; otherwise, go to step 6;

step 2, whether the equipment is virtualized or not, if so, turning to step 4; otherwise, turning to step 3;

Step 3, virtualizing a given number of virtual devices according to the virtual attribute of the devices and the virtualization method; the virtual attribute comprises the number of supported virtual devices;

Step 4, whether unassigned virtual devices exist or not, and if so, turning to step 5; otherwise, go to step 6;

step 5, mapping IO addresses and interrupts of the virtual device into the pause container of the pod through VFIO and eventfd;

and 6, failing to allocate the equipment and exiting.

Further, the application program and the VF device driver program communicate through two queues to realize the reading and writing of the application program, and the specific implementation mode comprises the following steps:

the method comprises the steps of providing a driving interface and an application interface, wherein the driving interface is deployed in a pause container of a pod and is connected with a VF device driver; the application interface is deployed in an application container and connected with an application program;

At the same time, the states of the two queues are described by a single array desc.

Further, the application program writing device comprises the following specific processes:

Step 1, an application program adds buffers with data into a write queue;

step 2, changing the state of the array desc to indicate that valid data exists;

Step 3, notifying the VF device that the driver data is updated;

and 4, the VF device driver checks the state of the array desc and reads out the effective data.

Compared with the prior art, the invention has the remarkable advantages that: 1) The user only needs to write the drive code of the specific device under the framework, so that the complexity of writing the device drive program is reduced; 2) By adopting a virtualization and transparent transmission mode, the isolation, the deliverability and the reliability of the system of the driving program are improved while the efficiency of the equipment is ensured.

The invention is described in further detail below with reference to the accompanying drawings.

Drawings

FIG. 1 is a diagram of a user-mode virtual device driver framework facing k8s in one embodiment.

FIG. 2 is a flow diagram of virtual device allocation in one embodiment.

FIG. 3 is a schematic diagram of an application program in communication with a device driver in one embodiment.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

In one embodiment, in combination with fig. 1, a k8 s-oriented user mode virtual device driving framework is provided, where the driving framework includes an IO device, a Linux kernel, i.e., kernel space, and a k8s pod, i.e., user space;

the IO device supports the SRIOV and can be virtualized into a plurality of virtual devices VFs, and the VFs share hardware resources;

The Linux kernel maps IO addresses and interrupts of the VF to the pause container of the k8s pod through VFIO and eventfd; the VF device driver operates in the path container of the pod, other application containers in the pod access the VF device driver in the path container in a memory sharing mode, and in addition, a virtual device management program is deployed in the application container of one of the pods, and the functions of the program comprise virtual device allocation, virtual device IO addresses and interrupt mapping.

The framework provides contents such as a virtual device management program, an application program and a driver communication interface, and the like, and a user only needs to write a specific device driver under the framework, so that the complexity of writing the device driver is reduced. The framework adopts a virtualization and transparent transmission mode to transfer most functions of the device driver into the pause container for realization, and can be delivered together with the application program, so that the isolation, the deliverability and the reliability of the system of the driver are improved while the efficiency of the device is ensured.

Further, in one embodiment, in conjunction with FIG. 2, when a pod requests a virtual device, the virtual device manager assigns the virtual device as follows:

Step 1, inquiring whether equipment supporting virtualization exists or not, and if so, turning to step 2; otherwise, go to step 6;

step 2, whether the equipment is virtualized or not, if so, turning to step 4; otherwise, turning to step 3;

Step 3, virtualizing a given number of virtual devices according to the virtual attribute of the devices and the virtualization method; the virtual attribute comprises the number of supported virtual devices;

Step 4, whether unassigned virtual devices exist or not, and if so, turning to step 5; otherwise, go to step 6;

step 5, mapping IO addresses and interrupts of the virtual device into the pause container of the pod through VFIO and eventfd;

and 6, failing to allocate the equipment and exiting.

Further, in one embodiment, in conjunction with fig. 3, the application program and the VF device driver communicate through two queues, so as to implement an application program read/write device, and the specific implementation manner includes:

the method comprises the steps of providing a driving interface and an application interface, wherein the driving interface is deployed in a pause container of a pod and is connected with a VF device driver; the application interface is deployed in an application container and connected with an application program;

At the same time, the states of the two queues are described by a single array desc.

Further, in one embodiment, the application writing device includes:

Step 1, an application program adds buffers with data into a write queue;

step 2, changing the state of the array desc to indicate that valid data exists;

Step 3, notifying the VF device that the driver data is updated;

and 4, the VF device driver checks the state of the array desc and reads out the effective data.

The invention provides a user-state device driving framework aiming at a k8s Pod framework based on VFIO framework, which supports the development of a user-state device driving program under the k8s framework. The framework encapsulates the device driver into the container, and delivers, deploys and runs the device driver together with the application, so that the isolation between the applications is stronger, the interference degree between different applications is reduced, and the reliability of the system is improved.

The foregoing has outlined and described the basic principles, features, and advantages of the present invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.

Claims (3)

1. The k8 s-oriented user mode virtual device driving framework is characterized by comprising IO devices, linux kernels, namely kernel space, and k8s pod, namely user space;

the IO device supports the SRIOV and can be virtualized into a plurality of virtual devices VFs, and the VFs share hardware resources;

The Linux kernel maps IO addresses and interrupts of the VF to the pause container of the k8s pod through VFIO and eventfd; the VF device driver is operated in the path container of the pod, other application containers in the pod access the VF device driver in the path container in a memory sharing mode, and in addition, the application container of one of the pods is provided with a virtual device management program;

when the pod requests a virtual device, the virtual device manager allocates the virtual device as follows:

Step 1, inquiring whether equipment supporting virtualization exists or not, and if so, turning to step 2; otherwise, go to step 6;

step 2, whether the equipment is virtualized or not, if so, turning to step 4; otherwise, turning to step 3;

Step 3, virtualizing a given number of virtual devices according to the virtual attribute of the devices and the virtualization method; the virtual attribute comprises the number of supported virtual devices;

Step 4, whether unassigned virtual devices exist or not, and if so, turning to step 5; otherwise, go to step 6;

step 5, mapping IO addresses and interrupts of the virtual device into the pause container of the pod through VFIO and eventfd;

Step 6, equipment fails to be allocated and exits;

The application program and the VF device driver program communicate through two queues to realize the reading and writing of the application program, and the specific implementation mode comprises the following steps:

the method comprises the steps of providing a driving interface and an application interface, wherein the driving interface is deployed in a pause container of a pod and is connected with a VF device driver; the application interface is deployed in an application container and connected with an application program;

At the same time, the states of the two queues are described by a single array desc.

2. The k8s oriented user mode virtual device driver framework of claim 1, wherein the functions of the virtual device manager include virtual device allocation, virtual device IO address and interrupt mapping.

3. The k8s oriented user mode virtual device driver framework of claim 1, wherein the application writing device comprises:

Step 1, an application program adds buffers with data into a write queue;

step 2, changing the state of the array desc to indicate that valid data exists;

Step 3, notifying the VF device that the driver data is updated;

and 4, the VF device driver checks the state of the array desc and reads out the effective data.