CN115904940A

CN115904940A - Self-adaptive regression random test method, device, terminal and medium

Info

Publication number: CN115904940A
Application number: CN202211317710.8A
Authority: CN
Inventors: 廖淳昊; 康雨晖; 杨文汉夫; 彭佳然; 王斯南; 刘烨庞
Original assignee: Southern University of Science and Technology
Current assignee: Southern University of Science and Technology
Priority date: 2022-10-26
Filing date: 2022-10-26
Publication date: 2023-04-04

Abstract

The invention discloses a method, a device, a terminal and a medium for self-adaptive regression random test, wherein the method comprises the following steps: acquiring an initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iteration sample set; acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set; screening the iteration sample set, and acquiring a sample which has the largest contribution to the coverage information of the iteration sample set as a sample to be executed; and calculating the performance value of the sample to be executed, and returning the calculation result. The invention defines the difference size according to the code coverage condition of the test case. The invention can greatly shorten the test period and save the test labor cost by reducing the number of test cases on the basis of ensuring the coverage information.

Description

Self-adaptive regression random test method, device, terminal and medium

Technical Field

The invention relates to the field of terminal application, in particular to a method, a device, a terminal and a medium for self-adaptive regression random test.

Background

With the increasing strength of software functions, software testing is more and more important, and the requirements are higher and higher. The search engine back-end test needs to guarantee high operation stability in a short time, for example, the search engine back-end test needs to maintain stable operation under tens of millions of data.

Driven by this requirement, testing the search engine back end with a large amount of data is a common way to ensure its operational stability. However, the test of large data volume requires a long test time, and in practical application, the test time is limited due to the limit of the development period of the product. Therefore, due to the constraint of testing time, a balance between testing period and data volume is often required. In addition, the complexity of the function and strategy at the back end of the search engine is high, the difficulty in manually selecting test data for a specific function or strategy is high, and great inconvenience is brought to a tester.

Therefore, how to shorten the testing time and improve the testing precision on the basis of ensuring the data volume requirement becomes an urgent problem to be solved.

Disclosure of Invention

The invention provides a method, a device, a terminal and a medium for adaptive regression random test, aiming at the defects of the prior art, so as to solve the technical problems that the test time is long and the data volume cannot meet the requirements at present.

The technical scheme adopted by the invention for solving the technical problem is as follows:

in a first aspect, the present invention provides an adaptive regression random test method, including the following steps:

an adaptive regression random test method, characterized in that the adaptive regression random test method comprises the steps of:

acquiring an initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iteration sample set; wherein the initial execution result is the coverage information of the initial sample;

acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set; wherein the new execution result is coverage information of the new sample;

screening the iteration sample set, and acquiring a sample which has the greatest contribution to the coverage information of the iteration sample set as a sample to be executed;

and calculating the performance value of the sample to be executed, and returning a calculation result.

In one implementation, the obtaining of the initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iterative sample set, includes;

acquiring the initial sample;

recording the coverage information of the initial sample and storing the coverage information into an iteration sample set; wherein the coverage information of the initial sample comprises the coverage rate of the initial sample and the coverage information of the code line.

In one implementation, the obtaining a new sample, obtaining a new execution result according to the new sample, and storing the new sample and the new execution result in the iterative sample set includes:

acquiring the new sample;

recording the coverage information of the new sample, and storing the coverage information into an iteration sample set; wherein the coverage information of the new sample comprises the coverage rate of the new sample and the coverage information of the code line.

In an implementation manner, the screening the iterative sample set to obtain a sample that contributes most to coverage information of the iterative sample set as a sample to be executed includes:

defining and updating distances between the initial samples and the new samples according to the overall coverage rate information of the iteration sample set.

In an implementation manner, the screening the iterative sample set to obtain a sample that contributes most to coverage information of the iterative sample set as a sample to be executed further includes:

and iteratively selecting the sample with the largest distance in the iterative sample set as a sample to be executed by using a greedy algorithm based on the distance corresponding to each initial sample and each new sample, and putting the sample into the iterative sample set.

In one implementation, the calculating the performance value of the to-be-executed sample and returning a calculation result includes:

transmitting the sample to be executed to a performance analysis model for calculation to obtain a model performance value of the sample to be executed, and returning; and the performance analysis model is calculated by adopting a model performance value calculation formula.

In one implementation, the model performance value calculation formula is:

wherein E is the model performance value, a is the coverage information of the screened samples, b is the coverage information of all the samples, c is the number of the screened samples, and d is the total number of the samples.

In a second aspect, the present invention provides an adaptive regression random testing apparatus, including:

the test sample generation module is used for acquiring an initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iteration sample set; the system is also used for acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set;

the test sample screening module is used for screening the iteration sample set to obtain a sample to be executed;

and the performance analysis module is used for performing performance analysis on the iteration sample set.

In a third aspect, the present invention provides a terminal, including: a processor and a memory, wherein the memory stores an adaptive regression random test program, and the adaptive regression random test program is used for implementing the adaptive regression random test method of the first aspect when executed by the processor.

In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores an adaptive regression random test program, and the adaptive regression random test program is used for implementing the adaptive regression random test method according to the first aspect when being executed by a processor.

The invention adopts the technical scheme and has the following effects:

according to the method, whether the corresponding input data (namely the initial sample and the new sample) contributes to the code coverage information is judged through the change condition of the code coverage information, if not, the input data of the test sample can be screened out, so that the original input data of ten million orders can be effectively compressed to hundred orders on the premise of ensuring that the code coverage information is not changed, the test speed can be greatly improved, the test period is shortened, manual intervention is not needed, and the labor cost can be saved.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the structures shown in the drawings without creative efforts.

FIG. 1 is a flow chart of a method of adaptive regression random testing in one implementation of the invention.

Fig. 2 is a functional schematic of a terminal in one implementation of the invention.

Fig. 3 is a block diagram of the apparatus provided by the present invention.

The implementation, functional features and advantages of the present invention will be further described with reference to the accompanying drawings.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention clearer and clearer, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The inventor finds that Software products have various defects through research, therefore, developers need to apply various means for ensuring the quality of Software, wherein the most important means is Software Testing (Software Testing), which refers to a process of operating a program to be tested under a specified condition and evaluating whether the program can meet the original requirement, and generally comprises unit Testing, integration Testing, system Testing and the like; random Testing (Random Testing) refers to randomly generating a series of Test cases according to an input field of a program to be tested, and is generally used for generating a Test Suite (Test Suite) for unit Testing, and an Adaptive Random Testing method (ART for short) is a Testing method for enabling a final Test Suite to have certain distribution characteristics (such as uniform distribution, stripe distribution, and aggregate distribution) by evaluating differences between generated Test samples and screening the samples based on Random Testing, and differences between different ART technologies are generally reflected in the definition of differences between Test samples and sample screening methods.

Because the function requirement of the network application program changes greatly, the function iteration is rapid, and the defects need to be repaired, the code at the back end needs to be modified frequently, and in order to avoid generating new errors or causing errors in other original codes, a Regression Test (Regression Test) needs to be introduced, that is, after the code is modified, the original Test case is rerun. Developing regression testing requires maintenance and management of the test suite. Since the number of test samples in a test suite is typically very large, running all of the test samples requires a significant amount of time overhead and computational resource usage (e.g., testing of some commercial software takes weeks to run all). This large overhead is unacceptable for fast iterative software projects. Therefore, regression testing generally selects and runs a part of the Test sample that may be affected by the modification, and this process is called Regression Test Selection (RTS). The existing RTS technology is generally directed to a general software type, but relatively few relevant researches or inventions are provided for a regression test scenario of RESTful API. Most of the original RTS technology needs to analyze code logic and calculate the dependency relationship among codes, and cannot be directly applied to the software form of RESTful API with high abstraction degree.

At present, a large amount of test data is often needed to ensure certain test code coverage information, so that the test time is too long, and the design cycle of a product cannot be met. However, some of these test data play the same role, so if these test data with the same role can be reduced, that is, duplicated test data can be deleted, the required test data can be greatly reduced. In an embodiment of the invention, it can be detected by the code coverage information which test data is the most contributing coverage information for the entire sample. Therefore, in the embodiment of the present invention, the sample with the largest contribution to the sample coverage information is reserved as the sample to be executed. By such a comparison, the required test data can be greatly reduced.

As shown in fig. 1, an embodiment of the present invention provides an adaptive regression random test method, including the following steps:

step S100, obtaining an initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iteration sample set; wherein the initial execution result is the coverage information of the initial sample.

In this embodiment, in the pre-preparation stage, the source code is compiled in this scheme, and the corresponding bytecode is generated. And (4) instrumentation is carried out in a part of source codes needing coverage information statistics by using a Soot tool, so that corresponding coverage information can be counted in the running process. And finally, combining the instrumented code and the non-instrumented code to obtain a new source code, and compiling the new source code to obtain an executable file of the system to be tested.

In this embodiment, in the information collection phase, in this scheme, firstly, for information of a network application to be tested, a yaml file describing interface information is generated by using openapi, and a test sample is randomly generated. These generated test samples correspond to the above-described initial samples, and may correspond to newly generated new samples when the system is changed. The test sample takes a message as a format and conforms to the specification of RESTful API. And then sending the message randomly generated in the test sample randomly generated by the controller to the system to be tested, and when the message is executed, representing that the workflow enters a test sample execution model. After the test sample is executed, the coverage information is counted through an information feedback model. There may be various algorithms for the transmitted coverage information, for example, one of the algorithms is: for each test sample, a string of 01 characters is generated, in which a 0 on a bit represents that the code of the corresponding line is uncovered, and a 1 represents that the code of the corresponding line is covered. Finally, the coverage information in the feedback model is extracted in the scheme for the sample screening stage.

In one implementation manner of the present embodiment, the step S100 includes the following steps:

step S101, acquiring the initial sample;

step S102, recording the coverage information of the initial sample, and storing the coverage information into an iteration sample set; wherein the coverage information of the initial sample comprises the coverage rate of the initial sample and the coverage information of the code line.

In this embodiment, before the system under test comes online: and acquiring an initial test sample set, executing all test samples in the sample set, monitoring the tested system, and recording the code coverage condition of the tested system. These coverage conditions generally refer to the number of rows or methods, etc., in the system under test that the test sample executes. And (4) correspondingly setting the samples and the respective coverage conditions one by one, and storing the samples into an iteration sample set.

In this embodiment, the server omits the annotation and the format change in the source data in the at least two overlay information statistics, and obtains the unique identifier of the function corresponding to the source data by calculating the source data in the at least two overlay information statistics.

In this embodiment, the server stores the unique identifier, and/or the server stores the first data in the at least two statistics of coverage information.

In this embodiment, the coverage information statistics include: and testing the initial samples in the file according to the serial numbers of the files to which the same initial samples belong, marking the tested blocks in the initial samples as covered blocks, and counting the covering information condition of the initial samples.

In this embodiment, the at least two times of latest coverage information statistics include: and respectively counting the coverage information conditions of the initial samples in the files at different times according to the time sequence aiming at the same file sequence number.

In this embodiment, the corresponding code coverage information data may be generated by a cov _ eventCount library function in the bullseye coverage software. Of course, in this embodiment, other software may be used to obtain the code coverage information data.

As shown in fig. 1, in an implementation manner of the embodiment of the present invention, the adaptive regression random testing method further includes the following steps:

step S200, acquiring a new sample, obtaining a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set; wherein the new execution result is the coverage information of the new sample;

in one implementation manner of this embodiment, the step S200 includes the following steps:

step S201, acquiring the new sample;

and step S202, recording the coverage information of the new sample, and storing the coverage information into an iteration sample set.

In this embodiment, the update phase of the system under test: when a tested system has events such as adding a new function, modifying an old function or repairing a software defect, a part of a new test sample (i.e. a new sample) is generated. When a new sample occurs, the test system executes and records the execution result of the new sample and the code coverage of the sample on the tested system, wherein the information format of the coverage is consistent with the information format recorded before the tested system is on line.

In this embodiment, new samples are always generated in the process of continuous iteration. Before each regression test, the iteration sample set is screened by using the evaluation and screening module, and only the part of original test cases which can supplement the coverage condition of the new sample in the iteration sample set is executed. By this means, the number of original test samples to be executed can be reduced as much as possible while original coverage information is maintained, and test samples with low coverage information can be gradually excluded. The finally selected executed test sample set can achieve the original coverage information and simultaneously avoid the infinite increase of the number of test samples, thereby reducing various consumptions generated when regression test samples are executed each time to the maximum extent.

step S300, the iteration sample set is screened, and a sample with the largest contribution to the coverage information of the iteration sample set is obtained and serves as a sample to be executed.

In the embodiment, in the ART sample screening and evaluating stage, firstly, the coverage information extracted by transmitting the coverage information to the test sample screening model is used as the guide information to guide the screening of the samples; and transmitting the coverage information to the candidate test sample file input in the test sample screening model to be used as the test sample to be screened.

In one implementation manner of this embodiment, step S300 includes the following steps:

step S301, defining and updating the distance between the initial sample and the new sample according to the overall coverage rate information of the iteration sample set.

Common discrepancy metric algorithms are: definition based on edit distance, weight addition, and the like. In the prior art, the measurement of the difference mostly starts from the test case itself: for example, the test case is regarded as a character string, the edit distance is used as a measure of the difference, or different weights are given to the attributes of the test case object, and the sum of the weights of the attributes having the difference is used as the difference. The method has the defects that only the codes of the tests are considered when the difference between the tests is calculated, and the coverage condition of the test cases on the tested system is ignored. Specifically, although the contents of some test cases are very different, the actual call paths to the system under test are almost the same. Such phenomena are commonly found in system modules such as a login module, an error processing module, and a form filling module.

In this embodiment, in view of the above phenomenon, the reaction of the system under test to the test case, i.e. the code coverage of the test case, is a method for defining the difference size. Compared with the prior art, the method has better adaptability to the tested system, and can overcome the defect of relative loss of attention to the tested system in the prior method to a certain extent.

In an implementation manner of this embodiment, step S300 further includes the following steps:

step S303, based on the distance corresponding to each initial sample and each new sample, using a greedy algorithm to iteratively select the sample with the largest distance in the iterative sample set as the sample to be executed, and putting the sample into the iterative sample set.

In this embodiment, in a specific screening process, the present invention uses a greedy algorithm to gradually select, according to coverage information corresponding to each test sample, a test sample that contributes most to coverage information of a current system as a sample to be executed, and puts the sample into an iterative sample set.

In this embodiment, when calculating the contribution to the system coverage information, the solution selects the distance between the coverage information of the current test sample and the coverage information of the executed test sample as a reference. The larger the distance, the more the contribution. The distance is defined in various ways. For example, for the 01 string representation described above, the difference in coverage for the two test samples may be the hamming distance or euclidean distance of their corresponding 01 strings. In addition, the distance may also be defined in terms of operations on the resource. For example, in the RESTful API, the user resource may be GET/POST/DELETE/PUT, etc., and in this scheme, it is necessary to preferentially ensure that all resources have operations covered. To achieve this goal, the "difference between the number of uncovered operations" of two resources can be used as a measure of the distance between them, for example, if two resources cover information 4 and 1 operation respectively, their distance is 3, so that there is a greater chance that only a small amount of the resources covered by the operations is selected than the resources already covering all the operations.

and step S400, calculating the performance value of the sample to be executed, and returning the calculation result.

In one implementation manner of this embodiment, step S400 includes the following steps:

step S401, transmitting the sample to be executed to a performance analysis model for calculation to obtain a model performance value of the sample to be executed, and returning; and the performance analysis model is calculated by adopting a model performance value calculation formula.

In one implementation manner of this embodiment, step S401 includes the following steps:

step S4011, the model performance value calculation formula is:

In this embodiment, when the screened test sample can cover all the methods or rows covered by the non-screened test, the screening is stopped. In the performance analysis stage, the system quantitatively calculates and evaluates the good and bad effect of a certain sample screening, specifically, the following formula is adopted:

for the two items before and after the division number, the larger the former item (the maximum value is 1), the screened sample can be closer to the coverage condition of the whole sample, and the screening effect is better; the smaller the latter term, the fewer the number of samples screened. Thus, when the value of the formula is larger, the result of sample screening is better, namely, the requirements of high coverage information and small sample number are met at the same time.

In the embodiment, whether corresponding input data (namely the initial sample and the new sample) contributes to the code coverage information is judged according to the change condition of the code coverage information, and if the corresponding input data does not contribute to the code coverage information, the input data of the test sample can be screened out, so that the original input data of ten-million magnitude can be effectively compressed to one-hundred magnitude under the premise of ensuring that the code coverage information is not changed, the test speed can be greatly improved, and the test period can be shortened. In addition, in the embodiment of the application, the method runs automatically completely without manual intervention, so that the labor cost can be saved.

Based on the foregoing adaptive regression random test method, this embodiment provides an adaptive regression random test apparatus, including:

a test sample generation module 1, configured to obtain an initial sample, obtain an initial execution result according to the initial sample, and store the initial sample and the initial execution result in an iterative sample set; the system is also used for acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set;

the test sample screening module 2 is used for screening the iteration sample set to obtain a sample to be executed;

and the performance analysis module 3 is used for performing performance analysis on the iteration sample set.

Exemplary device

Based on the above embodiments, the present invention further provides a terminal, and a schematic block diagram thereof may be as shown in fig. 2.

The terminal includes: the system comprises a processor, a memory, an interface, a display screen and a communication module which are connected through a system bus; wherein the processor of the terminal is configured to provide computing and control capabilities; the memory of the terminal includes a computer-readable storage medium and an internal memory; the computer-readable storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of an operating system and a computer program in the computer-readable storage medium; the interface is used for connecting external terminal equipment, such as mobile terminals and computers; the display screen is used for displaying corresponding self-adaptive regression random test information; the communication module is used for communicating with a cloud server or a mobile terminal.

The computer program is executed by a processor to implement an adaptive regression random test method.

It will be appreciated by those skilled in the art that the block diagram of fig. 2 is only a block diagram of a portion of the structure associated with the inventive arrangements and is not intended to limit the terminals to which the inventive arrangements may be applied, and that a particular terminal may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, a terminal is provided, which includes: a processor and a memory storing an adaptive regression random test program for implementing the adaptive regression random test method as described above when executed by the processor.

In one embodiment, a computer readable storage medium is provided, wherein the computer readable storage medium stores an adaptive regression random test mutual program, which when executed by a processor is configured to implement the adaptive regression random test method as described above.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, databases, or other media used in embodiments provided herein may include non-volatile and/or volatile memory.

In summary, the invention discloses a self-adaptive regression random test method, wherein the method comprises the following steps: acquiring an initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iteration sample set; the initial execution result is the coverage information of the initial sample; acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set; wherein the new execution result is the coverage information of the new sample; screening the iteration sample set to obtain a sample to be executed; and calculating the performance value of the sample to be executed, and returning the calculation result. The invention defines the difference size according to the code coverage condition of the test case. The invention can greatly improve the testing speed, shorten the testing period and save the testing labor cost.

It is to be understood that the invention is not limited to the examples described above, but that modifications and variations may be effected thereto by those of ordinary skill in the art in light of the foregoing description, and that all such modifications and variations are intended to be within the scope of the invention as defined by the appended claims.

Claims

1. An adaptive regression random test method, characterized in that the adaptive regression random test method comprises the steps of:

acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set; wherein the new execution result is the coverage information of the new sample;

screening the iteration sample set, and acquiring a sample which has the largest contribution to the coverage information of the iteration sample set as a sample to be executed;

2. The adaptive regression random test method of claim 1, wherein the obtaining of the initial sample, obtaining of the initial execution result according to the initial sample, and storing of the initial sample and the initial execution result in an iterative sample set comprises;

acquiring the initial sample;

3. The method according to claim 1, wherein the obtaining a new sample, obtaining a new execution result according to the new sample, and storing the new sample and the new execution result in the iterative sample set comprises:

acquiring the new sample;

recording the coverage information of the new sample, and storing the coverage information into an iteration sample set; wherein the coverage information of the new sample comprises coverage of the new sample and coverage information of the code line.

4. The adaptive regression random test method according to claim 1, wherein the screening the iterative sample set to obtain a sample with a largest contribution to coverage information of the iterative sample set as a sample to be executed includes:

5. The adaptive regression random test method according to claim 1, wherein the screening the iterative sample set to obtain a sample that contributes most to coverage information of the iterative sample set as a sample to be executed further comprises:

and iteratively selecting the sample with the largest distance in the iterative sample set as the sample to be executed by using a greedy algorithm based on the distance corresponding to each initial sample and each new sample, and putting the sample into the iterative sample set.

6. The method of claim 1, wherein the calculating the property values of the sample to be executed and returning the calculation results comprises:

7. The adaptive regression random test method of claim 6, wherein said model performance value calculation formula is:

8. An adaptive regression random test apparatus, comprising:

the test sample generation module is used for acquiring an initial sample, obtaining an initial execution result according to the initial sample, and storing the initial sample and the initial execution result into an iteration sample set;

the system is also used for acquiring a new sample, acquiring a new execution result according to the new sample, and storing the new sample and the new execution result into the iteration sample set;

and the performance analysis module is used for carrying out performance analysis on the iteration sample set.

9. A terminal, comprising: a processor and a memory storing an adaptive regression random test program for implementing the method of any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores an adaptive regression random test program for implementing the adaptive regression random test method according to any one of claims 1 to 7 when executed by a processor.