US9612947B2

US9612947B2 - Code-free testing framework

Info

Publication number: US9612947B2
Application number: US13/692,228
Authority: US
Inventors: Tomer Fishaimer
Original assignee: CA Inc
Current assignee: CA Inc
Priority date: 2012-12-03
Filing date: 2012-12-03
Publication date: 2017-04-04
Also published as: US20140157057A1; US9304894B2; US20140157241A1

Abstract

A method, system and computer program product for testing testable code of an application comprises sending a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator; and receiving, by the computer-implemented client from the remote test orchestrator, a list identifying the test agent registered with the remote test orchestrator.

Description

CROSS REFERENCE TO RELATED APPLICATION

This application is related to an application filed concurrently herewith entitled “Code-Free Testing Framework,” by Tomer Fishaimer, now issued as U.S. Pat. No. 9,304,894, the entire disclosure of which is incorporated herein by reference.

BACKGROUND

The present disclosure relates to deployed applications, and more specifically, to testing a deployed application.

One aspect of developing complex, and not-so-complex, software applications relates to testing of the software application. Typical software testing may involve providing a software tester with a complete version of a software application, in various stage of development, and then having them randomly use the application in an attempt to identify problems. There have been attempts, however to automate software testing. These attempts, however require knowing the structure of the software application and having sufficient programming ability to create an automatic testing framework that autonomously inputs different, pre-selected data sets into the application and analyzes the resulting output. Thus, developing and designing such an automated testing framework requires a software tester that is highly knowledgeable about the details of the software application and highly skilled in programming as well. While such an approach is, at best, difficult to implement even if started at the early stages of developing an application, it is nearly impossible to implement for legacy code that has already been deployed into operation.

BRIEF SUMMARY

According to one aspect of the present disclosure, a method for testing testable code of an application comprises sending a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator; and receiving, by the computer-implemented client from the remote test orchestrator, the list identifying any test agents registered with the remote test orchestrator.

According to another aspect of the disclosure, a system for testing testable code of an application comprises a computer processor and memory configured to store and execute instructions; a communication module configured to send a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator; and the communication module further configured to receive, by the computer-implemented client from the remote test orchestrator, the list identifying any test agents registered with the remote test orchestrator.

According to yet another aspect of the disclosure, a computer program product for testing testable code of an application, comprises a computer readable storage medium having computer readable program code embodied therewith. The computer readable program code comprising computer readable program code configured to send a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator; and computer readable program code configured to receive, by the computer-implemented client from the remote test orchestrator, the list identifying any test agents registered with the remote test orchestrator.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying figures with like references indicating like elements.

FIG. 1 illustrates an example computing environment in which a code-free testing framework can be deployed in accordance with the principles of the present disclosure.

FIG. 2A-FIG. 2C illustrate an example user interface of a code-free testing framework in accordance with the principles of the present disclosure.

FIG. 3A depicts a flowchart of an example method for a deployed agent providing a code-free testing framework in accordance with the principles of the present disclosure.

FIG. 3B depicts a flowchart of an example method for a test orchestrator to provide a code-free testing framework in accordance with the principles of the present disclosure.

FIG. 3C depicts a flowchart of an example method for a test client to utilize a code-free testing framework in accordance with the principles of the present disclosure.

FIG. 4 is a block diagram of a data processing system in accordance with the principles of the present disclosure.

DETAILED DESCRIPTION

As will be appreciated by one skilled in the art, aspects of the present disclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc.) or combining software and hardware implementation that may all generally be referred to herein as a “circuit,” “module,” “component,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer readable program code embodied thereon.

Any combination of one or more computer readable media may be utilized. The computer readable media may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Rash memory), an appropriate optical fiber with a repeater, a portable compact disc read-only memory (CORaM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, CII, VB.NET, Python or the like, conventional procedural programming languages, such as the “c” programming language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby and Groovy, or other programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS).

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that when executed can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions when stored in the computer readable medium produce an article of manufacture including instructions which when executed, cause a computer to implement the function/act specified in the flowchart and/or block diagram block or blocks. The computer program instructions may also be loaded onto a computer, other programmable instruction execution apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatuses or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

To aid in understanding aspects of the present disclosure, an example environment of J2EE managed servers is described, with respect to FIG. 1. However, one of ordinary skill will readily recognize that this environment is provided merely by way of example and the functionality and behavior described herein is not limited to this environment but can, for example, be implemented with .NET managed servers and the like. The example environment of FIG. 1 is merely one particular example of an environment that provides a code-free testing framework.

Referring to FIG. 1, different J2EE servers may be located at different physical or logical locations accessible using a network 100 and each of the J2EE servers provide one or more executing applications which can be utilized by a client. For example, one J2EE server 102, a second J2EE server 106, and other J2EE servers 110 can be deployed on various hardware platforms in communication with the network 100 (e.g., the Internet). Each J2EE server is not necessarily isolated at a unique hardware location, although that can occur. For example, a hardware platform 124 can have more than one concurrently executing J2EE server (e.g., server 126 and server 130). Examples of conventional J2EE servers include JBOSS, WebLogic, and WebSphere. One of ordinary skill will recognize that a J2EE server is one type of application server wherein an application server, in general, provides an environment in which various applications can execute. For example, application servers usually provide services such as security, data services, transaction support, and load balancing so that applications running on the server do not need to explicitly include such services.

In the environment of FIG. 1, there is a respective test agent associated with each of the J2EE servers. Specifically, test agent Agent1 104 is associated with the J2EE server 102; test agent Agent2 108 is associated with the J2EE server 106; test agent AgentN 112 is associated with the J2EE server 110; test agent AgentA 128 is associated with the J2EE server 126; and test agent AgentB 132 is associated with the J2EE server 130. As explained more fully below, each respective test agent provides access to testable code associated with its associated J2EE server. For example, the J2EE Server 106 may have two different components that execute within its environment. The test agent Agent2 108 is configured to discover portions of each component that can be tested and to facilitate interaction with that code by a client.

In a particular example, the testable code can comprise a plurality of components and each component (e.g., a Java component) available within the environment of the J2EE server 106 may be an Enterprise Bean defined by a respective .class file; in another example, the components may be part of a package that has a number of related .class files. In either instance, the test agent Agent2 108 executes on the J2EE server 106 and can identify the .class files defining the components. Once the .class files are identified, then the test agent Agent2 108 can identify respective methods that become available when each .class file is instantiated as a respective object.

Each test agent (e.g., 104, 108, 112, 128, 132) may also register with a test orchestrator 114. The test orchestrator 114 may be an application accessible through the network 100 and may simplify access by clients to the different J2EE servers (e.g., 102, 106, 110, 126, 130). For example, the test agent Agent1 102 may be developed with foreknowledge of the existence, and location, of the test orchestrator 114. Thus, when the test agent Agent1 104 is deployed on the application server 102 it may register with the test orchestrator 114. When a test agent (e.g., 104, 108, 112, 128, 132) registers with the test orchestrator 114, the test agent can identify the appropriate application server (e.g., 102, 106, 110, 126, 130) on which it is deployed as well as information about the components that comprise the testable code that each agent discovers. The test orchestrator 114 can have storage 116 available to store the information related to those test agents (e.g., 104, 108, 112, 128, 132) which have registered with the test orchestrator 114.

As a result of the test agents (e.g., 104, 108, 112, 128, 132) registering with the test orchestrator, the test orchestrator 114 has knowledge of how to locate the application servers (e.g., 102, 106, 110, 126, 130). There may be one or more clients such as client1 118, client2 120, and clientN 122 that desire to test testable code that is available on one or more of the application servers (e.g., 102, 106, 110, 126, 130). Each client (e.g., 118, 120, 122) may be a web browser that can be used by a quality assurance engineer, an application developer or a software engineer to test software applications in accordance with the principles of the present disclosure. Each client (e.g., 118, 120, 122), therefore, may communicate with just the test orchestrator 114 so that each client does not need to know how to directly communicate with each of the application servers. Accordingly, the dashed

lines

101 and 103 of FIG. 1 represent the apparent communication that takes place between the clients (e.g., 118, 120,122), the test orchestrator 114, and the application servers (e.g., 102, 106, 110, 126, 130).

FIG. 2A-FIG. 2C illustrate an example user interface of a code-free testing framework in accordance with the principles of the present disclosure. The orchestrator 114 can provide a client (e.g., 118, 120, 122) with a list of available test agents (e.g., 104, 108, 112, 128, 132) which the client may be authorized to access. The client may then select a particular one of the agents and then the client is presented with information about testable code which is accessible using that particular test agent. For example, the client may be presented a user interface screen 200 that lists components which the client may test. A component may, for example, comprise one or more .class files, which can be instantiated to define one or more objects.

Returning to FIG. 1, client1 118 may have selected Agent1 104 such that client1 118 is presented with a list of components that are installed (i.e., deployed) to be executed on the J2EE Server1 102. Each component may have a corresponding .class file residing in an appropriate location on the J2EE Server1 102. For example, the J2EE Server1 may have an environmental variable that defines the default path locations for where all possible .class files are stored on the J2EE Server1 102. These class files provide the definition of the different components that can be instantiated as objects and tested by the client1 118 via the test orchestrator 114 and the test agent Agent1 104.

In the example screen 200 of FIG. 2A, the client client1 118 is presented with a list having three components Component1 202A, Component2 202B, and Component3 202C. Assuming that the client1 118 clicked on Component1 202A, an expanded drop-down list can appear that lists the three methods declared within Component1 202A which are, for example, Method1 204A, Method2 204B, and Method3 204C.

When client client1 118 selects Method1 204A, for example, then the user interface screens 201 and 202 of FIG. 2B and FIG. 2C, respectively, can be provided to the client client1 118. The user interface screens 200, 201 and 202 are merely examples of what information can be provided by a test agent and how it may be arranged. The information may also vary according to a particular structure of the component or method that is being tested.

In the example screen 201, the testable component Component1 202A has two fields 206A and 206B. The fields represent variables within the component that can be assigned different values by the client client1 118 via the user interface screen 201. A field has a label (e.g., “Field1”) and an assigned value. The particular values assigned to Field1 206A and Field2 206B may affect multiple methods within Component1 202A and, thus, they may be grouped and displayed as shown in FIG. 2B. The Method1 204A may have its own fields that are variables applicable only within Method1 204A. The three fields of user interface screen 203, FieldA 216, FieldB 217, and FieldC 218 can be collectively referred to as the properties of Method1 204A. Field1 206A and Field2 206B apply to all methods within Component1 202A and, hence, are also considered properties of Method1 204.

Other information about the Method1 204A can also be provided via the user interface screen 201. For example, parameters 208 that are passed to the Method1 204A can be identified; and also the primitive type (e.g., int, double, string, etc.) of the Method1 204A can be provided as well. It is possible for a method to also refer to a component, thus ComponentA 212 and ComponentB 214 may also be available for selection and testing. The

various fields

206A, 206B, 216, 217, and 218 are depicted as boxes to indicate that their values may be changed. For example, the following example code provides a simple example of a .java file that could be compiled to create a .class file defining a component that can be instantiated as an object to be tested.


	public class Rectangle {
	public Rectangle(int width, int height) {
	myDimension.width = width;
	myDimension.height = height;
	}
	public double getArea( ) {
	return myDimension.width * myDimension.height;
	}
	public double getPerimeter {
	return 2*(myDimension.width+myDimension.height);
	}
	public String getDescription{
	private String myLabel = “Rectangle”
	return myLabel;
	}
	}

Relying on the above example, the user-interface screen 200 would list one testable component “Rectangle.” Within this testable component, there would be three methods identified as “getArea”, “getPerimeter” and “getDescription”. The fields of “Rectangle” corresponding to 206A and 206B of FIG. 2B would be “myDimension.width” and “myDimension.height”. If the method “getDescription” was selected for testing, then its list of properties in screen 203 would include one field labeled “myLabel”. The text box next to that label (e.g., 216) could be initially filled in with the value “Rectangle” but this value could be changed.

Once the client1 118 is finished assigning values to all the available fields, then a “test” button 220 can be selected. In response to the “test” button 220 being selected, the test agent Agent1 104 receives the assigned values and activates the testable code of Method1 204A. In particular, the Agent1 104 invokes the Method1 using the field values provided by the client1 118 to produce a result. Using the above example code, if the client1 makes no changes, then the result of invoking “getDescription” will be the return of the string “Rectangle”. If the client1 118 had used the interface screens 201 and 203 to change the value of “myLabel” to another word such as, for example, “Square”, then invoking “getDescription” would have returned the string “Square”.

In this manner, the client1 118 can be presented with a series of user interface screens that allows the client1 118 to test different methods of a testable component by assigning respective values to the different properties of a method and then invoking the method with those assigned respective values. No programming knowledge or skill is necessary to perform such testing and no prior knowledge about the structure of the testable components and their methods is needed.

FIG. 3A depicts a flowchart of an example method for a deployed agent providing a code-free testing framework in accordance with the principles of the present disclosure. In step 302 one or more applications are deployed on a managed server. The term “managed server” refers to a software platform in which an executable environment is provided by an application server such as, for example, a J2EE server. Within a managed server environment, executable components, or applications, have a common general format appropriate for the application server.

In step 304, a test agent is deployed on the managed server as well. By deploying the test agent on the managed server, the test agent executes within the environment of the managed server and has access to the other applications that are deployed on that server. Using this access, the test agent may determine, in step 306, all the components on the managed server that are available to be tested. One example, in a Java environment, is that the test agent may locate all the .class files in specified directory locations in the managed server environment.

The following code snippet, for example, may be one example of how a list of the class files in the current directory path could be generated by the test agent.

import java.io.File;


	public class ListFiles {
	public static void main (String[ ] args) {
	String path = “.”;
	String files;
	File folder = new File(path);
	File[ ] listOfFiles = folder.listFiles( );
	for (int i = 0; i <listOfFiles.length; i++) {
	files = listOfFiles[i].getName( );
	if (files.endsWith(“.class”) {
	System.out.printIn(files);
	}
	}
	}
	}

Once the names of the .class files are known, the test agent can then instantiate an object corresponding to each .class file. Once instantiated, the test agent can then, in step 308, determine what methods are available for that instantiated object. Although there are a variety of different ways to inspect within an instantiated object, Java's “Reflection” API provides beneficial techniques. Similar techniques are available for C## and other programming environments as well. For example, the following code snippet illustrates how to generate a list of methods for an instantiated object:


	import java.lang.reflect.*;
	public class DumpMethods {
	public static void main(String args[ ]) {
	try {
	Class c = Class.forName(args[0]);
	Method m[ ] = c.getDeclaredMethods( );
	for (int i=0; l <m.length; i++)
	System.out.printIn(m[i].toString( ));
	}
	}
	}

The line Class c=Class.forName(args[0]) constructs an object in the memory space of the managed server using the .class file named “args[0]”. Thus, the test agent would use the appropriate name of a .class file instead of “args[0]” to instantiate any desired object. The next line Method m[ ]=c.getDeclaredMethods( ) uses the built-in method “getDeclaredMethods” to fill an array m[ ] with all the methods available in the instantiated object.

Once the test agent has identified the available components and their respective methods, the test agent can register, in step 310, with the test orchestrator. When registering, the test agent can identify the identity of the application server (e.g., a URL) and provide the discovered information about components and their methods.

In operation, the test agent, in step 312, receives a client request related to testing code. The test orchestrator, or the test agent, could send information to the client, in step 314, identifying available, testable components and their testable methods. For example, the test orchestrator could act as a proxy in all communications between a client and a test agent. Alternatively, the test orchestrator can handle an initial inquiry from a client and then share enough information to allow the client to directly communicate with the test agent.

In sending information about a testable method, the test agent identifies the properties that may be associated with the method. The Java Reflection API provides a method “getDeclaredFields” that identifies each field declared in a particular method and, for that field identifies its name, its type, and other information. The test agent can, for example, construct a fillable HTML form that can be sent to a web browser identifying the information about the fields of a method and providing corresponding text boxes that can be filled in. From the test agent's perspective, a client will input data values for the different properties (i.e., fields) associated with a method and then send those data values back as a way to request that the test agent activate, or invoke the method, using those data values.

Thus, in step 318, the test agent invokes the method using the returned data values from the client. For example, the Java Reflection class provides for a method such as Object returnObject=meth.invoke(methodName, argList) to invoke the method “methodName” using a supplied list of arguments “argList” in order to generate a result called “returnObject”. This result can then be provided to the client, by the test agent in step 320, so that the client can observe that output of the method that results based on their input values they provided.

FIG. 3B depicts a flowchart of an example method for a test orchestrator to provide a code-free testing framework in accordance with the principles of the present disclosure. From the perspective of the test orchestrator, it may be deployed on a hardware platform, in step 330, so that clients have a single location that they may visit to discover information about a variety of potentially available test agents and testable code. Initially, the test orchestrator is contacted, in step 332, by a deployed test agent that desires to register with the test orchestrator. Thus, in step 334, the test orchestrator registers the deployed test agent. Example information that may be involved with registering a test agent includes an identifier of the test agent, an identifier and/or location, of a managed server on which the test agent is deployed, identifiers of testable components available through the test agent, and possibly one or more methods associated with the available, testable components. For example, the identifiers of testable components may include a list of .class files, or more generally a list of available testable code, which the test agent can use to instantiate different objects to be tested.

Subsequently, the test orchestrator will be contacted by a test client wishing to use one or more of the registered test agents to test available, testable components. In step 338, the test orchestrator can send a user-interface screen that identifies the test agents registered with the test orchestrator. For example, the user interface screen may be an HTML page, viewable in a web browser, that has a respective hyperlink to each of the test agents. When a client clicks on one of the hyperlinks the client may be connected to the appropriate test agent that is executing on a corresponding managed server. The test agent may then be responsible for communicating information about testable code to the client.

As mentioned above, another alternative is that the test orchestrator can act as a proxy so that all communication, in any direction, first passes through the test orchestrator before being forwarded to either the test agent of the client.

FIG. 3C depicts a flowchart of an example method for a test client to utilize a code-free testing framework in accordance with the principles of the present disclosure. From the perspective of the test client, in step 340, the client contacts the test orchestrator and, in response in step 342, the client receives a list of test agents that have previously registered with the test orchestrator. The client may then, in step 344, select one of the registered test agents.

Upon selection of a particular test agent, the client receives, in step 346, information about the components (e.g., objects defined by a class file) available for testing and their respective methods using that test agent. This information may be communicated with the client directly from a test agent or may be communicated indirectly via the test orchestrator. The client then, in step 348 selects one of the available methods of a component and, in response, receives information about that method. In particular, the information about the method relates to the properties, or fields, of that method for which the client can supply different values. In particular, the test agent provides this information in a user interface that allows the client to assign different values to each of the method's properties and then request that these assigned values be used.

Thus, in step 350, the client supplies data values, or test data to the test agent and requests that the test agent invoke the method using this test data. The test agent invokes the method using the test data which generates a result based on the particular values of the test data. In step 352, the client receives from the test agent the result generated based on the supplied test data. This process can be easily repeated to supply a variety of test data to a particular method or for selecting different methods, or even different components available through the test agent.

Referring to FIG. 4, a block diagram of a data processing system is depicted in accordance with the present disclosure. A data processing system 400, such as may be utilized to implement the hardware platform 108 or aspects thereof, e.g., as set out in greater detail in FIG. 1-FIG. 3C, may comprise a symmetric multiprocessor (SMP) system or other configuration including a plurality of processors 402 connected to system bus 404. Alternatively, a single processor 402 may be employed. Also connected to system bus 404 is memory controller/cache 406, which provides an interface to local memory 408. An I/O bridge 410 is connected to the system bus 404 and provides an interface to an I/O bus 412. The I/O bus may be utilized to support one or more busses and corresponding devices 414, such as bus bridges, input output devices (I/O devices), storage, network adapters, etc. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks.

Also connected to the I/O bus may be devices such as a graphics adapter 416, storage 418 and a computer usable storage medium 420 having computer usable program code embodied thereon. The computer usable program code may be executed to execute any aspect of the present disclosure, for example, to implement aspect of any of the methods, computer program products and/or system components illustrated in FIG. 1-FIG. 3C.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various aspects of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The terminology used herein is for the purpose of describing particular aspects only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

The corresponding structures, material s, acts, and equivalents of any means or step plus function elements in the claims below are intended to include any disclosed structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The aspects of the disclosure herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure with various modifications as are suited to the particular use contemplated.

Claims

The invention claimed is:

1. A method for testing testable code of an application comprising:

sending a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator, said any test agents each configured to provide access to respective associated testable code and said remote test orchestrator configured to provide the computer-implemented client with access to said any test agents; and

receiving, by the computer-implemented client from the remote test orchestrator, the list identifying said any test agents registered with the remote test orchestrator.

2. The method of claim 1, wherein a particular one of the test agents is configured to execute on an application server separate from the remote test orchestrator.

3. The method of claim 1, wherein the list identifies a plurality of different test agents registered with the remote test orchestrator.

4. The method of claim 1, further comprising:

in response to receiving the list, sending a first selection indication, by the computer-implemented client to the remote test orchestrator, thereby selecting a particular one of said any test agents; and

receiving, by the computer-implemented client from the remote test orchestrator, an identification of the respective testable code associated with the particular one test agent.

5. The method of claim 4, wherein the respective testable code comprises a plurality of components.

6. The method of claim 4, wherein the identification of the respective testable code comprises an identification of a testable method associated with the particular one test agent.

7. The method of claim 6, further comprising:

in response to receiving the identification of the testable method, sending a selection indication, by the computer-implemented client to the remote test orchestrator, thereby selecting the testable method; and

receiving, by the computer-implemented client from the remote test orchestrator, a list of properties associated with the testable method.

8. The method of claim 7, further comprising:

in response to receiving the list of properties associated with the testable method, sending, by the computer-implemented client to the remote test orchestrator, a respective data portion corresponding to each of the properties, whereby execution of the respective testable code using the respective data portions produces a test result; and

receiving by the computer-implemented client from the remote test orchestrator, the test result.

9. The method of claim 1, further comprising:

in response to receiving the list, selecting a particular one of said any test agents; and

receiving, by the computer-implemented client from the particular one of the test agents, an identification of the respective testable code associated with the particular one of the test agents.

10. The method of claim 9, further comprising:

receiving, by the computer-implemented client from the particular one of the test agents, a list of properties associated with the respective testable code.

11. The method of claim 10, further comprising:

in response to receiving the list of properties associated with the respective testable code, sending, by the computer-implemented client to the particular one of the test agents, a respective data portion corresponding to each of the properties, whereby execution of the testable code using the respective data portions produces a test result; and

receiving, by the computer-implemented client, the test result.

12. A system for testing testable code of an application comprising:

a processor;

a non-transitory memory in communication with the processor, wherein the non-transitory memory storing instructions executable by the processor to:

send a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator, said any test agents each configured to provide access to respective associated testable code and said remote test orchestrator configured to provide the computer-implemented client with access to said any test agents; and

receive, by the computer-implemented client from the remote test orchestrator, the list identifying said any test agents registered with the remote test orchestrator.

13. The system of claim 12, wherein a particular one of the test agents is configured to execute on an application server separate from the remote test orchestrator.

14. The system of claim 12, wherein the list identifies a plurality of different test agents registered with the remote test orchestrator.

15. The system of claim 12, wherein the instructions are executable by the processor to:

in response to receiving the list, send a first selection indication, by the computer-implemented client to the remote test orchestrator, thereby selecting a particular one of said any test agents; and

receive, by the computer-implemented client from the remote test orchestrator, an identification of the respective testable code associated with the particular one test agent.

16. The system of claim 15, wherein the respective testable code comprises a plurality of components.

17. The system of claim 15, wherein the identification of the respective testable code comprises an identification of a testable method associated with the particular one test agent.

18. The system of claim 17, wherein the instructions are executable by the processor to:

in response to receiving the identification of the testable method, send a selection indication, by the computer-implemented client to the remote test orchestrator, thereby selecting the testable method; and

receive, by the computer-implemented client from the remote test orchestrator, a list of properties associated with the testable method.

19. The system of claim 18, wherein the instructions are executable by the processor to:

in response to receiving the list of properties associated with the testable method, send, by the computer-implemented client to the remote test orchestrator, a respective data portion corresponding to each of the properties, whereby execution of the respective testable code using the respective data portions produces a test result; and

receive by the computer-implemented client from the remote test orchestrator, the test result.

20. The system of claim 12, wherein the instructions are executable by the processor to:

in response to receiving the list, select a particular one of said any test agents; and

receive, by the computer-implemented client from the particular one of the test agents, an identification of the respective testable code associated with the particular one of the test agents.

21. The system of claim 20, wherein the instructions are executable by the processor to:

receive, by the computer-implemented client from the particular one of the test agents, a list of properties associated with the respective testable code.

22. The system of claim 21, wherein the instructions are executable by the processor to:

in response to receiving the list of properties associated with the respective testable code, send, by the computer-implemented client to the particular one of the test agents, a respective data portion corresponding to each of the properties, whereby execution of the respective testable code using the respective data portions produces a test result; and

receive, by the computer-implemented client, the test result.

23. A computer program product for testing testable code of an application, comprising:

a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:

computer readable program code configured to send a request, from a computer-implemented client to a remote test orchestrator, for a list identifying any test agents registered with the remote test orchestrator, said any test agents each configured to provide access to respective associated testable code and said remote test orchestrator configured to provide the computer-implemented client with access to said any test agents; and

computer readable program code configured to receive, by the computer-implemented client from the remote test orchestrator, the list identifying said any test agents registered with the remote test orchestrator.

24. The computer program product of claim 23, wherein a particular one of the test agents is configured to execute on an application server separate from the remote test orchestrator.

25. The computer program product of claim 23, wherein the list identifies a plurality of different test agents registered with the remote test orchestrator.

26. The computer program product of claim 23, further comprising:

computer readable program code configured to, in response to receiving the list, send a first selection indication, by the computer-implemented client to the remote test orchestrator, thereby selecting a particular one of said any test agents; and

computer readable program code configured to receive, by the computer-implemented client from the remote test orchestrator, an identification of the respective testable code associated with the particular one test agent.

27. The computer program product of claim 26, wherein the respective testable code comprises a plurality of components.

28. The computer program product of claim 26, wherein the identification of the respective testable code comprises an identification of a testable method associated with the particular one test agent.

29. The computer program product of claim 28, further comprising:

computer readable program code configured to, in response to receiving the identification of the testable method, send a selection indication, by the computer-implemented client to the remote test orchestrator, thereby selecting the testable method; and

computer readable program code configured to receive, by the computer-implemented client from the remote test orchestrator, a list of properties associated with the testable method.

30. The computer program product m of claim 29, further comprising:

computer readable program code configured to, in response to receiving the list of properties associated with the testable method, send, by the computer-implemented client to the remote test orchestrator, a respective data portion corresponding to each of the properties, whereby execution of the respective testable code using the respective data portions produces a test result; and

computer readable program code configured to receive by the computer-implemented client from the remote test orchestrator, the test result.

31. The computer program product of claim 23, further comprising:

computer readable program code configured to, in response to receiving the list, select a particular one of said any test agents; and

computer readable program code configured to receive, by the computer-implemented client from the particular one of the test agents, an identification of the respective testable code associated with the particular one of the test agents.

32. The computer program product of claim 31, further comprising:

computer readable program code configured to receive, by the computer-implemented client from the particular one of the test agents, a list of properties associated with the respective testable code.

33. The computer program product of claim 32, further comprising:

computer readable program code configured to, in response to receiving he list of properties associated with the respective testable code, send, by the computer-implemented client to the particular one of the test agents, a respective data portion corresponding to each of the properties, whereby execution of the respective testable code using the respective data portions produces a test result; and

computer readable program code configured to receive, by the computer-implemented client, the test result.