CN110932918A - Log data acquisition method and device and storage medium - Google Patents
Log data acquisition method and device and storage medium Download PDFInfo
- Publication number
- CN110932918A CN110932918A CN201911368492.9A CN201911368492A CN110932918A CN 110932918 A CN110932918 A CN 110932918A CN 201911368492 A CN201911368492 A CN 201911368492A CN 110932918 A CN110932918 A CN 110932918A
- Authority
- CN
- China
- Prior art keywords
- information
- service node
- state
- log
- service
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/069—Management of faults, events, alarms or notifications using logs of notifications; Post-processing of notifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
- H04L41/064—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving time analysis
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/06—Management of faults, events, alarms or notifications
- H04L41/0677—Localisation of faults
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Debugging And Monitoring (AREA)
Abstract
The disclosure discloses a log data acquisition method, a device and a storage medium, wherein the method comprises the following steps: the method comprises the steps of receiving log data information corresponding to each service node sent by each service node, analyzing the log data information, obtaining basic information of each service node and a calling state of each service node, obtaining running state information of each service node when the calling state is an abnormal state, and storing the basic information and the running state information of each service node as log information in a storage system. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a log data acquisition method, a log data acquisition device and a storage medium.
Background
Nowadays, in the process of processing business, more and more applications need to be called, and the calling relationship between the applications is more and more complex, and a large amount of log data is generated in the process.
Currently, log data generated by the running of an application program can be collected according to a link to which a service belongs through a log collection tool, and the collected log data content is data related to the running of the application program.
However, in the related art, when collecting log data generated by performing a service, only data of each application program running the service is collected, so that only the data of each application program running the service can be analyzed, and there is a problem in analyzing a link with a complex call relationship.
Disclosure of Invention
The disclosure provides a log data acquisition method and device of Internet of things equipment and a storage medium. The technical scheme is as follows:
according to a first aspect of the embodiments of the present disclosure, there is provided a log data acquisition method, where the method includes:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, when the call state of the service node is an abnormal state, before the obtaining of the running state information of the service node, the method further includes:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, after the storing the basic information of each service node and the running state information as log information in a storage system, the method further includes:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
According to a second aspect of the embodiments of the present disclosure, there is provided a log data collecting apparatus, the apparatus including:
the data receiving module is used for receiving the log data information which is sent by each service node and corresponds to each service node;
the basic information acquisition module is used for analyzing the log data information and acquiring the basic information of each service node and the calling state of each service node;
a state obtaining module, configured to obtain running state information of the service node when the call state of the service node is an abnormal state, where the running state information is used to indicate a running state of a system in which the service node is currently located;
and the information storage module is used for storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, the apparatus further comprises:
and the exception determining module is used for determining that the calling state of the service node is an abnormal state when the calling state of the service node is the abnormal state and the return value information is failure information after analyzing the basic information of the service node before acquiring the running state information of the service node.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, the apparatus further comprises:
the log information acquisition module is used for acquiring the corresponding log information in the storage system according to query conditions, and the query conditions are input through a client;
and the log information display module is used for displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
According to a third aspect of the embodiments of the present disclosure, there is provided a log data collecting apparatus, the apparatus including:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
According to a fourth aspect of embodiments of the present disclosure, a computer-device-readable storage medium is provided, where the computer-device-readable storage medium contains executable instructions, and the executable instructions are invoked and executed by a processor to implement the log data collection method according to the first aspect or any one of the alternatives of the first aspect.
The technical scheme provided by the embodiment of the disclosure can have the following beneficial effects:
receiving log data information corresponding to each service node sent by each service node, acquiring basic information of each service node and a calling state of each service node by analyzing the log data information, acquiring running state information of the service node when the calling state is an abnormal state, wherein the running state information is used for indicating the running state of a system where the service node is located at present, and storing the basic information and the running state information of each service node as log information in a storage system. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
FIG. 1 is a schematic diagram of an implementation environment shown in accordance with an exemplary embodiment;
FIG. 2 is a flow diagram illustrating a method of log data collection in accordance with an exemplary embodiment;
FIG. 3 is a flow chart illustrating a method of log data collection in accordance with another exemplary embodiment;
FIG. 4 is a diagram illustrating a full link service node invocation relationship according to the embodiment shown in FIG. 3;
FIG. 5 is a schematic diagram of a full link service node invocation failure in accordance with the embodiment shown in FIG. 3;
FIG. 6 is a schematic diagram of an interface for a client to input query conditions according to the embodiment shown in FIG. 3;
FIG. 7 is a block diagram illustrating a log data collection apparatus in accordance with an exemplary embodiment;
FIG. 8 is a schematic diagram illustrating a configuration of a computer device, according to an example embodiment.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.
It is to be understood that reference herein to "a number" means one or more and "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
FIG. 1 is a schematic diagram illustrating one implementation environment in accordance with an example embodiment. This implementation environment includes several devices 110, a log server 120, and a client 130.
The device 110 may be a terminal device, a computer device, or a server with data transmission capabilities.
For example, the device 110 may be a mobile portable terminal such as a smart phone, a tablet computer, and an e-book reader, or a computer device such as a notebook computer, or the device 110 may also be an intelligent wearable device such as smart glasses and a smart watch.
The devices 110 may be connected to each other through a wireless or wired network, and the devices 110 may perform one or several services through mutual invocation.
The device 110 and the log server 120 may be connected via a wireless or wired network.
The log server 120 may retrieve data in the device 110 and store in a memory of the log server 120.
The log server 120 may be a server, or the log server 120 may also be a cloud computing service center, and at this time, the data acquired by the terminal 110 may be stored in a database of the cloud computing service center.
Data transmission can be performed between the client 130 and the log server 120, and the client 130 can query the data in the log server 120 and obtain the log data stored in the memory of the log server 120.
Optionally, the wireless network or wired network described above uses standard communication techniques and/or protocols. The Network is typically the Internet, but may be any Network including, but not limited to, a Local Area Network (LAN), a Metropolitan Area Network (MAN), a Wide Area Network (WAN), a mobile, wireline or wireless Network, a private Network, or any combination of virtual private networks. In some embodiments, data exchanged over a network is represented using techniques and/or formats including Hypertext Mark-up Language (HTML), Extensible markup Language (XML), and the like. All or some of the links may also be encrypted using conventional encryption techniques such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet protocol Security (IPsec). In other embodiments, custom and/or dedicated data communication techniques may also be used in place of, or in addition to, the data communication techniques described above.
FIG. 2 is a flow chart illustrating a method of log data collection in accordance with an exemplary embodiment. The log data acquisition method can be applied to a log server, and the log information is stored in a storage system of the log server by acquiring the log information on the equipment. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. As shown in fig. 2, the log data collection method may include the steps of:
in step 201, receiving log data information corresponding to each service node sent by each service node;
in step 202, analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
in step 203, when the calling state of the service node is an abnormal state, obtaining running state information of the service node, where the running state information is used to indicate a running state of a system in which the service node is currently located;
in step 204, the basic information of each service node and the running state information of the service node whose calling state is the abnormal state are stored in the storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in the form of letter and number codes.
Optionally, when the call state of the service node is an abnormal state, before the obtaining of the running state information of the service node, the method further includes:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
Optionally, the operation status information includes at least one of central processing unit status information, storage system status information, network status information, input/output status information, and disk status information of the current system.
Optionally, after storing the basic information of each service node and the running state information as log information in a storage system, the method further includes:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
In summary, in the log data collection method provided in this disclosure, log data information corresponding to each service node sent by each service node is received, the log data information is analyzed, so as to obtain basic information of each service node and a call state of each service node, when the call state is an abnormal state, operation state information of the service node is obtained, the operation state information is used to indicate an operation state of a system where the service node is currently located, and the basic information of each service node and the operation state information are stored in a storage system as log information. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
Fig. 3 is a flowchart illustrating a log data collection method according to another exemplary embodiment, which may be applied to a log server for storing log information in a storage system of the log server by acquiring the log information on a device. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. As shown in fig. 3, the log data collection method may include the steps of:
in step 301, the log server receives log data information corresponding to each service node sent by each service node.
In the embodiment of the disclosure, in order to perform one or more services, each terminal or server runs an application program related to the service, and in the process of running the corresponding application program, each terminal or server generates a data packet containing data information of the running application program, and uploads the data packet generated by running the application program to a cloud platform or a cloud server of a log collection tool through the log collection tool on each terminal or server. The log server receives a data packet containing log data information through a cloud platform or a cloud server of the log collection tool.
Among them, the log collection tool can be a Logstash tool, a Filebeat tool, a Flume tool, a fluent tool, a Logement tool, an rsyslog tool, and a syslog-ng tool.
Alternatively, the respective service nodes may be located in the same or different servers or terminals.
Wherein, any log collecting tool is installed in the same or different servers or terminals where each service node is located.
For example, a service needs to call A, B and C, the three service nodes are located in three different servers, a log collection tool installed in a server where A, B and C are located is a loggent tool, according to the collection principle of the loggent tool, the loggent is a transmission tool provided by the setup and used for transmitting logs to the logsend, and the logsend is an elastic search API based on the SaaS platform, because the logsend exposes an elastic search API (Application Programming Interface), the loggent can easily push data to the elastic search.
In step 302, the log server parses the log data information, and obtains the basic information of each service node and the call status of each service node.
In the embodiment of the disclosure, the log server downloads a data packet containing data information of each application program from a cloud platform or a cloud server of the log collection tool, and analyzes the data packet to obtain basic information of each service node and a call state of each service node.
The basic information of each service node comprises link identification information of the current service node, identification information of a source service node, return value information of the current service node and the running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a digitally encoded form.
The link identification information is used to indicate the link to which the current serving node belongs, and a complete link represents a service. At least one service node is required to be called to complete one service, the link identification of at least one service node called to complete one service is the same and is different from the link identification of service nodes called to complete other services, and the link identification can be marked and distinguished by using a numerical sequence number.
For example, a service a needs to invoke four service nodes, namely, a service a, a service B, a service C, and a service D, link identification information carried in data packets sent by the four service nodes may be 1, a service B needs to invoke two service nodes, namely, a service E and a service F, and link identification information carried in data packets sent by the two service nodes may be 2.
The current service node identification information is used for representing the identity of the current service node, and the source service node identification information is used for representing the identity of the service node calling the current service node. The current service node identification information and the source service node identification information are marked and distinguished by numerical sequence numbers.
For example, if service a calls service B and service B calls C, then service a is the source service node of service B, and the service node identification information of service a may be 0, then the service node identification information of service B is 1, and service B is the source service node of service C, so the service node identification information of service C is 2.
For example, please refer to fig. 4, which shows a schematic diagram of a call relationship of a full link service node according to an embodiment of the present disclosure, and as shown in fig. 4, the link starts from service a and ends at service E. Wherein: service A calls service B1, service B1 calls service B2 first, then service C1, then service C1 calls service C2, service C2 calls service C3, then service C1 calls service D, finally service D calls service E, and the link call is finished. The service node identifier of the service A is 0, and the source service node identifier is null; the service node identifier of service B1 is 1, and the source service node identifier is 0; the service node identification of service B2 is 2, the source service node identification is 1; the service node identifier of service C1 is 3, and the source service node identifier is 1; service node identification of service C2 is 4, source service node identification is 3; service node identification of service C3 is 5, source service node identification is 4; the service node identifier of the service D is 6, and the source service node identifier is 3; service node identification of service E is 4 and source service node identification is 3.
The return value information of the current service node is used for representing the data information returned from the next service node when the current service node calls the next service node.
Optionally, the return value information of the current service node may include call success information of call success or call failure information of call failure.
For example, when the service a calls the service B, the service B sends a data message to the service a, and if the call is successful, the service B sends a call success message to the service a; if the call fails, service B sends a call failure message to service A.
For example, please refer to fig. 5, which illustrates a schematic diagram of a call failure of a full link service node according to an embodiment of the present disclosure, as shown in fig. 5, the link starts from service a and ends at service E. Wherein: service B is initially invoked by service a, then service C by service B, then service D by service C, and finally service E by service D. When an error occurs in the process of calling the service D by the service C, the service C fails to call the service D, and at the moment, the service D sends a return value to the service C to inform the service C of call failure information.
Optionally, the running time of the current service node is a starting time and an ending time of the current service running.
In step 303, when the log server parses the basic information of the service node and obtains that the return value information is failure information, it determines that the calling state of the service node is an abnormal state.
In the embodiment of the present disclosure, the log server analyzes the log data information to obtain the call state of each service node, and the call state of each service node is determined according to the return value information of the current service node and is represented in the form of a state code. And if the return value information is calling success information, the calling state is a normal state. And if the return value information is calling failure information, the calling state is an abnormal state.
In step 304, when the calling state is an abnormal state, the log server obtains the running state information of the service node.
In the embodiment of the present disclosure, the operation state information is used to indicate an operation state of a system in which the current service node is located.
Optionally, the running state information includes at least one of central processor state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
When the log server determines that the calling state is an abnormal state according to the obtained return value information by analyzing the basic information of the service node, the log server obtains the running states of the terminals or the systems of the two service nodes in the process that the current service node calls the next service node, wherein the running states of the terminals or the systems of the two service nodes comprise the data information of a Central Processing Unit (CPU) of the terminals or the systems of the servers in which the current two service nodes are located; acquiring data information of a storage system of a terminal or a server where two current service nodes are located; acquiring network data information of terminals or servers where two current service nodes are located; acquiring data information of input/output values of terminals or servers where two current service nodes are located; and acquiring the disk running data information of the terminals or servers where the current two service nodes are located.
In step 305, the log server stores the basic information of the respective service nodes and the operation state information as log information in the storage system.
In the embodiment of the present disclosure, the log server records the acquired basic information of each service node and the running state information of the service node with abnormal call in the storage system.
The storage system may be a memory or a database.
In step 306, the log server obtains the corresponding log information in the storage system according to a query condition, and the query condition is input through the client.
In the embodiment of the present disclosure, the storage system of the log server stores the basic information of each service node and the running state information of the service node with abnormal call. The user inputs the query condition at the client, and the log server acquires the basic information of the corresponding service node and the running state information of the abnormal service node called therein according to the input query condition.
Referring to fig. 6, which shows an interface schematic diagram of a client inputting query conditions according to an embodiment of the present disclosure, as shown in fig. 6, four query condition input boxes, i.e., a link identifier input box, a service identifier input box, a start time input box, and an end time input box, are arranged in a log query interface of the client. The user can select at least one query condition to input into the corresponding query condition input box, and the log server is controlled to acquire corresponding log information according to the query condition input by the user by triggering the search control.
For example, as shown in fig. 6, the user inputs 3 in the link identifier input box, and then performs a trigger operation on the search control, and the log server may query all log information of each service node with the link identifier of 3. If the user inputs 3 in the link identifier input box, then inputs 2019.01.01 in the start time input box, and inputs 2019.01.03 in the end time input box, the log server can query all log information of each service node with the link identifier of 3 from 1/2019 to 3/2019 by triggering the search control.
In step 307, the log server displays the log information on the display interface of the client.
In the embodiment of the disclosure, the log server integrates and sorts the log information according to a certain sequence according to the obtained log information corresponding to the query condition, and finally displays the log information sorted according to the certain sequence on the display interface of the client.
Optionally, the log information may be presented on a display interface of the client in a list form.
For example, as shown in table 1, when the query condition is a link identifier, the service node identifiers, the service names, the states, the entries, the return values, and the source service node identifiers of all the service nodes on the link may be displayed on the client display interface, the call relationship of each service node and the states of each service node during call may be restored, and the service node with a problem and the part with an abnormality during call of the service node may be accurately found.
TABLE 1
Optionally, the log information is sorted and integrated according to a certain order, and may be adjusted according to the query condition.
For example, when the query condition is a time period, that is, the query condition is a start time and an end time, the log information may be integrated and sorted according to the time sequence, and the log information of the time period meeting the query condition is acquired and displayed on the display interface of the client. As shown in table 2, when the query condition is the start time and the end time, the time information of each service node, the log information of each service node, that is, the log information of each application program, and the running state information of the service node that failed in the call may be displayed on the display interface of the client. The service nodes are arranged according to the time sequence, and the time nodes with the calling failure problem and the service node system running state corresponding to the time nodes can be accurately found according to the arrangement, so that the service problem is positioned.
TABLE 2
In summary, in the log data collection method provided in this disclosure, log data information corresponding to each service node sent by each service node is received, the log data information is analyzed, so as to obtain basic information of each service node and a call state of each service node, when the call state is an abnormal state, operation state information of the service node is obtained, the operation state information is used to indicate an operation state of a system where the service node is currently located, and the basic information of each service node and the operation state information are stored in a storage system as log information. By the scheme, the basic information of the service nodes in each device and the running state of the device system in the abnormal state can be acquired and stored as the log information, the problems can be specifically positioned in the subsequent analysis of the log information, and the quality of log information analysis is improved.
Fig. 7 is a block diagram illustrating a log data collection apparatus according to an exemplary embodiment, and as shown in fig. 7, the log data collection apparatus may be implemented as all or part of a log server in hardware or a combination of hardware and software to perform the steps shown in any one of the embodiments shown in fig. 2 or fig. 3. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. The log data acquisition device may include:
a data receiving module 710, configured to receive log data information corresponding to each service node sent by each service node;
a basic information obtaining module 720, configured to analyze the log data information, and obtain basic information of each service node and a call state of each service node;
a state obtaining module 730, configured to obtain running state information of the service node when the call state of the service node is an abnormal state, where the running state information is used to indicate a running state of a system in which the current service node is located;
an information storage module 740, configured to store the basic information of each service node and the running state information of the service node whose calling state is the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, the apparatus further comprises:
and the exception determining module is used for determining that the calling state of the service node is an abnormal state when the calling state of the service node is the abnormal state and the return value information is failure information after analyzing the basic information of the service node before acquiring the running state information of the service node.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, the apparatus further comprises:
the log information acquisition module is used for acquiring the corresponding log information in the storage system according to query conditions, and the query conditions are input through a client;
and the log information display module is used for displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
It should be noted that, when the apparatus provided in the foregoing embodiment implements the functions thereof, only the division of the above functional modules is illustrated, and in practical applications, the above functions may be distributed by different functional modules according to actual needs, that is, the content structure of the device is divided into different functional modules, so as to complete all or part of the functions described above.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
An exemplary embodiment of the present disclosure provides a log data collecting apparatus, which may be implemented as all or part of a log server in a hardware or software and hardware combination manner to perform the steps shown in any one of the embodiments shown in fig. 2 or fig. 3. For example, the device may be the device 110 shown in fig. 1, and the log server may be the log server 120 shown in fig. 1. The log data acquisition device further comprises: a processor, a memory for storing processor-executable instructions;
wherein the processor is configured to:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
Optionally, the basic information of each service node includes link identification information where the current service node is located, identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
Optionally, the link identification information and the service node identification information are identified in a form of letter and number codes.
Optionally, when the call state of the service node is an abnormal state, before the obtaining of the running state information of the service node, the method further includes:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
Optionally, the running state information includes at least one of central processing unit state information, storage system state information, network state information, input/output state information, and disk state information of the current system.
Optionally, after the storing the basic information of each service node and the running state information as log information in a storage system, the method further includes:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
Optionally, the query condition includes at least one of the operation time period information, the link identification information, and the operation time of the service node.
FIG. 8 is a schematic diagram illustrating a configuration of a computer device, according to an example embodiment. The computer apparatus 800 includes a Central Processing Unit (CPU) 801, a system Memory 804 including a Random Access Memory (RAM) 802 and a Read-Only Memory (ROM) 803, and a system bus 805 connecting the system Memory 804 and the CPU 801. The computer device 800 also includes a basic Input/Output (I/O) system 806 for facilitating information transfer between various devices within the computer device, and a mass storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.
The basic input/output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse, keyboard, etc. for user input of information. Wherein the display 808 and the input device 809 are connected to the central processing unit 801 through an input output controller 810 connected to the system bus 805. The basic input/output system 806 may also include an input/output controller 810 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input-output controller 810 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 807 is connected to the central processing unit 801 through a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer device-readable media provide non-volatile storage for the computer device 800. That is, the mass storage device 807 may include a computer device readable medium (not shown) such as a hard disk or Compact Disc-Only Memory (CD-ROM) drive.
Without loss of generality, the computer device readable media may comprise computer device storage media and communication media. Computer device storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer device readable instructions, data structures, program modules or other data. Computer device storage media includes RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash Memory or other solid state Memory technology, CD-ROM, Digital Video Disk (DVD), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer device storage media is not limited to the foregoing. The system memory 804 and mass storage 807 described above may be collectively referred to as memory.
The computer device 800 may also operate as a remote computer device connected to a network through a network, such as the internet, in accordance with various embodiments of the present disclosure. That is, the computer device 800 may be connected to the network 812 through the network interface unit 811 coupled to the system bus 805, or may be connected to other types of networks or remote computer device systems (not shown) using the network interface unit 811.
The memory further includes one or more programs, the one or more programs are stored in the memory, and the central processing unit 801 executes the one or more programs to implement all or part of the steps of the method shown in fig. 2 or fig. 3.
Those skilled in the art will recognize that, in one or more of the examples described above, the functions described in embodiments of the disclosure may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-device-readable medium. Computer device readable media includes both computer device storage media and communication media including any medium that facilitates transfer of a computer device program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer device.
The embodiment of the present disclosure further provides a computer device storage medium, which is used for storing computer device software instructions for the testing apparatus, and includes a program designed for executing the log data acquisition method.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims (10)
1. A method for log data collection, the method comprising:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state of the service node is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the service node is located;
and storing the basic information of each service node and the running state information of the service node with the calling state being the abnormal state in a storage system as log information.
2. The method of claim 1, wherein the basic information of each service node comprises link identification information of the current service node, identification information of a source service node, return value information of the current service node, and running time of the current service node.
3. The method of claim 2, wherein the link identification information and the serving node identification information are identified in an alpha and numeric code.
4. The method according to claim 1, wherein before obtaining the running state information of the service node when the calling state of the service node is an abnormal state, the method further comprises:
and when the basic information of the service node is analyzed and the return value information is obtained as failure information, determining that the calling state of the service node is an abnormal state.
5. The method of claim 1, wherein the operational state information comprises at least one of central processor state information, storage system state information, network state information, input/output state information, and disk state information of a current system.
6. The method according to claim 1, wherein after storing the basic information of each service node and the operating state information as log information in a storage system, the method further comprises:
acquiring corresponding log information in the storage system according to a query condition, wherein the query condition is input through a client;
and displaying the log information on a display interface of the client.
7. The method of claim 6, wherein the query condition comprises at least one of the operating time period information, the link identification information, and an operating time of the service node.
8. An apparatus for collecting log data, the apparatus comprising:
the data receiving module is used for receiving the log data information which is sent by each service node and corresponds to each service node;
the basic information acquisition module is used for analyzing the log data information and acquiring the basic information of each service node and the calling state of each service node;
a state obtaining module, configured to obtain running state information of the service node when the call state is an abnormal state, where the running state information is used to indicate a running state of a system in which the service node is currently located;
and the information storage module is used for storing the basic information of each service node and the running state information as log information in a storage system.
9. An apparatus for collecting log data, the apparatus comprising:
a processor;
a memory for storing executable instructions of the processor;
wherein the processor is configured to:
receiving log data information corresponding to each service node sent by each service node;
analyzing the log data information, and acquiring the basic information of each service node and the calling state of each service node;
when the calling state is an abnormal state, acquiring running state information of the service node, wherein the running state information is used for indicating the running state of a system where the current service node is located;
and storing the basic information and the running state information of each service node in a storage system as log information.
10. A computer device readable storage medium, wherein the computer device readable storage medium contains executable instructions, and the executable instructions are called and executed by a processor to implement the log data collection method of any one of the above claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368492.9A CN110932918B (en) | 2019-12-26 | 2019-12-26 | Log data acquisition method and device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911368492.9A CN110932918B (en) | 2019-12-26 | 2019-12-26 | Log data acquisition method and device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110932918A true CN110932918A (en) | 2020-03-27 |
| CN110932918B CN110932918B (en) | 2023-01-10 |
Family
ID=69860995
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911368492.9A Active CN110932918B (en) | 2019-12-26 | 2019-12-26 | Log data acquisition method and device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110932918B (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111740860A (en) * | 2020-06-15 | 2020-10-02 | 中国工商银行股份有限公司 | Log data transmission link monitoring method and device |
| CN112764997A (en) * | 2021-01-28 | 2021-05-07 | 北京字节跳动网络技术有限公司 | Log storage method and device, computer equipment and storage medium |
| CN113190583A (en) * | 2021-05-14 | 2021-07-30 | 华北电力大学扬中智能电气研究中心 | Data acquisition system, method, electronic equipment and storage medium |
| CN114185502A (en) * | 2021-12-15 | 2022-03-15 | 平安科技（深圳）有限公司 | Log printing method, device, equipment and medium based on production line environment |
| CN114650252A (en) * | 2022-03-25 | 2022-06-21 | 华润数字科技(深圳)有限公司 | Routing method and device based on enterprise service bus and computer equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5995981A (en) * | 1997-06-16 | 1999-11-30 | Telefonaktiebolaget Lm Ericsson | Initialization of replicated data objects |
| CN105912587A (en) * | 2016-03-31 | 2016-08-31 | 乐视控股（北京）有限公司 | Data acquisition method and system |
| CN106021064A (en) * | 2016-05-16 | 2016-10-12 | 浪潮电子信息产业股份有限公司 | Monitoring information collection method, baseboard management controller and system |
| CN106254144A (en) * | 2016-09-06 | 2016-12-21 | 华为技术有限公司 | Fault location platform, Fault Locating Method and device |
| CN106339379A (en) * | 2015-07-07 | 2017-01-18 | 阿里巴巴集团控股有限公司 | Website running status monitoring method and device |
| US20170091008A1 (en) * | 2015-09-29 | 2017-03-30 | International Business Machines Corporation | Detecting and analyzing performance anomalies of client-server based applications |
| CN106815115A (en) * | 2017-01-13 | 2017-06-09 | 郑州云海信息技术有限公司 | A kind of operation condition of server monitoring system |
| CN109656778A (en) * | 2018-11-28 | 2019-04-19 | 金蝶软件（中国）有限公司 | Data capture method, device, computer equipment and storage medium |
| CN110113200A (en) * | 2019-04-29 | 2019-08-09 | 平安科技（深圳）有限公司 | The correlating method of chain-circuit system and log system, device and storage medium |
| CN110297746A (en) * | 2019-07-05 | 2019-10-01 | 北京慧眼智行科技有限公司 | A kind of data processing method and system |
| US20190354422A1 (en) * | 2018-05-16 | 2019-11-21 | Nec Laboratories America, Inc. | Joint semantic and format similarity for large scale log retrieval |
| CN110502494A (en) * | 2019-08-30 | 2019-11-26 | 北京思维造物信息科技股份有限公司 | Log processing method, device, computer equipment and storage medium |
-
2019
- 2019-12-26 CN CN201911368492.9A patent/CN110932918B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5995981A (en) * | 1997-06-16 | 1999-11-30 | Telefonaktiebolaget Lm Ericsson | Initialization of replicated data objects |
| CN106339379A (en) * | 2015-07-07 | 2017-01-18 | 阿里巴巴集团控股有限公司 | Website running status monitoring method and device |
| US20170091008A1 (en) * | 2015-09-29 | 2017-03-30 | International Business Machines Corporation | Detecting and analyzing performance anomalies of client-server based applications |
| CN105912587A (en) * | 2016-03-31 | 2016-08-31 | 乐视控股（北京）有限公司 | Data acquisition method and system |
| CN106021064A (en) * | 2016-05-16 | 2016-10-12 | 浪潮电子信息产业股份有限公司 | Monitoring information collection method, baseboard management controller and system |
| CN106254144A (en) * | 2016-09-06 | 2016-12-21 | 华为技术有限公司 | Fault location platform, Fault Locating Method and device |
| CN106815115A (en) * | 2017-01-13 | 2017-06-09 | 郑州云海信息技术有限公司 | A kind of operation condition of server monitoring system |
| US20190354422A1 (en) * | 2018-05-16 | 2019-11-21 | Nec Laboratories America, Inc. | Joint semantic and format similarity for large scale log retrieval |
| CN109656778A (en) * | 2018-11-28 | 2019-04-19 | 金蝶软件（中国）有限公司 | Data capture method, device, computer equipment and storage medium |
| CN110113200A (en) * | 2019-04-29 | 2019-08-09 | 平安科技（深圳）有限公司 | The correlating method of chain-circuit system and log system, device and storage medium |
| CN110297746A (en) * | 2019-07-05 | 2019-10-01 | 北京慧眼智行科技有限公司 | A kind of data processing method and system |
| CN110502494A (en) * | 2019-08-30 | 2019-11-26 | 北京思维造物信息科技股份有限公司 | Log processing method, device, computer equipment and storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111740860A (en) * | 2020-06-15 | 2020-10-02 | 中国工商银行股份有限公司 | Log data transmission link monitoring method and device |
| CN112764997A (en) * | 2021-01-28 | 2021-05-07 | 北京字节跳动网络技术有限公司 | Log storage method and device, computer equipment and storage medium |
| CN112764997B (en) * | 2021-01-28 | 2024-02-20 | 抖音视界有限公司 | Log storage method and device, computer equipment and storage medium |
| CN113190583A (en) * | 2021-05-14 | 2021-07-30 | 华北电力大学扬中智能电气研究中心 | Data acquisition system, method, electronic equipment and storage medium |
| CN113190583B (en) * | 2021-05-14 | 2024-02-20 | 长春理工大学 | Data acquisition system, method, electronic equipment and storage medium |
| CN114185502A (en) * | 2021-12-15 | 2022-03-15 | 平安科技（深圳）有限公司 | Log printing method, device, equipment and medium based on production line environment |
| CN114185502B (en) * | 2021-12-15 | 2024-05-14 | 平安科技（深圳）有限公司 | Log printing method, device, equipment and medium based on production line environment |
| CN114650252A (en) * | 2022-03-25 | 2022-06-21 | 华润数字科技(深圳)有限公司 | Routing method and device based on enterprise service bus and computer equipment |
| CN114650252B (en) * | 2022-03-25 | 2023-11-21 | 华润数字科技(深圳)有限公司 | Routing method and device based on enterprise service bus and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110932918B (en) | 2023-01-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110932918B (en) | Log data acquisition method and device and storage medium | |
| US10769228B2 (en) | Systems and methods for web analytics testing and web development | |
| CN111683066B (en) | Heterogeneous system integration method, heterogeneous system integration device, computer equipment and storage medium | |
| CN107957940B (en) | Test log processing method, system and terminal | |
| CN112988485B (en) | Simulation test method and device for electric power Internet of things equipment | |
| KR100865015B1 (en) | Realtime unification management information data conversion and monitoring apparatus and method for thereof | |
| CN112039701A (en) | Interface call monitoring method, device, equipment and storage medium | |
| US10775751B2 (en) | Automatic generation of regular expression based on log line data | |
| CN110928934A (en) | Data processing method and device for business analysis | |
| CN113656194A (en) | Account checking result data notification method and device, electronic device and storage medium | |
| CN112631924A (en) | Automatic testing method and device, computer equipment and storage medium | |
| CN113760730A (en) | Automatic testing method and device | |
| CN113630418A (en) | Network service identification method, device, equipment and medium | |
| CN111639059A (en) | Log information storage and positioning method, electronic equipment and storage medium | |
| KR20210000041A (en) | Method and apparatus for analyzing log data in real time | |
| CN115604343A (en) | Data transmission method, system, electronic equipment and storage medium | |
| CN109634931B (en) | Log uploading method and device | |
| CN114143308A (en) | File uploading information processing method and device, computer equipment and storage medium | |
| CN110166636B (en) | Pressure testing device and method and storage medium | |
| CN114416420A (en) | Equipment problem feedback method and system | |
| CN112364121A (en) | Automatic creation method and device of questionnaire PDF, storage medium and computer equipment | |
| CN114827197B (en) | Service communication method, device, equipment and medium based on Internet of things | |
| CN112215587B (en) | Method, device and system for automatically executing business process interaction tasks by multiple devices | |
| CN113141366B (en) | Message submitting method and device, storage medium and electronic equipment | |
| WO2018134680A1 (en) | System and method for integrating disparate computer systems and applications |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Hu Nan Inventor after: Zhou Xiaomin Inventor after: Yu Haifeng Inventor before: Hu Nan Inventor before: Zhou Xiaomin |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |