CN115328730A - Distributed storage cluster-based inspection and processing device, method, apparatus, and medium - Google Patents
Distributed storage cluster-based inspection and processing device, method, apparatus, and medium Download PDFInfo
- Publication number
- CN115328730A CN115328730A CN202210969191.7A CN202210969191A CN115328730A CN 115328730 A CN115328730 A CN 115328730A CN 202210969191 A CN202210969191 A CN 202210969191A CN 115328730 A CN115328730 A CN 115328730A
- Authority
- CN
- China
- Prior art keywords
- distributed storage
- cluster
- inspection
- checking
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007689 inspection Methods 0.000 title claims abstract description 94
- 238000012545 processing Methods 0.000 title claims abstract description 82
- 238000000034 method Methods 0.000 title claims abstract description 49
- 230000002159 abnormal effect Effects 0.000 claims abstract description 120
- 238000007405 data analysis Methods 0.000 claims abstract description 51
- 230000008569 process Effects 0.000 claims abstract description 27
- 230000010365 information processing Effects 0.000 claims abstract description 22
- 238000004886 process control Methods 0.000 claims abstract description 10
- 230000008439 repair process Effects 0.000 claims description 25
- 238000004590 computer program Methods 0.000 claims description 16
- 238000003672 processing method Methods 0.000 claims description 16
- 238000004458 analytical method Methods 0.000 claims description 15
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 239000002699 waste material Substances 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3003—Monitoring arrangements specially adapted to the computing system or computing system component being monitored
- G06F11/3034—Monitoring arrangements specially adapted to the computing system or computing system component being monitored where the computing system component is a storage system, e.g. DASD based or network based
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/30—Monitoring
- G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
- G06F11/3072—Monitoring arrangements determined by the means or processing involved in reporting the monitored data where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Quality & Reliability (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Computer Security & Cryptography (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Debugging And Monitoring (AREA)
- Hardware Redundancy (AREA)
Abstract
The application discloses an inspection and processing device, method, equipment and medium based on distributed storage cluster, relating to the technical field of computer and comprising: the system comprises a flow control module, an inspection module and a data analysis module; the inspection module is positioned at each target node, and the process control module is used for generating a preset inspection item list, grouping the target nodes to obtain preset node groups and determining a management node of each preset node group; the checking module is used for acquiring a checking command sent by the management node, checking a target node based on the checking command and a preset checking item list and sending checking information to the management node so that the management node can determine abnormal information from the checking information; the data analysis module is used for acquiring the abnormal information sent by the management node, and sending a processing scheme corresponding to the abnormal information to the management node based on a preset abnormal information processing scheme library so that the management node can automatically process the abnormal information. The method and the device can complete the inspection and processing of the distributed storage cluster, and improve the upgrading success rate of the cluster.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an inspection and processing apparatus, method, device, and medium based on a distributed storage cluster.
Background
Currently, with the development and application of distributed storage technology in recent years, in order to improve the stability of the system and the functional characteristics of the application system version, the system needs to be periodically upgraded and maintained. How to ensure normal service during upgrading, how to ensure efficient execution of upgrading, and how to ensure that the influence of some emergency or cluster potential problems on cluster upgrading becomes a problem to be solved urgently.
In summary, how to complete the inspection and processing of the distributed storage cluster to improve the success rate of cluster upgrade is an urgent problem to be solved at present.
Disclosure of Invention
In view of the above, an object of the present invention is to provide an inspection and processing apparatus based on a distributed storage cluster, which can complete inspection and processing of the distributed storage cluster, so as to improve the success rate of cluster upgrade.
The specific scheme is as follows:
in a first aspect, the present application discloses an inspection and processing apparatus based on a distributed storage cluster, including: the system comprises a flow control module, an inspection module and a data analysis module; the inspection module is located in each target node of the distributed storage cluster, wherein,
the flow control module is used for generating a preset check item list before the distributed storage cluster is upgraded, grouping the target nodes to obtain a plurality of preset node groups, and then determining a management node in each preset node group;
the checking module is used for acquiring a checking command sent by each management node, checking a plurality of target nodes based on the checking command and the preset checking item list and sending checking information to the corresponding management node so that the management node can determine abnormal information from the checking information;
the data analysis module is used for acquiring the abnormal information sent by the management nodes and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme.
Optionally, the data analysis module is further configured to provide a repair suggestion for the abnormal information if the processing scheme corresponding to the abnormal information does not exist in the preset abnormal information processing scheme library, so that the client performs manual repair based on the repair suggestion.
Optionally, the data analysis module is further configured to automatically learn a manual processing scheme for performing manual repair, and store the manual processing scheme in the preset abnormal information processing scheme library, so as to automatically repair the abnormal information based on the manual processing scheme.
Optionally, the flow control module is configured to generate a target check item based on first cluster information of the distributed storage cluster before the distributed storage cluster is upgraded, and generate the preset check item list based on the target check item and a fixed check item; the first cluster information comprises the current cluster state, cluster historical alarm information and the abnormal information in the historical upgrading process of the distributed storage cluster.
Optionally, the data analysis module is further configured to generate an upgrade prediction report based on second cluster information of the distributed storage cluster; the cluster information comprises a current cluster state, a service state and service pressure; the upgrade forecast report comprises an upgrade success rate, potential abnormal information, service performance analysis during upgrade, overall upgrade time consumption analysis and single-node upgrade time consumption analysis.
Optionally, the process control module summarizes the check information and the processing result of the abnormal information corresponding to each target node in the distributed storage cluster, and sends the result to the client.
In a second aspect, the application discloses a distributed storage cluster-based inspection and processing method, which is applied to a distributed storage cluster-based inspection and processing device, wherein the device comprises a flow control module, an inspection module and a data analysis module; the checking module is positioned in each target node of the distributed storage cluster; the method comprises the following steps:
generating a preset check item list before upgrading the distributed storage cluster through the flow control module, grouping the target nodes to obtain a plurality of preset node groups, and then determining a management node in each preset node group;
acquiring a checking command sent by each management node through the checking module, checking a plurality of target nodes based on the checking command and the preset checking item list, and sending checking information to the corresponding management node so that the management node determines abnormal information from the checking information;
and acquiring the abnormal information sent by the management nodes through the data analysis module, and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme.
Optionally, the generating, by the process control module, a preset check item list before the upgrade of the distributed storage cluster includes:
generating, by the flow control module, a target check item based on first cluster information of the distributed storage cluster before the distributed storage cluster is upgraded, and generating the preset check item list based on the target check item and a fixed check item; the first cluster information comprises the current cluster state, cluster historical alarm information and the abnormal information in the historical upgrading process of the distributed storage cluster.
In a third aspect, the present application discloses an electronic device comprising a processor and a memory; when the processor executes the computer program stored in the memory, the inspection and processing method based on the distributed storage cluster disclosed in the foregoing is implemented.
In a fourth aspect, the present application discloses a computer readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the distributed storage cluster based inspection and processing method disclosed in the foregoing.
As can be seen, the flow control module in the present application is configured to generate a preset check item list before upgrading the distributed storage cluster, group each target node to obtain a plurality of preset node groups, and then determine a management node in each preset node group; the checking module is used for acquiring a checking command sent by each management node, checking a plurality of target nodes based on the checking command and the preset checking item list and sending checking information to the corresponding management node so that the management node can determine abnormal information from the checking information; the data analysis module is used for acquiring the abnormal information sent by the management nodes and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme. Therefore, the method and the device can complete the inspection and processing of the distributed storage cluster so as to improve the success rate of cluster upgrading; the method and the device have the advantages that the target nodes are grouped to obtain the preset node groups, so that the inspection efficiency can be effectively improved, and the occupation of the inspection function on cluster performance and resources can be reduced; the data analysis module automatically restores the abnormal information based on the abnormal information processing scheme base, can effectively improve the processing efficiency of the abnormal information, and avoids the waste of human resources.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
Fig. 1 is a structural diagram of an inspection and processing device based on a distributed storage cluster according to the present application;
FIG. 2 is a flowchart of a distributed storage cluster-based inspection and processing method provided in the present application;
fig. 3 is a schematic flowchart of a specific inspection and processing method based on a distributed storage cluster according to the present application;
fig. 4 is a block diagram of an electronic device provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Currently, with the continuous development and application of distributed storage technology in recent years, in order to improve the stability of the system and the functional characteristics of the application system version, the system needs to be upgraded and maintained regularly. How to ensure normal service during upgrading, how to ensure efficient execution of upgrading, and how to ensure that the influence of some emergency or cluster potential problems on cluster upgrading becomes a problem to be solved urgently.
In order to overcome the above problems, the present application provides an inspection and processing apparatus based on a distributed storage cluster, which can complete inspection and processing of the distributed storage cluster, so as to improve the success rate of cluster upgrade.
Referring to fig. 1, an embodiment of the present application discloses an inspection and processing device based on a distributed storage cluster, including: a flow control module 11, an inspection module 12 and a data analysis module 13; the inspection module is located in each target node of the distributed storage cluster, wherein,
the flow control module 11 is configured to generate a preset check item list before the distributed storage cluster is upgraded, group each target node to obtain a plurality of preset node groups, and then determine a management node in each preset node group;
the checking module 12 is configured to obtain a checking command sent by each management node, check a plurality of target nodes based on the checking command and the preset checking item list, and send checking information to the corresponding management node, so that the management node determines abnormal information from the checking information;
the data analysis module 13 is configured to acquire the abnormal information sent by the plurality of management nodes, and send a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library, so that the management nodes automatically process the abnormal information according to the processing scheme.
In the embodiment of the application, in order to complete checking and processing of a distributed storage cluster and improve the success rate of cluster upgrading, a checking and processing device before large-scale distributed cluster upgrading is provided, which aims to quickly collect information of each node, service state and hardware equipment state in distributed storage, comprehensively judge whether an abnormal problem which can affect cluster upgrading exists according to a checking result, and check and process the abnormal problem existing in the cluster and a potential problem which can affect the success rate of cluster upgrading before upgrading. And the problem is exposed and processed before upgrading, so that the robustness of the cluster upgrading function is ensured.
In this embodiment of the present application, the flow control module 11 is configured to generate a target check item based on first cluster information of the distributed storage cluster before the distributed storage cluster is upgraded, and generate the preset check item list based on the target check item and a fixed check item; the first cluster information comprises the current cluster state, cluster historical alarm information and the abnormal information in the historical upgrading process of the distributed storage cluster. The content of the check item can be dynamically generated and adjusted, so that the problem existing in the current cluster can be accurately found by checking before upgrading.
In this embodiment of the application, the process control module 11 summarizes the inspection information and the processing result of the abnormal information corresponding to each target node in the distributed storage cluster, and sends the inspection information and the processing result to the client.
To sum up, in the embodiment of the present application, the flow control module 11 mainly works three times, and first, dynamically generates an inspection item list, except for a fixed inspection item, the flow control module may also synthesize a current cluster state, cluster history alarm information, abnormal problems occurring in a history upgrading process, and the like to automatically generate a corresponding inspection item, and perform a key inspection on such abnormalities; secondly, generating a node inspection group (a preset node group) by comprehensively dividing the node group and receiving and forwarding information of the inspection module node to the data analysis module, and grouping large-scale cluster nodes for inspection in order to improve the overall inspection and the processing efficiency and reduce the influence of the inspection device on the cluster performance. Each node group can automatically promote one node to become a management node, the node can gather the inspection results of other nodes in the node group and report the abnormal results to the data analysis module; and thirdly, summarizing the inspection and exception handling results of each node group of the cluster (the result of the summarized data analysis module) and sending the handling results to the user.
In this embodiment of the application, the checking module 12 is mainly distributed on each node, and is mainly used for receiving and executing the checking command issued by the node group management node, summarizing the node checking result, and feeding back the result to the management node.
In this embodiment of the application, the data analysis module 13 is further configured to provide a repair suggestion for the abnormal information if the processing scheme corresponding to the abnormal information does not exist in the preset abnormal information processing scheme library, so that the client performs manual repair based on the repair suggestion.
In the embodiment of the present application, the data analysis module 13 is further configured to automatically learn a manual processing scheme during manual repair, and store the manual processing scheme to the preset abnormal information processing scheme library, so as to automatically repair the abnormal information based on the manual processing scheme. The automatic learning capability of the data analysis module can automatically collect abnormal problem processing schemes, and abnormal problems can be automatically repaired when appearing again.
In this embodiment of the present application, the data analysis module 13 is further configured to generate an upgrade prediction report based on the second cluster information of the distributed storage cluster; the cluster information comprises a current cluster state, a service state and service pressure; the upgrade forecasting report comprises upgrade success rate, potential abnormal information, service performance analysis during upgrade, overall upgrade time consumption analysis and single-node upgrade time consumption analysis. The prediction function of the data analysis module can enable a user to have more visual understanding on cluster performance, service influence, upgrading time consumption and the like generated in the upgrading process.
In summary, the data analysis module 13 of the present application mainly works five, and firstly, statistics, summarization, and process the abnormal information pushed by the management nodes of each node group; second, an exception problem handling scenario library is provided and the data analysis module can automatically fix exception problems based on the resolution of specific problems within the scenario library. Thirdly, for the abnormity without definite solutions, summarizing abnormal information and then giving a repair suggestion; fourthly, the data analysis module can automatically explore and learn the manual processing scheme and the operation steps of the complex problem or the problem lacking the solution and update the manual processing scheme and the operation steps to the abnormal problem processing scheme library, and when the abnormal problem occurs again, the data analysis module can automatically carry out repair processing; and fifthly, generating an upgrade prediction report by integrating the current cluster state, the service pressure and other conditions, wherein the report content mainly comprises an upgrade success rate, a potential abnormal problem point, service performance analysis during upgrade, integral upgrade time consumption analysis, single-node upgrade time consumption analysis and the like. In summary, the data analysis module 13 is mainly used to analyze various abnormal information reported by the inspection module, automatically repair and process the abnormal information with definite solutions, and comprehensively judge the abnormal information without definite solutions and give rough repair suggestions.
As can be seen, the flow control module in the present application is configured to generate a preset check item list before the distributed storage cluster is upgraded, group each target node to obtain a plurality of preset node groups, and then determine a management node in each preset node group; the checking module is used for acquiring a checking command sent by each management node, checking a plurality of target nodes based on the checking command and the preset checking item list and sending checking information to the corresponding management node so that the management node can determine abnormal information from the checking information; the data analysis module is used for acquiring the abnormal information sent by the management nodes and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme. Therefore, the method and the device can complete the inspection and processing of the distributed storage cluster so as to improve the success rate of cluster upgrading; the method and the device have the advantages that the target nodes are grouped to obtain the preset node groups, so that the inspection efficiency can be effectively improved, and the occupation of the inspection function on cluster performance and resources can be reduced; the data analysis module automatically restores the abnormal information based on the abnormal information processing scheme base, can effectively improve the processing efficiency of the abnormal information, and avoids the waste of human resources.
Referring to fig. 2, an embodiment of the present application discloses an inspection and processing method based on a distributed storage cluster, which is applied to an inspection and processing device based on a distributed storage cluster, where the device includes a process control module, an inspection module, and a data analysis module; the checking module is positioned in each target node of the distributed storage cluster; the method comprises the following steps:
step S11: generating a preset check item list by the flow control module before upgrading the distributed storage cluster, grouping the target nodes to obtain a plurality of preset node groups, and then determining a management node in each preset node group.
In the embodiment of the application, a large-scale cluster is split into a plurality of inspection node groups (preset node groups), so that inspection efficiency can be effectively improved, and occupation of an inspection function on cluster performance and resources can be reduced.
In this embodiment of the application, generating, by the process control module, a preset check item list before the upgrade of the distributed storage cluster includes: generating, by the process control module, a target check item based on first cluster information of the distributed storage cluster before the distributed storage cluster is upgraded, and generating the preset check item list based on the target check item and a fixed check item; the first cluster information comprises the current cluster state, cluster historical alarm information and the abnormal information in the historical upgrading process of the distributed storage cluster. It should be noted that, the content of the inspection item can be dynamically generated and adjusted, so that the inspection before upgrading can accurately find the problems existing in the current cluster.
In the embodiment of the application, the check information and the processing result of the abnormal information corresponding to each target node in the distributed storage cluster are summarized through the process control module, and are sent to the client.
It should be noted that three main tasks can be accomplished by the process control module, including: firstly, an inspection item list is dynamically generated, besides a fixed inspection item, a flow control module can also automatically generate a corresponding inspection item by integrating the current cluster state, cluster historical alarm information, abnormal problems in the history upgrading process and the like, and perform key inspection on the abnormal conditions; secondly, generating a node inspection group (a preset node group) by comprehensively dividing the node group and receiving and forwarding information of the inspection module node to the data analysis module, and grouping large-scale cluster nodes for inspection in order to improve the overall inspection and the processing efficiency and reduce the influence of the inspection device on the cluster performance. Each node group can automatically promote one node to become a management node, the node can gather the inspection results of other nodes in the node group and report the abnormal results to the data analysis module; and thirdly, summarizing the inspection and exception handling results of each node group of the cluster (the result of the summarized data analysis module) and sending the handling results to the user.
Step S12: and acquiring a checking command sent by each management node through the checking module, checking a plurality of target nodes based on the checking command and the preset checking item list, and sending checking information to the corresponding management node so that the management node determines abnormal information from the checking information.
In the embodiment of the application, the inspection module is positioned in each target node, and the vehicle inspection module is mainly used for receiving and executing an inspection command issued by the node group management node, summarizing a node inspection result and feeding the result back to the management node.
Step S13: and acquiring the abnormal information sent by the management nodes through the data analysis module, and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme.
In the embodiment of the application, the automatic learning capability of the data analysis module can automatically collect abnormal problem processing schemes, and when abnormal problems occur again, the abnormal problems can be automatically repaired.
In this embodiment of the application, an upgrade prediction report may be generated by the data analysis module based on the second cluster information of the distributed storage cluster; the cluster information comprises a current cluster state, a service state and service pressure; the upgrade forecast report comprises an upgrade success rate, potential abnormal information, service performance analysis during upgrade, overall upgrade time consumption analysis and single-node upgrade time consumption analysis. It should be noted that the prediction function of the data analysis module can enable the user to have more intuitive knowledge about the cluster performance, the influence of the service, the time consumed for upgrading, and the like generated in the upgrading process.
In the embodiment of the application, when the processing scheme corresponding to the abnormal information does not exist in the preset abnormal information processing scheme library through the data analysis module, a repair suggestion is provided for the abnormal information, so that a client performs manual repair based on the repair suggestion.
In the embodiment of the application, the data analysis module can automatically learn the manual processing scheme during manual repair, and store the manual processing scheme to the preset abnormal information processing scheme library so as to automatically repair the abnormal information based on the manual processing scheme. It should be noted that the automatic learning capability of the data analysis module can automatically collect abnormal problem handling schemes, and can automatically repair abnormal problems when they occur again.
As can be seen, in the present application, a preset check item list is generated by the flow control module before the distributed storage cluster is upgraded, and each target node is grouped to obtain a plurality of preset node groups, and then a management node in each preset node group is determined; acquiring a checking command sent by each management node through the checking module, checking a plurality of target nodes based on the checking command and the preset checking item list, and sending checking information to the corresponding management node so that the management node determines abnormal information from the checking information; and acquiring the abnormal information sent by the management nodes through the data analysis module, and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme. Therefore, the method and the device can complete the inspection and processing of the distributed storage cluster so as to improve the success rate of cluster upgrading; the method and the device have the advantages that the target nodes are grouped to obtain the preset node groups, so that the inspection efficiency can be effectively improved, and the occupation of the inspection function on cluster performance and resources can be reduced; the data analysis module automatically restores the abnormal information based on the abnormal information processing scheme base, can effectively improve the processing efficiency of the abnormal information, and avoids the waste of human resources.
Fig. 3 is a schematic flow chart of a specific inspection and processing method based on a distributed storage cluster; in the graph, a flow control node sends a check command to a management node in each node group, the management node sends the check command to a check module, the check module checks corresponding nodes and sends check information to corresponding management nodes, the management node determines abnormal information from the check information and sends the abnormal information to a data analysis module, the data analysis module sends a processing scheme corresponding to the abnormal information to the management node based on a preset abnormal information processing scheme library, and the management node automatically processes the abnormal information according to the processing scheme; in addition, the data analysis module may also generate an upgrade forecast report.
Further, an electronic device is provided in the embodiments of the present application, and fig. 4 is a block diagram of the electronic device 20 according to an exemplary embodiment, which should not be construed as limiting the scope of the application.
Fig. 4 is a schematic structural diagram of an electronic device 20 according to an embodiment of the present disclosure. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, an input output interface 24, a communication interface 25, and a communication bus 26. The memory 22 is configured to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps of the distributed storage cluster-based inspection and processing method disclosed in any of the foregoing embodiments.
In this embodiment, the power supply 23 is configured to provide a working voltage for each hardware device on the electronic device 20; the communication interface 25 can create a data transmission channel between the electronic device 20 and an external device, and a communication protocol followed by the communication interface is any communication protocol applicable to the technical solution of the present application, and is not specifically limited herein; the input/output interface 24 is configured to obtain external input data or output data to the outside, and a specific interface type thereof may be selected according to specific application requirements, which is not specifically limited herein.
In addition, the storage 22 is used as a carrier for resource storage, and may be a read-only memory, a random access memory, a magnetic disk or an optical disk, and the storage 22 is used as a non-volatile storage that may include a random access memory as a running memory and a storage purpose for an external memory, and the storage resources on the storage include an operating system 221, a computer program 222, and the like, and the storage manner may be a transient storage or a permanent storage.
The operating system 221 is used for managing and controlling each hardware device and the computer program 222 on the electronic device 20 on the source host, and the operating system 221 may be Windows, unix, linux, or the like. The computer program 222 may further include a computer program that can be used to perform other specific tasks in addition to the computer program that can be used to perform the distributed storage cluster based inspection and processing method performed by the electronic device 20 disclosed in any of the foregoing embodiments.
In this embodiment, the input/output interface 24 may specifically include, but is not limited to, a USB interface, a hard disk reading interface, a serial interface, a voice input interface, a fingerprint input interface, and the like.
Further, the embodiment of the application also discloses a computer readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the distributed storage cluster based inspection and processing method disclosed in the foregoing.
For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The computer-readable storage medium includes a Random Access Memory (RAM), a Memory, a Read-Only Memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a magnetic or optical disk, or any other form of storage medium known in the art. Wherein the computer program, when executed by a processor, implements the aforementioned distributed storage cluster-based inspection and processing method. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the foregoing embodiments, which are not described herein again.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the checking and processing method based on the distributed storage cluster disclosed by the embodiment, so that the description is relatively simple, and the relevant points can be referred to the description of the method part.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
The steps of an algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Finally, it should also be noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising a … …" does not exclude the presence of another identical element in a process, method, article, or apparatus that comprises the element.
The above detailed description is provided for an inspection and processing device, method, device and medium based on a distributed storage cluster, and a specific example is applied in this document to explain the principle and implementation of the present invention, and the description of the above embodiment is only used to help understanding the method and its core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. An inspection and processing apparatus based on distributed storage clusters, comprising: the system comprises a flow control module, an inspection module and a data analysis module; the inspection module is located in each target node of the distributed storage cluster, wherein,
the flow control module is used for generating a preset check item list before the distributed storage cluster is upgraded, grouping the target nodes to obtain a plurality of preset node groups, and then determining a management node in each preset node group;
the checking module is used for acquiring a checking command sent by each management node, checking a plurality of target nodes based on the checking command and the preset checking item list and sending checking information to the corresponding management node so that the management node can determine abnormal information from the checking information;
the data analysis module is used for acquiring the abnormal information sent by the management nodes and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme.
2. The distributed storage cluster-based inspection and processing apparatus of claim 1, wherein,
the data analysis module is further configured to provide a repair suggestion for the abnormal information if the processing scheme corresponding to the abnormal information does not exist in the preset abnormal information processing scheme library, so that the client performs manual repair based on the repair suggestion.
3. The distributed storage cluster-based inspection and processing apparatus of claim 2, wherein,
the data analysis module is further configured to automatically learn a manual processing scheme during manual repair, and store the manual processing scheme in the preset abnormal information processing scheme library, so as to automatically repair the abnormal information based on the manual processing scheme.
4. The distributed storage cluster-based inspection and processing apparatus of claim 1, wherein,
the flow control module is used for generating a target check item based on first cluster information of the distributed storage cluster before the distributed storage cluster is upgraded, and generating the preset check item list based on the target check item and a fixed check item; the first cluster information comprises the current cluster state, cluster historical alarm information and the abnormal information in the historical upgrading process of the distributed storage cluster.
5. The distributed storage cluster-based inspection and processing apparatus of claim 1, wherein,
the data analysis module is further configured to generate an upgrade prediction report based on second cluster information of the distributed storage cluster; the cluster information comprises a current cluster state, a service state and service pressure; the upgrade forecasting report comprises upgrade success rate, potential abnormal information, service performance analysis during upgrade, overall upgrade time consumption analysis and single-node upgrade time consumption analysis.
6. The distributed storage cluster-based inspection and processing apparatus of any one of claims 1 to 5, wherein,
and the flow control module summarizes the check information and the processing result of the abnormal information corresponding to each target node in the distributed storage cluster and sends the check information and the processing result to a client.
7. The inspection and processing method based on the distributed storage cluster is characterized by being applied to an inspection and processing device based on the distributed storage cluster, wherein the device comprises a flow control module, an inspection module and a data analysis module; the checking module is positioned in each target node of the distributed storage cluster; the method comprises the following steps:
generating a preset check item list before the distributed storage cluster is upgraded through the flow control module, grouping the target nodes to obtain a plurality of preset node groups, and then determining a management node in each preset node group;
acquiring a checking command sent by each management node through the checking module, checking a plurality of target nodes based on the checking command and the preset checking item list, and sending checking information to the corresponding management node so that the management node determines abnormal information from the checking information;
and acquiring the abnormal information sent by the management nodes through the data analysis module, and sending a processing scheme corresponding to the abnormal information to the management nodes based on a preset abnormal information processing scheme library so that the management nodes can automatically process the abnormal information according to the processing scheme.
8. The distributed storage cluster-based inspection and processing method according to claim 7, wherein the generating, by the flow control module, a preset inspection item list before the upgrading of the distributed storage cluster includes:
generating, by the process control module, a target check item based on first cluster information of the distributed storage cluster before the distributed storage cluster is upgraded, and generating the preset check item list based on the target check item and a fixed check item; the first cluster information comprises the current cluster state, cluster historical alarm information and the abnormal information in the historical upgrading process of the distributed storage cluster.
9. An electronic device comprising a processor and a memory; wherein the processor, when executing the computer program stored in the memory, implements the distributed storage cluster based inspection and processing method according to any one of claims 7 to 8.
10. A computer-readable storage medium for storing a computer program; wherein the computer program when executed by a processor implements the distributed storage cluster based inspection and processing method of any of claims 7 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210969191.7A CN115328730A (en) | 2022-08-12 | 2022-08-12 | Distributed storage cluster-based inspection and processing device, method, apparatus, and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210969191.7A CN115328730A (en) | 2022-08-12 | 2022-08-12 | Distributed storage cluster-based inspection and processing device, method, apparatus, and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115328730A true CN115328730A (en) | 2022-11-11 |
Family
ID=83923840
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210969191.7A Pending CN115328730A (en) | 2022-08-12 | 2022-08-12 | Distributed storage cluster-based inspection and processing device, method, apparatus, and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115328730A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116484373A (en) * | 2023-05-08 | 2023-07-25 | 合芯科技(苏州)有限公司 | Abnormal process checking and killing method, system, device, computer equipment and storage medium |
-
2022
- 2022-08-12 CN CN202210969191.7A patent/CN115328730A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116484373A (en) * | 2023-05-08 | 2023-07-25 | 合芯科技(苏州)有限公司 | Abnormal process checking and killing method, system, device, computer equipment and storage medium |
| CN116484373B (en) * | 2023-05-08 | 2024-02-23 | 合芯科技(苏州)有限公司 | Abnormal process checking and killing method, system, device, computer equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109753356A (en) | A kind of container resource regulating method, device and computer readable storage medium | |
| CN110515647B (en) | Static resource management method, device, equipment and storage medium | |
| CN110119306B (en) | Method, device and equipment for balancing automatic scheduling of jobs and storage medium | |
| CN107291950B (en) | Form data updating method and device and computer equipment | |
| US11676114B2 (en) | Automated control of distributed computing devices | |
| JP2007334580A (en) | Support device, program, information processing system and support method | |
| US12032477B2 (en) | Method and apparatus for generating and applying deep learning model based on deep learning framework | |
| EP3779701B1 (en) | Data monitoring method, electronic device, and computer readable storage medium | |
| CN109684309A (en) | A kind of quality of data evaluating method and device, computer equipment and storage medium | |
| CN115328730A (en) | Distributed storage cluster-based inspection and processing device, method, apparatus, and medium | |
| KR20210034558A (en) | Artificial intelligence development platform managing method, device, and medium | |
| CN111966382A (en) | Online deployment method and device of machine learning model and related equipment | |
| CN110855484B (en) | Method, system, electronic device and storage medium for automatically detecting traffic change | |
| CN112817687A (en) | Data synchronization method and device | |
| CN114925283A (en) | Management method and system of push task, electronic device and medium | |
| CN110932926B (en) | Container cluster monitoring method, system and device | |
| CN114265605A (en) | Version rollback method and device for functional component of business system | |
| CN110971664A (en) | Interface service management system | |
| CN111625262A (en) | Rollback method, system, equipment and storage medium of workload resources | |
| CN115062233B (en) | Combined label generation method for civil aviation data | |
| CN114615144B (en) | Network optimization method and system | |
| CN113986469A (en) | Task monitoring method, device and system | |
| CN118278383A (en) | Log analysis method, device, medium and product during database synchronization | |
| CN115202825A (en) | Large-scale cluster request processing method, device and medium | |
| CN113568818A (en) | Alarm method, alarm system and computer equipment for continuous integration of software platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |