CN111881164B - Data processing method based on edge computing and path analysis and big data cloud platform - Google Patents
Data processing method based on edge computing and path analysis and big data cloud platform Download PDFInfo
- Publication number
- CN111881164B CN111881164B CN202010675996.1A CN202010675996A CN111881164B CN 111881164 B CN111881164 B CN 111881164B CN 202010675996 A CN202010675996 A CN 202010675996A CN 111881164 B CN111881164 B CN 111881164B
- Authority
- CN
- China
- Prior art keywords
- data
- information
- target
- service
- intelligent
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 title claims abstract description 31
- 238000003672 processing method Methods 0.000 title claims abstract description 11
- 230000005540 biological transmission Effects 0.000 claims abstract description 111
- 238000004519 manufacturing process Methods 0.000 claims abstract description 51
- 238000012544 monitoring process Methods 0.000 claims abstract description 48
- 238000013515 script Methods 0.000 claims abstract description 47
- 238000009826 distribution Methods 0.000 claims abstract description 37
- 238000013024 troubleshooting Methods 0.000 claims abstract description 32
- 238000010586 diagram Methods 0.000 claims abstract description 27
- 238000005538 encapsulation Methods 0.000 claims description 20
- 238000012163 sequencing technique Methods 0.000 claims description 18
- 238000000034 method Methods 0.000 claims description 17
- 238000004891 communication Methods 0.000 claims description 15
- 238000013507 mapping Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 15
- 230000008859 change Effects 0.000 claims description 12
- 238000004806 packaging method and process Methods 0.000 claims description 12
- 238000012216 screening Methods 0.000 claims description 12
- 230000008569 process Effects 0.000 claims description 8
- 238000003860 storage Methods 0.000 claims description 8
- 230000006835 compression Effects 0.000 claims description 6
- 238000007906 compression Methods 0.000 claims description 6
- 238000006243 chemical reaction Methods 0.000 description 9
- 239000011159 matrix material Substances 0.000 description 9
- 238000009776 industrial production Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000004590 computer program Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000007405 data analysis Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000012806 monitoring device Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
- G06F16/24568—Data stream processing; Continuous queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2458—Special types of queries, e.g. statistical queries, fuzzy queries or distributed queries
- G06F16/2462—Approximate or statistical queries
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/04—Manufacturing
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Probability & Statistics with Applications (AREA)
- General Business, Economics & Management (AREA)
- Mathematical Physics (AREA)
- Strategic Management (AREA)
- Primary Health Care (AREA)
- Marketing (AREA)
- Human Resources & Organizations (AREA)
- Fuzzy Systems (AREA)
- Tourism & Hospitality (AREA)
- Software Systems (AREA)
- General Health & Medical Sciences (AREA)
- Economics (AREA)
- Health & Medical Sciences (AREA)
- Manufacturing & Machinery (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The data processing method based on edge computing and path analysis and the big data cloud platform provided by the embodiment of the application firstly analyze data transmission records uploaded by data stream monitoring scripts deployed in intelligent equipment to obtain a data operation list and a data transmission path of target data, secondly determine a data stream distribution diagram of the target data among a plurality of intelligent equipment and obtain a production data map of an intelligent production system formed by the plurality of intelligent equipment based on the data operation list and the data transmission path, and finally determine the target intelligent equipment with faults according to the fault instruction and find out a target data set corresponding to the target intelligent equipment in the production data map. Therefore, the target intelligent equipment can be subjected to troubleshooting according to the target data set, the troubleshooting time can be reduced, and the data transmission path between the intelligent equipment can be taken into account so as to accurately perform troubleshooting.
Description
Technical Field
The application relates to the technical field of big data analysis, in particular to a data processing method based on edge computing and path analysis and a big data cloud platform.
Background
Big data (bigdata) plays a guiding role in social production and people's life as an important information asset at present. Big data analysis enables deep mining of data to gain significant value behind the data.
With the rapid development of industrial automation, the application of big data to industrial production tends to be great, and the efficiency of industrial production can be effectively improved by combining the application of edge calculation in industrial production.
In industrial production, a plurality of intelligent devices exchange data and communicate with each other to realize ordered promotion of a production process. However, in practical applications, an intelligent production system composed of a plurality of intelligent devices still fails, but it is difficult to effectively troubleshoot the intelligent production system in the prior art.
Disclosure of Invention
The application provides a data processing method and a big data cloud platform based on edge computing and path analysis, and aims to solve the technical problem that in the prior art, effective troubleshooting on an intelligent production system is difficult.
Firstly, a data processing method based on edge computing and path analysis is provided, which is applied to a big data cloud platform which is in communication connection with a plurality of intelligent devices, and the method comprises the following steps:
generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device, and issuing the data stream monitoring script to the corresponding intelligent device;
acquiring a data transmission record uploaded by a data stream monitoring script corresponding to each intelligent device, and analyzing the data transmission record to obtain a data operation list of target data corresponding to the data transmission record and a data transmission path;
determining a data flow distribution diagram of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and performing correlation storage on each data flow distribution diagram to obtain a production data diagram of the intelligent production system formed by the plurality of intelligent devices;
when a fault instruction reported by fault monitoring equipment corresponding to the intelligent production system is received, determining target intelligent equipment with a fault according to the fault instruction, and finding out a target data set corresponding to the target intelligent equipment in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data.
Secondly, a big data cloud platform is provided for executing the method, the big data cloud platform is in communication connection with a plurality of intelligent devices, and the big data cloud platform comprises the following functional modules:
the script issuing module is used for generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device and issuing the data stream monitoring script to the corresponding intelligent device;
the record analysis module is used for acquiring data transmission records uploaded by the data stream monitoring scripts corresponding to each intelligent device, and analyzing the data transmission records to obtain a data operation list of target data corresponding to the data transmission records and a data transmission path;
the map generation module is used for determining a data flow distribution map of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and performing correlation storage on each data flow distribution map to obtain a production data map of the intelligent production system formed by the plurality of intelligent devices;
the fault troubleshooting module is used for determining target intelligent equipment with faults according to the fault instruction when the fault instruction reported by the fault monitoring equipment corresponding to the intelligent production system is received, and finding out a target data set corresponding to the target intelligent equipment in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data.
The data processing method based on edge computing and path analysis and the big data cloud platform provided by the embodiment of the application firstly analyze data transmission records uploaded by data stream monitoring scripts deployed in intelligent equipment to obtain a data operation list and a data transmission path of target data, secondly determine a data stream distribution diagram of the target data among a plurality of intelligent equipment and obtain a production data map of an intelligent production system formed by the plurality of intelligent equipment based on the data operation list and the data transmission path, and finally determine the target intelligent equipment with faults according to the fault instruction and find out a target data set corresponding to the target intelligent equipment in the production data map. Therefore, the target intelligent equipment can be subjected to troubleshooting according to the target data set, the troubleshooting time can be reduced, and the data transmission path between the intelligent equipment can be taken into account so as to accurately perform troubleshooting.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application.
FIG. 1 is a schematic diagram of a data processing system based on edge computation and path analysis, according to an exemplary embodiment of the present application.
Fig. 2 is a flowchart illustrating a data processing method based on edge computation and path analysis according to an exemplary embodiment of the present application.
Fig. 3 is a functional block diagram of a big data cloud platform according to an exemplary embodiment of the present application.
Fig. 4 is a hardware block diagram of a big data cloud platform according to an exemplary embodiment of the present application.
Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Depending on the development of edge computing, data interaction and data processing between intelligent devices are usually deployed nearby on the device side, and if data of one ring is wrong, the wrong data can be transmitted between different intelligent devices, so that a fault of the intelligent production system is caused. In this case, since the smart devices are distributed systems, data processing records need to be retrieved from each smart device and checked, on one hand, time consumption for troubleshooting is increased, and on the other hand, data transmission paths between the smart devices cannot be taken into consideration, so that it is difficult to accurately perform troubleshooting.
In order to solve the above problems, embodiments of the present invention are directed to disclose a data processing method and a big data cloud platform based on edge computing and path analysis, which can not only reduce the time consumption of troubleshooting, but also take the data transmission path between intelligent devices into consideration so as to accurately perform troubleshooting.
Referring first to fig. 1, a system architecture diagram of a data processing system 100 based on edge computing and path analysis according to an embodiment of the present invention is shown, where the system may include a big data cloud platform 200 and a plurality of smart devices 300, which are in communication with each other. The smart device 300 may be an industrial production device having a data processing function, and the smart device 300 may be applied to a plurality of industrial fields, which is not limited herein. On the basis of fig. 1, fig. 2 provides a flowchart of a data processing method based on edge computation and path analysis, which may be applied to the big data cloud platform 200 in fig. 1, and specifically may include the contents described in the following steps S21-S24.
And step S21, generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device, and issuing the data stream monitoring script to the corresponding intelligent device.
In a specific example, the data stream monitoring script running in the smart device 300 can crawl the data transmission record of the smart device 300 and upload the data transmission record to the big data cloud platform 200, so that the big data cloud platform 200 can analyze the data transmission record.
And step S22, acquiring data transmission records uploaded by the data stream monitoring scripts corresponding to each intelligent device, and analyzing the data transmission records to obtain a data operation list of target data corresponding to the data transmission records and a data transmission path.
In step S22, the target data is the data processed by the smart device 300, the data operation list includes operation logic information such as splitting, cleaning, marking, format splitting, and feature extraction of the smart device 200 on the target data, and the data transmission path includes communication addresses of all downstream smart devices 200 that are issued by the smart device 200 after the data operation on the target data is completed. It can be understood that the data transmission path corresponding to the target data can be obtained by connecting the plurality of communication addresses according to the transmission identifier corresponding to the target data.
Step S23, determining a data flow distribution diagram of each group of target data among the plurality of intelligent devices based on the data operation list corresponding to each group of target data and the data transmission path, and associating and storing each data flow distribution diagram to obtain a production data diagram of the intelligent production system formed by the plurality of intelligent devices.
It can be understood that the data flow distribution diagram includes the service forms of the target data in different intelligent devices 300 and the upstream and downstream relations between different intelligent devices 300, and the data flow distribution diagram is stored in an associated manner, so that the data association between different target data can be taken into account, and an accurate and reliable judgment basis is provided for subsequent troubleshooting. Further, the production data map may be stored in the form of map data.
Step S24, when a fault instruction reported by a fault monitoring device corresponding to the intelligent production system is received, determining a target intelligent device with a fault according to the fault instruction, and finding out a target data set corresponding to the target intelligent device in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data.
By applying the above steps S21 to S24, the data transmission record uploaded by the data flow monitoring script deployed in the intelligent device is firstly analyzed to obtain the data operation list and the data transmission path of the target data, then the data flow distribution diagram of the target data among the plurality of intelligent devices is determined based on the data operation list and the data transmission path, and the production data map of the intelligent production system formed by the plurality of intelligent devices is obtained, and finally the target intelligent device with the fault is determined according to the fault instruction, and the target data set corresponding to the target intelligent device is found in the production data map. Therefore, the target intelligent equipment can be subjected to troubleshooting according to the target data set, the troubleshooting time can be reduced, and the data transmission path between the intelligent equipment can be taken into account so as to accurately perform troubleshooting.
In the process of implementing the above-mentioned scheme, the inventors found that when generating the data stream distribution diagram, the traffic patterns of the target data in different intelligent devices need to be considered, otherwise, the loss of part of the diagram data of the data stream distribution diagram may result. In order to improve the above problem, in step S23, the data flow profile of each set of target data among the plurality of intelligent devices is determined based on the data operation list corresponding to each set of target data and the data transmission path, which may specifically include the contents described in steps S231 to S233 below.
Step S231, obtaining result data corresponding to the target data generated according to the data operation list, and determining a service form of the result data in a plurality of downstream intelligent devices according to the communication address and the transfer identifier obtained by extracting the data transfer path.
Step S232, performing nodularization on the description information corresponding to the multiple service forms according to the path parameters corresponding to the data transmission path to obtain a service node corresponding to each service form.
Step S233, the execution data of the result data in the downstream intelligent device corresponding to each service node is encapsulated into the corresponding service node, and the plurality of service nodes are connected according to the transfer identifier to obtain the data flow distribution map.
It can be understood that, through the above steps S231 to S233, the traffic forms of the target data in different intelligent devices can be taken into account, so that it can be ensured that no part of graph data (traffic nodes) in the data stream profile is missing when the data stream profile is generated.
On the basis of the foregoing steps S231 to S233, in order to reduce the similarity between the service nodes and ensure the feature identification degree of each service node, the step S232 may further include the following steps S2321 to S2325 of nodularizing the description information corresponding to the multiple service formats according to the path parameter corresponding to the data transmission path to obtain the service node corresponding to each service format.
Step S2321, listing the path priorities corresponding to the path parameters of the data transmission path, and generating a parameter network corresponding to the path parameters according to the path priorities, and determining, according to the parameter network, a time sequence difference coefficient between each information field of the corresponding description information of each service form in the received information category and each information field of the corresponding description information of each service form in the sent information category according to the information field of the corresponding description information of each service form in the sent information category and the field code of the information field, when it is determined that the corresponding description information of each service form includes the sent information category.
Step S2322, based on the time sequence difference coefficient, dividing the information field of the corresponding description information of each service form in the received information category and the information field in the sent information category, which has continuity in time sequence, into the corresponding sent information category.
Step S2323, in a case that the received information category corresponding to the corresponding description information of each service type includes a plurality of information fields, determining a time sequence difference coefficient between the information fields of the corresponding description information of each service type under the received information category according to the information field of the corresponding description information of each service type under the transmitted information category and the field coding of the information field; screening the information fields under the received information category through the time sequence difference coefficient among the information fields to obtain target information fields; and dividing part of target information fields obtained by screening into the sent information categories according to the information fields of the corresponding description information of each service form under the sent information categories and the field codes of the information fields.
Step S2324, determine the nodularization logic information of the corresponding description information of each service form in the information field of the corresponding sent information category according to the corresponding description information of each service form.
Step S2325, according to the logic sequence extracted from the nodularized logic information, once performing information compression on the corresponding description information of each service form in the information field of the corresponding sent information category to obtain an information characteristic value, and integrating the information characteristic value to obtain a service node corresponding to each service form.
Based on the content described in the above steps S2321 to S2325, the information field corresponding to each service form can be classified and adjusted, so that the similarity between the service nodes is reduced, and the feature recognition degree of each service node is ensured.
In a more specific embodiment, in order to ensure compatibility of the execution data with the service node when the execution data is encapsulated, and avoid an error in the encapsulation of the execution data due to a compatibility problem between the execution data and the service node, the step S233 may be described as encapsulating the execution data of the result data in the downstream smart device corresponding to each service node into the corresponding service node, and may exemplarily include the following steps S2331 to S2334.
Step S2331, determining device configuration information used for characterizing a data processing process of downstream intelligent devices corresponding to each service node, and extracting target configuration information which does not change along with the change of the time slice resource occupancy rate of the downstream intelligent devices in the device configuration information; judging whether the information coding format of the target configuration information is consistent with the data coding format of the result data; when the information coding format is judged to be inconsistent with the data coding format, generating a format label corresponding to the information coding format and loading the format label into a corresponding service node, and when the information coding format is judged to be consistent with the data coding format, generating a data splitting identifier according to the data coding format and loading the data splitting identifier into the corresponding service node.
Step S2332, if the format label exists in the service node, constructing a data packaging list of the service node based on compatibility distribution information corresponding to the format label; if the data splitting identifier exists in the service node, mapping the data splitting identifier to a node container corresponding to the service node, and generating a data packaging list of the service node based on the mapping identifier of the data splitting identifier in the node container.
Step S2333, for a downstream intelligent device corresponding to each service node, determining, from a database of the downstream intelligent device, cache address information in which a data index value corresponding to the result data exists, determining, according to the cache address information, corresponding original data from the result data, and integrating execution data corresponding to the result data in the database and the original data determined from the result data to obtain data to be encapsulated.
Step S2334, determining the encapsulation priority of each data segment in the data to be encapsulated, sequencing the data segments according to the sequence of the encapsulation priorities from large to small to obtain a data segment sequencing sequence, and sequentially encapsulating the data segments in the data segment sequencing sequence into corresponding service nodes according to the data encapsulation list.
When the contents described in the above steps S2331 to S2334 are executed, the compatibility of the execution data with the service node is ensured when the execution data is encapsulated, and an error of the execution data in the encapsulation due to a compatibility problem between the execution data and the service node is avoided, thereby accurately encapsulating the execution data into the corresponding service node.
In a specific implementation process, in order to ensure the integrity of the target data set and thus improve the reliability of subsequent troubleshooting, the target intelligent device that has a fault is determined according to the fault instruction as described in step S24, and the target data set corresponding to the target intelligent device is found in the production data map, which may specifically include the contents described in steps S241 to S243.
Step S241, extracting instruction stream information of the fault instruction, determining equipment registration information from the instruction stream information, and determining the target intelligent equipment from a preset information base according to the equipment registration information; and extracting the signature key of the target intelligent device based on the api interface of the target intelligent device.
In step S242, first map data in which the signing key exists and second map data of a target key whose correlation coefficient with the signing key is larger than a set value are marked in the production data map.
Step S243, generating a target data set corresponding to the target smart device based on the first graph data and the second graph data.
It can be understood that through the above steps S241 to S243, not only the first graph data directly related to the target smart device but also the second graph data having a high correlation with the target smart device can be marked, so that the target data set can be completely determined based on the first graph data and the second graph data, thereby improving the reliability of subsequent troubleshooting.
Optionally, in order to ensure matching between the data stream monitoring script and the intelligent device and avoid the intelligent device misjudging the data stream monitoring script as an abnormal crawler when receiving the data stream monitoring script, in step S21, the data stream monitoring script corresponding to each intelligent device is generated according to the device class information and the device interface information of each intelligent device, which may exemplarily include the contents described in the following steps S211 to S214.
Step S211, respectively extracting a category text set corresponding to the equipment category information and an interface text set corresponding to the equipment interface information; the category text set and the interface text set respectively contain a plurality of text files with different text weight values.
Step S212, judging whether the number of the first text files corresponding to the category text set is the same as the number of the second text files corresponding to the interface text set; if not, determining a first average value of the text weight values corresponding to the first text file and a second average value of the text weight values corresponding to the second text file; when the first average value is larger than the second average value, merging the second text files by taking the number of the first text files as a reference so that the number of the second text files after merging is the same as the number of the first text files; and when the second average value is larger than the first average value, merging the first text files by taking the number of the second text files as a reference so that the number of the first text files after merging is the same as the number of the second text files.
Step S213, under the condition that the number of the first text files is the same as that of the second text files, extracting code stream information of one first text file in the category text set, and determining the second text file with the maximum text weight value in the interface text set as a reference text file in parallel; mapping the code stream information to the reference text file to obtain mirror image code stream information, and generating a text stream conversion matrix between the category text set and the interface text set according to configuration weight between the code stream information and the mirror image code stream information.
Step S214, determining a corresponding relation between each first text file in the category file set and a second text file in the interface text set according to the text stream conversion matrix, generating a data stream monitoring script corresponding to each intelligent device through a logic topology of the corresponding relation, and adding a feature key corresponding to the text stream conversion matrix into the data stream monitoring script; wherein the feature key uniquely corresponds to the smart device.
When the contents described in the above steps S211 to S214 are applied, the feature key uniquely corresponding to the intelligent device can be correspondingly added after the data stream monitoring script is generated, so that the matching between the data stream monitoring script and the intelligent device is ensured, and the intelligent device is prevented from misjudging the data stream monitoring script as an abnormal crawler when receiving the data stream monitoring script.
Optionally, the parsing the data transmission record to obtain the data operation list and the data transmission path of the target data corresponding to the data transmission record described in step S22 may specifically include the contents described in step S221 to step S223 below.
Step S221, importing the data transmission record into a preset record analysis thread.
Step S222, determining whether an update identifier exists in the thread parameter of the record analysis thread, and if not, updating the thread parameter based on the transmission link parameter corresponding to the data transmission record; and the thread parameters after being updated have updating marks.
Step S223, when the thread parameter has the update identifier, starting the record analysis thread to output a data operation list of the target data corresponding to the data transmission record and a data transmission path.
It can be understood that, through the above steps S221 to S223, the real-time performance of the record analysis thread in analyzing the data transmission record can be ensured, so as to ensure the accuracy and reliability of the data operation list and the data transmission path.
Based on the same inventive concept, please refer to fig. 3 in combination, a big data cloud platform 200 is provided for executing the method shown in fig. 2, where the big data cloud platform 200 includes the following functional modules:
the script issuing module 210 is configured to generate a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device, and issue the data stream monitoring script to the corresponding intelligent device;
the record analysis module 220 is configured to obtain a data transmission record uploaded by the data stream monitoring script corresponding to each intelligent device, and analyze the data transmission record to obtain a data operation list of target data corresponding to the data transmission record and a data transmission path;
the map generation module 230 is configured to determine a data flow distribution map of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and perform association storage on each data flow distribution map to obtain a production data map of the intelligent production system formed by the plurality of intelligent devices;
a troubleshooting module 240, configured to, when a fault instruction reported by a fault monitoring device corresponding to the intelligent production system is received, determine a target intelligent device having a fault according to the fault instruction, and find out a target data set corresponding to the target intelligent device in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data.
Optionally, the troubleshooting module 240 is specifically configured to:
extracting instruction stream information of the fault instruction, determining equipment registration information from the instruction stream information, and determining the target intelligent equipment from a preset information base according to the equipment registration information; extracting a signature key of the target intelligent device based on an api interface of the target intelligent device;
marking first graph data with the signature key and second graph data of a target key with a correlation coefficient larger than a set value;
generating a target data set corresponding to the target smart device based on the first graph data and the second graph data.
Optionally, the map generating module 230 is specifically configured to:
acquiring result data corresponding to the target data generated according to the data operation list, and determining the service forms of the result data in a plurality of downstream intelligent devices through communication addresses and transmission identifiers obtained by extracting the data transmission paths;
performing nodularization on the description information corresponding to the plurality of service forms according to the path parameters corresponding to the data transmission path to obtain service nodes corresponding to each service form;
and encapsulating the execution data of the result data in the downstream intelligent equipment corresponding to each service node into the corresponding service node, and connecting a plurality of service nodes according to the transmission identifier to obtain the data flow distribution map.
Optionally, the map generation module 230 is further configured to:
listing the path priority corresponding to the path parameters of the data transmission path, generating a parameter network corresponding to the path parameters according to the path priority, and determining a time sequence difference coefficient between each information field of the corresponding description information of each service form under the received information category and each information field of the corresponding description information of each service form under the sent information category according to the information field of the corresponding description information of each service form under the corresponding sent information category and the field coding of the information field under the condition that the corresponding description information of each service form contains the sent information category according to the parameter network;
dividing the information field of the corresponding description information of each traffic form under the received information category and the information field under the sent information category which has continuity in time sequence into the corresponding sent information category based on the time sequence difference coefficient;
under the condition that a plurality of information fields are contained in the received information category corresponding to the corresponding description information of each service form, determining a time sequence difference coefficient between the information fields of the corresponding description information of each service form in the received information category according to the information fields of the corresponding description information of each service form in the sent information category and the field codes of the information fields; screening the information fields under the received information category through the time sequence difference coefficient among the information fields to obtain target information fields; dividing part of target information fields obtained by screening into the sent information categories according to the information fields of the corresponding description information of each service form under the sent information categories and the field codes of the information fields;
determining the node logic information of the corresponding description information of each service form in the information field of the corresponding sent information type according to the corresponding description information of each service form;
and according to the logic sequence extracted from the nodularization logic information, performing information compression on the corresponding description information of each service form in the information field of the corresponding sent information type once to obtain an information characteristic value, and integrating the information characteristic value to obtain a service node corresponding to each service form.
Optionally, the map generation module 230 is further configured to:
determining device configuration information used for representing a data processing process of downstream intelligent devices corresponding to each service node, and extracting target configuration information which does not change along with the change of the time slice resource occupancy rate of the downstream intelligent devices in the device configuration information; judging whether the information coding format of the target configuration information is consistent with the data coding format of the result data; when the information coding format is judged to be inconsistent with the data coding format, generating a format label corresponding to the information coding format and loading the format label into a corresponding service node, and when the information coding format is judged to be consistent with the data coding format, generating a data splitting identifier according to the data coding format and loading the data splitting identifier into the corresponding service node;
if the format label exists in the service node, constructing a data packaging list of the service node based on compatibility distribution information corresponding to the format label; if the data splitting identifier exists in the service node, mapping the data splitting identifier to a node container corresponding to the service node and generating a data packaging list of the service node based on the mapping identifier of the data splitting identifier in the node container;
determining, for a downstream intelligent device corresponding to each service node, cache address information in which a data index value corresponding to the result data exists from a database of the downstream intelligent device, determining corresponding original data from the result data according to the cache address information, and integrating execution data corresponding to the result data in the database and the original data determined from the result data to obtain data to be encapsulated;
determining the encapsulation priority of each data segment in the data to be encapsulated, sequencing the data segments according to the sequence of the encapsulation priorities from large to small to obtain a data segment sequencing sequence, and sequentially encapsulating the data segments in the data segment sequencing sequence into corresponding service nodes according to the data encapsulation list.
On the basis of the above, a data processing device based on edge calculation and path analysis is also provided, and the specific description about the device is as follows.
A1. A data processing device based on edge computing and path analysis is applied to a big data cloud platform, the big data cloud platform is in communication connection with a plurality of intelligent devices, and the device comprises the following functional modules:
the script issuing module is used for generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device and issuing the data stream monitoring script to the corresponding intelligent device;
the record analysis module is used for acquiring data transmission records uploaded by the data stream monitoring scripts corresponding to each intelligent device, and analyzing the data transmission records to obtain a data operation list of target data corresponding to the data transmission records and a data transmission path;
the map generation module is used for determining a data flow distribution map of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and performing correlation storage on each data flow distribution map to obtain a production data map of the intelligent production system formed by the plurality of intelligent devices;
the fault troubleshooting module is used for determining target intelligent equipment with faults according to the fault instruction when the fault instruction reported by the fault monitoring equipment corresponding to the intelligent production system is received, and finding out a target data set corresponding to the target intelligent equipment in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data.
A2. The apparatus according to a1, wherein the script issuing module is specifically configured to:
respectively extracting a category text set corresponding to the equipment category information and an interface text set corresponding to the equipment interface information; the interface text set comprises a category text set and an interface text set, wherein the category text set and the interface text set respectively comprise a plurality of text files with different text weight values;
judging whether the number of the first text files corresponding to the category text set is the same as the number of the second text files corresponding to the interface text set; if not, determining a first average value of the text weight values corresponding to the first text file and a second average value of the text weight values corresponding to the second text file; when the first average value is larger than the second average value, merging the second text files by taking the number of the first text files as a reference so that the number of the second text files after merging is the same as the number of the first text files; when the second average value is larger than the first average value, merging the first text files by taking the number of the second text files as a reference so that the number of the first text files after merging is the same as the number of the second text files;
under the condition that the number of the first text files is the same as that of the second text files, extracting code stream information of one first text file in the category text set, and determining the second text file with the largest text weight value in the interface text set as a reference text file in parallel; mapping the code stream information to the reference text file to obtain mirror image code stream information, and generating a text stream conversion matrix between the category text set and the interface text set according to configuration weight between the code stream information and the mirror image code stream information;
determining a corresponding relation between each first text file in the category file set and a second text file in the interface text set according to the text stream conversion matrix, generating a data stream monitoring script corresponding to each intelligent device through a logic topology of the corresponding relation, and adding a characteristic key corresponding to the text stream conversion matrix to the data stream monitoring script; wherein the feature key uniquely corresponds to the smart device.
A3 the apparatus of a1, wherein the record parsing module is specifically configured to:
importing the data transmission record into a preset record analysis thread;
judging whether the thread parameter of the record analysis thread has an updating identifier, if not, updating the thread parameter based on the transmission link parameter corresponding to the data transmission record; wherein, the thread parameter after completing updating has an updating mark;
and when the thread parameter has the updating identifier, starting the record analysis thread to output a data operation list of target data corresponding to the data transmission record and a data transmission path.
A4. The apparatus of a1, the troubleshooting module, in particular, being configured to:
extracting instruction stream information of the fault instruction, determining equipment registration information from the instruction stream information, and determining the target intelligent equipment from a preset information base according to the equipment registration information; extracting a signature key of the target intelligent device based on an api interface of the target intelligent device;
marking first graph data with the signature key and second graph data of a target key with a correlation coefficient larger than a set value;
generating a target data set corresponding to the target smart device based on the first graph data and the second graph data.
A5. The apparatus of a1, the map generation module is specifically configured to:
acquiring result data corresponding to the target data generated according to the data operation list, and determining the service forms of the result data in a plurality of downstream intelligent devices through communication addresses and transmission identifiers obtained by extracting the data transmission paths;
performing nodularization on the description information corresponding to the plurality of service forms according to the path parameters corresponding to the data transmission path to obtain service nodes corresponding to each service form;
and encapsulating the execution data of the result data in the downstream intelligent equipment corresponding to each service node into the corresponding service node, and connecting a plurality of service nodes according to the transmission identifier to obtain the data flow distribution map.
A6. The apparatus of a5, the atlas generation module, further to:
listing the path priority corresponding to the path parameters of the data transmission path, generating a parameter network corresponding to the path parameters according to the path priority, and determining a time sequence difference coefficient between each information field of the corresponding description information of each service form under the received information category and each information field of the corresponding description information of each service form under the sent information category according to the information field of the corresponding description information of each service form under the corresponding sent information category and the field coding of the information field under the condition that the corresponding description information of each service form contains the sent information category according to the parameter network;
dividing the information field of the corresponding description information of each traffic form under the received information category and the information field under the sent information category which has continuity in time sequence into the corresponding sent information category based on the time sequence difference coefficient;
under the condition that a plurality of information fields are contained in the received information category corresponding to the corresponding description information of each service form, determining a time sequence difference coefficient between the information fields of the corresponding description information of each service form in the received information category according to the information fields of the corresponding description information of each service form in the sent information category and the field codes of the information fields; screening the information fields under the received information category through the time sequence difference coefficient among the information fields to obtain target information fields; dividing part of target information fields obtained by screening into the sent information categories according to the information fields of the corresponding description information of each service form under the sent information categories and the field codes of the information fields;
determining the node logic information of the corresponding description information of each service form in the information field of the corresponding sent information type according to the corresponding description information of each service form;
and according to the logic sequence extracted from the nodularization logic information, performing information compression on the corresponding description information of each service form in the information field of the corresponding sent information type once to obtain an information characteristic value, and integrating the information characteristic value to obtain a service node corresponding to each service form.
A7. The apparatus of a5, the atlas generation module, further to:
determining device configuration information used for representing a data processing process of downstream intelligent devices corresponding to each service node, and extracting target configuration information which does not change along with the change of the time slice resource occupancy rate of the downstream intelligent devices in the device configuration information; judging whether the information coding format of the target configuration information is consistent with the data coding format of the result data; when the information coding format is judged to be inconsistent with the data coding format, generating a format label corresponding to the information coding format and loading the format label into a corresponding service node, and when the information coding format is judged to be consistent with the data coding format, generating a data splitting identifier according to the data coding format and loading the data splitting identifier into the corresponding service node;
if the format label exists in the service node, constructing a data packaging list of the service node based on compatibility distribution information corresponding to the format label; if the data splitting identifier exists in the service node, mapping the data splitting identifier to a node container corresponding to the service node and generating a data packaging list of the service node based on the mapping identifier of the data splitting identifier in the node container;
determining, for a downstream intelligent device corresponding to each service node, cache address information in which a data index value corresponding to the result data exists from a database of the downstream intelligent device, determining corresponding original data from the result data according to the cache address information, and integrating execution data corresponding to the result data in the database and the original data determined from the result data to obtain data to be encapsulated;
determining the encapsulation priority of each data segment in the data to be encapsulated, sequencing the data segments according to the sequence of the encapsulation priorities from large to small to obtain a data segment sequencing sequence, and sequentially encapsulating the data segments in the data segment sequencing sequence into corresponding service nodes according to the data encapsulation list.
Further, a data processing system based on edge calculation and path analysis is also provided, which is described in detail below.
B1. A data processing system based on edge computing and path analysis, comprising a big data cloud platform and a plurality of smart devices in communication with each other, the big data cloud platform being configured to:
generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device, and issuing the data stream monitoring script to the corresponding intelligent device;
acquiring a data transmission record uploaded by a data stream monitoring script corresponding to each intelligent device, and analyzing the data transmission record to obtain a data operation list of target data corresponding to the data transmission record and a data transmission path;
determining a data flow distribution diagram of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and performing correlation storage on each data flow distribution diagram to obtain a production data diagram of the intelligent production system formed by the plurality of intelligent devices;
when a fault instruction reported by fault monitoring equipment corresponding to the intelligent production system is received, determining target intelligent equipment with a fault according to the fault instruction, and finding out a target data set corresponding to the target intelligent equipment in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data.
B2. The system of B1, the big data cloud platform is specifically configured to:
respectively extracting a category text set corresponding to the equipment category information and an interface text set corresponding to the equipment interface information; the interface text set comprises a category text set and an interface text set, wherein the category text set and the interface text set respectively comprise a plurality of text files with different text weight values;
judging whether the number of the first text files corresponding to the category text set is the same as the number of the second text files corresponding to the interface text set; if not, determining a first average value of the text weight values corresponding to the first text file and a second average value of the text weight values corresponding to the second text file; when the first average value is larger than the second average value, merging the second text files by taking the number of the first text files as a reference so that the number of the second text files after merging is the same as the number of the first text files; when the second average value is larger than the first average value, merging the first text files by taking the number of the second text files as a reference so that the number of the first text files after merging is the same as the number of the second text files;
under the condition that the number of the first text files is the same as that of the second text files, extracting code stream information of one first text file in the category text set, and determining the second text file with the largest text weight value in the interface text set as a reference text file in parallel; mapping the code stream information to the reference text file to obtain mirror image code stream information, and generating a text stream conversion matrix between the category text set and the interface text set according to configuration weight between the code stream information and the mirror image code stream information;
determining a corresponding relation between each first text file in the category file set and a second text file in the interface text set according to the text stream conversion matrix, generating a data stream monitoring script corresponding to each intelligent device through a logic topology of the corresponding relation, and adding a characteristic key corresponding to the text stream conversion matrix to the data stream monitoring script; wherein the feature key uniquely corresponds to the smart device.
B3 the system of B1, the big data cloud platform being configured to:
importing the data transmission record into a preset record analysis thread;
judging whether the thread parameter of the record analysis thread has an updating identifier, if not, updating the thread parameter based on the transmission link parameter corresponding to the data transmission record; wherein, the thread parameter after completing updating has an updating mark;
and when the thread parameter has the updating identifier, starting the record analysis thread to output a data operation list of target data corresponding to the data transmission record and a data transmission path.
B4. The system of B1, the big data cloud platform is specifically configured to:
extracting instruction stream information of the fault instruction, determining equipment registration information from the instruction stream information, and determining the target intelligent equipment from a preset information base according to the equipment registration information; extracting a signature key of the target intelligent device based on an api interface of the target intelligent device;
marking first graph data with the signature key and second graph data of a target key with a correlation coefficient larger than a set value;
generating a target data set corresponding to the target smart device based on the first graph data and the second graph data.
B5. The system of B1, the big data cloud platform is specifically configured to:
acquiring result data corresponding to the target data generated according to the data operation list, and determining the service forms of the result data in a plurality of downstream intelligent devices through communication addresses and transmission identifiers obtained by extracting the data transmission paths;
performing nodularization on the description information corresponding to the plurality of service forms according to the path parameters corresponding to the data transmission path to obtain service nodes corresponding to each service form;
and encapsulating the execution data of the result data in the downstream intelligent equipment corresponding to each service node into the corresponding service node, and connecting a plurality of service nodes according to the transmission identifier to obtain the data flow distribution map.
B6. The system of B5, the big data cloud platform is specifically configured to:
listing the path priority corresponding to the path parameters of the data transmission path, generating a parameter network corresponding to the path parameters according to the path priority, and determining a time sequence difference coefficient between each information field of the corresponding description information of each service form under the received information category and each information field of the corresponding description information of each service form under the sent information category according to the information field of the corresponding description information of each service form under the corresponding sent information category and the field coding of the information field under the condition that the corresponding description information of each service form contains the sent information category according to the parameter network;
dividing the information field of the corresponding description information of each traffic form under the received information category and the information field under the sent information category which has continuity in time sequence into the corresponding sent information category based on the time sequence difference coefficient;
under the condition that a plurality of information fields are contained in the received information category corresponding to the corresponding description information of each service form, determining a time sequence difference coefficient between the information fields of the corresponding description information of each service form in the received information category according to the information fields of the corresponding description information of each service form in the sent information category and the field codes of the information fields; screening the information fields under the received information category through the time sequence difference coefficient among the information fields to obtain target information fields; dividing part of target information fields obtained by screening into the sent information categories according to the information fields of the corresponding description information of each service form under the sent information categories and the field codes of the information fields;
determining the node logic information of the corresponding description information of each service form in the information field of the corresponding sent information type according to the corresponding description information of each service form;
and according to the logic sequence extracted from the nodularization logic information, performing information compression on the corresponding description information of each service form in the information field of the corresponding sent information type once to obtain an information characteristic value, and integrating the information characteristic value to obtain a service node corresponding to each service form.
B7. The system of B5, the big data cloud platform is specifically configured to:
determining device configuration information used for representing a data processing process of downstream intelligent devices corresponding to each service node, and extracting target configuration information which does not change along with the change of the time slice resource occupancy rate of the downstream intelligent devices in the device configuration information; judging whether the information coding format of the target configuration information is consistent with the data coding format of the result data; when the information coding format is judged to be inconsistent with the data coding format, generating a format label corresponding to the information coding format and loading the format label into a corresponding service node, and when the information coding format is judged to be consistent with the data coding format, generating a data splitting identifier according to the data coding format and loading the data splitting identifier into the corresponding service node;
if the format label exists in the service node, constructing a data packaging list of the service node based on compatibility distribution information corresponding to the format label; if the data splitting identifier exists in the service node, mapping the data splitting identifier to a node container corresponding to the service node and generating a data packaging list of the service node based on the mapping identifier of the data splitting identifier in the node container;
determining, for a downstream intelligent device corresponding to each service node, cache address information in which a data index value corresponding to the result data exists from a database of the downstream intelligent device, determining corresponding original data from the result data according to the cache address information, and integrating execution data corresponding to the result data in the database and the original data determined from the result data to obtain data to be encapsulated;
determining the encapsulation priority of each data segment in the data to be encapsulated, sequencing the data segments according to the sequence of the encapsulation priorities from large to small to obtain a data segment sequencing sequence, and sequentially encapsulating the data segments in the data segment sequencing sequence into corresponding service nodes according to the data encapsulation list.
On the basis of the above, please refer to fig. 4 in combination, a hardware structure diagram of a big data cloud platform 200 is provided, where the big data cloud platform 200 includes a processor 270 and a memory 280, which are in communication with each other, and the processor 270 implements the method shown in fig. 2 by running a computer program called from the memory 280.
Further, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when executed, implements the method shown in fig. 2.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
Claims (6)
1. A data processing method based on edge computing and path analysis is applied to a big data cloud platform which is in communication connection with a plurality of intelligent devices, and comprises the following steps:
generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device, and issuing the data stream monitoring script to the corresponding intelligent device;
acquiring a data transmission record uploaded by a data stream monitoring script corresponding to each intelligent device, and analyzing the data transmission record to obtain a data operation list of target data corresponding to the data transmission record and a data transmission path;
determining a data flow distribution diagram of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and performing correlation storage on each data flow distribution diagram to obtain a production data diagram of the intelligent production system formed by the plurality of intelligent devices;
when a fault instruction reported by fault monitoring equipment corresponding to the intelligent production system is received, determining target intelligent equipment with a fault according to the fault instruction, and finding out a target data set corresponding to the target intelligent equipment in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data;
determining a data flow distribution diagram of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, wherein the data flow distribution diagram comprises: acquiring result data corresponding to the target data generated according to the data operation list, and determining the service forms of the result data in a plurality of downstream intelligent devices through communication addresses and transmission identifiers obtained by extracting the data transmission paths; performing nodularization on the description information corresponding to the plurality of service forms according to the path parameters corresponding to the data transmission path to obtain service nodes corresponding to each service form; encapsulating the execution data of the result data in the downstream intelligent device corresponding to each service node into the corresponding service node, and connecting a plurality of service nodes according to the transmission identifier to obtain the data flow distribution map;
wherein, the nodularizing the description information corresponding to a plurality of service forms according to the path parameters corresponding to the data transmission path to obtain the service node corresponding to each service form, further comprising: listing the path priority corresponding to the path parameters of the data transmission path, generating a parameter network corresponding to the path parameters according to the path priority, and determining a time sequence difference coefficient between each information field of the corresponding description information of each service form under the received information category and each information field of the corresponding description information of each service form under the sent information category according to the information field of the corresponding description information of each service form under the corresponding sent information category and the field coding of the information field under the condition that the corresponding description information of each service form contains the sent information category according to the parameter network; dividing the information field of the corresponding description information of each traffic form under the received information category and the information field under the sent information category which has continuity in time sequence into the corresponding sent information category based on the time sequence difference coefficient; under the condition that a plurality of information fields are contained in the received information category corresponding to the corresponding description information of each service form, determining a time sequence difference coefficient between the information fields of the corresponding description information of each service form in the received information category according to the information fields of the corresponding description information of each service form in the sent information category and the field codes of the information fields; screening the information fields under the received information category through the time sequence difference coefficient among the information fields to obtain target information fields; dividing part of target information fields obtained by screening into the sent information categories according to the information fields of the corresponding description information of each service form under the sent information categories and the field codes of the information fields; determining the node logic information of the corresponding description information of each service form in the information field of the corresponding sent information type according to the corresponding description information of each service form; and according to the logic sequence extracted from the nodularization logic information, performing information compression on the corresponding description information of each service form in the information field of the corresponding sent information type once to obtain an information characteristic value, and integrating the information characteristic value to obtain a service node corresponding to each service form.
2. The method of claim 1, wherein determining a target smart device with a fault according to the fault instruction, and finding a target data set corresponding to the target smart device in the production data map comprises:
extracting instruction stream information of the fault instruction, determining equipment registration information from the instruction stream information, and determining the target intelligent equipment from a preset information base according to the equipment registration information; extracting a signature key of the target intelligent device based on an api interface of the target intelligent device;
marking first graph data with the signature key and second graph data of a target key with a correlation coefficient larger than a set value;
generating a target data set corresponding to the target smart device based on the first graph data and the second graph data.
3. The method of claim 1, wherein encapsulating the execution data of the result data in the downstream intelligent device corresponding to each service node into the corresponding service node comprises:
determining device configuration information used for representing a data processing process of downstream intelligent devices corresponding to each service node, and extracting target configuration information which does not change along with the change of the time slice resource occupancy rate of the downstream intelligent devices in the device configuration information; judging whether the information coding format of the target configuration information is consistent with the data coding format of the result data; when the information coding format is judged to be inconsistent with the data coding format, generating a format label corresponding to the information coding format and loading the format label into a corresponding service node, and when the information coding format is judged to be consistent with the data coding format, generating a data splitting identifier according to the data coding format and loading the data splitting identifier into the corresponding service node;
if the format label exists in the service node, constructing a data packaging list of the service node based on compatibility distribution information corresponding to the format label; if the data splitting identifier exists in the service node, mapping the data splitting identifier to a node container corresponding to the service node and generating a data packaging list of the service node based on the mapping identifier of the data splitting identifier in the node container;
determining, for a downstream intelligent device corresponding to each service node, cache address information in which a data index value corresponding to the result data exists from a database of the downstream intelligent device, determining corresponding original data from the result data according to the cache address information, and integrating execution data corresponding to the result data in the database and the original data determined from the result data to obtain data to be encapsulated;
determining the encapsulation priority of each data segment in the data to be encapsulated, sequencing the data segments according to the sequence of the encapsulation priorities from large to small to obtain a data segment sequencing sequence, and sequentially encapsulating the data segments in the data segment sequencing sequence into corresponding service nodes according to the data encapsulation list.
4. A big data cloud platform for performing the method of any one of claims 1 to 3, wherein the big data cloud platform is communicatively connected to a plurality of smart devices, and the big data cloud platform comprises the following functional modules:
the script issuing module is used for generating a data stream monitoring script corresponding to each intelligent device according to the device type information and the device interface information of each intelligent device and issuing the data stream monitoring script to the corresponding intelligent device;
the record analysis module is used for acquiring data transmission records uploaded by the data stream monitoring scripts corresponding to each intelligent device, and analyzing the data transmission records to obtain a data operation list of target data corresponding to the data transmission records and a data transmission path;
the map generation module is used for determining a data flow distribution map of each group of target data among the plurality of intelligent devices based on a data operation list corresponding to each group of target data and a data transmission path, and performing correlation storage on each data flow distribution map to obtain a production data map of the intelligent production system formed by the plurality of intelligent devices;
the fault troubleshooting module is used for determining target intelligent equipment with faults according to the fault instruction when the fault instruction reported by the fault monitoring equipment corresponding to the intelligent production system is received, and finding out a target data set corresponding to the target intelligent equipment in the production data map; the target data set is used for troubleshooting the target intelligent device, and the target data set comprises at least one group of service data processed by the target intelligent device and data flow direction information of the service data;
the map generation module is specifically configured to:
acquiring result data corresponding to the target data generated according to the data operation list, and determining the service forms of the result data in a plurality of downstream intelligent devices through communication addresses and transmission identifiers obtained by extracting the data transmission paths; performing nodularization on the description information corresponding to the plurality of service forms according to the path parameters corresponding to the data transmission path to obtain service nodes corresponding to each service form; encapsulating the execution data of the result data in the downstream intelligent device corresponding to each service node into the corresponding service node, and connecting a plurality of service nodes according to the transmission identifier to obtain the data flow distribution map;
wherein the map generation module is further configured to:
listing the path priority corresponding to the path parameters of the data transmission path, generating a parameter network corresponding to the path parameters according to the path priority, and determining a time sequence difference coefficient between each information field of the corresponding description information of each service form under the received information category and each information field of the corresponding description information of each service form under the sent information category according to the information field of the corresponding description information of each service form under the corresponding sent information category and the field coding of the information field under the condition that the corresponding description information of each service form contains the sent information category according to the parameter network; dividing the information field of the corresponding description information of each traffic form under the received information category and the information field under the sent information category which has continuity in time sequence into the corresponding sent information category based on the time sequence difference coefficient; under the condition that a plurality of information fields are contained in the received information category corresponding to the corresponding description information of each service form, determining a time sequence difference coefficient between the information fields of the corresponding description information of each service form in the received information category according to the information fields of the corresponding description information of each service form in the sent information category and the field codes of the information fields; screening the information fields under the received information category through the time sequence difference coefficient among the information fields to obtain target information fields; dividing part of target information fields obtained by screening into the sent information categories according to the information fields of the corresponding description information of each service form under the sent information categories and the field codes of the information fields; determining the node logic information of the corresponding description information of each service form in the information field of the corresponding sent information type according to the corresponding description information of each service form; and according to the logic sequence extracted from the nodularization logic information, performing information compression on the corresponding description information of each service form in the information field of the corresponding sent information type once to obtain an information characteristic value, and integrating the information characteristic value to obtain a service node corresponding to each service form.
5. The big data cloud platform of claim 4, wherein the troubleshooting module is specifically configured to:
extracting instruction stream information of the fault instruction, determining equipment registration information from the instruction stream information, and determining the target intelligent equipment from a preset information base according to the equipment registration information; extracting a signature key of the target intelligent device based on an api interface of the target intelligent device;
marking first graph data with the signature key and second graph data of a target key with a correlation coefficient larger than a set value;
generating a target data set corresponding to the target smart device based on the first graph data and the second graph data.
6. The big data cloud platform of claim 4, wherein the graph generation module is further to:
determining device configuration information used for representing a data processing process of downstream intelligent devices corresponding to each service node, and extracting target configuration information which does not change along with the change of the time slice resource occupancy rate of the downstream intelligent devices in the device configuration information; judging whether the information coding format of the target configuration information is consistent with the data coding format of the result data; when the information coding format is judged to be inconsistent with the data coding format, generating a format label corresponding to the information coding format and loading the format label into a corresponding service node, and when the information coding format is judged to be consistent with the data coding format, generating a data splitting identifier according to the data coding format and loading the data splitting identifier into the corresponding service node;
if the format label exists in the service node, constructing a data packaging list of the service node based on compatibility distribution information corresponding to the format label; if the data splitting identifier exists in the service node, mapping the data splitting identifier to a node container corresponding to the service node and generating a data packaging list of the service node based on the mapping identifier of the data splitting identifier in the node container;
determining, for a downstream intelligent device corresponding to each service node, cache address information in which a data index value corresponding to the result data exists from a database of the downstream intelligent device, determining corresponding original data from the result data according to the cache address information, and integrating execution data corresponding to the result data in the database and the original data determined from the result data to obtain data to be encapsulated;
determining the encapsulation priority of each data segment in the data to be encapsulated, sequencing the data segments according to the sequence of the encapsulation priorities from large to small to obtain a data segment sequencing sequence, and sequentially encapsulating the data segments in the data segment sequencing sequence into corresponding service nodes according to the data encapsulation list.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110094546.8A CN112765217A (en) | 2020-07-14 | 2020-07-14 | Data processing method and system based on edge calculation and path analysis |
| CN202010675996.1A CN111881164B (en) | 2020-07-14 | 2020-07-14 | Data processing method based on edge computing and path analysis and big data cloud platform |
| CN202110091899.2A CN112800090A (en) | 2020-07-14 | 2020-07-14 | Data processing method combining edge computing and path analysis and big data cloud platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010675996.1A CN111881164B (en) | 2020-07-14 | 2020-07-14 | Data processing method based on edge computing and path analysis and big data cloud platform |
Related Child Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110091899.2A Division CN112800090A (en) | 2020-07-14 | 2020-07-14 | Data processing method combining edge computing and path analysis and big data cloud platform |
| CN202110094546.8A Division CN112765217A (en) | 2020-07-14 | 2020-07-14 | Data processing method and system based on edge calculation and path analysis |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111881164A CN111881164A (en) | 2020-11-03 |
| CN111881164B true CN111881164B (en) | 2021-03-23 |
Family
ID=73151440
Family Applications (3)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010675996.1A Active CN111881164B (en) | 2020-07-14 | 2020-07-14 | Data processing method based on edge computing and path analysis and big data cloud platform |
| CN202110091899.2A Withdrawn CN112800090A (en) | 2020-07-14 | 2020-07-14 | Data processing method combining edge computing and path analysis and big data cloud platform |
| CN202110094546.8A Withdrawn CN112765217A (en) | 2020-07-14 | 2020-07-14 | Data processing method and system based on edge calculation and path analysis |
Family Applications After (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110091899.2A Withdrawn CN112800090A (en) | 2020-07-14 | 2020-07-14 | Data processing method combining edge computing and path analysis and big data cloud platform |
| CN202110094546.8A Withdrawn CN112765217A (en) | 2020-07-14 | 2020-07-14 | Data processing method and system based on edge calculation and path analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (3) | CN111881164B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113515367B (en) * | 2020-08-23 | 2022-08-30 | 浪潮工业互联网股份有限公司 | Data integration method based on big data and edge calculation and storage medium |
| CN113645075A (en) * | 2021-08-11 | 2021-11-12 | 杭州安恒信息技术股份有限公司 | Network link fault positioning method, device, equipment and readable storage medium |
| CN116651306B (en) * | 2023-08-01 | 2023-10-03 | 山西中科冶金建设有限公司 | Intelligent coking coal proportioning system |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101170454B (en) * | 2006-10-27 | 2010-05-12 | 亿阳信通股份有限公司 | A method and system for monitoring data collection and summary status |
| CN101640612B (en) * | 2009-09-07 | 2011-12-07 | 杭州华三通信技术有限公司 | Method and device for flow path discovery and fault fast positioning |
| WO2013145493A1 (en) * | 2012-03-30 | 2013-10-03 | 日本電気株式会社 | Pipeline administration assistance device and pipeline administration assistance system |
| CN106774240B (en) * | 2016-11-25 | 2019-12-10 | 华南理工大学 | service-oriented industrial production control and monitoring method and system |
-
2020
- 2020-07-14 CN CN202010675996.1A patent/CN111881164B/en active Active
- 2020-07-14 CN CN202110091899.2A patent/CN112800090A/en not_active Withdrawn
- 2020-07-14 CN CN202110094546.8A patent/CN112765217A/en not_active Withdrawn
Also Published As
| Publication number | Publication date |
|---|---|
| CN111881164A (en) | 2020-11-03 |
| CN112765217A (en) | 2021-05-07 |
| CN112800090A (en) | 2021-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111881164B (en) | Data processing method based on edge computing and path analysis and big data cloud platform | |
| CN115174231B (en) | Network fraud analysis method and server based on AI Knowledge Base | |
| CN113590451B (en) | Root cause positioning method, operation and maintenance server and storage medium | |
| CN114500690B (en) | Interface data processing method and device, electronic equipment and storage medium | |
| CN116415206B (en) | Operator multiple data fusion method, system, electronic equipment and computer storage medium | |
| CN111339363B (en) | Image recognition method and device and server | |
| CN111338876B (en) | Decision method, system and storage medium for fault mode and influence analysis | |
| CN116414948A (en) | Abnormal data mining method and software product based on cloud data and artificial intelligence | |
| CN111355696A (en) | Message identification method and device, DPI (deep packet inspection) equipment and storage medium | |
| CN114153980A (en) | Knowledge graph construction method and device, inspection method and storage medium | |
| CN112070508B (en) | Block chain payment processing method based on block chain finance and block chain payment platform | |
| CN113821418A (en) | Fault tracking analysis method and device, storage medium and electronic equipment | |
| CN111913824A (en) | Method for determining data link fault reason and related equipment | |
| CN117251563A (en) | Quality inspection method, equipment and storage medium for fault worksheets | |
| CN113923268B (en) | Resolution method, device and storage medium for multi-version communication protocol | |
| CN111949720B (en) | Data analysis method based on big data and artificial intelligence and cloud data server | |
| CN111740494B (en) | Data management method based on edge computing and cloud computing and edge computing platform | |
| CN111459104B (en) | Data tracking method based on industrial Internet and electronic equipment | |
| CN117632659A (en) | Log exception processing method, device, equipment and medium | |
| CN111193631A (en) | Information processing method, system, and computer-readable storage medium | |
| CN111784319B (en) | Communication data processing method based on block chain payment network and big data server | |
| CN115842861A (en) | Edge connection device adaptation method, device and computer readable storage medium | |
| WO2024012186A1 (en) | Root cause positioning method, and communication device and computer-readable storage medium | |
| CN111917847A (en) | Information processing method based on 5G and artificial intelligence and big data cloud computing center | |
| CN114598588B (en) | Server fault determination method and device and terminal equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Li Daqing Inventor after: Yuan Yuan Inventor before: Yuan Yuan |
|
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210304 Address after: No. 888, Yuexiu Road, Lili Town, Wujiang District, Suzhou City, Jiangsu Province Applicant after: JIANGSU YONGDING COMMUNICATIONS Co.,Ltd. Address before: 655000 Xicheng Industrial Park, Qujing Economic and Technological Development Zone, Yunnan Province Applicant before: Yuan Yuan |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |