CN117349256A - File anticorrosion method, device and system based on distributed platform - Google Patents
File anticorrosion method, device and system based on distributed platform Download PDFInfo
- Publication number
- CN117349256A CN117349256A CN202311540824.3A CN202311540824A CN117349256A CN 117349256 A CN117349256 A CN 117349256A CN 202311540824 A CN202311540824 A CN 202311540824A CN 117349256 A CN117349256 A CN 117349256A
- Authority
- CN
- China
- Prior art keywords
- file
- platform
- host
- check
- operation stage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000005260 corrosion Methods 0.000 claims abstract description 62
- 238000013515 script Methods 0.000 claims abstract description 18
- 238000004590 computer program Methods 0.000 claims description 30
- 238000003860 storage Methods 0.000 claims description 25
- 238000004321 preservation Methods 0.000 claims description 21
- 239000003755 preservative agent Substances 0.000 claims description 16
- 230000002335 preservative effect Effects 0.000 claims description 16
- 238000004519 manufacturing process Methods 0.000 abstract description 10
- 238000012545 processing Methods 0.000 description 36
- 238000010586 diagram Methods 0.000 description 21
- 230000006870 function Effects 0.000 description 18
- 230000008569 process Effects 0.000 description 11
- 238000004891 communication Methods 0.000 description 10
- 230000010354 integration Effects 0.000 description 10
- 238000012795 verification Methods 0.000 description 8
- 239000000872 buffer Substances 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 238000013500 data storage Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 230000007797 corrosion Effects 0.000 description 3
- 238000005536 corrosion prevention Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000009826 distribution Methods 0.000 description 3
- 238000003780 insertion Methods 0.000 description 2
- 230000037431 insertion Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012958 reprocessing Methods 0.000 description 2
- 108010001267 Protein Subunits Proteins 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000032683 aging Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013475 authorization Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000007795 chemical reaction product Substances 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000011031 large-scale manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 239000000047 product Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000011144 upstream manufacturing Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/182—Distributed file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/16—File or folder operations, e.g. details of user interfaces specifically adapted to file systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/10—File systems; File servers
- G06F16/18—File system types
- G06F16/1805—Append-only file systems, e.g. using logs or journals to store data
- G06F16/1815—Journaling file systems
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a file anti-corrosion method, device and system based on a distributed platform, relates to the technical field of cloud computing, and can be applied to the technical field of finance or other technical fields. The file anti-corrosion method based on the distributed platform comprises the following steps: acquiring a platform file, a host file and an operation stage; generating a check file according to the platform file, the host file and the operation stage, and selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file; pushing the anti-corrosion file to downstream applications. The invention can effectively reduce the workload of downstream replacement and the error probability of large-scale replacement logic, and reduce the production accidents.
Description
Technical Field
The invention relates to the technical field of cloud computing, in particular to a file anti-corrosion method, device and system based on a distributed platform.
Background
Technical proposal of the prior art
With the popularization of the distributed platform technology in banks, data originally provided by host data alone is provided by each distributed platform, and the original data is provided by one provider and is changed into a plurality of to tens of providers.
Disadvantages of the prior art
Firstly, each platform has extremely large time difference, and secondly, data are too scattered and usability is extremely poor. The advent of multiple data providers has resulted in extensive adjustment and switching effort for the document user, and even large-scale production incidents due to logic errors caused by too many data sources.
Disclosure of Invention
The embodiment of the invention mainly aims to provide a file anti-corrosion method, device and system based on a distributed platform, so as to shield the timeliness and structural difference of data exchange among multiple platforms, effectively reduce the workload of downstream replacement and the error probability of large-scale replacement logic, and reduce the production accidents.
In order to achieve the above object, an embodiment of the present invention provides a file preservation method based on a distributed platform, including:
acquiring a platform file, a host file and an operation stage;
generating a check file according to the platform file, the host file and the operation stage;
selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
pushing the anti-corrosion file to downstream applications.
In one embodiment, generating the collation file from the platform file, the host file and the run phase comprises:
Comparing the platform file with the host file to generate a comparison result;
and generating the check file according to the comparison result and the operation stage.
In one embodiment, the collation file comprises a platform collation file and a host collation file;
generating the collation file according to the comparison result and the operation stage includes:
determining the number of the platform check files and the number of the host check files according to the comparison result and the operation stage;
and determining the platform check file according to the number of the platform files and the platform check files, and determining the host check file according to the number of the host files and the host check files.
In one embodiment, comparing the platform file and the host file, the generating a comparison result includes:
determining a comparison mode according to the structure of the platform file and the structure of the host file;
and comparing the platform file with the host file according to the comparison mode to generate the comparison result.
In one embodiment, the method further comprises:
receiving a customer service instruction, and inserting a corresponding record in a log according to the transaction type in the customer service instruction;
And checking the client business instruction, and generating the platform file according to the checking result and the record.
The embodiment of the invention also provides a file anti-corrosion device based on the distributed platform, which comprises:
the acquisition module is used for acquiring the platform file, the host file and the operation stage;
the check file module is used for generating a check file according to the platform file, the host file and the operation stage;
the anticorrosion module is used for selecting corresponding anticorrosion scripts to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
and the file pushing module is used for pushing the anti-corrosion file to the downstream application.
In one embodiment, the collation file module comprises:
the comparison unit is used for comparing the platform file with the host file to generate a comparison result;
and the check file unit is used for generating the check file according to the comparison result and the operation stage.
In one embodiment, the collation file comprises a platform collation file and a host collation file;
the collation file unit includes:
the verification document quantity subunit is used for determining the quantity of the platform verification documents and the quantity of the host verification documents according to the comparison result and the operation stage;
And the checking file subunit is used for determining the platform checking file according to the number of the platform files and the platform checking files and determining the host checking file according to the number of the host files and the host checking files.
In one embodiment, the alignment unit includes:
the comparison method subunit is used for determining a comparison method according to the structure of the platform file and the structure of the host file;
and the comparison result subunit is used for comparing the platform file and the host file according to the comparison mode to generate the comparison result.
In one embodiment, the method further comprises:
the record insertion module is used for receiving a customer service instruction and inserting a corresponding record into the log according to the transaction type in the customer service instruction;
and the platform file generation module is used for checking the client business instruction and generating the platform file according to the checking result and the record.
The embodiment of the invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and running on the processor, wherein the steps of the file anti-corrosion method based on the distributed platform are realized when the processor executes the computer program.
The embodiment of the invention also provides a computer readable storage medium, on which a computer program is stored, which when being executed by a processor, realizes the steps of the file preservation method based on the distributed platform.
The embodiment of the invention also provides a computer program product, which comprises a computer program/instruction, wherein the computer program/instruction realizes the steps of the file preservation method based on the distributed platform when being executed by a processor.
The embodiment of the invention also provides a file anti-corrosion system based on the distributed platform, which comprises the following steps:
the file anti-corrosion device based on the distributed platform is applied to the platform;
the host is used for sending host files to the file preservative device;
and the downstream application is used for receiving the anti-corrosion file pushed by the file anti-corrosion device.
According to the file anti-corrosion method, device and system based on the distributed platform, the verification file is generated according to the platform file, the host computer file and the operation stage, the corresponding anti-corrosion script is selected according to the operation stage to carry out anti-corrosion on the verification file, and finally the generated anti-corrosion file is pushed to downstream application, so that the workload of downstream replacement and the error probability of large-scale replacement logic can be effectively reduced, and production accidents are reduced.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a distributed platform based file preservation method in an embodiment of the invention;
FIG. 2 is a schematic diagram of preservative treatment logic in an embodiment of the invention;
FIG. 3 is a flow chart of generating a platform file in an embodiment of the invention;
FIG. 4 is a flowchart of S102 in an embodiment of the invention;
FIG. 5 is a flowchart of S201 in an embodiment of the present invention;
FIG. 6 is a flowchart of S202 in an embodiment of the invention;
FIG. 7 is a block diagram of a distributed platform based file preservative device in an embodiment of the invention;
FIG. 8 is a block diagram of a data integration module in an embodiment of the invention;
FIG. 9 is a block diagram of an open platform server cluster in accordance with an embodiment of the present invention;
FIG. 10 is a block diagram of the structure of a data lake in an embodiment of the invention;
FIG. 11 is a schematic block diagram of a system configuration of an electronic device 9600 of an embodiment of the present application;
FIG. 12 is a distributed platform based file preservative system in an embodiment of the invention;
FIG. 13 is a block diagram of a host in an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Those skilled in the art will appreciate that embodiments of the invention may be implemented as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the following forms, namely: complete hardware, complete software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.
The technical scheme of the invention obtains, stores, uses, processes and the like the data, which all meet the relevant regulations of national laws and regulations. The user information in the embodiment of the application is obtained through legal compliance approaches, and the user information is obtained, stored, used, processed and the like through client authorization and consent.
The invention can shield the timeliness and structural difference of data exchange among multiple platforms, thereby improving the problem of frequent modification of downstream data users. The invention changes the original data from many-to-many to many downstream use into many-to-many distribution based on the HADOOP, firstly, the source pasting mode of the multi-data sources is carried out to the HADOOP, and then a corrosion-resistant layer is established after each multi-data source through a certain logic (the logic is sealed in each data source). The data of the anti-corrosion layer is the same as the single data of the original host. The anti-corrosion layer realizes synchronization of data sources to the layer in a real-time and quasi-real-time mode, and ensures that timeliness and structural performance of the anti-corrosion layer are consistent with original single data sources in dimension design, record uniqueness, attribute design and extension attribute design in a source pasting mode; and meanwhile, the file is distributed by the anticorrosive layer in the same way as the single data source of the original host by using the extended recording time field. The logic of the file at the application end is unchanged, the content of the file is unchanged, the modification of a downstream data user can be shielded, only the file replacement is needed, the use efficiency of the file user is improved, and flexible adjustment according to supervision needs is supported.
Fig. 1 is a flowchart of a file preservation method based on a distributed platform in an embodiment of the present invention. Fig. 2 is a schematic diagram of the preservative treatment logic in an embodiment of the invention. As shown in fig. 1-2, the file preservation method based on the distributed platform comprises the following steps:
s101: platform files, host files and run phases are obtained.
FIG. 3 is a flow chart of generating a platform file in an embodiment of the invention. As shown in fig. 3, the file preservation method based on the distributed platform further includes:
s201: and receiving a customer service instruction, and inserting a corresponding record into a log according to the transaction type in the customer service instruction.
S202: and checking the client business instruction, and generating the platform file according to the checking result and the record.
In specific implementation, transaction elements in the platform file are basic lake entering data and anti-corrosion elements which are displayed and supervised according to a downstream report/detail of banking business. When the check result is correct, the record is updated, and the state of the updated record is an intermediate state. When the business is completed or closed-loop, the record is updated again, and the state of the updated record is the final state, and the platform file is generated according to the updated record.
S102: and generating a check file according to the platform file, the host file and the operation stage.
Fig. 4 is a flowchart of S102 in the embodiment of the present invention. As shown in fig. 4, S102 includes:
s301: and comparing the platform file with the host file to generate a comparison result.
Fig. 5 is a flowchart of S201 in the embodiment of the present invention. As shown in fig. 5, S201 includes:
s401: and determining a comparison mode according to the structure of the platform file and the structure of the host file.
S402: and comparing the platform file with the host file according to the comparison mode to generate the comparison result.
In the specific implementation, based on the downstream use logic and file operation principle of lake table data, the banking transaction basic elements including the amount, time, transaction information, places and the like are subjected to one-by-one comparison, and the recorded states are subjected to one-by-one comparison.
When the host lake table (host file) and the platform lake table (platform file) are consistent in structure, the platform file and the host file are compared one by adopting the full table field. When the structures of the host lake table (host file) and the platform lake table (platform file) are inconsistent, the platform file and the host file are compared by adopting the same-latitude full-scale comparison method.
S302: and generating the check file according to the comparison result and the operation stage.
In the specific implementation, in the bank core system, the consistency principle is used as the bottom logic of the anti-corrosion technology in the processes of host machine downloading and host machine removing, and the consistency comparison of files from single data sources to multiple data sources, specifically, the comparison of host machine platform files is realized in three stages of a double-machine parallel with host machine as a standard stage, a double-machine parallel with platform as a standard stage and a broken double-writing stage, and the corresponding anti-corrosion script loading is carried out based on the comparison consistency result, so that the consistency of multiple data source files is realized, and the continuous and steady from end to end of the commercial bank core service system is ensured.
Wherein the collation file comprises a platform collation file and a host collation file.
Fig. 6 is a flowchart of S202 in the embodiment of the present invention. As shown in fig. 6, S202 includes:
s501: and determining the number of the platform check files and the number of the host check files according to the comparison result and the operation stage.
When the operation stage is a stage taking a host as a reference, most comparison results are taken the host as a reference, and the rest comparison results are taken a platform as a reference, wherein the number of the platform check files is smaller than that of the host check files; when the operation stage is a double-machine parallel stage, most comparison results are based on the platform, and the rest comparison results are based on the host, wherein the number of the platform check files is larger than that of the host check files. When the operation stage is the double writing stage, taking the platform as the reference, and all comparison results are platform check files.
S502: and determining the platform check file according to the number of the platform files and the platform check files, and determining the host check file according to the number of the host files and the host check files.
S103: and selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage, and generating an anticorrosion file.
In specific implementation, the method adds the anti-corrosion layer on the lake surface, aims at stable operation in the process of the host, respectively makes different anti-corrosion treatment scripts for the three different phases (the host is the standard phase, the dual-host is the platform is the standard phase and the dual-write breaking phase) and ensures that downstream treatment of multiple data sources is displayed without difference by adopting corresponding anti-corrosion treatment based on the check result of the multiple source data and the consistency as the bottom logic.
S104: pushing the anti-corrosion file to downstream applications.
In the implementation, the anti-corrosion file can be generated in a single field or multiple fields and synchronously pushed to the downstream in real time or near real time. The anti-corrosion file is final lake entering data, and corresponding data lake batch processing can be carried out according to customer detail query use (real-time and historical) and supervision report use (full-quantity and increment) so as to support downstream display or report.
The execution subject of the file preservation method based on the distributed platform shown in fig. 1 may be a file preservation device applied to the distributed platform. As can be seen from the flow shown in fig. 1, the file anticorrosion method based on the distributed platform according to the embodiment of the present invention generates the check file according to the platform file, the host file and the operation stage, then selects the corresponding anticorrosion script according to the operation stage to perform anticorrosion on the check file, and finally pushes the generated anticorrosion file to the downstream application, so that the workload of downstream replacement and the error probability of large-scale replacement logic can be effectively reduced, and the production accident can be reduced.
Based on the same inventive concept, the embodiment of the invention also provides a file anti-corrosion device based on the distributed platform, and because the principle of the device for solving the problem is similar to that of the file anti-corrosion method based on the distributed platform, the implementation of the device can refer to the implementation of the method, and the repetition is omitted.
Fig. 7 is a block diagram of a file preservative device based on a distributed platform according to an embodiment of the invention. As shown in fig. 7, the file preservative device based on the distributed platform includes:
the acquisition module is used for acquiring the platform file, the host file and the operation stage;
The check file module is used for generating a check file according to the platform file, the host file and the operation stage;
the anticorrosion module is used for selecting corresponding anticorrosion scripts to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
and the file pushing module is used for pushing the anti-corrosion file to the downstream application.
In one embodiment, the collation file module comprises:
the comparison unit is used for comparing the platform file with the host file to generate a comparison result;
and the check file unit is used for generating the check file according to the comparison result and the operation stage.
In one embodiment, the collation file comprises a platform collation file and a host collation file;
the collation file unit includes:
the verification document quantity subunit is used for determining the quantity of the platform verification documents and the quantity of the host verification documents according to the comparison result and the operation stage;
and the checking file subunit is used for determining the platform checking file according to the number of the platform files and the platform checking files and determining the host checking file according to the number of the host files and the host checking files.
In one embodiment, the alignment unit includes:
the comparison method subunit is used for determining a comparison method according to the structure of the platform file and the structure of the host file;
and the comparison result subunit is used for comparing the platform file and the host file according to the comparison mode to generate the comparison result.
In one embodiment, the method further comprises:
the record insertion module is used for receiving a customer service instruction and inserting a corresponding record into the log according to the transaction type in the customer service instruction;
and the platform file generation module is used for checking the client business instruction and generating the platform file according to the checking result and the record.
In practical application, the file anti-corrosion device based on the distributed platform comprises a data integration module, an open platform server cluster and a data lake. Fig. 8 is a block diagram illustrating a structure of a data integration module in an embodiment of the present invention. FIG. 9 is a block diagram of an open platform server cluster in accordance with an embodiment of the invention. Fig. 10 is a block diagram showing the structure of a data lake in the embodiment of the present invention. As shown in fig. 8-10:
the data integration module 1 comprises an acquisition module, a file checking module and an anti-corrosion module, and is responsible for receiving business transaction data files sent by various channels of a bank and carrying out data anti-corrosion treatment on the files according to the required processing requirements. The module is divided into a data generation module 6, a data acquisition module 7 and a data processing judgment module 8. The data integration module 1 can judge whether the acquired data needs to be subjected to corrosion protection according to the data structure logic of the current service. If the corrosion prevention treatment is needed, the data integration module 1 will switch the data collected by the current business transaction to the data lake corrosion prevention treatment for centralized treatment.
The open platform server cluster 2 comprises a file pushing module, a record inserting module and a platform file generating module, and comprises platform table data routing contents, so that data of platform routing, service processing logic, storage in a cache and storage in a table are covered, a basic supporting platform and data storage which can support closed-loop operation of services in a development platform are built, and data support can be provided for corrosion protection processing of different requirements. The data routing module 21 defines basic data such as corresponding account cards through platform routing, so that the transaction runs in an open platform closed loop. Specific online or batch business processing is initiated by the business processing module 22, updated in the buffer module 23 and stored in the data storage module 24 in real time, and the online or batch updating to the anti-corrosion preprocessing is realized by the module.
The data lake 4 can perform data collection, copy and process service source system data into the lake, the platform batch file collection module 41 performs platform batch file processing on the data provided by the data storage module 24 of the platform, the host batch file collection module 42 performs host batch file processing on the data provided by the host batch processing module 34, and performs storage processing on the collected data, and the storage integrated processing module 43 performs real-time processing or batch processing according to specific service and aging requirements.
In summary, the file anticorrosion device based on the distributed platform in the embodiment of the invention generates the check file according to the platform file, the host file and the operation stage, then selects the corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage, and finally pushes the generated anticorrosion file to downstream application, thereby effectively reducing the workload of downstream replacement and the error probability of large-scale replacement logic and reducing the production accidents.
Fig. 11 is a schematic block diagram of a system configuration of an electronic device 9600 of an embodiment of the present application. As shown in fig. 11, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 11 is exemplary; other types of structures may also be used in addition to or in place of the structures to implement telecommunications functions or other functions.
In one embodiment, the distributed platform based file preservative method functionality may be integrated into the central processor 9100.
The central processor 9100 may be configured to perform the following control:
acquiring a platform file, a host file and an operation stage;
generating a check file according to the platform file, the host file and the operation stage;
Selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
pushing the anti-corrosion file to downstream applications.
As can be seen from the above description, the file anticorrosion method based on the distributed platform provided by the application generates the check file according to the platform file, the host file and the operation stage, then selects the corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage, and finally pushes the generated anticorrosion file to the downstream application, so that the workload of downstream replacement and the error probability of large-scale replacement logic can be effectively reduced, and the production accident is reduced.
In another embodiment, the file preservative device based on the distributed platform may be configured separately from the central processor 9100, for example, the file preservative device based on the distributed platform may be configured as a chip connected to the central processor 9100, and the function of the file preservative method based on the distributed platform is implemented by the control of the central processor.
As shown in fig. 11, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 need not include all of the components shown in fig. 11; in addition, the electronic device 9600 may further include components not shown in fig. 11, and reference may be made to the related art.
As shown in fig. 11, the central processor 9100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, which central processor 9100 receives inputs and controls the operation of the various components of the electronic device 9600.
The memory 9140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information about failure may be stored, and a program for executing the information may be stored. And the central processor 9100 can execute the program stored in the memory 9140 to realize information storage or processing, and the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, but not limited to, an LCD display.
The memory 9140 may be a solid state memory such as Read Only Memory (ROM), random Access Memory (RAM), SIM card, etc. But also a memory which holds information even when powered down, can be selectively erased and provided with further data, an example of which is sometimes referred to as EPROM or the like. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer 9141 (sometimes referred to as a buffer memory). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 storing application programs and function programs or a flow for executing operations of the electronic device 9600 by the central processor 9100.
The memory 9140 may also include a data store 9143, the data store 9143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, address book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. A communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, as in the case of conventional mobile communication terminals.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, etc., may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132 to implement usual telecommunications functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100 so that sound can be recorded locally through the microphone 9132 and sound stored locally can be played through the speaker 9131.
The embodiment of the present invention further provides a computer readable storage medium capable of implementing all the steps in the file preservation method based on the distributed platform in the embodiment in which the execution subject is a server or a client, where the computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the computer program implements all the steps in the file preservation method based on the distributed platform in the embodiment, for example, the processor implements the following steps when executing the computer program:
acquiring a platform file, a host file and an operation stage;
generating a check file according to the platform file, the host file and the operation stage;
selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
pushing the anti-corrosion file to downstream applications.
In summary, the computer readable storage medium of the embodiment of the invention generates the check file according to the platform file, the host file and the operation stage, then selects the corresponding anti-corrosion script to carry out anti-corrosion on the check file according to the operation stage, and finally pushes the generated anti-corrosion file to the downstream application, thereby effectively reducing the workload of downstream replacement and the error probability of large-scale replacement logic and reducing the production accident.
The embodiment of the present invention further provides a computer program product capable of implementing all the steps in the file preservation method based on the distributed platform in the embodiment in which the execution subject is a server or a client, where the computer program product includes a computer program/instruction, where the computer program/instruction implements all the steps in the file preservation method based on the distributed platform in the embodiment in which the processor implements the following steps when the processor executes the computer program:
acquiring a platform file, a host file and an operation stage;
generating a check file according to the platform file, the host file and the operation stage;
selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
pushing the anti-corrosion file to downstream applications.
In summary, the computer program product of the embodiment of the invention generates the check file according to the platform file, the host file and the operation stage, then selects the corresponding anti-corrosion script to carry out anti-corrosion on the check file according to the operation stage, and finally pushes the generated anti-corrosion file to the downstream application, thereby effectively reducing the workload of downstream replacement and the error probability of large-scale replacement logic and reducing the production accident.
Based on the same inventive concept, the embodiment of the invention also provides a file anti-corrosion system based on the distributed platform, and because the principle of solving the problem of the system is similar to that of the file anti-corrosion method based on the distributed platform, the implementation of the system can refer to the implementation of the method, and the repetition is omitted.
Fig. 12 is a schematic diagram of a distributed platform based file preservative system in an embodiment of the invention. As shown in fig. 12, the distributed platform-based file preservative system includes:
the file anti-corrosion device based on the distributed platform is applied to the platform; the platform comprises a data integration module 1, an open platform server cluster 2 and a data lake 4.
The host is used for sending host files to the file preservative device;
and the downstream application is used for receiving the anti-corrosion file pushed by the file anti-corrosion device.
As shown in fig. 12, compared with the prior art, the scheme focuses on the complete consistency of upstream and downstream data processing in the process of the lower host of the banking core system, ensures the file operation of the service, and uses different anti-corrosion processes based on the service characteristics of the banking core system and different stages of the lower host. The anti-corrosion layer is packaged in the database for the database of multiple data sources, so that the final database is consistent with the single data condition, and the downstream file content and timeliness are ensured. It comprises a data integration module 1, an open platform server cluster 2, a mainframe 3, a data lake 4 and downstream applications 5. The data integration module 1 is responsible for transaction data generation file reprocessing and distribution, the open platform server cluster 2 is responsible for processing distributed system business of a platform, the large host 3 is a host data processing center, the data lake 4 is responsible for storing and reprocessing files provided by the open platform server cluster 2 and the large host 3 into a lake, and the downstream application 5 is the most downstream application which finally uses the data.
FIG. 13 is a block diagram of a host in an embodiment of the invention. As shown in fig. 13, the mainframe 3 includes data routed by the mainframe, basic data running on the core mainframe, on-line batch service occurring and corresponding batch transmission processing data. The host performs batch processing operations, storage and integration of memory increment combination according to the service requirement and the output of the host interface module 31, and forms basic data of the storage processing module 33 according to the output of the service processing module 32. The basic data covers online batch logic and data storage of the business running on the core host computer, and provides data support for the anti-corrosion treatment of corresponding requirements.
In specific implementation, the host interface module 31 defines basic data such as account data of a host class card, transaction details and the like, and the business processing module 32 defines business logic relations among the basic data, so that reasonable logic references are provided for corrosion prevention processing in subsequent batch processing. The storage processing module 33 is updated by initiating a specific online or batch process at the business processing module 32 and the batch file is downloaded and updated to the data lake 4 by the batch processing module 34.
In summary, the file anti-corrosion system based on the distributed platform can effectively solve the problem that a single host computer data source is distributed to multiple platforms and multiple time periods of file distribution at present, thereby shielding large-scale replacement and modification of downstream data users, effectively reducing the workload of downstream replacement and the error probability of large-scale replacement logic, reducing production accidents, avoiding unnecessary data abnormal accidents and ensuring the validity of data.
In this specification, each embodiment is described in a progressive manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for a hardware+program class embodiment, the description is relatively simple, as it is substantially similar to the method embodiment, as relevant see the partial description of the method embodiment.
The foregoing describes specific embodiments of the present disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
Although the present application provides method operational steps as described in the examples or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive labor. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When implemented by an actual device or client product, the instructions may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment) as shown in the embodiments or figures.
Although the present description provides method operational steps as described in the examples or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps recited in the embodiments is merely one way of performing the order of steps and does not represent a unique order of execution. When implemented in an actual device or end product, the instructions may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even in a distributed data processing environment) as illustrated by the embodiments or by the figures. The terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, it is not excluded that additional identical or equivalent elements may be present in a process, method, article, or apparatus that comprises a described element.
For convenience of description, the above devices are described as being functionally divided into various modules, respectively. Of course, when implementing the embodiments of the present disclosure, the functions of each module may be implemented in the same or multiple pieces of software and/or hardware, or a module that implements the same function may be implemented by multiple sub-modules or a combination of sub-units, or the like. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
Those skilled in the art will also appreciate that, in addition to implementing the controller in a pure computer readable program code, it is well possible to implement the same functionality by logically programming the method steps such that the controller is in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Such a controller can be regarded as a hardware component, and means for implementing various functions included therein can also be regarded as a structure within the hardware component. Or even means for achieving the various functions may be regarded as either software modules implementing the methods or structures within hardware components.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It will be appreciated by those skilled in the art that embodiments of the present description may be provided as a method, system, or computer program product. Accordingly, the present specification embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present description embodiments may take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
Embodiments in this specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The embodiments of the specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the embodiments of the present specification. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.
The foregoing is merely an example of an embodiment of the present disclosure and is not intended to limit the embodiment of the present disclosure. Various modifications and variations of the illustrative embodiments will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, or the like, which is within the spirit and principles of the embodiments of the present specification, should be included in the scope of the claims of the embodiments of the present specification.
Claims (10)
1. The file anti-corrosion method based on the distributed platform is characterized by comprising the following steps of:
acquiring a platform file, a host file and an operation stage;
generating a check file according to the platform file, the host file and the operation stage;
selecting a corresponding anticorrosion script to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
pushing the anti-corrosion file to downstream applications.
2. The distributed platform-based file preservation method of claim 1, wherein generating a collation file from the platform file, the host file and the run phase comprises:
comparing the platform file with the host file to generate a comparison result;
and generating the check file according to the comparison result and the operation stage.
3. The distributed platform based file preservation method of claim 2, wherein the collation file comprises a platform collation file and a host collation file;
generating the collation file according to the comparison result and the operation stage includes:
determining the number of the platform check files and the number of the host check files according to the comparison result and the operation stage;
and determining the platform check file according to the number of the platform files and the platform check files, and determining the host check file according to the number of the host files and the host check files.
4. The distributed platform-based file preservation method of claim 2, wherein comparing the platform file and the host file, generating a comparison result comprises:
determining a comparison mode according to the structure of the platform file and the structure of the host file;
and comparing the platform file with the host file according to the comparison mode to generate the comparison result.
5. The distributed platform-based file preservation method of claim 1, further comprising:
receiving a customer service instruction, and inserting a corresponding record in a log according to the transaction type in the customer service instruction;
And checking the client business instruction, and generating the platform file according to the checking result and the record.
6. Document anticorrosion device based on distributed platform, characterized by comprising:
the acquisition module is used for acquiring the platform file, the host file and the operation stage;
the check file module is used for generating a check file according to the platform file, the host file and the operation stage;
the anticorrosion module is used for selecting corresponding anticorrosion scripts to carry out anticorrosion on the check file according to the operation stage to generate an anticorrosion file;
and the file pushing module is used for pushing the anti-corrosion file to the downstream application.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and running on the processor, characterized in that the processor implements the steps of the distributed platform based file preservation method of any one of claims 1 to 5 when the computer program is executed by the processor.
8. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the distributed platform based file preservation method of any one of claims 1 to 5.
9. A computer program product comprising computer programs/instructions which, when executed by a processor, implement the steps of the distributed platform based file preservation method of any one of claims 1 to 5.
10. A distributed platform based file preservation system, comprising:
the file preservative device based on the distributed platform as claimed in claim 6, which is applied to the platform;
the host is used for sending host files to the file preservative device;
and the downstream application is used for receiving the anti-corrosion file pushed by the file anti-corrosion device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311540824.3A CN117349256A (en) | 2023-11-17 | 2023-11-17 | File anticorrosion method, device and system based on distributed platform |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311540824.3A CN117349256A (en) | 2023-11-17 | 2023-11-17 | File anticorrosion method, device and system based on distributed platform |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117349256A true CN117349256A (en) | 2024-01-05 |
Family
ID=89369521
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311540824.3A Pending CN117349256A (en) | 2023-11-17 | 2023-11-17 | File anticorrosion method, device and system based on distributed platform |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117349256A (en) |
-
2023
- 2023-11-17 CN CN202311540824.3A patent/CN117349256A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111047430B (en) | Accounting information processing method and device | |
| US10572685B1 (en) | Protecting sensitive data | |
| CN112163946A (en) | Accounting processing method and device based on distributed transaction system | |
| CN103164523A (en) | Inspection method, device and system of data consistency inspection | |
| CN114048034A (en) | Accounting data processing method and device | |
| CN112395253A (en) | Index file generation method, terminal device, electronic device and medium | |
| CN113742212A (en) | New and old system migration test method and device | |
| US11157456B2 (en) | Replication of data in a distributed file system using an arbiter | |
| US20240303413A1 (en) | Document editing method and apparatus, and terminal and non-transitory storage medium | |
| CN111427918A (en) | Transaction detail data comparison method and device | |
| CN113110829B (en) | Multi-UI component library data processing method and device | |
| CN111930690B (en) | File generation method and device | |
| CN117349256A (en) | File anticorrosion method, device and system based on distributed platform | |
| CN111930624A (en) | Test link message data processing method and device | |
| CN111897849A (en) | Data query method and device | |
| CN112738550A (en) | Content data processing method and device, electronic equipment and medium | |
| CN113434423B (en) | Interface testing method and device | |
| CN112785201B (en) | Heterogeneous system quasi-real-time high-reliability interaction system and method | |
| CN115421976A (en) | Remote disaster recovery data processing method and device | |
| CN111221817B (en) | Service information data storage method, device, computer equipment and storage medium | |
| CN114092226A (en) | Method and device for recommending foreign exchange products of bank outlets | |
| CN113419957A (en) | Rule-based big data offline batch processing performance capacity scanning method and device | |
| CN111161047A (en) | Bank business data processing and inquiring method and device | |
| CN113515447A (en) | System automation test method and device | |
| CN111949355B (en) | Column menu shielding method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |