CN117874145B - Strong agreement method, device, equipment and storage medium for master-slave database - Google Patents
Strong agreement method, device, equipment and storage medium for master-slave database Download PDFInfo
- Publication number
- CN117874145B CN117874145B CN202410285023.5A CN202410285023A CN117874145B CN 117874145 B CN117874145 B CN 117874145B CN 202410285023 A CN202410285023 A CN 202410285023A CN 117874145 B CN117874145 B CN 117874145B
- Authority
- CN
- China
- Prior art keywords
- data
- database
- master
- slave
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000003860 storage Methods 0.000 title claims abstract description 20
- 238000012790 confirmation Methods 0.000 claims abstract description 39
- 230000000903 blocking effect Effects 0.000 claims abstract description 17
- 238000010200 validation analysis Methods 0.000 claims description 45
- 230000007246 mechanism Effects 0.000 claims description 20
- 230000010076 replication Effects 0.000 claims description 19
- 230000008569 process Effects 0.000 claims description 12
- 238000003780 insertion Methods 0.000 claims description 5
- 230000037431 insertion Effects 0.000 claims description 5
- 238000012986 modification Methods 0.000 abstract description 6
- 230000004048 modification Effects 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 22
- 238000012545 processing Methods 0.000 description 18
- 230000006870 function Effects 0.000 description 9
- 230000008859 change Effects 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 4
- 230000001934 delay Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 230000001960 triggered effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000013474 audit trail Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 238000013523 data management Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 239000006185 dispersion Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 238000011010 flushing procedure Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2365—Ensuring data consistency and integrity
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Computing Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present disclosure relates to a strong consistency method, apparatus, device and storage medium for master-slave databases, the method comprising storing replica data to a mirror library; after the copy data is successfully stored in the mirror image library, inserting a confirmation record into a first confirmation table created in the main database; under the condition of master-slave switching, acquiring difference data according to the mirror image library and a second confirmation table in the slave database; constructing a blacklist comprising the difference data based on the difference data; and after the blacklist is constructed, taking the slave database as a new master database, and blocking the operation of the difference data recorded in the blacklist. The method and the device can ensure the consistency of the data between the master database and the slave database, reduce service interruption time and improve service continuity; the operation of data in the blacklist is blocked, the access and modification of inconsistent data are effectively avoided, the business risk caused by the inconsistent data is reduced, and the safety of data operation is improved.
Description
Technical Field
The disclosure relates to the technical field of internet, and in particular relates to a strong consistency method, device, equipment and storage medium of master-slave databases.
Background
The distributed storage system is composed of a plurality of storage nodes, each node is responsible for storing a part of data, and the nodes are connected with each other through a network and work cooperatively to provide high-efficiency and high-reliability data service. In such a system, data may be stored in disparate physical locations in a distributed manner while still being logically managed and accessed as a whole.
In a distributed storage system, a master-slave database architecture is one way to achieve data management and consistency. The architecture typically includes a master database that is responsible for handling all write operations and maintaining the latest state and integrity of the data, and at least one slave database that is responsible for synchronizing the data changes of the master database and handling read requests. The master-slave database architecture not only allows for the dispersion of data storage and processing loads among multiple nodes, but also ensures the consistency of data throughout the system through replication and synchronization mechanisms.
The master-slave replication mechanism is the key to realize a master-slave database architecture, and maintains data consistency by synchronizing data changes. The master database and the slave database can effectively cooperate to maintain the consistency and the latest state of the data. In a master-slave replication mechanism, the master database records changes to a particular log file, typically a binary log, and the slave database synchronizes the data by periodically pulling and applying these log records to ensure that its own copy of the data remains consistent with the master database. However, master-slave replication mechanisms are not strong coherency protocols, and data in the slave database may lag the master database in the event of network delays or failures, resulting in data inconsistency problems. Although master-slave database architectures introduce semi-synchronous and full replication mechanisms for enhancing data consistency, semi-synchronous replication mechanisms do not completely eliminate the risk of data inconsistencies due to network problems, which may lead to data inconsistencies at master-slave switching, thus raising production problems, while full replication mechanisms may cause master databases to fall into a blocked state due to network latency problems, affecting the usability of the overall system.
Disclosure of Invention
In order to solve at least one technical problem set forth above, the present disclosure provides a method, an apparatus, a device, and a storage medium for strong agreement of master-slave databases.
According to a first aspect of an embodiment of the present disclosure, there is provided a strong consistency method of a master-slave database, including:
Storing the copy data to a mirror library;
after the copy data is successfully stored in the mirror image library, inserting a confirmation record into a first confirmation table created in the main database;
Under the condition of master-slave switching, obtaining difference data according to a second confirmation table in the mirror image library and the slave database, wherein the second confirmation table is a table created in the slave database based on a master-slave replication mechanism;
constructing a blacklist comprising the difference data based on the difference data;
And after the blacklist is constructed, taking the master database as an old master database, taking the slave database as a new master database, and blocking the operation of the difference data recorded in the blacklist.
In one embodiment, before the storing the replica data to the mirror library, the method further comprises:
intercepting a data write operation statement in the transaction execution process;
extracting core data in the data write operation statement based on a preset rule, wherein the core data comprises table names, key field names and values corresponding to the key field names in the data write operation statement; the table names and the key field names belong to data defined in the preset rule;
Storing the core data in a specific format to a transaction context.
In one embodiment, after the storing the core data in the specific format to the transaction context and before storing the replica data to the mirror library, the method further comprises:
Generating a transaction unique identifier, wherein the transaction unique identifier is used for identifying and tracking a corresponding transaction;
and combining the transaction unique identifier and the core data stored in the transaction context to obtain the copy data.
In one embodiment, after the duplicate data is successfully stored in the mirror database, inserting a validation record into a first validation table created in the master database, including:
Creating the first validation table in the master database, the first validation table including the transaction unique identifier field and a timestamp field for recording a time of insertion of the validation record;
And after the copy data is successfully stored in the mirror image library, inserting the transaction unique identifier and the timestamp into the first confirmation table.
In one embodiment, in the case of master-slave switching, the obtaining difference data according to the mirror library and the second acknowledgement table in the slave database includes:
comparing the mirror image library with the second confirmation table under the condition of master-slave switching, and determining an unrecorded transaction unique identifier in the second confirmation table;
And acquiring the difference data based on the unrecorded transaction unique identifier, wherein the difference data is the core data corresponding to the unrecorded transaction unique identifier in the mirror image library.
In one embodiment, the method further comprises:
after the old main library is restored, the missing data of the old main library and the new main library are supplemented;
Releasing the data in the blacklist.
In one embodiment, before the master-slave switching occurs, the method further includes:
Submitting the transaction if the validation record is successfully inserted into the first validation table created in the master database;
in the event that the validation record is not successfully inserted into the first validation table created in the primary database, the transaction rolls back to an initial state.
According to a second aspect of embodiments of the present disclosure, there is provided a strong agreement device of a master-slave database, comprising:
The storage module is used for storing the copy data to the mirror image library;
the inserting module is used for inserting a confirmation record into a first confirmation table created in the main database after the duplicate data is successfully stored in the mirror image library;
The acquisition module is used for acquiring difference data according to a second confirmation table in the mirror image library and the slave database under the condition of master-slave switching, wherein the second confirmation table is a table created in the slave database based on a master-slave replication mechanism;
A building module, configured to build a blacklist including the difference data based on the difference data;
And the blocking module is used for taking the master database as an old master database and the slave database as a new master database after the blacklist is constructed, and blocking the operation of the difference data recorded in the blacklist.
According to a third aspect of embodiments of the present disclosure, there is provided a computer readable storage medium having stored therein at least one instruction or at least one program loaded and executed by a processor to implement a strong agreement method of a master-slave database as described in the first aspect above.
According to a fourth aspect of embodiments of the present disclosure, there is provided an electronic device comprising at least one processor, and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing a strong agreement method of a master-slave database as described in the first aspect above by executing the instructions stored by the memory.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
The implementation of the present disclosure has the following beneficial effects:
Storing the duplicate data into a mirror image library, and ensuring the integrity of the data between the master database and the slave database; the transaction which is successfully copied to the mirror image library is recorded by using the confirmation table, and difference data is obtained by comparing the mirror image library with the confirmation table in the slave database during master-slave switching, so that the consistency of the data between the master database and the slave database can be ensured; after the master-slave switching, the slave database is promoted to be a new master database, so that the service interruption time is reduced, and the continuity of the service and the experience of the user are improved; the blacklist is constructed, and the operation on the data in the blacklist is blocked after the new main library takes effect, so that the access and modification of inconsistent data can be effectively avoided, the business risk caused by inconsistent data is reduced, and the safety of data operation is improved; is simpler and lighter in deployment and maintenance than semi-synchronous and full-synchronous replication mechanisms.
Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments, which proceeds with reference to the accompanying drawings.
Drawings
In order to more clearly illustrate the embodiments of the present description or the technical solutions and advantages of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are only some embodiments of the present description, and other drawings may be obtained according to these drawings without inventive effort to a person skilled in the art.
FIG. 1 illustrates a flow diagram of a strong agreement method for a master-slave database according to an embodiment of the present disclosure.
FIG. 2 illustrates a flow diagram for obtaining replica data, according to an embodiment of the present disclosure.
FIG. 3 illustrates a flow diagram of transaction commit according to an embodiment of the present disclosure.
Fig. 4 shows a flow diagram of acquiring difference data according to an embodiment of the present disclosure.
FIG. 5 illustrates a block diagram of a strong compliance device for a master-slave database according to an embodiment of the present disclosure.
Fig. 6 shows a block diagram of an electronic device, according to an embodiment of the disclosure.
Detailed Description
The technical solutions of the embodiments of the present specification will be clearly and completely described below with reference to the drawings in the embodiments of the present specification, and it is apparent that the described embodiments are only some embodiments of the present specification, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are intended to be within the scope of the present invention based on the embodiments herein.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or server that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Various exemplary embodiments, features and aspects of the disclosure will be described in detail below with reference to the drawings. In the drawings, like reference numbers indicate identical or functionally similar elements. Although various aspects of the embodiments are illustrated in the accompanying drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The word "exemplary" is used herein to mean "serving as an example, embodiment, or illustration. Any embodiment described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments.
The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, may mean including any one or more elements selected from the group consisting of A, B and C.
Furthermore, numerous specific details are set forth in the following detailed description in order to provide a better understanding of the present disclosure. It will be understood by those skilled in the art that the present disclosure may be practiced without some of these specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art have not been described in detail in order not to obscure the present disclosure.
In the financial field, consistency, integrity of data and reliability of the system are critical. Semi-synchronous replication enhances data consistency to some extent by ensuring that each transaction is replicated to at least one slave database. However, network instability or failure of the slave database may lead to data inconsistencies during master-slave switching, which may cause serious consequences in financial transactions, such as duplicate transactions, transaction omission or erroneous transaction amounts. Full replication, while providing data consistency guarantees, may reduce the overall availability of the system in cases where network latency is high or database performance is poor. Because the master database needs to wait for the confirmation of all the slave databases, the master database may be blocked, so that the whole system cannot respond to the user request, and further the execution speed and efficiency of the transaction are affected.
FIG. 1 is a flow diagram of a method for strong consistency of master-slave databases, according to an embodiment of the present disclosure, as in FIG. 1, the method includes:
s101, storing the copy data into a mirror image library.
FIG. 2 shows a schematic flow diagram of obtaining replica data, as in FIG. 2, according to an embodiment of the present disclosure, the obtaining replica data including:
s201, intercepting a data write operation statement in the transaction execution process.
And selecting a proper interception technology, and intercepting all data write operation sentences sent to the database after the transaction begins. The present disclosure is not limited to interception techniques, and in one embodiment, a Java database connectivity (Java Database Connectivity, JDBC) driver layer is employed to intercept SQL write statements. The JDBC driving layer is an intermediary for interaction between the Java application program and the database, and is responsible for transmitting SQL commands in the Java application program to the database and transmitting the execution results of the database back to the Java application program. By implementing interception at the JDBC driver layer, it is possible to intervene before the Java application submits the SQL command to the database for execution, enabling interception of all SQL write statements via the JDBC layer.
When interception is implemented at the JDBC driver layer, two main techniques of Java dynamic proxy mechanism or custom wrapper can be employed.
The Java dynamic proxy mechanism allows proxy objects of implementation classes to be dynamically generated for a particular interface at application run-time, suitable for JDBC core interfaces such as Connection, statement. Creation of the proxy object is accomplished through implementation InvocationHandler interface, invoking the invoke method to be forwarded to the associated InvocationHandler when any method of the proxy object is invoked. In the application scenario of JDBC, for example, when an application performs a SQL write operation through a proxy's state object execution executeUpdate () method, the proxy object associated InvocationHandler invoke method will be triggered. Inside the invoke method, interception logic is executed. SQL parsing tools, such as JSQLParser or ANTLR, can be embedded in the invoke method to parse SQL statements transferred to the executeUpdate () method, and corresponding processing is performed according to preset logic.
The custom wrapper can create custom Connection, statement wrapper classes, rewrite methods for executing SQL sentences, such as executeUpdate () methods, add interception logic in the methods, and perform corresponding processing according to preset logic.
S202, extracting core data in the data write operation statement based on a preset rule, wherein the core data comprises table names, key field names and values corresponding to the key field names in the data write operation statement; and the table names and the key field names belong to the data defined in the preset rule.
After the JDBC driving layer intercepts the data writing operation statement, the data writing operation statement is analyzed, key components such as table names, field names and corresponding values thereof are extracted, and basic information is provided for subsequent data processing and decision. After analyzing the data writing operation statement, extracting core data in the data writing operation statement based on a preset rule. The present disclosure is not limited to a method of configuring the preset rules, and in one embodiment, the preset rules are configured using Apollo. Apollo is an open-source distributed configuration center, and is mainly used for centrally managing configuration information of applications. Apollo provides functions of configuration real-time update, version change history, gray level release and the like, and can be suitable for a micro-service architecture. By configuring interception rules in Apollo, for example specifying specific tables and fields for interception, it is possible to implement updating rules without changing codes. When business requirements change, such as monitoring new tables and fields, or stopping intercepting certain tables and fields, the configuration is updated in the Apollo, and dynamic adjustment of rules can be achieved without modifying any code. The present disclosure is not limited to preset rules, and the rules are designed according to actual requirements, and in one embodiment, for example, an account system, where the account system and the flow table are related to an account number field for identifying an account, and the account system needs to update with "account number" as a key dimension, so that the account table name, the flow table name, and the account number field name may be configured in an Apollo rule. And extracting the table names and the field names designated by the preset rules and the values corresponding to the field names from the intercepted data writing operation sentences according to the rules configured in the Apollo. By combining the Apollo with the JDBC driver layer interception, a flexible and efficient data interception and processing mechanism can be constructed, and the capability of data monitoring and management is remarkably improved.
S203, storing the core data in a specific format into a transaction context.
In an application, a transaction context is defined as a data structure, which may be a class or structure, that holds and conveys critical information throughout the execution of a transaction. The transaction context may contain a number of information related to the transaction, such as transaction ID, execution time, table and field information involved, etc. The core data extracted according to the preset rule needs to be formatted into a form acceptable to the transaction context. The present disclosure is not limited to a particular format of core data in the context of transactions, in one embodiment, core data may be formatted as [ { table names: { field name: [ field value...] ] }, structure of... In Java, the above structure can be implemented by using a List, map, or the like collection type.
By intercepting the data writing operation statement in the transaction execution process and extracting the core data based on the preset rule, the modified data can be accurately identified and recorded, and the detailed tracking of the data change is ensured by including the information such as the related table name, the key field name and the corresponding value thereof; storing core data in a specific format into a transaction context, and providing key information for subsequent consistency checks; the accuracy and the efficiency of data processing in master-slave switching are enhanced, so that the data consistency and the reliability of the whole system are ensured.
S204, generating a transaction unique identifier, wherein the transaction unique identifier is used for identifying and tracking the corresponding transaction.
The present disclosure is not limited to the manner in which the transaction unique identifier is generated, and in one embodiment, the transaction unique identifier is an identifier (Universally Unique Identifier, UUID) used in a computer system to ensure global uniqueness, and the transaction unique identifier is generated by an application layer, typically using a UUID algorithm, and may be directly generated using a Java. The transaction unique identifier provides an unrepeatable identification code for each transaction, ensuring traceability and manageability of each transaction.
S205, combining the transaction unique identifier and the core data stored in the transaction context to obtain copy data. By combining the transaction unique identifier with the core data, each data change is ensured to be accurately recorded and tracked, thereby effectively maintaining the integrity and consistency of the data.
Mirror libraries are an integral part of a distributed storage system for storing copies of data. And storing the copy data into a mirror image library, and ensuring the integrity and consistency of the data between the master database and the slave database by recording detailed changes of each transaction.
Storing the duplicate data in a mirror library, wherein a database system is selected as the mirror library, the disclosure is not limited to the mirror library, and the selection of the mirror library depends on a plurality of factors including the existing technical stack, performance requirements, reliability, maintainability, etc., and in one embodiment, a MySQL database is selected as the mirror library; after determining the database to use, the tables related to storing the duplicate data are required, and the present disclosure is not limited to the specific structure of the tables used to store the duplicate data, and in one embodiment, the tables are used to store the duplicate data, and further include other additional fields to meet a wide range of business requirements. The table is specifically designed as follows:
CREATE TABLE `example_prepare` (
`id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT,
`uid` bigint(20) NOT NULL COMMENT,
`source_key` varchar(100) NOT NULL COMMENT,
`data` text NOT NULL COMMENT,
`status` tinyint(1) DEFAULT NULL COMMENT,
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT,
`update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT,
PRIMARY KEY (`id`),
UNIQUE key `uidx_uid` (`uid`),
KEY `idx_create_time_source_key` (`create_time`,`source_key`)
) ENGINE=InnoDBDEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC；
the above-mentioned fields of the table storing copy data include:
primary key (id): uniquely identifying each row record in the table;
Transaction unique identifier field (uid): storing a transaction unique identifier corresponding to each transaction;
Source identification field (source_key): storing a source identification, which may be, for example, an IP address of a master database that triggered the transaction;
Data field (data): storing core data in a specific format, such as serialized JSON formatted data { "test" { "account_no": [ "123456" ] };
status field (status): recording the state of the line record, for example, 0 indicates a ready state, 2 indicates a rollback state, and the like;
Creation time field (create_time): a time stamp type, recording the time of data creation;
Update time field (update_time): the time stamp type records the last updated time of the data.
S102, after the copy data is successfully stored in the mirror image library, inserting a confirmation record into a first confirmation table created in the main database.
The first validation table is created in the master database, which may be manually performed by a database administrator or designed to be automatically detected and created at the start-up of the application. The present disclosure is not limited to a first validation table that, in one embodiment, includes a transaction unique identifier field and a timestamp field for recording the time at which the validation record was inserted. In one embodiment, the timestamp field includes a create time field (create_time) for recording a creation time of the validation record and an update time field (update_time) for recording a last update time of the validation record, and the first validation table is specifically designed as follows:
CREATE TABLE `dama_ack` (
`id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT,
`uid` bigint(20) NOT NULL COMMENT,
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT,
`update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT,
PRIMARY KEY (`id`),
UNIQUE key `uidx_uid` (`uid`),
KEY `inx_create_time` (`create_time`)
) ENGINE=InnoDBDEFAULT CHARSET=utf8mb4 ROW_FORMAT=DYNAMIC;
After inserting the replica data into the mirror library, a reliable mechanism is required to verify whether the data has been successfully stored, either by checking the number of rows affected by the return status validation of the insert operation, or by performing a targeted query to verify that the data does exist in the mirror library. After confirming that the duplicate data is successfully stored in the mirror library, an insertion operation is performed to insert the transaction unique identifier and the current timestamp into a first validation table in the master database. The duplicate data is stored in the mirror image library immediately, and the confirmation information is recorded, so that the data consistency is maintained effectively, the synchronization of the data between the main database and the mirror image library is ensured, and the risk of inconsistent data is reduced; meanwhile, by inserting the transaction unique identifier and the timestamp into the first confirmation table, auditable data change history is provided, and reliable data is provided for subsequent monitoring.
FIG. 3 illustrates a flow diagram of transaction commit, as in FIG. 3, where the insertion of validation records into the first validation table is performed to ensure tracking and recording of data changes in the event that the replica data is successfully stored into the mirror library, in accordance with an embodiment of the present disclosure. In the event that the duplicate data is not successfully stored to the mirror library, the transaction rolls back to the original state. If the copy data cannot be successfully stored in the mirror image library, in order to maintain the consistency and the integrity of the data, executing the transaction rollback, and restoring the database state to the initial state before the transaction starts, thereby preventing incomplete or erroneous data from being recorded and ensuring the integrity and the reliability of the data.
The transaction is committed in the event that the validation record is successfully inserted into the first validation table created in the master database. If the record is successfully inserted, the operation is completed, and the data states are consistent. Commit transactions involve commit commands that invoke databases, ensuring that all relevant data changes are permanently saved. The transaction is submitted after the validation record is successfully inserted into the first validation table, so that the transaction is considered to be successfully ended after the data change of the transaction is validated and recorded.
In the event that a validation record is not successfully inserted into the first validation table created in the primary database, the transaction rolls back to the initial state. If the validation record is not successfully inserted, it may be due to various reasons, such as database connection problems, rights issues, or other anomalies. In this case, the transaction rollback will be performed, all data changes made since the transaction began will be canceled, possible data inconsistencies or errors are prevented, and the accuracy and reliability of the data are maintained.
And S103, under the condition of master-slave switching, acquiring difference data according to the mirror image library and a second confirmation table in the slave database.
Fig. 4 shows a flow diagram of acquiring difference data, according to an embodiment of the present disclosure, as in fig. 4, the acquiring difference data includes:
s401, comparing a mirror image library with a second confirmation table, and determining an unrecorded transaction unique identifier in the second confirmation table; the second validation table is a table created in the slave database based on the master-slave replication mechanism.
In a master-slave database architecture, the first validation table is a table created and maintained in the master database, the structure and data of which are replicated into the slave database based on the master-slave replication mechanism of the database, and is referred to as the second validation table in the slave database. The second validation table maintains a copy of the validation record of the transaction in the master database from the slave database for use in participating in the verification of data consistency at the time of a master-slave switch or other specific instance.
When a master-slave switch occurs, such as when the master database is offline for a failure or other reason, the slave database will be promoted to a new master database. Since master-slave replication mechanisms typically do not guarantee strong consistency, particularly in the event of network delays or failures, the slave database may fail to synchronize the latest state of the data in time, resulting in the information it reflects lagging the master database. Although the second validation table provides a validation record of transactions in the secondary database, the information contained therein does not represent the final data change in the primary database. By comparing the data in the mirror library with the records in the second validation table, transaction unique identifiers corresponding to transactions that are not being processed from the database due to replication delays are identified and obtained.
The method comprises the specific operations of obtaining a transaction unique identifier corresponding to a transaction which is not processed in a database, wherein the specific operations comprise:
extracting transaction unique identifiers corresponding to all core data from a mirror image library;
Extracting the transaction unique identifiers in all validation records from the second validation table;
the two sets of transaction unique identifiers are compared to find the transaction unique identifiers that do not appear in the second validation table. The present disclosure is not limited to the comparison method, and an appropriate comparison method needs to be selected according to the specific situation. In one embodiment, the comparison at the application layer is performed by retrieving data from a database and storing the data as a collection or list of data structures in the application memory; processing is then performed using the logic of the application, such as comparing the two sets using loops, conditional statements, or set operations, and finding a non-matching transaction unique identifier. The method provides a high degree of flexibility and adaptability, especially for cases where the processing logic is complex. IN another embodiment, the collection operation is performed using SQL or a database-specific query language, such as using NOT IN, EXCEPT, or LEFT JOIN statements, IN a manner that compares at the database level, identifying transaction unique identifiers directly IN the database that are present IN one collection and NOT IN the other. The method is generally more efficient, is particularly suitable for processing large amounts of data, and reduces the load of network transmission and application servers.
S402, based on the unrecorded transaction unique identifier, obtaining difference data, wherein the difference data is core data corresponding to the unrecorded transaction unique identifier in the mirror image library.
In one embodiment, a specific implementation of obtaining difference data includes constructing one or more query statements from an unrecorded transaction unique identifier for retrieving corresponding core data from a mirror library; and executing the query statement in the database, and retrieving the core data corresponding to the unique identifier of each unrecorded transaction.
The difference data is obtained by comparing the mirror image database with a second confirmation table in the slave database during the master-slave switching, so that the consistency of the data between the master database and the slave database can be ensured; by directly locating and acquiring the core data corresponding to the transaction which is not synchronized in the slave database through the transaction unique identifier, unnecessary data processing and resource consumption are reduced.
S104, constructing a blacklist comprising the difference data based on the difference data.
In one embodiment, a particular implementation of constructing the blacklist includes defining a data structure of the blacklist, which may be in the form of a list or collection, wherein each element represents a data item that needs to be blocked. Specifically, each data item includes a table name, a field name, and a value corresponding to the field name, and the data format stored in the blacklist is [ { table name: { field name: [ field value..+ -.) },.+ -. ]. Filling the blacklist according to the obtained difference data, wherein a specific implementation manner is to traverse all the difference data represented by the unrecorded transaction unique identifier and add the difference data into the blacklist. The pseudo code is as follows:
blacklist = new List()
for each identifier in difference_identifiers:
it is assumed that diffence_ identifiers is the unique identifier of all unrecorded transactions
Data = RETRIEVEDATA (IDENTIFIER)// retrieval of core data from mirror library
Blacklist (data)/adding core data to blacklist
S105: and after the blacklist is constructed, taking the master database as an old master database, taking the slave database as a new master database, and blocking the operation of the difference data recorded in the blacklist.
After the blacklist is built, the current main database is used as an old main database, the selected slave database is promoted to be a new main database, the data is not written into the old main database any more usually through the operation executed by an automatic script, and the read-write operation is started to be accepted on the new main database. After the master-slave switching, the slave database is promoted to be a new master database, the service interruption time is reduced, and the service continuity and the user experience are improved.
In one embodiment, data operation sentences of the database are continuously intercepted, and key components such as table names, field names and corresponding values of the table names, the field names and the like are analyzed and extracted for each intercepted data operation sentence. Comparing the extracted data with records in the blacklist, and checking whether a matched table name and field name exist or not and whether the field value is matched with a field value list of the blacklist or not. If a match is found, the currently intercepted data operation statement attempts to access or modify a certain data item recorded in the blacklist, and immediately performs blocking operation. Specific blocking operations include directly cancelling the current data operation to prevent the data from being accessed or modified, and recording details of the occurrence of the blocking operation, such as time, type of operation, tables and fields involved, etc., to facilitate subsequent auditing and analysis.
In implementing the blocking operation, implementation at the application layer or database layer may be selected according to system architecture and requirements. At the database level, triggers or stored procedures can be used to immediately throw exceptions and cancel operations when a blacklist match is detected in the execution of a transaction. The method directly controls the data source, and can effectively prevent the data access or modification behavior which does not meet the data consistency requirement. At the application layer, access to inconsistent data or modification operations may be blocked by interrupting and canceling subsequent database operation commands when a blacklist match is detected in the business logic. This approach provides more flexible control and error handling. Once blocking occurs, a detailed record of the event is required. The information recorded to the log system typically includes time stamps, operation types, table and field names involved, reasons for interception, etc., to facilitate subsequent auditing and analysis, an important resource for future analysis and optimization of system behavior. Typically, the detailed information will be stored in a specific audit trail or log repository to ensure reliable preservation of the information and ease of retrospective analysis.
In one embodiment, if a table with a strong consistency requirement does not contain core data defined in a preset rule, or the whole data of the table needs to be blocked, a policy of blocking the whole table may be adopted. Data in the format of { "table name": { "# - # all# - # - ["1"] } }) is stored in the blacklist, and" 1 "is a value of no practical significance set to satisfy the format requirement. The blocking operation will be performed immediately when any statement is intercepted that attempts to access or modify the table marked as full-table intercept. The full table intercept strategy is a powerful measure for handling tables that need to be fully protected to ensure data consistency and security.
The blacklist is constructed and the operation of data in the blacklist is blocked after the new main library takes effect, so that the access and modification of inconsistent data can be effectively avoided, the business risk caused by the inconsistent data is reduced, and the safety of data operation is improved.
After the old main library is restored, the missing data of the old main library and the new main library are supplemented; releasing the data in the blacklist.
In one embodiment, after the old master library is restored, the database administrator obtains a set of global transaction identifiers (Global Transaction Identifiers, GTID) on the old master library and the new master library. GTID is an identifier that is used in the database system to uniquely identify each transaction. When a transaction is committed, the database automatically generates a unique GTID. By comparing GTID sets of old and new master libraries, a database administrator can accurately identify transaction differences between the two, including transactions that are unique to the old and new master libraries.
After GTID needing to be supplemented is determined, the database manager extracts the corresponding binary log from the log system of the corresponding database, and performs playback operation. The binary log details the sequence of all change operations in the database and is arranged in time order. For transactions unique to the new master library, the database administrator applies the binary log corresponding to the transaction unique to the new master library to the old master library to ensure that the old master library contains all data changes that occurred on the new master library during the failure. For the transaction unique to the old main library, namely, the transaction is only executed and completed on the old main library before downtime, but is not synchronized to other slave databases in time, a database manager applies the binary log corresponding to the transaction unique to the old main library to the new main library, so that the new main library can comprehensively and accurately reflect all data updates occurring in the old main library.
After the old main library is restored, the consistency and the integrity of the data in the distributed storage system are effectively ensured by synchronizing the difference data between the old main library and the new main library, the risks of data conflict and inconsistency are reduced, and the stability and the reliability of the system are enhanced. Meanwhile, a solid foundation is provided for maintaining the continuity of the service and the accuracy of the data.
After the playback operation is completed, a data consistency check is performed to confirm the consistency of the data in the master and slave databases. And after finishing the data consistency check and confirming that the data is correct, releasing the data in the blacklist. The present disclosure is not limited to a particular manner of release, and in one embodiment, the blacklist is stored in a database form, and all records may be removed by performing a DELETE operation, thereby clearing the blacklist. In another embodiment, the blacklist is built at the application layer, and in the form of an in-memory data structure, such as a list or collection, all restricted data items can be effectively released by flushing the blacklist variables by re-initializing or setting them to an empty collection. By releasing the data in the blacklist, the operation that was temporarily intercepted due to the consistency problem can be resumed, thereby ensuring continuity of data access and processing. Not only improves the responsiveness and usability of the system, but also ensures the normal operation of the business process and enhances the stability and reliability of the whole system.
FIG. 5 illustrates a block diagram of a strong compliance device for a master-slave database according to an embodiment of the present disclosure. As shown in fig. 5, the strong agreement means of the master-slave database includes:
The storage module is used for storing the copy data to the mirror image library;
the inserting module is used for inserting a confirmation record into a first confirmation table created in the main database after the duplicate data is successfully stored in the mirror image library;
The acquisition module is used for acquiring difference data according to a second confirmation table and the mirror image library in the slave database under the condition of master-slave switching, wherein the second confirmation table is created in the slave database based on a master-slave replication mechanism;
A building module, configured to build a blacklist including the difference data based on the difference data;
And the blocking module is used for taking the master database as an old master database and the slave database as a new master database after the blacklist is constructed, and blocking the operation of the difference data recorded in the blacklist.
In some embodiments, functions or modules included in an apparatus provided by the embodiments of the present disclosure may be used to perform a method described in the foregoing method embodiments, and specific implementations thereof may refer to descriptions of the foregoing method embodiments, which are not repeated herein for brevity.
The embodiment of the disclosure also provides a computer readable storage medium, wherein at least one instruction or at least one section of program is stored in the computer readable storage medium, and the at least one instruction or the at least one section of program realizes the method when being loaded and executed by a processor. The computer readable storage medium may be a non-volatile computer readable storage medium.
The embodiment of the disclosure also provides an electronic device, which comprises: a processor; a memory for storing processor-executable instructions; wherein the processor is configured as the method.
The electronic device may be provided as a terminal, server or other form of device.
Fig. 6 shows a block diagram of an electronic device, according to an embodiment of the disclosure. For example, electronic device 1900 may be provided as a server. Referring to FIG. 6, electronic device 1900 includes a processing component 1922 that further includes one or more processors and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by processing component 1922. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. Further, processing component 1922 is configured to execute instructions to perform the methods described above.
The electronic device 1900 may also include a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input/output (I/O) interface 1958. The electronic device 1900 may operate based on an operating system stored in memory 1932, such as Windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, or the like.
Various aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium having the instructions stored therein includes an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The foregoing description of the embodiments of the present disclosure has been presented for purposes of illustration and description, and is not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or the technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
Claims (7)
1. A method of strong agreement of a master-slave database, comprising:
Storing the copy data to a mirror library; the process for obtaining the copy data comprises the following steps: intercepting a data write operation statement in the transaction execution process; extracting core data in the data writing operation statement based on a preset rule, wherein the core data comprises table names, key field names and values corresponding to the key field names in the data writing operation statement, and the table names and the key field names belong to data defined in the preset rule; storing the core data in a specific format to a transaction context; generating a transaction unique identifier, wherein the transaction unique identifier is used for identifying and tracking a corresponding transaction; combining the transaction unique identifier with the core data stored in the transaction context to obtain the copy data;
After the copy data is successfully stored in the mirror image library, inserting a confirmation record into a first confirmation table created in the main database; the first validation table includes the transaction unique identifier field and a timestamp field for recording a time of insertion of the validation record;
Under the condition of master-slave switching, obtaining difference data according to a second confirmation table in the mirror image library and the slave database, wherein the second confirmation table is a table created in the slave database based on a master-slave replication mechanism;
constructing a blacklist comprising the difference data based on the difference data;
And after the blacklist is constructed, taking the master database as an old master database, taking the slave database as a new master database, and blocking the operation of the difference data recorded in the blacklist.
2. The method according to claim 1, wherein in the case of master-slave switching, the step of obtaining difference data according to the mirror library and the second acknowledgement table in the slave database comprises:
comparing the mirror image library with the second confirmation table under the condition of master-slave switching, and determining an unrecorded transaction unique identifier in the second confirmation table;
And acquiring the difference data based on the unrecorded transaction unique identifier, wherein the difference data is the core data corresponding to the unrecorded transaction unique identifier in the mirror image library.
3. A method of strong agreement for a master-slave database according to claim 1, the method further comprising:
after the old main library is restored, the missing data of the old main library and the new main library are supplemented;
Releasing the data in the blacklist.
4. A method of strong agreement of a master-slave database according to claim 1, wherein the method further comprises, prior to the occurrence of a master-slave switch:
Submitting the transaction if the validation record is successfully inserted into the first validation table created in the master database;
in the event that the validation record is not successfully inserted into the first validation table created in the primary database, the transaction rolls back to an initial state.
5. A strong agreement device for a master-slave database, comprising:
The storage module is used for storing the copy data to the mirror image library; the process for obtaining the copy data comprises the following steps: intercepting a data write operation statement in the transaction execution process; extracting core data in the data writing operation statement based on a preset rule, wherein the core data comprises table names, key field names and values corresponding to the key field names in the data writing operation statement, and the table names and the key field names belong to data defined in the preset rule; storing the core data in a specific format to a transaction context; generating a transaction unique identifier, wherein the transaction unique identifier is used for identifying and tracking a corresponding transaction; combining the transaction unique identifier with the core data stored in the transaction context to obtain the copy data;
The inserting module is used for inserting a confirmation record into a first confirmation table created in the main database after the duplicate data is successfully stored in the mirror image library; the first validation table includes the transaction unique identifier field and a timestamp field for recording a time of insertion of the validation record;
The acquisition module is used for acquiring difference data according to a second confirmation table in the mirror image library and the slave database under the condition of master-slave switching, wherein the second confirmation table is a table created in the slave database based on a master-slave replication mechanism;
A building module, configured to build a blacklist including the difference data based on the difference data;
And the blocking module is used for taking the master database as an old master database and the slave database as a new master database after the blacklist is constructed, and blocking the operation of the difference data recorded in the blacklist.
6. A computer readable storage medium having stored therein at least one instruction or at least one program loaded and executed by a processor to implement a strong agreement method of a master-slave database according to any one of claims 1-4.
7. An electronic device comprising at least one processor, and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the at least one processor implementing a strong coherency method for a master-slave database according to any of claims 1-4 by executing the instructions stored by the memory.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410285023.5A CN117874145B (en) | 2024-03-13 | 2024-03-13 | Strong agreement method, device, equipment and storage medium for master-slave database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410285023.5A CN117874145B (en) | 2024-03-13 | 2024-03-13 | Strong agreement method, device, equipment and storage medium for master-slave database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117874145A CN117874145A (en) | 2024-04-12 |
| CN117874145B true CN117874145B (en) | 2024-05-28 |
Family
ID=90592246
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410285023.5A Active CN117874145B (en) | 2024-03-13 | 2024-03-13 | Strong agreement method, device, equipment and storage medium for master-slave database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117874145B (en) |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CA2545532A1 (en) * | 2005-05-04 | 2006-11-04 | Rajesh Kapur | Method and system for providing network attached system for disaster recovery and upgrade testing |
| JP2006311064A (en) * | 2005-04-27 | 2006-11-09 | Decenter:Kk | Portrait sales system through communication network |
| CN104618127A (en) * | 2013-11-01 | 2015-05-13 | 深圳市腾讯计算机系统有限公司 | Master and standby storage node switching method and system |
| CN104750755A (en) * | 2013-12-31 | 2015-07-01 | 阿里巴巴集团控股有限公司 | Method and system for recovering data after switching between main database and standby database |
| CN107015885A (en) * | 2016-07-12 | 2017-08-04 | 阿里巴巴集团控股有限公司 | Service providing method, system and configuration center when primary standby data storehouse switches |
| CN107357800A (en) * | 2017-05-18 | 2017-11-17 | 杭州沃趣科技股份有限公司 | A kind of database High Availabitity zero loses solution method |
| CN108319617A (en) * | 2017-01-17 | 2018-07-24 | 阿里巴巴集团控股有限公司 | Determine the method, apparatus and method for handover control, device of database principal and subordinate's difference |
| CN111930465A (en) * | 2020-05-28 | 2020-11-13 | 武汉达梦数据库有限公司 | Kubernetes-based dreams master-slave cluster deployment method and device |
| CN111966699A (en) * | 2020-07-09 | 2020-11-20 | 微民保险代理有限公司 | Method and device for checking data, computer equipment and storage medium |
| CN112486718A (en) * | 2020-11-30 | 2021-03-12 | 深圳市移卡科技有限公司 | Database fault automatic switching method and device and computer storage medium |
| CN112685234A (en) * | 2020-11-26 | 2021-04-20 | 辽宁振兴银行股份有限公司 | Financial-level two-place three-center high-availability MySQL database implementation method |
| CN113141370A (en) * | 2021-04-30 | 2021-07-20 | 国家计算机网络与信息安全管理中心山西分中心 | Malicious DNS tunnel identification method for internal network traffic |
| CN114493097A (en) * | 2021-12-20 | 2022-05-13 | 深圳供电局有限公司 | Power dispatching system operation method based on master-slave mode |
| CN114691771A (en) * | 2020-12-28 | 2022-07-01 | 顺丰科技有限公司 | Master-slave copying method and device for database, computer equipment and storage medium |
| CN115421976A (en) * | 2022-09-05 | 2022-12-02 | 中国工商银行股份有限公司 | Remote disaster recovery data processing method and device |
| US11537314B1 (en) * | 2021-10-07 | 2022-12-27 | Netapp, Inc. | Resynchronization of individual volumes of a consistency group (CG) within a cross-site storage solution while maintaining synchronization of other volumes of the CG |
| CN117555966A (en) * | 2023-12-08 | 2024-02-13 | 中电云计算技术有限公司 | Database master-slave synchronization method, device, equipment and readable storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP4484618B2 (en) * | 2004-07-30 | 2010-06-16 | 株式会社日立製作所 | Disaster recovery system, program, and data replication method |
| US7650369B2 (en) * | 2006-03-30 | 2010-01-19 | Fujitsu Limited | Database system management method and database system |
| US8074107B2 (en) * | 2009-10-26 | 2011-12-06 | Amazon Technologies, Inc. | Failover and recovery for replicated data instances |
| US20150347547A1 (en) * | 2014-05-30 | 2015-12-03 | Zardosht Kasheff | Replication in a NoSQL System Using Fractal Tree Indexes |
| WO2020113314A1 (en) * | 2018-12-04 | 2020-06-11 | Zeu Crypto Networks Inc. | System and method for augmenting database applications with blockchain technology |
| US20240004902A1 (en) * | 2022-06-29 | 2024-01-04 | International Business Machines Corporation | Performance of row table to columnar table replication |
-
2024
- 2024-03-13 CN CN202410285023.5A patent/CN117874145B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2006311064A (en) * | 2005-04-27 | 2006-11-09 | Decenter:Kk | Portrait sales system through communication network |
| CA2545532A1 (en) * | 2005-05-04 | 2006-11-04 | Rajesh Kapur | Method and system for providing network attached system for disaster recovery and upgrade testing |
| CN104618127A (en) * | 2013-11-01 | 2015-05-13 | 深圳市腾讯计算机系统有限公司 | Master and standby storage node switching method and system |
| CN104750755A (en) * | 2013-12-31 | 2015-07-01 | 阿里巴巴集团控股有限公司 | Method and system for recovering data after switching between main database and standby database |
| CN107015885A (en) * | 2016-07-12 | 2017-08-04 | 阿里巴巴集团控股有限公司 | Service providing method, system and configuration center when primary standby data storehouse switches |
| CN108319617A (en) * | 2017-01-17 | 2018-07-24 | 阿里巴巴集团控股有限公司 | Determine the method, apparatus and method for handover control, device of database principal and subordinate's difference |
| CN107357800A (en) * | 2017-05-18 | 2017-11-17 | 杭州沃趣科技股份有限公司 | A kind of database High Availabitity zero loses solution method |
| CN111930465A (en) * | 2020-05-28 | 2020-11-13 | 武汉达梦数据库有限公司 | Kubernetes-based dreams master-slave cluster deployment method and device |
| CN111966699A (en) * | 2020-07-09 | 2020-11-20 | 微民保险代理有限公司 | Method and device for checking data, computer equipment and storage medium |
| CN112685234A (en) * | 2020-11-26 | 2021-04-20 | 辽宁振兴银行股份有限公司 | Financial-level two-place three-center high-availability MySQL database implementation method |
| CN112486718A (en) * | 2020-11-30 | 2021-03-12 | 深圳市移卡科技有限公司 | Database fault automatic switching method and device and computer storage medium |
| CN114691771A (en) * | 2020-12-28 | 2022-07-01 | 顺丰科技有限公司 | Master-slave copying method and device for database, computer equipment and storage medium |
| CN113141370A (en) * | 2021-04-30 | 2021-07-20 | 国家计算机网络与信息安全管理中心山西分中心 | Malicious DNS tunnel identification method for internal network traffic |
| US11537314B1 (en) * | 2021-10-07 | 2022-12-27 | Netapp, Inc. | Resynchronization of individual volumes of a consistency group (CG) within a cross-site storage solution while maintaining synchronization of other volumes of the CG |
| CN114493097A (en) * | 2021-12-20 | 2022-05-13 | 深圳供电局有限公司 | Power dispatching system operation method based on master-slave mode |
| CN115421976A (en) * | 2022-09-05 | 2022-12-02 | 中国工商银行股份有限公司 | Remote disaster recovery data processing method and device |
| CN117555966A (en) * | 2023-12-08 | 2024-02-13 | 中电云计算技术有限公司 | Database master-slave synchronization method, device, equipment and readable storage medium |
Non-Patent Citations (4)
| Title |
|---|
| 一种基于云的应用层容错机制设计与实现;于翔;叶德建;;微型电脑应用;20160220(第02期);全文 * |
| 基于SQL-92和JDBC的SpaceOS数据库数据迁移的实现;宁泰安;刘金刚;;计算机应用与软件;20141215(12);全文 * |
| 打造与"两地三中心"相适应的应用架构体系;李旭风;;中国金融电脑;20150907(第09期);全文 * |
| 表广播机制在MyCat中的实现;王锦;梁正和;王法强;;计算机技术与发展;20171205(03);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117874145A (en) | 2024-04-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12001420B2 (en) | Disconnected operation within distributed database systems | |
| US11429641B2 (en) | Copying data changes to a target database | |
| EP1062569B1 (en) | Isolation levels and compensating transactions in an information system | |
| US6873995B2 (en) | Method, system, and program product for transaction management in a distributed content management application | |
| US20150347250A1 (en) | Database management system for providing partial re-synchronization and partial re-synchronization method of using the same | |
| EP3391244B1 (en) | Replication control among redundant data centers | |
| CN108021338B (en) | System and method for implementing a two-layer commit protocol | |
| US10248709B2 (en) | Promoted properties in relational structured data | |
| EP2194467A1 (en) | Extend CRUD to support lifecycle management and business continuity | |
| US11226985B2 (en) | Replication of structured data records among partitioned data storage spaces | |
| US20240143386A1 (en) | Using multiple blockchains for applying transactions to a set of persistent data objects in persistent storage systems | |
| US20220197761A1 (en) | Cloud architecture for replicated data services | |
| WO2022242372A1 (en) | Object processing method and apparatus, computer device, and storage medium | |
| CN117874145B (en) | Strong agreement method, device, equipment and storage medium for master-slave database | |
| WO2023111910A1 (en) | Rolling back database transaction | |
| US20090319525A1 (en) | Lost write protection via stream-based replication | |
| Zhou et al. | FoundationDB: A Distributed Key Value Store | |
| Kampen | Zero Downtime Schema Migrations in Highly Available Databases | |
| Timothy et al. | Polyglot Persistence in Distributed Databases for Point in Time Recovery and Failure Handling in MySQL Replicated Environment | |
| CN118260267A (en) | Data recovery method, system and electronic equipment | |
| CN118733604A (en) | Database updating method and device, electronic device and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |