Detailed Description
Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the specification, as detailed in the appended claims.
The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the description. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information, without departing from the scope of the present specification. The word "if" as used herein may be interpreted as "at … …" or "when … …" or "in response to a determination", depending on the context.
Referring to FIG. 1, shown in FIG. 1 is AN architecture diagram of a system 100 implementing data processing shown in AN exemplary embodiment of the present description, the system 100 may include AN operations server 105 in data communication with one or more clients 106 via a network 112, and databases 115, 116 independent of the operations server 105 or integrated with the operations server 105. the network 112 may include, for example, the Internet, AN intranet, AN extranet, a Wide Area Network (WAN), a local area network (L AN), a wired network, a wireless network, or other suitable network, or any combination of two or more such networks.
Commercially available hypertext transfer protocol (HTTP) server applications, such as HTTP servers, Internet Information Services (IIS), and/or other servers may be included on operator server 105.
The client 106 may be a network device in which the application is installed. The applications referred to herein may be applications involved in business systems that provide various services, such as financial systems that provide accounting in-and out-of-services, communications systems that provide communications services, etc., and the network devices may include desktop computers, laptop computers, tablet computers, smart phones, handheld computers, personal digital assistants ("PDAs"), or any other wired or wireless processor-driven device in hardware.
Once a user uses a client 106 and initiates a registration request to the operation server 105 through the network 112, the operation server 105 may record registration information of the user and set an account with preset rights in the operation server 105 according to the registration request. In addition, an account registered by each user, various passwords (login password and/or business processing password, etc.) of each account, and authentication information uploaded at the time of user registration may be stored.
Each subsequent time a user initiates a service request for a service via the client 106, for example: the service data access request, the service data retrieval request, and the like, so that when the operation server 105 is accessed, the operation server 105 can learn and record such service request based on the user ID and other identification indicating the user identity carried in each service request. In particular, the request time of the user may be recorded, and optionally, one or more of the IP address at the time of initiating the request, or the hardware type of the network device used, or the client version/os version on which the client is based may be recorded. In particular, for a service request in which data update occurs, such as a service request for transferring resources in or out, a request for modifying a service status, and the like, the operation server 105 may also record the service request and give a user a specific response by performing a corresponding internal or external operation. Generally, all service requests will leave a corresponding record in the operation server 105. The operation server 105 may classify all service requests of different users according to user ID, service description information, and the like. Service requests for interactive activities, such as interactions between different registered users, may also be classified and grouped according to the user ID.
The operation server 105 may use the database 115 or the database 116 to store the service request, the operation responding to the service request, and the service data generated by the operation, and the database 115 and the database 116 may be used as the database responding to the service request of the client 106.
For the application scenario of redundant backup, the database 115 may be one of a primary database and a redundant backup database of the primary database, and the database 116 may be one of the primary database and the redundant backup database of the primary database.
For an application scenario of data migration, the database 115 may be one of a master database and a migration database of the master database, and the database 116 may be one of a master database and a migration database of the master database, wherein the master database or the migration database may be a Qracle database, an OceanBase database, a MySQ L database, and the like.
System 100 may include other databases in addition to database 115 and database 116. The data stored in database 115 or database 116 is associated with the operational steps of the various embodiments described below.
In practical applications, the need for switching databases may arise from the considerations of data security, data processing efficiency, database operation and maintenance cost, and the like. Taking the system shown in fig. 1 as an example, the database 115 is a database that is read and written by the operation server 105 all the time, and may be called a source database, and after receiving a service request from the client 106, the operation server 105 preferentially accesses the source database. Since the data processing efficiency, security, operation and maintenance cost, etc. of the database 116 are superior to those of the database 115, the database 116 is required to replace the database 115, the database preferentially accessed by the operation server 105 is switched from the database 115 to the database 116, and the database 116 may be referred to as a target database for replacing the source data.
In order to ensure that the operation server 105 provides normal service for the client 106 after the target database replaces the source database, before switching the databases, the service data needs to be migrated from the source database to the target database. The following describes the database switching process in the embodiments of the present specification in detail with reference to the drawings.
Referring to fig. 2, fig. 2 is a flowchart illustrating a database switching method according to an exemplary embodiment of the present disclosure, where the embodiment may be applied to the operation server shown in fig. 1, and includes the following steps S201 to S203:
step S201, when data is written into the source database, marking the written data as data to be synchronized, where the source database is a database that is preferentially accessed by the service system.
Step S202, if the preset synchronization time is reached, the marked data in the source database is synchronized to the target database.
Step S203, after the marked data in the source database are all synchronized to the target database, switching the target database to the database that the service system preferentially accesses.
In this embodiment, the service system may refer to each service device that the service provider provides the user with the relevant service, such as: and the operation server is used for responding to the service request of the client and providing corresponding service for the client.
In practical applications, when the operation server receives a service request sent by a client or detects a service request directly triggered by a user (such as a service administrator) on the operation server, the operation server may perform an operation corresponding to the service request on the source database, where the operation mentioned here may be a read operation, a write operation, a lock operation, and the like, and the write operation may also refer to at least one of an insert operation, a modify operation, and a delete operation. When the service scenarios are different, the service requests received by the operation server are different, and when the operation is performed on the source database in response to the service requests, the operated data may also be different, for example: when the service scene is a scene in the field of electronic commerce, the received service request is an order processing request, and the operated data is order data; when the service scene is a scene in the financial field, the received service request is a balance inquiry request, and the operated data can be account balance data. In other service scenarios, other types of service requests and other types of data may also be received, which is not limited in this embodiment of the present specification.
When the operation server responds to the received or detected service request and writes data into the source database, whether an original SQ L query condition corresponding to the operation to be executed contains a unique index configured for the data to be operated or not can be judged, if yes, the write operation is directly executed on the target database, if not, a service table of the slave database is back-checked according to a where query condition, whether the unique index of the service to which the data to be operated belongs is included in the service table or not is judged, if not, the original SQ L has no query condition, the write operation on the target database is forbidden, and if yes, the write operation is executed on the target database according to the unique index.
The written data can be marked as data to be synchronized, and when the written data is specifically marked, a synchronization data table can be established, and a synchronization mark capable of representing the written data is added in the synchronization data table, such as: adding the index of the written data into a synchronous data table, or directly adding the index of the written data and information describing the writing operation into the synchronous data table; when marking, a synchronous mark can be directly added in the data table for storing the written data corresponding to the written data, and the synchronous mark can be characters such as letters and numbers which are preset by designers and can represent that the corresponding data is data to be synchronized. In other embodiments, the data to be synchronized may also be marked in other manners, which is not limited in this illustrative embodiment.
When writing data, the operations performed on the source database are different, and the marked data may also be different, for example: when the deletion operation is executed on the source database, the indexes of the deleted data and the information describing the deletion operation can be added to the synchronous data table, or the indexes of a group of data obtained after the deletion operation is executed can be added to the synchronous data table; when the insertion operation is performed on the source database, the index of the inserted data and the information describing the insertion operation may be added to the synchronous data table, or the index of a group of data obtained after the insertion operation is performed may be added to the synchronous data table.
After the written data is marked, if the preset synchronization time is not reached, the written data can be marked as the data to be synchronized again as long as the data is written into the source database. The synchronization time mentioned here can be predetermined by the designer of the present solution according to the actual application scenario. In some examples, the predetermined time period may be a fixed time period of each day, such as 3 to 6 hours each morning. In other examples, the period of time that the probability that the source database is operated is lower than the predetermined probability threshold may be predetermined, and the probability threshold may be other values after 5%.
After the written data is marked, if the synchronization time is up, the marked data can be synchronized to the target database, when the target database does not have the data to be synchronized, the data to be synchronized is directly written into the target database, and when the target database has historical data related to the data to be synchronized, the related historical data is updated according to the synchronization marks or the data to be synchronized.
In practical application, when the marked data are synchronized, the latest marked data can be preferentially synchronized to the target database according to the marking time, and each time a group or a piece of data is synchronized, the group or the piece of data can be updated to be synchronized data so as to distinguish the synchronized data from unsynchronized data. Further, the subsequent data may be continuously synchronized from the last synchronized data. In other examples, the marked data may also be synchronized in other orders, which is not limited in this embodiment of the present specification.
When the marked data is synchronized, the source database can be kept as the database which is preferentially accessed by the service system, so that when data needs to be written into the source database, the data is written into the source database and the written data is marked while the data is synchronously marked. After the synchronization time is exceeded, the data synchronization may be temporarily terminated, and the data synchronization may be continued after the next predetermined synchronization time is reached.
In some scenarios, when the first group of data is not written in the source database, the data to be synchronized is marked, so that the data to be synchronized is marked and written in the data of the source database before the data to be synchronized is written, and at least part of the data may need to be synchronized in the target database, so that after the marked data in the source database are all synchronized in the target database, the data to be synchronized which is not marked in the source database can be synchronized in the target database; after the unmarked data to be synchronized in the source database are all synchronized to the target database, the target database is switched to the database preferentially accessed by the service system. The data to be synchronized, which is not marked in the source database, can be determined by the designer or system administrator of the present solution according to actual business requirements, for example: data related to marked data in the unmarked data may be determined as data to be synchronized, and data related to marked data may be the same data as a service to which the marked data belongs. In other examples, important service data, service data that may be repeatedly accessed, and the like in the untagged data may also be determined as data to be synchronized, which is not limited by the embodiments of the present specification.
After the target database is switched to the database preferentially accessed by the service system, considering that the newly switched target database does not stably run under the actual operation environment of the service system, and a fault may occur, so as to influence the timely processing of the service, in order to avoid the influence of the fault of the target database on the service, when data is written into the target database within a preset time period after switching, the written data is marked as data to be synchronized, and when the preset synchronization time is reached, the marked data in the target database is synchronized to the source database. The predetermined time mentioned here can be set by the designer or service manager according to the service requirement, such as 1 hour, 1 day or other time periods.
And synchronizing new data in the target database to the source database within a preset time period, and switching the source database to a database which is preferentially accessed by the service system when the target database has an abnormal operating condition within the preset time period. Then when data are written into the source database again, marking the written data as data to be synchronized; when the preset synchronization time is reached, synchronizing the marked data in the target database to the source database, and synchronizing the marked data in the source database to the target database; and after the marked data in the source database are all synchronized to the target database, switching the target database to the database which is preferentially accessed by the service system. Through database back-switching in the case of a fault, the service system and the target database can be better matched to operate, the influence of the fault on the service after the database is switched is avoided, the target database is convenient to adjust in time, the fault occurrence rate of the target database can be effectively reduced, and the service processing efficiency is further improved.
Due to the back-cut of the databases, when the marked data are synchronized, the data in the target database can be synchronized to the source database, and the data in the source database can also be synchronized to the target database.
In other examples, the steps of switching back the database, marking the data to be synchronized, synchronizing the data to the target database, and switching the target database to the database preferentially accessed by the service system may be performed in a loop until the target database does not have an abnormal operating condition within the predetermined time period.
In addition, if the target database has no abnormal operation condition within the preset time period, the source database can be directly abandoned. However, in order to avoid that the target database fails within a short time after the predetermined time period and affects the service processing, when data is written into the target database after the predetermined time period, the written data may be selectively synchronized to the source database, and in some examples, when data is written into the target database after the predetermined time period, data to be synchronized corresponding to the current time may be acquired, where the greater the time distance between the current time and the predetermined time period is, the less the corresponding data to be synchronized is; and when the written data is matched with the data to be synchronized corresponding to the current time, marking the written data as the data to be synchronized. The data to be synchronized corresponding to the current time can be preset by the personnel in the scheme, and different data to be synchronized are set according to different times, for example: the less important the data is, the less time corresponds to, which may slowly decrease the amount of marked data to be synchronized until the data to be synchronized is no longer marked.
Considering that after the database is switched to the database preferentially accessed by the service system, when the target database fails in a preset time period, the source database is switched to the database preferentially accessed by the service system again, and after the marked data to be synchronized are all synchronized to the target database, the target database is switched to the database preferentially accessed by the service system again. Therefore, the database that the service system has access to preferentially is not fixed for at least a period of time, and then during this period of time, the operation server may receive multiple service requests for the same service (multiple requests belong to the same service), the first service request is received before the source database and the target database are switched, and the operation server 105 responds to the first service request, performs corresponding operation on the source database, and updates the status data of the service to 1. And after the source database is replaced by the target database, the target database becomes a database which is preferentially accessed by the operation server, the second service request is sent to the operation server, the operation server responds to the second service request, corresponding operation is carried out on the target database, and the state data of the service is updated to be 2.
In summary, the operation server responds to two service requests and performs operations on different databases, which may cause inconsistency of data of the same service in the two databases and consistency of the pipeline data of the same service in the different databases, thereby easily causing power failure and causing user investment loss. In order to solve the problem, in the embodiments of the present disclosure, when data written into a source database is pipelined data, a service and a database to which the written data belongs may be stored in a predetermined service operation record, where the database to which the written data belongs is the source database.
In addition, the data belonging to the same service is recorded once in the service operation record, so that the problem can be solved, and the service operation record can be called before the service to which the written data belongs and the database are stored in the preset service operation record; judging whether the service and the database to which the written data belongs are recorded in the service operation record; and when the service and the database to which the written data belongs do not exist in the service operation record, recording the service and the database to which the written data belongs to a preset service operation record. In addition, in order to reduce the idempotency problem between different databases, the business operation records can be stored in the target database, so that the idempotency between the database where the business operation records are located and the target database is not required to be guaranteed through other measures.
In addition to the pipelined data, there may be non-pipelined data that needs to be written to the target database or source database, which may be finger-like data, possibly frequently modified, such as accounting data like balances within an electronic account, for non-pipelined data, if a part of data of the same service is distributed in the target database and another part of data is distributed in the source database, it will cause error reporting in the service processing related to the non-pipelined data, therefore, in a period after switching the source database or the target database to the database preferentially accessed by the service system each time, if it is required to operate the master database, which is a database preferentially accessed by the service system in the source database and the target database, and the other database in the source database and the target database is a slave database, before the master database is operated, an operation policy may be determined based on at least one of:
the data type of the data to be operated, the business to which the data to be operated belongs, and a preset business operation record; and executing corresponding operation on the master database according to the determined operation strategy.
The business operation records comprise businesses and databases to which the operated data belong; the determined operation policy is a policy that prohibits a corresponding operation from being performed on the master database or a policy that permits a corresponding operation to be performed on the master database.
Further, considering that the operation server may perform a plurality of operations on the database to which the operation server preferentially accesses in response to a single service request, if the operation policies of the plurality of operations are different, it is liable to cause data of the same service in different databases to be identical, and therefore, when performing corresponding operations on the master database according to the determined operation policies, if the service requests to which the respective operations are responded are the same, the operation policies according to which the respective operations are performed on the master database are the same, and this same operation policy may be an operation policy determined according to at least one of the data type of the data to be operated, the service to which the data to be operated belongs, and a predetermined service operation record before performing a first operation on the master database in response to the service request.
When the operation policy is determined, when the application scenarios are different, the contents according to which the operation policy is determined are different, and several application scenarios and the manner of determining the operation policy are listed as follows:
scene one: the data related to the service in the scenario is non-pipeline data, or when the operation to be performed on the master database in the scenario is a read operation or a locking operation, the operation policy may be determined only according to the service to which the data to be operated belongs. The operation mentioned here may be any one of a write operation, a read operation, and a lock operation, and the write operation may be any one of an insert operation, a delete operation, and a modify operation.
When determining an operation policy according to a service to which data to be operated belongs, obtaining a service to which the service request is directed from contents carried in the service request received or detected by an operation server, responding to the data to be operated by the service request, taking the obtained service as the service to which the data to be operated belongs, and then judging whether all the data belonging to the service in a slave database are synchronized to a master database, wherein if so, the determined operation policy can be a policy allowing corresponding operations to be performed on the master database; if not, the determined operation policy may be a policy prohibiting a corresponding operation from being performed on the master database.
In actual judgment, a synchronization mark of data belonging to the service can be inquired in a synchronization data table of the slave database, if the synchronization mark is found, the data belonging to the service in the slave database is determined not to be completely synchronized to the master database, if the synchronization mark is not found, the service can be searched in the slave database according to the index of the service or the index of data to be operated, after the service is searched, whether the data of the service are all synchronized to the master database is judged, if the synchronization mark is not found, the data belonging to the service in the slave database is determined not to be completely synchronized to the master database, and if the synchronization mark is found, the data belonging to the service in the slave database is determined to be completely synchronized to the master database.
In other embodiments, when determining the operation policy, it may be determined whether the raw SQ L query condition corresponding to the operation to be performed includes a unique index configured for the data to be performed, and if so, it may be determined whether the data of the service in the slave database are all synchronized to the master database according to the unique index, if so, the determined operation policy may be a policy that allows a corresponding operation to be performed on the master database, and if not, the determined operation policy may be a policy that progresses to perform a corresponding operation on the master database.
If the original SQ L query condition does not contain the unique index configured for the data to be operated, judging whether the primary key ID configuration of the original SQ L query condition is null, if so, determining that the SQ L query condition is the query condition of the batch operation, querying whether the data of the service in the slave database are all synchronized to the master database according to the table name and the synchronization mark of the synchronization data table, if not, the determined operation strategy can be a strategy for performing corresponding operations on the master database in advance, and if so, the determined operation strategy can be a strategy for allowing the corresponding operations to be performed on the master database.
If the primary key ID configuration of the original SQ L query condition is not null, querying whether the data of the service in the secondary database is null according to the primary key ID, if not, determining whether the data queried according to the primary key ID are all synchronized to the primary database, if so, the determined operation policy may be a policy allowing a corresponding operation to be performed on the primary database, and if not, the determined operation policy may be a policy progressing a corresponding operation to be performed on the primary database.
If the data of the service in the slave database is empty according to the primary key ID query, the service table of the slave database is reversely queried according to the where query condition, whether the unique index of the service to which the data to be operated belongs is included in the service table is judged, if not, the SQ L query condition is determined to be the query condition of batch operation, whether the data of the service in the slave database are all synchronized to the master database is queried according to the table name and the synchronization mark of the synchronous data table, if so, the determined operation strategy can be a strategy for allowing corresponding operations to be performed on the master database, and if not, the determined operation strategy can be a strategy for progressing corresponding operations to be performed on the master database.
If the business table comprises the unique index of the business to which the data to be operated belongs, the unique index of the business to which the operated data belongs is read from the business table, and then whether the data of the business in the slave database are all synchronized to the master database is inquired according to the unique index, if so, the determined operation strategy can be a strategy for allowing the master database to perform corresponding operation, and if not, the determined operation strategy can be a strategy for performing corresponding operation on the master database.
In the above-described process of determining the operation policy, if the determined operation policy is to prohibit the corresponding operation from being performed on the master database, the determined operation policy may be represented by a throw error. If the operation required to be performed on the master database is a read operation, the embodiment of the present specification may allow the operation server to read data of the slave database when the determined operation policy is to prohibit the corresponding operation from being performed on the master database.
In other scenarios, if the operation to be performed on the master database is a write operation, and the data to be written into the master database may be pipeline data or non-pipeline data, when determining the operation policy, before determining whether the original SQ L query condition corresponding to the operation to be performed includes a unique index configured for the data to be operated, the data type of the data to be operated may be determined, if the data is non-pipeline data, it may be determined whether the original SQ L query condition corresponding to the operation to be performed includes a unique index configured for the data to be operated, and if the data is pipeline data, the operation policy may be determined according to a service to which the data to be operated belongs and a predetermined service record table.
Scene two: the data related to the service under the scene includes both non-pipelined data and pipelined data, and the operation policy may be determined according to the data type of the data to be operated and the service to which the data belongs, or may be determined according to the data type of the data to be operated, the service to which the data belongs, and a predetermined service operation record.
When it is determined that the data to be operated is not pipeline data according to the data type of the data to be operated and the service to which the data to be operated belongs, and the data in the slave database and the data in the same service as the data to be operated are completely synchronized to the master database, the determined operation policy may be a policy allowing a corresponding operation to be performed on the master database.
When it is determined that the data to be operated is not pipeline data and the data in the slave database and the data of the same service as the data to be operated are not fully synchronized to the master database according to the service and the data type of the data to be operated, the determined operation policy may be a policy for prohibiting the master database from performing a corresponding operation.
When it is determined that the data to be operated is the pipeline data according to the data type of the data to be operated, the service to which the data to be operated belongs and a predetermined service operation record, and the service to which the data to be operated belongs does not exist in the service operation record, the determined operation policy may be a policy that allows a corresponding operation to be performed on the master database.
In addition, when it is determined that the data to be operated is pipeline data and the business operation record has a business to which the data to be operated belongs, it is further determined whether the data belonging to the business in the business operation record belongs to a master database, and if so, the determined operation policy may be a policy allowing a corresponding operation to be performed on the master database.
If the data belonging to the service in the service operation record belongs to the slave database, it may be further determined whether the data belonging to the service in the slave database is totally synchronized to the master database through the service to which the data to be operated belongs, if totally synchronized to the master database, the determined operation policy may be a policy that allows a corresponding operation to be performed on the master database, and if not totally synchronized to the slave database, the determined operation policy may be a policy that prohibits a corresponding operation to be performed on the master database.
In the process of determining the operation policy, if the data to be operated is the pipeline data, when the operation to be performed on the database is a write operation, the operation policy may be determined according to the data type of the data to be operated, the service to which the data belongs, and a predetermined service operation record, and the prohibited operation is the write operation.
In addition, the data to be operated is pipeline data, the determined operation policy is to allow corresponding operation to be performed on the master database, and after the corresponding operation is performed on the master database, the business and the database to which the operated data belongs may be recorded to the business operation record, wherein the belonging database is the master database. Therefore, the operation strategy can be conveniently determined for the pipeline data next time.
In other scenarios, if the data related to the service only includes the pipeline data, the operation policy may be determined according to the service to be operated and the predetermined service operation record, and the specific determination manner may refer to the determination manner in the above embodiment, which is not described herein again.
In the following, referring to fig. 3 to 6, for example, in a service scenario involving database switching, how to handle writing operations of pipelined data before and after switching a database that is preferentially accessed by a service system. The pipelining data can refer to business data such as electronic account pipelining, order data and the like, and related businesses have dynamic processes like pipelining.
In each service scenario shown in fig. 3 to 6, the service processing record includes two databases, one of which is a database Master, and the other is a database Slave, the service processing record is a blacklist, and is used to record a service (service for which a service request is directed) to which operated data belongs and a database (database operated in response to the service request) to which the operated data belongs, the data to be synchronized is marked in a synchronization data table, and the data to be synchronized can be determined according to the content recorded in the synchronization data table.
Referring to fig. 3, in the scenario shown in fig. 3, the database Master is a source database, which is an initial database that is preferentially accessed by the operation server, the database Slave is a target database, which may be a redundant backup database or a migration database of the database Master, and data in the database Master is not synchronized with the database Slave, and a service processing record (blacklist) is not generated.
In practical applications, if a user logs in a client at a client device and sends a first service request for service a to an operation server through the client (step 301): setting the state of the service a to 1, after the operation server receives the service request, the operation server may write data for setting the state of the service a to 1 into the database Master (step S302): a character string indicating "state of service a is 1". Then, for later synchronization of the data, the written data may be marked, resulting in the synchronization data table 31 shown in fig. 3. Since the data in the database Master has not started to be synchronized with the database Slave and the operation server has not performed an operation on the database Slave, the synchronized data of the database Slave is empty, as shown in the table 32 in fig. 3.
In addition, in order to determine that the operation is the database Master when the service request of the service a is received again, the identifier a of the service a and the database identifier Master may be correspondingly recorded, a blacklist as shown in table 33 in fig. 3 is generated, and the blacklist is stored in the database Slave.
Referring to fig. 4, in the scenario shown in fig. 4, the database Master is a source database and is an initial database that is preferentially accessed by the operation server, the database Slave is a target database, and may specifically be a redundant backup database or a migration database of the database Master, the database Master stores the data synchronization record generated in the embodiment corresponding to fig. 3, as shown in a table 41 shown in fig. 4, at least part of data of the database Master is synchronized to the database Slave according to the table 41, the database Slave is a database that is currently and preferentially accessed by the operation server and stores a blacklist generated in the embodiment corresponding to fig. 3, as shown in a table 42 shown in fig. 4.
In practical applications, if a user logs in a client at a client device and sends a first service request for a service B to an operation server through the client (step 401): setting the state of the service B to 1, after the operation server receives the service request, the operation server may retrieve the blacklist shown in the table 42 from the database Slave (step 402), determine whether the service B exists in the retrieved blacklist (step 403), and write data for setting the state of the service B to 1 into the database Slave (step S404): a character string indicating "state of service B is 1". Then, for later synchronization of the data, the written data may be marked, resulting in the synchronization data table 43 shown in fig. 4.
In addition, in order to determine that the operation is the database Slave when the service request of the service B is received again, the identifier B of the service B and the database identifier Slave may be recorded correspondingly, so as to generate a blacklist as shown in table 44 in fig. 4.
Referring to fig. 5, in the scenario shown in fig. 5, the database Master is a source database and is an initial database that is preferentially accessed by the operation server, the database Slave is a target database, and may specifically be a redundant backup database or a migration database of the database Master, the database Master stores the data synchronization record generated in the embodiment corresponding to fig. 3, as shown in a table 51 shown in fig. 5, at least part of data of the database Master is synchronized to the database Slave according to the table 51, the database Slave is a database that is currently and preferentially accessed by the operation server and stores the synchronization data record and the blacklist generated in the embodiment corresponding to fig. 4, as shown in a table 52 and a table 53 shown in fig. 5, respectively.
In practical applications, if a user logs in a client at a client device and sends a second service request for service a to an operation server through the client (step 501): setting the state of the service a to 2, after the operation server receives the service request, the operation server may retrieve the blacklist shown in the table 53 from the database Slave (step 502), and then determine whether the service a exists in the retrieved blacklist and whether the corresponding server identifier is Slave when the service a exists (step 503), and determine that the service a exists and the corresponding server identifier is not Slave, at this time, the operation server is not allowed to write data for setting the state of the service a to 2 into the database Slave, and then the service request is rejected (step S504).
In other scenarios, if the database Master is a database that is initially accessed first by the operation server, the database Slave is a redundant backup database or a migration database of the database Master, and the database Master stores the data synchronization record generated in the embodiment corresponding to fig. 3, as shown in table 51 shown in fig. 5, all data of the database Master that needs to be synchronized have been synchronized to the database Slave according to the table 51, then even if there is a service a in the called blacklist and the server identifier corresponding to the service a is not the Slave, the operation server is allowed to write data for setting the state of the service a to 2 into the database Slave.
Referring to fig. 6, in the scenario shown in fig. 6, the database Master is a source database, which is an initial database that is preferentially accessed by the operation server, and the database Slave is a target database, which may specifically be a redundant backup database or a migration database of the database Master, and at least a part of data of the database Master is synchronized to the database Slave according to the data synchronization record generated in the embodiment corresponding to fig. 3.
After the database Slave is switched to the database which is preferentially accessed by the operation server, the blacklist and the synchronous data record which are generated in the embodiment corresponding to fig. 4 are stored, which are respectively shown as a table 62 and a table 63 in fig. 6, at least part of data of the database Slave is synchronized to the database Master according to the data synchronous record shown in the table 63, and after the database Slave is switched again, the database Master is the database which is preferentially accessed by the operation server currently.
In practical applications, if a user logs in a client at a client device and sends a second service request for service a to an operation server through the client (step 601): setting the state of the service a to 2, after the operation server receives the service request, the operation server may call a blacklist shown in table 62 from the database Slave (step 602), and then determine whether the called blacklist has the service a and whether the corresponding server identifier is a Master when the blacklist exists (step 603), and determine that the service a exists and the corresponding server identifier is not a Master, at this time, the operation server may write data for setting the state of the service a to 2 into the database Master (step 604): a character string indicating "state of service a is 2". Then, for later synchronization of the data, the written data may be marked, generating a synchronization data table 61 shown in fig. 6.
In addition, because the request record has the service a and the operation is the identification Master of the database, the blacklist may not be modified.
For the application scenario shown in any one of fig. 3 to 6, when a predetermined timing condition is satisfied, the marked data in the database Master may be synchronized into the database Slave, or the marked data in the database Slave may be synchronized into the database Master.
In addition, if an idempotent service is required to be implemented when the service a or the service B is required, before allowing the operation corresponding to the service request to be performed on the database Master or the database Slave, it may be determined whether the service request is received for the first time, if so, the operation corresponding to the service request is allowed to be performed on the database Master or the database Slave, and if not, the operation corresponding to the service request is prohibited to be performed on the database Master or the database Slave.
From the above embodiment, it can be seen that: the data switching scheme provided by the specification can not only realize the data consistency of the same service in each database, but also ensure the service idempotent in the database switching process.
Embodiments of an apparatus are also provided in the present specification, corresponding to embodiments of the method described above.
Referring to fig. 7, fig. 7 is a block diagram illustrating a database switching apparatus according to an exemplary embodiment of the present disclosure, which may include: a data tagging module 710, a data synchronization module 720, and a database switching module 730.
The data marking module 710 is configured to mark written data as data to be synchronized when data is written into a source database, where the source database is a database that is preferentially accessed by a service system.
And a data synchronization module 720, configured to synchronize the marked data in the source database with the target database when a predetermined synchronization time is reached.
The database switching module 730 is configured to switch the target database to the database that is preferentially accessed by the service system after all the marked data in the source database are synchronized to the target database.
In some examples, the database switching apparatus of this specification may further include:
the deep synchronization module is used for synchronizing unmarked data to be synchronized in the source database to the target database after the marked data in the source database are all synchronized to the target database;
the database switching module 730 may also be configured to:
and after the unmarked data to be synchronized in the source database are all synchronized to the target database, switching the target database into the database preferentially accessed by the service system.
In other examples, the database switching apparatus according to this embodiment may further include:
and the subsequent marking module is used for marking the written data as the data to be synchronized when data are written into the target database within a preset time period after the target database is switched to the database preferentially accessed by the service system.
And the subsequent synchronization module is used for synchronizing the marked data in the target database to the source database when the preset synchronization time is reached.
In other examples, the database switching apparatus according to this embodiment may further include:
and the data type acquisition module is used for acquiring the data to be synchronized corresponding to the current time when the target database does not have an abnormal operation condition in the preset time period and data is written into the target database after the preset time period, wherein the larger the time distance between the current time and the preset time period is, the less the corresponding data to be synchronized is.
And the supplementary marking module is used for marking the written data as the data to be synchronized when the written data is matched with the data to be synchronized corresponding to the current time.
In other examples, the database switching apparatus according to this embodiment may further include a failover module, a bidirectional synchronization module, and a database rollback module:
the failure handling module is configured to, when the target database has an abnormal operating condition within the predetermined time period, switch the source database to the database that the service system has access to preferentially, and notify the data synchronization module that the data synchronization module is still used to mark written data as data to be synchronized when data is written into the source database.
And the bidirectional synchronization module is used for synchronizing the marked data in the target database to the source database and synchronizing the marked data in the source database to the target database when the preset synchronization time is reached.
And the database back-cutting module is used for switching the target database to the database which is preferentially accessed by the service system again after the marked data in the source database are all synchronized to the target database.
As an example, the bidirectional synchronization module is further configured to synchronize any marked set of data from a database with a high optimistically locked version to a database with a low optimistically locked version when synchronizing the set of data.
In other examples, the source database is an Oracle database and the target database is an OceanBase database.
In other examples, the database switching apparatus according to this embodiment may further include:
and the service recording module is used for storing the service and the database to which the written data belongs to a preset service operation record when the written data is the pipelining data, wherein the database to which the written data belongs is a source database.
In other examples, the database switching apparatus according to this embodiment may further include:
the record calling module is used for calling the business operation record before storing the business and the database to which the written data belongs into a preset business operation record;
the record judging module is used for judging whether the service and the database to which the written data belongs are recorded in the service operation record;
and the business recording module is used for recording the business and the database to which the written data belongs to a preset business operation record when the business and the database to which the written data belongs do not exist in the business operation record.
As an example, the business operation record is stored in a target database.
In other examples, the database switching apparatus according to this embodiment may further include:
the strategy determining module is used for determining an operation strategy based on at least one of the following items after the target database is switched to the database preferentially accessed by the service system and before the main database is operated:
the data type of the data to be operated, the business to which the data to be operated belongs, and a preset business operation record;
the operation execution module is used for executing corresponding operation on the main database according to the determined operation strategy;
the database which is preferentially accessed by the service system in the source database and the target database is a master database, and the other database is a slave database;
the business operation record comprises business and a database to which the operated data belong;
the determined operating strategy is:
a policy prohibiting corresponding operations from being performed on the master database; or the like, or, alternatively,
policies that allow corresponding operations to be performed on the master database.
As an example, when the data of the same service with the data to be operated in the slave database is fully synchronized to the master database, the determined operation policy is a policy allowing the master database to perform corresponding operations;
and when the data of the same service with the data to be operated in the slave database is not completely synchronized to the master database, the determined operation strategy is a strategy for forbidding the master database to execute corresponding operations.
As an example, when the data to be operated is not pipeline data, and the data in the slave database and the data in the same service as the data to be operated are completely synchronized to the master database, the determined operation policy is a policy allowing corresponding operations to be performed on the master database;
and when the data to be operated is not the pipeline data, and the data of the same service with the data to be operated in the slave database is not completely synchronized to the master database, the determined operation strategy is a strategy for forbidding the master database to execute corresponding operation.
As an example, when the data to be operated is pipeline data and there is no service to which the data to be operated belongs in the service operation record, the determined operation policy is a policy that allows corresponding operations to be performed on a master database;
the database switching apparatus according to the embodiment of the present specification may further include:
the first recording module is used for recording a business master database to which the operated data belongs to the business operation record after the master database is executed with corresponding operation, wherein the belonging database is the master database.
As an example, when the data to be operated is pipeline data, the service operation record includes a service to which the data to be operated belongs, and the data belonging to the service in the service operation record does not belong to the master database, the determined operation policy is a policy for prohibiting corresponding operations from being performed on the master database;
when the data to be operated is pipeline data, the business to which the data to be operated belongs is in the business operation record, and the data belonging to the business in the business operation record all belong to the master database, the determined operation strategy is a strategy for allowing the master database to execute corresponding operations;
the database switching apparatus according to the embodiment of the present specification may further include:
and the second recording module is used for recording the business and the database to which the operated data belongs to the business operation record after the main database is executed with corresponding operation, wherein the belonging database is the main database.
As an example, the data to be operated is pipeline data, the business operation record includes a business to which the data to be operated belongs, the data belonging to the business in the business operation record does not belong to the master database, and when the data in the slave database and the data of the business to be operated share the same data, and are not fully synchronized to the master database, the determined operation policy is a policy for prohibiting corresponding operations to be performed on the master database;
the data to be operated is flow data, the business to which the data to be operated belongs is recorded in the business operation record, the data belonging to the business in the business operation record does not belong to the master database, and when the data of the business is completely synchronized with the data to be operated in the slave database, the determined operation strategy is a strategy for allowing the master database to execute corresponding operation;
the database switching apparatus according to the embodiment of the present specification may further include:
and the third recording module is used for recording the business and the database to which the operated data belongs to the business operation record after the main database is executed with corresponding operation, wherein the database to which the data belongs is the main database.
As an example, the business operation record is stored in a slave database.
As an example, if the data to be operated on is pipelined data, the operation that is inhibited is a write operation.
As an example, the database switching apparatus of the embodiments of the present specification may further include:
and the data reading module is used for allowing the data to be read from the database when the forbidden operation is the read operation.
As an example, the operation execution module is further to:
when the service requests responded by the operations are the same, executing the operations on the main database according to the same operation strategy;
the operation strategy is determined according to at least one of the data type of the data to be operated, the service of the data to be operated and a preset service operation record before the first operation is performed on the main database in response to the service request.
The implementation process of the functions and actions of each unit (or module) in the above device is specifically described in the implementation process of the corresponding step in the above method, and is not described herein again.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units or modules described as separate parts may or may not be physically separate, and the parts displayed as the units or modules may or may not be physical units or modules, may be located in one place, or may be distributed on a plurality of network units or modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution in the specification. One of ordinary skill in the art can understand and implement it without inventive effort.
The embodiment of the device in the specification can be applied to computer equipment. In particular, it may be implemented by a computer chip or entity, or by an article of manufacture having some functionality. In a typical implementation, the computer device is a computer, which may take the form of a personal computer, a laptop computer, a personal digital assistant, a media player, or any combination of these devices.
The device embodiments may be implemented by software, or by hardware, or by a combination of hardware and software. The software implementation is taken as an example, and is formed by reading corresponding computer program instructions in a readable medium such as a nonvolatile memory into a memory for running through a processor of a computer device in which the software implementation is located. From a hardware aspect, as shown in fig. 8, the hardware structure diagram of the computer device in which the apparatus in this specification is located is shown, except for the processor, the memory, the network interface, and the nonvolatile memory shown in fig. 8, the computer device in which the apparatus is located in the embodiment may also include other hardware according to the actual function of the computer device, which is not described again.
In one example, a memory of a computer device may store processor-executable program instructions; the processor may be coupled to the memory for reading program instructions stored in the memory and, in response, performing the following: when data is written into a source database, marking the written data as data to be synchronized, wherein the source database is a database which is preferentially accessed by a service system; if the preset synchronization time is reached, synchronizing the marked data in the source database to the target database; and after the marked data in the source database are all synchronized to the target database, switching the target database into the database which is preferentially accessed by the service system.
In other embodiments, the operations performed by the processor may refer to the description related to the above method embodiments, which is not repeated herein.
The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.
The above description is only a preferred embodiment of the present disclosure, and should not be taken as limiting the present disclosure, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.