CN112765180B - Method and device for analyzing column names of table building logs of DB2 database - Google Patents
Method and device for analyzing column names of table building logs of DB2 database Download PDFInfo
- Publication number
- CN112765180B CN112765180B CN202110109747.0A CN202110109747A CN112765180B CN 112765180 B CN112765180 B CN 112765180B CN 202110109747 A CN202110109747 A CN 202110109747A CN 112765180 B CN112765180 B CN 112765180B
- Authority
- CN
- China
- Prior art keywords
- name
- analysis
- column
- log
- task
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2282—Tablespace storage structures; Management thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
- G06F16/2433—Query languages
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a method and a device for analyzing column names of table-building logs of a DB2 database, wherein the method comprises the following steps: step S1, when the table building log is analyzed, analyzing the original SQL log, separating a table name and a schema name from an SQL statement, and storing the table name and the schema name into an analysis structural body for representing basic information of the table; s2, acquiring a table ID, a table space ID and attribute information of each column of the table from the log of the initialization table, storing the table ID, the table space ID and the attribute information of each column of the table into an analysis structure, constructing a task structure, setting a flag number of table construction operation in the task structure, and storing the acquired table ID and the table space ID into the task structure; and S3, reading the task structure according to the sequence of the task queue, acquiring the table name, the schema name, the column name and the column attribute information according to the task structure, the analysis structure and the database system table, assembling the table name, the schema name, the column name and the column attribute information into the SQL operation basic information in the JSON format, and sending the SQL operation basic information to KAFKA.
Description
Technical Field
The invention relates to the technical field of computer data backup disaster recovery, in particular to a method and a device for analyzing list names of table-building logs of a DB2 database.
Background
DB2 is a large relational database platform developed by IBM, which supports multiple users or applications to query data in different databases or even different DBMSs in the same SQL statement, and is one of the most widely used relational database systems at present.
Currently, when analyzing a Create Table (Table) log, a DB2 database sequentially generates three types of log data: one is the original SQL LOG record with Component identifier of 11 and Function identifier of 4, and the database attribute LOG _ DDL _ STMTS needs to be set to YES; one is a table building log record with Component identifier of 4 and Function identifier of 101; in addition, the log record of the initialization table with Component identifier 1 and Function identifier 128 requires to declare data capture changes when building or modifying the table, wherein the original SQL log record includes the original text of the input SQL, the data of the table building log is used inside the database, and the initialization table log mainly includes the column attributes of the table, the number of columns, and the like.
When the Table creating operation is analyzed in real time, table names and Table structures are required to be obtained, wherein the Table structures mainly comprise column names and column attributes, the Table names can be obtained from the original SQL log record analysis, the column attributes can be obtained in the initialized Table log but no column names are required, and when the analysis is finished, the basic information of the JSON format newly created Table is sent to KAFKA, the information of the Table names, the schema names, the column attributes and the like is required to be sent.
In order to solve the above problems, the current direct method generally adopts the steps of directly parsing the SQL statement from the original SQL log, and decomposing each word of the SQL statement according to the grammar of Create Table or other features, thereby obtaining the column name. However, analyzing various information in the Create Table statement by using the SQL syntax is basically the same as implementing a compiler for analyzing the Create Table statement, so that although the program is ensured to be complete, the requirement on the technical level of developers is high, and the implementation is too complex and time-consuming; various information in the Create Table statement is analyzed by other features (such as a space, a comma, left and right brackets and the like) in the SQL statement in a character string matching manner, and although the implementation is simple, various situations of using the Create Table operation are difficult to cover, which may cause the Create Table operation to fail to analyze and even cause the program to crash.
Disclosure of Invention
In order to overcome the above-mentioned deficiencies of the prior art, the present invention provides a method and an apparatus for analyzing a column name of a table-building log of a DB2 database, so as to achieve the purpose of obtaining a column name in a table-building log analysis.
In order to achieve the above object, the present invention provides a method for column name resolution of a table-building log of a DB2 database, comprising the following steps:
step S1, when the table building log is analyzed, analyzing the original SQL log, separating a table name and a schema name from an SQL statement, and storing the table name and the schema name into an analysis structural body for representing basic information of the table;
s2, acquiring a table ID, a table space ID and each row of attribute information of the table from the log of the initialized table, storing the table ID, the table space ID and each row of attribute information of the table into the analysis structure, constructing a task structure, setting a flag number of table construction operation in the task structure, and storing the acquired table ID and the table space ID into the task structure;
and S3, reading the task structure according to the sequence of the task queue, judging whether the table building operation is performed, and for the table building operation, obtaining the basic information of the SQL operation in which the table name, the schema name, the column attribute and other information are assembled into a JSON format according to the task structure, the analysis structure and the database system table and sending the basic information to KAFKA.
Preferably, in step S1, for the table building statement with parentheses in the original SQL log, the table name and the schema name are obtained by separating the first left parentheses and the space before the first left parentheses.
Preferably, for the table building statements without parentheses in the original SQL log, the table name and the schema name are separated according to the table building syntax.
Preferably, in step S3, first, the task structure is read according to the sequence of the task queue, and whether the operation of the current task structure is a table building operation is determined according to the flag number set in step S2, and if the operation is the table building operation, a corresponding analysis structure is found according to the table ID and the table space ID stored in the task structure; inquiring in a database system table according to the table name provided by the analysis structural body and the schema name of the table to obtain a corresponding column name, and storing the column name into the analysis structural body so as to enable the column attribute and the column name in the analysis structural body to be corresponding to each other; and finally, splicing the table name, the schema name, the column names, the column attributes and the like in the analysis structure body into basic information of the table building operation in the JSON format, and sending the basic information to KAFKA.
Preferably, before the corresponding column name is stored in the analysis structure, the queried information is stored in a new temporary structure, and is compared with the analysis structure for exception handling.
Preferably, in step S3, before storing the corresponding column names into the analysis structure, it is determined whether the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are the same, and if the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are the same, the same table is considered, and then the column names in the temporary structure are sequentially copied into the analysis structure, thereby completing the column name analysis of the table creation operation.
Preferably, if the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are not uniform, an error is considered, and a prompt message is generated to prompt that the current table may be deleted and exited.
In order to achieve the above object, the present invention further provides an apparatus for parsing column names of table-building logs of a DB2 database, comprising:
the system comprises an original SQL log analyzing unit, a table creating unit and a table creating unit, wherein the original SQL log analyzing unit is used for analyzing the original SQL log when analyzing a table creating log, separating a table name and a schema name from an SQL statement and storing the table name and the schema name into an analyzing structural body which represents table basic information;
the device comprises an initialization table log analysis unit, a task structure body and a data analysis unit, wherein the initialization table log analysis unit is used for acquiring a table ID, a table space ID and attribute information of each column of a table from a log of an initialization table, storing the table ID, the table space ID and the attribute information of each column of the table into the analysis structure body, establishing the task structure body, setting a flag number of table establishment operation in the task structure body, and storing the acquired table ID and the table space ID into the task structure body;
and the analysis query unit is used for reading the task structure according to the sequence of the task queue txn, judging whether the table building operation is performed or not, and for the table building operation, obtaining the basic information of the SQL operation in which the table name, the schema name, the column attribute and the like are assembled into a JSON format according to the task structure, the analysis structure and the database system table and sending the basic information to KAFKA.
Preferably, the parsing and querying unit determines whether the current operation is a table building operation according to the number of flags set by the initialized table log parsing unit, and finds a corresponding parsing structure according to the table ID and the table space ID stored in the task structure; inquiring in a database system table according to the table name provided by the analysis structural body and the schema name of the table to obtain a corresponding column name, and storing the column name into the analysis structural body so as to enable the column attribute and the column name in the analysis structural body to be corresponding to each other; and finally, splicing the table name, the schema name, the column names and the column attributes in the analysis structure body into basic information of the table building operation in the JSON format, and sending the basic information to KAFKA.
Preferably, the apparatus further includes an exception handling unit, configured to determine whether the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are the same before storing the corresponding column names in the analysis structure, and if the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are the same, the same table is considered, and the column names in the temporary structure are sequentially copied to the analysis structure, so as to complete the column name analysis of the table building operation.
Compared with the prior art, when the method and the device for analyzing the column names of the table establishing logs of the DB2 database analyze the logs, firstly, the table names and the schema names are separated from SQL sentences, the table ID, the table space ID and the column attribute information of the tables are obtained from the logs of an initialized table, the information is uniformly stored in an analysis structure body, a task structure body is simultaneously constructed, the flag number of the table establishing operation is set in the task structure body, the obtained table ID and the table space ID are stored in the task structure body, the basic information of the SQL operation, in which the table names, the schema names, the column attributes and the like are assembled into a JSON format according to the task structure body, the analysis structure body and the database system table, is obtained according to the sequence of a task queue txn, and is sent to KAFKA, and the purpose of obtaining the column names in the table creating table log analysis is realized.
Drawings
FIG. 1 is a flow chart of the steps of a method of the present invention for parsing a DB2 database table log column name;
FIG. 2 is a system architecture diagram of an apparatus for log column name resolution of a DB2 database according to the present invention.
Detailed Description
Other advantages and capabilities of the present invention will be readily apparent to those skilled in the art from the present disclosure by describing the embodiments of the present invention with specific embodiments thereof in conjunction with the accompanying drawings. The invention is capable of other and different embodiments and its several details are capable of modification in various other respects, all without departing from the spirit and scope of the present invention.
FIG. 1 is a flowchart illustrating the steps of a method for parsing the column name of a database table-building log of DB2 according to the present invention. As shown in FIG. 1, the method for column name resolution of the table-building log of the DB2 database of the present invention comprises the following steps:
step S1, when analyzing a table creating (creating) log, analyzing an original SQL log, separating a table name and a schema name from an SQL statement, and storing the table name and the schema name into an analysis structure tab representing basic information of a table.
Specifically, when the table building log is analyzed, the original SQL log is analyzed, for the table building statement with brackets, the SQL statement is decomposed according to the first left bracket and the space before the first left bracket, so that the table name and the schema name can be separated, and the table name and the schema name are stored in an analysis structural body tab representing the table basic information, such as a simple table building statement: "Create table myschema. Table name (id int)", according to the SQL syntax of the table creation (Create table), if there is a left bracket in the table creation statement, the first left bracket is immediately behind the table name of the newly created table, so that the character string "Create table myschema. Table name" can be obtained through decomposition; because the "create table" is a character string which is in a fixed position and the content is always unchanged, the 13 characters of the "create table" can be directly skipped over to obtain a character string "myschema. And because the table name can not have a space, the first space in the character string is used for segmentation to obtain the character string' myschema. When the decimal point exists, the decimal point is divided according to the decimal point, the schema name before the decimal point is myschema, and the table name after the decimal point is tabname; for tabular statements without parentheses, for example: "create table name like table name2", because there is no left bracket in the table building statement, so "create table" is skipped directly, and the character string "table name like table name2" is obtained; and because the table name cannot have a space, the table name is divided by using the first space in the character string, and the sub-character string from the character string to the first space of the character string is the' schema name.
Step S2, obtaining information such as a table ID, a table space ID, and attributes of each column of the table from a log of an initialized table, storing the information into an analysis structure tab, simultaneously constructing a task structure txn, storing a task queue, storing a table building (Create table) operation into the task structure txn of the task queue, where the task structure txn of the task queue includes a flag number of a table building (Create table) operation, the table ID, and the table space ID, specifically, setting a flag number of a table building (Create table) operation, and storing the obtained table ID and table space ID into the task structure txn, where the flag number is obtained by an enumeration type defined in a header file at program start time and including various SQL operations (such as insert data, delete data, update data, newly build a table, delete table, and the like), and in a specific embodiment of the present invention, the flag number of the table building (Create table) operation may be set to 15 (i.e., representing a table building operation), the default flag number is generally from 0, that is a current flag number of the SQL operation, that is set to be a flag number corresponding to the step S2, and a step S3, where the flag number is determined according to which the current flag number of the operation.
That is, a plurality of task structures are sequentially stored in the task queue; the task structure stores basic information of various SQL operations, and ensures that the sequence of the information is consistent with the operation sequence of a user in a database when the JSON format SQL basic information is sent to KAFKA through a task queue. In the invention, when the program is started, a task queue is generated and initialized, and when the whole transaction is analyzed, all task structures constructed when the transaction is analyzed are stored in the task queue (note that each operation generates a task structure, but only the table building operation generates an analysis structure tab representing the basic information of the table); when the task structure is found to be stored in the task queue, the process proceeds to step S3.
And S3, reading the task structure according to the sequence of the task queue, judging whether the table building operation is performed, and for the table building operation, obtaining information such as a table name, a schema name, a column attribute and the like according to the task structure txn, the tab structure and the database system table to assemble basic information of the SQL operation in the JSON format and sending the basic information of various SQL operations in the JSON format to KAFKA. Specifically, first, the flag number of the task structure txn is read and determined according to the sequence of the task queue, and it is determined whether the operation of the currently read task structure is a table building operation according to the flag number set in step S2 (for example, the flag number of the table building operation defined in step S2 is 15, where it is determined whether the operation is 15 according to the flag number of the read task structure to determine whether the operation is a table building operation), and then a corresponding tab structure is found according to the table ID and the table space ID stored in the txn structure; inquiring in a system table of a database according to the table name provided by the tab structure body and the schema name of the table to obtain a corresponding column name, and storing the column name into the tab structure body to enable the column attribute and the column name in the tab structure body to be corresponding to each other, wherein the column name is analyzed completely, and all information in the tab structure body is complete; and splicing the table name, the schema name, the column names and the column attributes in the tab structure into basic information of the table building operation in the JSON format, and sending the basic information to KAFKA, thereby completing the table analyzing and building operation and sending the basic information of the table building operation in the JSON format to KAFKA.
In a specific embodiment of the present invention, the database is a DB2 database, and when JSON data for table creation operation needs to be sent, a corresponding tab resolution structure is read from a memory according to a table id and a tablespace id in a txn task structure, column information of a corresponding table, including a column name, a column attribute, and the like, is queried in a system table sysibm.
Since the DB2 database has the multiplexing of table id, the parsing and sending may be slower than the execution speed of the database, and the user may perform the operations of table creation and re-deletion and continuous table creation and deletion, there may be a possibility that the table which is not desired to be searched and is searched in the database in step S3 may not be searched at all, so that the exception handling is also required. Therefore, in step S3, it is further determined whether the number of columns and the attributes of the columns stored in the analysis structure tab and the temporary structure temp are the same, and if the number of columns and the attributes of the columns stored in the analysis structure tab and the temporary structure temp are the same, the tables are regarded as the same table, the column names in the temporary structure temp are sequentially copied to the structure tab, and the column name analysis of the table building operation is completed, otherwise, an error is regarded as being in error, and it is suggested that the table may be deleted and exited.
FIG. 2 is a system architecture diagram of an apparatus for log column name resolution of a DB2 database according to the present invention. As shown in fig. 2, the device for column name resolution of the table-building log of the DB2 database according to the present invention includes:
the original SQL log analyzing unit 201 is configured to, when analyzing a table creation (created table) log, analyze the original SQL log, separate a table name and a schema name from an SQL statement, and store the table name and the schema name in an analysis structure tab representing table basic information.
Specifically, when analyzing the table building log, the original SQL log analyzing unit 201 first analyzes the original SQL log, and separates the table name and the schema name according to the first left bracket and the previous space of the SQL original text, and stores the table name in the analysis structure tab representing the basic information of the table.
An initialized table log analyzing unit 202, configured to obtain, from a log of an initialized table, information such as a table ID, a table space ID, and attributes of each column of the table, store the obtained table creating operation in a task structure txn of a task queue, where the task structure txn of the task queue includes a flag number of various SQL operations, the table ID, and the table space ID, specifically, a flag number of a table creating operation is set, and the obtained table ID and the table space ID are stored in the task structure txn, where the flag number is obtained by an enumeration type defined in a header file at program start and including various SQL operations (such as insertion data, deletion data, update data, new table creating operation, and deletion table), and in a specific embodiment of the present invention, the flag number of the table creating operation (table creating operation) may be set to 15 (i.e., representing a table creating operation), where enumeration in the enumeration type generally starts from 0, that is, that a flag number corresponding to a current task structure operation is 16, and a step S2 and a step S3 are determined according to which flag number of the operation is required to the operation. .
And the analysis query unit 203 is used for reading the task structure according to the sequence of the task queue, judging whether the table building operation is performed, acquiring the basic information of the SQL operation which is assembled into a JSON format according to the task structure, the structure tab and the database system table, wherein the basic information is sent to KAFKA. Specifically, first, the flag number of the task structure txn is read and determined according to the sequence of the task queue, and it is determined whether the operation of the currently read task structure is a table building operation according to the flag number set in the initialized table log analyzing unit 202 (for example, the flag number of the table building operation defined in the initialized table log analyzing unit 202 is 15, where it is determined whether the operation is 15 according to the flag number of the read task structure to determine whether the operation is a table building operation), and then a corresponding tab structure is found according to the table ID and the tablespace ID stored in the txn structure; inquiring in a system table of a database according to the table name provided by the tab structure body and the schema name of the table to obtain a corresponding column name, and storing the column name into the tab structure body to enable the column attribute and the column name in the tab structure body to be corresponding to each other, wherein the column name is analyzed completely, and all information in the tab structure body is complete; and then, splicing the table name, the schema name, the column names and the column attributes in the tab structure into basic information of the table building operation in the JSON format, and sending the basic information to KAFKA, thereby completing the table analysis and building operation and sending the basic information of the table building operation in the JSON format to KAFKA.
In a specific embodiment of the present invention, the database is a DB2 database, and when JSON data for table creation operation needs to be sent, a corresponding analysis structure tab is read from a memory according to a table id and a table space id in a task structure txn, column information of a corresponding table, including a column name, a column attribute, and the like, is queried in a system table sysibm.
Since the DB2 database has the multiplexing of table id, the parsing and sending may be slower than the execution speed of the database, and the user may perform the operations of table creation and re-deletion and continuous table creation and deletion, the parsing and querying unit 203 may have the possibility that the table which is not desired to be queried and is queried in the database may not be found at all, so that the exception handling is also required.
Therefore, an apparatus for parsing the column name of the table-building log of the DB2 database according to the present invention further comprises
And the exception processing unit 204 is configured to determine whether the number of columns and the attributes of the columns stored in the analysis structure tab and the temporary structure temp are the same, if the number of columns and the attributes of the columns stored in the analysis structure tab and the temporary structure temp are the same, the same table may be considered, the column names in the temporary structure temp are sequentially copied to the analysis structure tab, and the column name analysis of the table building operation is completed, otherwise, an error is considered, and a prompt that the table may have been deleted and exited is given.
In summary, when analyzing the log, the method and the device for analyzing the column name of the table-building log of the DB2 database according to the present invention first separate the table name and the schema name from the SQL statement, obtain the table ID, the table space ID and the attribute information of each column of the table from the log of the initialized table, store these information in the analysis structure, simultaneously construct a task structure, set a flag number of the table-building operation in the task structure, store the obtained table ID and the table space ID therein, assemble the table name, the schema name, the column attribute and other information into the basic information of the SQL operation in JSON format according to the task structure, the analysis structure and the database system table in the order of the task queue, and send the basic information to KAFKA, thereby achieving the purpose of obtaining the column name in the table-building log analysis.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Modifications and variations can be made to the above-described embodiments by those skilled in the art without departing from the spirit and scope of the present invention. Therefore, the scope of the invention should be determined from the following claims.
Claims (10)
1. A method for analyzing column names of table-building logs of a DB2 database comprises the following steps:
step S1, when the table building log is analyzed, analyzing the original SQL log, separating a table name and a schema name from an SQL statement, and storing the table name and the schema name into an analysis structural body for representing basic information of the table;
s2, acquiring a table ID, a table space ID and each row of attribute information of the table from the log of the initialized table, storing the table ID, the table space ID and each row of attribute information of the table into the analysis structure, constructing a task structure, setting a flag number of table construction operation in the task structure, and storing the acquired table ID and the table space ID into the task structure;
and S3, reading the task structure according to the sequence of the task queue, judging whether the operation of the currently read task structure is a table building operation or not according to the number of the marks, and for the table building operation, obtaining basic information of the SQL operation in which the table name, the schema name, the column name and the column attribute information are assembled into a JSON format according to the task structure, the analysis structure and the database system table and sending the basic information to KAFKA.
2. The method of claim 1, wherein the method comprises: in step S1, for the table building statement with parentheses in the original SQL log, the table name and the schema name are obtained by separating the first left parentheses and the space before the left parentheses.
3. The method of claim 2, wherein the method comprises: and for the table building statements without parentheses in the original SQL log, separating the table names and the schema names according to the table building syntax.
4. A method of table log column name resolution for DB2 databases as claimed in claim 3, wherein: in step S3, reading the task structure according to the sequence of the task queue, determining whether the operation of the current task structure is a table building operation according to the number of the flags set in step S2, and finding a corresponding analysis structure according to the table ID and the table space ID stored in the task structure if the operation is the table building operation; inquiring in a database system table according to the table name provided by the analysis structural body and the schema name of the table to obtain a corresponding column name, and storing the column name into the analysis structural body so as to enable the column attribute and the column name in the analysis structural body to be corresponding to each other; and finally, splicing the table name, the schema name, the column names and the column attributes in the analysis structure body into basic information of the table building operation in the JSON format, and sending the basic information to KAFKA.
5. The method of claim 4, wherein the method comprises: before the corresponding column name is stored in the analysis structure, the inquired information is stored in a new temporary structure, and is compared with the analysis structure for exception handling.
6. The method of claim 5, wherein the method comprises: in step S3, before storing the corresponding column names into the analysis structure, it is determined whether the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are the same, and if the number of columns and the attributes of the columns stored in the analysis structure and the temporary structure are the same, the same table is considered, and the column names in the temporary structure are sequentially copied into the analysis structure, thereby completing the column name analysis of the table building operation.
7. The method of claim 6, wherein the method comprises: and if the number of the columns and the attributes of the columns stored in the analysis structure body and the temporary structure body are not uniform and the same, judging that an error occurs, generating prompt information and prompting that the current table is deleted and quitting possibly.
8. An apparatus for column name resolution of a DB2 database tabulation log, comprising:
the system comprises an original SQL log analyzing unit, a table creating unit and a table creating unit, wherein the original SQL log analyzing unit is used for analyzing the original SQL log when analyzing a table creating log, separating a table name and a schema name from an SQL statement and storing the table name and the schema name into an analyzing structural body which represents table basic information;
the device comprises an initialization table log analysis unit, a task structure body and a data analysis unit, wherein the initialization table log analysis unit is used for acquiring a table ID, a table space ID and attribute information of each column of a table from a log of an initialization table, storing the table ID, the table space ID and the attribute information of each column of the table into the analysis structure body, establishing the task structure body, setting a flag number of table establishment operation in the task structure body, and storing the acquired table ID and the table space ID into the task structure body;
and the analysis query unit is used for reading the task structure bodies according to the sequence of the task queue txn, judging whether the operation of the currently read task structure body is a table building operation or not according to the number of the marks, and acquiring basic information of the SQL operation which is formed by splicing the table name, the schema name, the column name and the column attribute information into a JSON format according to the task structure bodies, the analysis structure bodies and the database system table and sending the basic information to KAFKA.
9. The apparatus for table log column name resolution of a DB2 database as claimed in claim 8, wherein: the analysis query unit determines whether the current operation is a table building operation according to the number of the flags set by the initialized table log analysis unit, and then finds a corresponding analysis structure according to the table ID and the table space ID stored in the task structure; inquiring in a database system table according to the table name provided by the analysis structural body and the schema name of the table to obtain a corresponding column name, and storing the column name into the analysis structural body so as to enable the column attribute and the column name in the analysis structural body to be corresponding to each other; and finally, splicing the table name, the schema name, the column names and the column attributes in the analysis structure body into basic information of the table building operation in the JSON format, and sending the basic information to KAFKA.
10. The apparatus for table log column name resolution of a DB2 database as claimed in claim 9, wherein: the device also comprises an exception processing unit which is used for judging whether the number of columns stored in the analysis structure body and the temporary structure body is the same before the corresponding column names are stored in the analysis structure body, judging whether the attributes of the columns stored in the analysis structure body and the temporary structure body are the same, if the number of the columns stored in the analysis structure body and the temporary structure body are the same and the attributes of the columns are also the same, the column names in the temporary structure body are copied to the analysis structure body in sequence to finish the column name analysis of the table building operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110109747.0A CN112765180B (en) | 2021-01-27 | 2021-01-27 | Method and device for analyzing column names of table building logs of DB2 database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110109747.0A CN112765180B (en) | 2021-01-27 | 2021-01-27 | Method and device for analyzing column names of table building logs of DB2 database |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112765180A CN112765180A (en) | 2021-05-07 |
| CN112765180B true CN112765180B (en) | 2023-01-17 |
Family
ID=75706019
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110109747.0A Active CN112765180B (en) | 2021-01-27 | 2021-01-27 | Method and device for analyzing column names of table building logs of DB2 database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112765180B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114118055B (en) * | 2021-12-06 | 2024-05-10 | 中电万维信息技术有限责任公司 | Optimized analysis method for complex SQL (structured query language) statement |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110879813A (en) * | 2019-11-20 | 2020-03-13 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN111241094A (en) * | 2019-12-31 | 2020-06-05 | 武汉达梦数据库有限公司 | Database deleted column synchronization method and device based on log analysis |
| CN111415199A (en) * | 2020-03-20 | 2020-07-14 | 重庆锐云科技有限公司 | Customer prediction updating method and device based on big data and storage medium |
| CN111752804A (en) * | 2020-06-29 | 2020-10-09 | 中国电子科技集团公司第二十八研究所 | Database cache system based on database log scanning |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7990897B2 (en) * | 2009-03-11 | 2011-08-02 | Sony Corporation | Method and apparatus for a wireless home mesh network with network topology visualizer |
| US9361338B2 (en) * | 2013-11-26 | 2016-06-07 | International Business Machines Corporation | Offloaded, incremental database statistics collection and optimization |
| CN105243067B (en) * | 2014-07-07 | 2019-06-28 | 北京明略软件系统有限公司 | A kind of method and device for realizing real-time incremental synchrodata |
| CN108255837B (en) * | 2016-12-28 | 2020-09-04 | 中国移动通信集团浙江有限公司 | SQL parser and method |
| CN109558452B (en) * | 2018-11-19 | 2020-12-08 | 武汉达梦数据库有限公司 | Synchronization method for query table building operation |
| CN111221907B (en) * | 2019-12-31 | 2021-03-30 | 武汉达梦数据库股份有限公司 | Database added column synchronization method and device based on log analysis |
| CN111240897B (en) * | 2020-01-07 | 2023-04-14 | 腾讯科技（深圳）有限公司 | Data processing method and related equipment |
-
2021
- 2021-01-27 CN CN202110109747.0A patent/CN112765180B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110879813A (en) * | 2019-11-20 | 2020-03-13 | 浪潮软件股份有限公司 | Binary log analysis-based MySQL database increment synchronization implementation method |
| CN111241094A (en) * | 2019-12-31 | 2020-06-05 | 武汉达梦数据库有限公司 | Database deleted column synchronization method and device based on log analysis |
| CN111415199A (en) * | 2020-03-20 | 2020-07-14 | 重庆锐云科技有限公司 | Customer prediction updating method and device based on big data and storage medium |
| CN111752804A (en) * | 2020-06-29 | 2020-10-09 | 中国电子科技集团公司第二十八研究所 | Database cache system based on database log scanning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112765180A (en) | 2021-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110908997B (en) | Data blood relationship construction method and device, server and readable storage medium | |
| US8959122B2 (en) | Data processing device | |
| US6449620B1 (en) | Method and apparatus for generating information pages using semi-structured data stored in a structured manner | |
| CN109710220B (en) | Relational database query method, relational database query device, relational database query equipment and storage medium | |
| EP3435256A2 (en) | Optimal sort key compression and index rebuilding | |
| CN112015722A (en) | Database management method, data blood relationship analysis method and related device | |
| CN111897824B (en) | Data operation method, device, equipment and storage medium | |
| CN111241800A (en) | MySQL database table structure document generation method, storage medium and intelligent terminal | |
| CN115543402B (en) | Software knowledge graph increment updating method based on code submission | |
| CN113326286A (en) | Semantic analysis method supporting dialect SQL blood margin analysis | |
| CN112765180B (en) | Method and device for analyzing column names of table building logs of DB2 database | |
| CN111460000B (en) | Backtracking data query method and system based on relational database | |
| CN117807106A (en) | Database operating system and method based on JSON and SQL | |
| WO2020024778A1 (en) | Method, system and device for modifying xml file in batch and computer-readable storage medium | |
| CN110147396B (en) | Mapping relation generation method and device | |
| CN114003231B (en) | SQL syntax parse tree optimization method and system | |
| CN111078273B (en) | Information acquisition method and device and storage medium | |
| CN107622070B (en) | Database management method and device | |
| CN111753045B (en) | Hive two-level full-text index technical method and system based on elastic search | |
| US11074401B2 (en) | Merging delta object notation documents | |
| US9916154B2 (en) | File-based transport of table content | |
| US11023674B2 (en) | Generation and application of object notation deltas | |
| CN114238326A (en) | Data batch comparison method, device, equipment and storage medium | |
| CN114461454A (en) | Data recovery method and device, storage medium and electronic equipment | |
| CN112925726A (en) | Method for automatically generating database test case and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |