[BUG]After deleting and rebuilding tenants with the same name, the old data still exists

[lutengda]

Describe the bug

version: cnosdb 2.3.1, revision 74fdaa3
mode: singleton

nohup ./target/debug/cnosdb run --config ./config/config_8902.toml -M singleton &

The default cnosdb tenant and root user login:

./target/debug/cnosdb-cli

Create tenants and users:

CREATE TENANT tenant_a;
CREATE USER user_a;
ALTER TENANT tenant_a ADD USER user_a AS owner;

Log in with tenant a and user a:

./target/debug/cnosdb-cli --tenant "tenant_a" --user "user_a"

Build database, build table and write data:

create database db1;
CREATE TABLE air_a (visibility DOUBLE,temperature DOUBLE,pressure DOUBLE,TAGS(station));
INSERT INTO air_a (TIME, station, visibility, temperature, pressure) VALUES(1666165200290401000, 'XiaoMaiDao', 56, 69, 77);

The default cnosdb tenant and root user login:

./target/debug/cnosdb-cli

Remove tenants and users:

drop tenant tenant_a;
drop user user_a;

Create tenants and users with the same name again:

CREATE TENANT tenant_a;
CREATE USER user_a;
ALTER TENANT tenant_a ADD USER user_a AS owner;

Log in with tenant a and user a:

./target/debug/cnosdb-cli --tenant "tenant_a" --user "user_a"

You'll find that the old data is still there

To Reproduce

No response

Expected behavior

I believe that after deleting a tenant, even if a new tenant with the same name is created, the old data should not exist

Additional context

No response

[lutengda]

My advice:
When executing the SQL command drop tenant tenantA, the tenantA is marked as deleting status and the SQL command returns immediately. A background thread is started to delete the data. After the data is deleted, the metadata of tenantA is deleted. During this period, attempts to delete or create a tenant with the same name as tenantA will return a message indicating that it is being deleted. If the data deletion fails, an error log is printed. Subsequent attempts to delete or create a tenant with the same name as tenantA will return the same error message.

[lutengda]

I has another idea:
There are two resources under the tenant, role and database, so when you execute drop tenant tenantA, delete the role and database under tenantA, and then delete the tenantA metadata.
This plan is also consistent with other drops.

[lutengda]

when drop tenant, drop database and role in this tenant, so close this issue