支持doris sequence列模型

[baisui1981]

在doris chunjun实现中添加 Sequece列模型 , 批量删除来防止，Doris 数据脏写

doris 写入相关类：

1. com.dtstack.chunjun.connector.doris.rest.DorisStreamLoad
2. com.dtstack.chunjun.connector.doris.rest.DorisLoadClient

[baisui1981]

试验delete操作，按照Doris 的文档 并不能将已有数据删除：
create DDL:

CREATE TABLE `instancedetail` (
  `instance_id` varchar(96) NOT NULL COMMENT "",
  `order_id` varchar(96) NULL COMMENT "",
  `batch_msg` varchar(765) NULL COMMENT "",
  `type` smallint(6) NULL COMMENT "",
  `ext` varchar(65000) NULL COMMENT "",
  `waitinginstance_id` varchar(96) NULL COMMENT "",
  `kind` smallint(6) NULL COMMENT "",
  `parent_id` varchar(96) NULL COMMENT "",
  `pricemode` smallint(6) NULL COMMENT "",
  `name` varchar(150) NULL COMMENT "",
  `makename` varchar(150) NULL COMMENT "",
  `taste` varchar(765) NULL COMMENT "",
  `spec_detail_name` varchar(150) NULL COMMENT "",
  `num` decimal(10, 2) NULL COMMENT "",
  `account_num` decimal(12, 4) NULL COMMENT "",
  `unit` varchar(150) NULL COMMENT "",
  `account_unit` varchar(150) NULL COMMENT "",
  `price` decimal(18, 2) NULL COMMENT "",
  `member_price` decimal(18, 2) NULL COMMENT "",
  `fee` decimal(18, 2) NULL COMMENT "",
  `ratio` decimal(10, 2) NULL COMMENT "",
  `ratio_fee` decimal(18, 2) NULL COMMENT "",
  `ratio_cause` varchar(765) NULL COMMENT "",
  `status` smallint(6) NULL COMMENT "",
  `kindmenu_id` varchar(96) NULL COMMENT "",
  `kindmenu_name` varchar(150) NULL COMMENT "",
  `menu_id` varchar(96) NULL COMMENT "",
  `memo` varchar(765) NULL COMMENT "",
  `is_ratio` smallint(6) NULL COMMENT "",
  `entity_id` varchar(96) NULL COMMENT "",
  `is_valid` smallint(6) NULL COMMENT "",
  `create_time` bigint(20) NULL COMMENT "",
  `op_time` bigint(20) NULL COMMENT "",
  `last_ver` bigint(20) NULL COMMENT "",
  `load_time` int(11) NULL COMMENT "",
  `modify_time` int(11) NULL COMMENT "",
  `draw_status` tinyint(4) NULL COMMENT "",
  `bookmenu_id` varchar(96) NULL COMMENT "",
  `make_id` varchar(96) NULL COMMENT "",
  `make_price` decimal(18, 2) NULL COMMENT "",
  `prodplan_id` varchar(96) NULL COMMENT "",
  `is_wait` tinyint(4) NULL COMMENT "",
  `specdetail_id` varchar(96) NULL COMMENT "",
  `specdetail_price` decimal(18, 2) NULL COMMENT "",
  `makeprice_mode` tinyint(4) NULL COMMENT "",
  `original_price` varchar(96) NULL COMMENT "",
  `is_buynumber_changed` tinyint(4) NULL COMMENT "",
  `ratio_operator_id` varchar(96) NULL COMMENT "",
  `child_id` varchar(96) NULL COMMENT "",
  `kind_bookmenu_id` varchar(96) NULL COMMENT "",
  `specprice_mode` tinyint(4) NULL COMMENT "",
  `worker_id` varchar(96) NULL COMMENT "",
  `is_backauth` tinyint(4) NULL COMMENT "",
  `service_fee_mode` tinyint(4) NULL COMMENT "",
  `service_fee` varchar(96) NULL COMMENT "",
  `orign_id` varchar(96) NULL COMMENT "",
  `addition_price` decimal(18, 2) NULL COMMENT "",
  `has_addition` tinyint(4) NULL COMMENT "",
  `seat_id` varchar(96) NULL COMMENT "",
  `__DORIS_DELETE_SIGN__` tinyint(4) NOT NULL DEFAULT "0" COMMENT "doris delete flag hidden column"
) ENGINE=OLAP
UNIQUE KEY(`instance_id`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`instance_id`) BUCKETS 10
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"in_memory" = "false",
"storage_format" = "V2"
)

通过cdc的方式向mysql 表中新增两条记录，能够append 到Doris库中：

INSERT INTO `instancedetail` 
VALUES ('000a8','10000015361171282085747c1f92005a',NULL,0
                   ,'{\"hitPrice\":0,\"isWait\":0,\"optionalType\":0}',NULL,3,NULL,1,'test02',NULL,NULL,NULL,1.00,1.0000
          ,'?',NULL,1.00,1.00,1.00,100.00,1.00,NULL,2,'8000001564d710600164f9cd0c510172','美丽','0',NULL,0,'80000015'
          ,1,1535257651481,1535257670907,3,1536117832,1678704159,0,'','',0.00,'',0,'',0.00,1,'1.0',1,'','',''
        ,2,'05eb689d72304a8d9064b20f58953d91',1,0,'0.0',NULL,0.00,0,'');

随后 将该条主键为000a8的记录需要删除掉，另外新增一条新的记录。使用streamload的方式：
构建新增记录的 data.json 文件：

[{"original_price":"1.0","taste":"","num":"1.00","fee":"1.00","op_time":"1535257670907","addition_price":"0.00","memo":"","type":"0","makename":"","is_buynumber_changed":"1","makeprice_mode":"1","price":"1.00","specprice_mode":"2","has_addition":"0","make_price":"0.00","ext":"{\"hitPrice\":0,\"isWait\":0,\"optionalType\":0}","bookmenu_id":"","create_time":"1535257651481","is_backauth":"1","kind":"3","entity_id":"80000015","draw_status":"0","make_id":"","ratio_cause":"","worker_id":"05eb689d72304a8d9064b20f58953d91","unit":"?","instance_id":"000a9","parent_id":"","is_valid":"1","name":"test02","pricemode":"1","order_id":"10000015361171282085747c1f92005a","status":"2","kindmenu_name":"美丽","ratio_fee":"1.00","seat_id":"","modify_time":"1678704159","service_fee_mode":"0","prodplan_id":"","member_price":"1.00","batch_msg":"","account_num":"1.0000","load_time":"1536117832","account_unit":"","spec_detail_name":"","menu_id":"0","kindmenu_id":"8000001564d710600164f9cd0c510172","specdetail_price":"0.00","service_fee":"0.0","is_ratio":"0","last_ver":"3","specdetail_id":"","child_id":"","kind_bookmenu_id":"","is_wait":"0","ratio_operator_id":"","waitinginstance_id":"","orign_id":"","ratio":"100.00"}]

触发streamload 请求：

curl -X PUT  -u root:  -H "format: json" -H "strip_outer_array: true"  -H "merge_type: MERGE" -H "delete: instance_id='000a8'" -T ./data.json  http:https://192.168.28.201:8040/api/tis/instancedetail/_stream_load

服务端返回结果，执行成功：

{
    "TxnId": 1025,
    "Label": "7ca6b928-ea2d-494b-bcfe-2101c1a6a2c8",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 1,
    "NumberLoadedRows": 1,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 1212,
    "LoadTimeMs": 36,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 2,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 19,
    "CommitAndPublishTimeMs": 12
}

但是通过doris MySQL 后台查看需要被删除的数据仍然存在：

select * from instancedetail where instance_id='000a8'

期望已经被删除了。

[baisui1981]

不能删除数据的原因找到了，在data.json文件中需要将 被删除的数据也要录入进去，不然是删不掉，光在head中出现-H "delete: instance_id='000a8'"，是没法删除数据的

[baisui1981]

支持 seq的表ddl例子：

create table `users`
(
siteid int not null,
citycode int ,
username varchar(100) ,
pv int 
)
ENGINE=olap
UNIQUE KEY(`siteid`)
DISTRIBUTED BY HASH(`siteid`)
BUCKETS 10
PROPERTIES("replication_num" = "1" , "function_column.sequence_col" = 'pv', "function_column.sequence_type"='int')

[baisui1981]

已经支持了