add satrn #8433

zhiminzhang0830 · 2022-11-24T03:49:31Z

复现论文：On Recognizing Texts of Arbitrary Shapes with 2D Self-Attention
参考代码：https://github.com/open-mmlab/mmocr/blob/1.x/configs/textrecog/satrn/README.md

paddle-bot · 2022-11-24T03:49:35Z

Thanks for your contribution!

zhiminzhang0830 · 2022-11-24T04:01:35Z

数据集：
训练集：https://aistudio.baidu.com/aistudio/datasetdetail/166485
验证集：https://aistudio.baidu.com/aistudio/datasetdetail/182867
实验结果：
IIIK-3000:94.53，SVT：91.04，IC13:94.68，IC15:78.24，SVTP：83.72，CUTE80:86.11，Avg:88.05

模型训练：
python3 -m paddle.distributed.launch --log_dir=./debug/ --gpus '0,1,2,3' tools/train.py -c configs/rec/rec_satrn.yml

模型验证：
python tools/eval.py -c {your config file}
-o Global.pretrained_model={your model file}
Eval.dataset.data_dir={your dataset path}/IIIT5k_3000

模型测试：
python3 tools/infer_rec.py -c {your config file}
-o Global.pretrained_model={your model file}
Global.infer_img="doc/imgs_words_en/"

存在问题：
1.使用python tools/export_model.py导出模型时，速度非常慢，大概需要7分钟才能导出模型
2.使用导出的模型做推理的时候报错，报错信息如下：
[libprotobuf ERROR /paddle/build/third_party/protobuf/src/extern_protobuf/src/google/protobuf/io/coded_stream.cc:208] A protocol message was rejected because it was too big (more than 67108864 bytes). To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
[libprotobuf ERROR /paddle/build/third_party/protobuf/src/extern_protobuf/src/google/protobuf/io/coded_stream.cc:208] A protocol message was rejected because it was too big (more than 67108864 bytes). To increase the limit (or to disable these warnings), see CodedInputStream::SetTotalBytesLimit() in google/protobuf/io/coded_stream.h.
Traceback (most recent call last):
File "tools/infer/predict_rec.py", line 690, in
main(utility.parse_args())
File "tools/infer/predict_rec.py", line 652, in main
text_recognizer = TextRecognizer(args)
File "tools/infer/predict_rec.py", line 127, in init
utility.create_predictor(args, 'rec', logger)
File "/data/code/PaddleOCR_satrn/tools/infer/utility.py", line 277, in create_predictor
predictor = inference.create_predictor(config)
ValueError: (InvalidArgument) Failed to parse program_desc from binary string.
[Hint: Expected desc_.ParseFromString(binary_str) == true, but received desc_.ParseFromString(binary_str):0 != true:1.] (at /paddle/paddle/fluid/framework/program_desc.cc:103)

zhiminzhang0830 · 2022-11-24T04:13:22Z

模型链接：https://pan.baidu.com/s/10J-Bsd881bimKaclKszlaQ?pwd=lk8a

Topdu · 2023-01-31T02:53:50Z

satrn_head.py的526行修改为：for step in range(0, paddle.to_tensor(self.max_seq_len)):
这样可以解决导出inference model 慢的问题呢，
推理时修改predict_rec.py 449行为： elif self.rec_algorithm in ["SVTR", "SATRN"]:
推理的结果是：
Predicts of ./doc/imgs_words_en/word_19.png:('slowuknuknuknuknuknuknuknuknuknuknukniuknuknuknuknuknuknuknuknukn', 0.5304282307624817)
看结果分析，后处理似乎没有找到eos。
推理命令：
python tools/infer/predict_rec.py --image_dir='./doc/imgs_words_en/word_19.png' --rec_model_dir='./inference/satrn/' --rec_algorithm='SATRN' --rec_image_shape='3,32,100' --rec_char_dict_path='./ppocr/utils/dict90.txt'

Topdu · 2023-01-31T02:56:25Z

satrn 和 nrtr是同类型的识别模型，如果可以的话尽量复用nrtr的代码，例如shallow cnn可以写到MTB中，attention和encoder layer和decoder layer 如果差别不大的话也可以复用nrtr的代码

Topdu · 2023-01-31T03:01:21Z

ppocr/modeling/heads/rec_satrn_head.py

+ init_target_seq[:, 0] = self.start_idx
+
+ outputs = []
+ for step in range(0, self.max_seq_len):


for step in range(0, paddle.to_tensor(self.max_seq_len)):

zhiminzhang0830 · 2023-02-07T10:38:12Z

导出模型：
python3 tools/export_model.py -c configs/rec/rec_satrn.yml
-o Global.pretrained_model=inference/satrn/rec_satrn/best_accuracy.pdparams
Global.save_inference_dir=./inference/satrn/
模型推理：
python tools/infer/predict_rec.py --image_dir='./doc/imgs_words_en/word_19.png' --rec_model_dir='./inference/satrn/' --rec_algorithm='SATRN' --rec_image_shape='3,32,100' --rec_char_dict_path='./ppocr/utils/dict90.txt' --use_space_char='False'

Topdu · 2023-02-07T11:04:02Z

configs/rec/rec_satrn.yml

+ epoch_num: 5
+ log_smooth_window: 20
+ print_batch_step: 50
+ save_model_dir: ../work_dir/ppocr/satrn_branch/rec_satrn/


save_model_dir最好与其他方法修改一致：./output/rec/rec_satrn

Topdu · 2023-02-07T11:04:49Z

configs/rec/rec_satrn.yml

+Train:
+ dataset:
+ name: LMDBDataSet
+ data_dir: /data/Dataset/OCR_Rec/visual_data/rfl_dataset2/training


./train_data/data_lmdb_release/training/

Topdu · 2023-02-07T11:05:04Z

configs/rec/rec_satrn.yml

+Eval:
+ dataset:
+ name: LMDBDataSet
+ data_dir: /data/Dataset/OCR_Rec/visual_data/rfl_dataset2/evaluation_academic


./train_data/data_lmdb_release/evaluation/

Topdu · 2023-02-07T11:06:08Z

ppocr/data/imaug/rec_img_aug.py

@@ -465,6 +465,21 @@ def __call__(self, data):
 return data


+class SATRNRecResizeImg(object):
+ def __init__(self, image_shape, padding=True, **kwargs):


如果没用到SATRNRecResizeImg的话可以删除

tink2123

LGTM

tink2123 · 2023-02-08T03:11:48Z

ppocr/modeling/heads/rec_satrn_head.py

+ if mask is not None:
+ attn = masked_fill(attn, mask == 0, -1e9)
+ # attn = attn.masked_fill(mask == 0, float('-inf'))
+ # attn += mask


todo：不必要的注释可以删除

tink2123 · 2023-02-08T03:19:39Z

需要补充文档并接入TIPC

add satrn

3a58440

paddle-bot bot added contributor status: proposed labels Nov 24, 2022

Topdu reviewed Jan 31, 2023

View reviewed changes

修复satrn导出问题

10361a6

Topdu reviewed Feb 7, 2023

View reviewed changes

zhiminzhang0830 added 2 commits February 7, 2023 19:13

规范satrn config文件

e588389

删除SATRNRecResizeImg

65fc28b

tink2123 approved these changes Feb 8, 2023

View reviewed changes

tink2123 merged commit 30201ef into PaddlePaddle:dygraph Feb 8, 2023

chenjjcccc mentioned this pull request Oct 24, 2023

补充Satrn识别模型文档 #11131

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add satrn #8433

add satrn #8433

zhiminzhang0830 commented Nov 24, 2022

paddle-bot bot commented Nov 24, 2022

zhiminzhang0830 commented Nov 24, 2022 •

edited

Loading

zhiminzhang0830 commented Nov 24, 2022

Topdu commented Jan 31, 2023 •

edited

Loading

Topdu commented Jan 31, 2023

Topdu Jan 31, 2023

zhiminzhang0830 commented Feb 7, 2023

Topdu Feb 7, 2023

Topdu Feb 7, 2023

Topdu Feb 7, 2023

Topdu Feb 7, 2023

tink2123 left a comment

tink2123 Feb 8, 2023

tink2123 commented Feb 8, 2023

add satrn #8433

add satrn #8433

Conversation

zhiminzhang0830 commented Nov 24, 2022

paddle-bot bot commented Nov 24, 2022

zhiminzhang0830 commented Nov 24, 2022 • edited Loading

zhiminzhang0830 commented Nov 24, 2022

Topdu commented Jan 31, 2023 • edited Loading

Topdu commented Jan 31, 2023

Topdu Jan 31, 2023

Choose a reason for hiding this comment

zhiminzhang0830 commented Feb 7, 2023

Topdu Feb 7, 2023

Choose a reason for hiding this comment

Topdu Feb 7, 2023

Choose a reason for hiding this comment

Topdu Feb 7, 2023

Choose a reason for hiding this comment

Topdu Feb 7, 2023

Choose a reason for hiding this comment

tink2123 left a comment

Choose a reason for hiding this comment

tink2123 Feb 8, 2023

Choose a reason for hiding this comment

tink2123 commented Feb 8, 2023

zhiminzhang0830 commented Nov 24, 2022 •

edited

Loading

Topdu commented Jan 31, 2023 •

edited

Loading