Skip to content

机器翻译平行语料过滤的脚本

Notifications You must be signed in to change notification settings

prigioni/corpus_filter

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

该脚本有过滤语料的作用

过滤特征有:

1.双语句子长度比率

2.重复句子

目前支持的语种类:

  • 英文<-->中文
  • 日文<-->中文
  • 韩文<-->中文

By Liam 9th/Aug/2018

About

机器翻译平行语料过滤的脚本

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%