Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor data loading workflow and support single evaluation mode #6

Merged
merged 12 commits into from
Apr 21, 2021

Conversation

sian-chen
Copy link

  1. Change the data format
  2. Remove code for handling MIMIC
  3. Refactor data loading workflow to support single evaluation mode

@sian-chen sian-chen requested review from a team as code owners April 17, 2021 08:43
@sian-chen sian-chen changed the title Single eval Refactor data loading workflow and support single evaluation mode Apr 17, 2021
@henryyang42 henryyang42 removed request for a team April 19, 2021 04:52
data_utils.py Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Show resolved Hide resolved
data_utils.py Show resolved Hide resolved
data_utils.py Outdated Show resolved Hide resolved
data_utils.py Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
data_utils.py Outdated
vocabs = Vocab(counter, specials=['<pad>', '<unk>'], min_freq=min_vocab_freq)

for data in dataset:
counter.update(set(data['text']))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original way of having "unique_tokens" may give better readability?

data_utils.py Show resolved Hide resolved
evaluate.py Outdated Show resolved Hide resolved
cjlin1
cjlin1 previously approved these changes Apr 21, 2021
README.md Outdated Show resolved Hide resolved
@sian-chen sian-chen requested a review from cjlin1 April 21, 2021 06:13
@sian-chen sian-chen merged commit 0d40051 into master Apr 21, 2021
@sian-chen sian-chen deleted the single_eval branch April 21, 2021 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants