Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The class with the same name was imported twice #999

Closed
D-X-Y opened this issue Jul 19, 2023 · 0 comments · Fixed by #1042
Closed

The class with the same name was imported twice #999

D-X-Y opened this issue Jul 19, 2023 · 0 comments · Fixed by #1042
Labels
bug Something isn't working

Comments

@D-X-Y
Copy link

D-X-Y commented Jul 19, 2023

Describe the bug
RT, in the header of megatron/tokenizer/tokenizer.py

from transformers import GPT2Tokenizer, GPT2TokenizerFast
import numpy as np
import sentencepiece as spm
from typing import List, Union
from .gpt2_tokenization import GPT2Tokenizer

GPT2Tokenizer are imported twice, will it cause some potential issues?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove the NeoX implementation of GPT2Tokenizer EleutherAI/gpt-neox
1 participant
@D-X-Y