Skip to content

Commit

Permalink
Fix typo
Browse files Browse the repository at this point in the history
  • Loading branch information
segyges committed Nov 8, 2023
1 parent f84c956 commit 05fd181
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion utils/scrape.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import requests
import re

# This scraps huggingface to get the pythia sha256 hashes because, so far as I can tell, there's no way to make huggingface deliver this information via their api
# This scrapes huggingface to get the pythia sha256 hashes because, so far as I can tell, there's no way to make huggingface deliver this information via their api
base_url = 'https://huggingface.co/datasets/EleutherAI/pythia_deduped_pile_idxmaps/blob/main/pile_0.87_deduped_text_document-'

sha256_list = [] # List to store extracted SHA256 hashes
Expand Down

0 comments on commit 05fd181

Please sign in to comment.