Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MidiDataset optimizations #68

Merged
merged 11 commits into from
Nov 22, 2023
Merged

MidiDataset optimizations #68

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0230715
MidiDataset can initialize with an iterator and only expand when nece…
honglu2875 Nov 9, 2023
c1d170d
reduce some memory overhead (we are starting to have >100k MidiDict a…
honglu2875 Nov 9, 2023
deec979
classmethod+property is better...
honglu2875 Nov 10, 2023
6e58f09
remove functools import
honglu2875 Nov 10, 2023
00cef03
Merge branch 'EleutherAI:main' into dev
honglu2875 Nov 10, 2023
7073a1a
use separate workers to build dataset instead of process pool
honglu2875 Nov 10, 2023
69922b4
merge
honglu2875 Nov 10, 2023
8bbc14b
add jsonl.zst support; unit test; fix bug
honglu2875 Nov 10, 2023
95f492d
receive context length via commandline. It's more convenient than dig…
honglu2875 Nov 10, 2023
93301fa
fix a minor output format mismatch when grad_checkpoint is true
honglu2875 Nov 12, 2023
607c438
format and small changes
loubbrad Nov 22, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
classmethod+property is better...
  • Loading branch information
@honglu2875
honglu2875 committed Nov 10, 2023
commit deec979fa8a13c66c9f314b286bbe9444597b1db
5 changes: 3 additions & 2 deletions aria/data/midi.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,8 +114,9 @@ def __init__(
}
]

@functools.cached_property
def program_to_instrument(self):
@classmethod
@property
def program_to_instrument(cls):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming this is here to speed up the process of building MidiDatasets. Does it make much of a difference?

# This combines the individual dictionaries into one
return (
{i: "piano" for i in range(0, 7 + 1)}
Expand Down