Known issues (and next release)

[mdeff]

Below are issues affecting the rc1 data release that cannot be fixed without a data update. As updating is disruptive (it'll break code and make results non-comparable), it should be done sparingly, e.g., to fix a fatal flaw or many small ones discovered over time.

• zip decompression fails because of unsupported bzip2 compression (zip: compress with deflate instead of bzip2 #5)
  • workaround (master): note in README to try with 7zip (5700859)
  • fix (next): zip with deflate (instead of bzip2) (zip: compress with deflate instead of bzip2 #5) or zstd (Torrent download and Zstd compression #32)
• excerpts shorter than 30s and erroneous audio length metadata (update technical metadata #4, Files (much) shorter than 30s in fma-small #8, Possibly corrupted files in fma_small #36, Corrupted Files? #44)
  • workaround (master): small subset's list, medium subset's list (Files (much) shorter than 30s in fma-small #8)
  • fix (next): metadata from mp3 not API, ensure 30s (8077afe, 00d5b71, 840b337)
• erroneous ID3 tags (erroneous ID3 tag info #27)
  • workaround (master): list (erroneous ID3 tag info #27)
  • fix (next): dump ID3 tags with technical metadata and remove from mp3
• exact duplicate tracks (Duplicate mp3s in fma_full? #23)
  • workaround (master): list the 937 duplicates
  • fix (next): remove them (try other methods and detect near duplicates)

Workarounds are explained in more details in the wiki.

[mdeff]

Branches:

• The master branch contains the code that produced the latest released data. The usage code and documentation can be updated but should work with the released data.
• The next branch contains the code to produce the hypothetical next release of the data. The usage code is updated to any new data format.
• The outputs branch is based on master and contains generated data (e.g., notebook outputs and figures) for convenience (most notably to run on binder).

[mdeff]

Potential todos for a dataset update:

• schema with type and explanation of every field (definition of favorites, listens and interest in track features #14)
• Data format: relational tables (tracks.csv, artists.csv, albums.csv) instead of a single huge tracks.csv? Consider standards like JAMS.
• Since https://freemusicarchive.org has been acquired it became a static archive. It might make sense to dump it one last time.
• Recompute all features available in the latest librosa (Tempo metadata? #37).
• Consider another hosting provider (host on cloud computing provider #26) or torrents (Torrent download and Zstd compression #32). It would be nice to have storage guarantees and a DOI. Maybe Zenodo would now accept it (they wouldn't in 2017).
• Add to additional dataset lists (List on Google Dataset Search #35).