Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace memoizing cache for parsed CIGARs with core.memoize's LU cache #189

Merged
merged 1 commit into from
Feb 13, 2020

Conversation

athos
Copy link
Member

@athos athos commented Feb 12, 2020

cljam has a memoizing cache for parsed CIGAR strings, and it tends to limitlessly increase in size after a long-term process such as pileup. The following chart shows how large the whole memory usage got during a certain run of variant call using cljam's pileup. Note that the bottom line of the used heap size is steadily getting higher as time goes along:

Memory usage (using memoize)

This PR replaces the memoizing cache with core.memoize's LU cache, limiting the cache size to an appropriate upper bound. Also, it defines a new system property named cljam.sam.cigar.cache-size to specify the upper bound of cache size when launching the JVM.

After applying the change, memory usage would grow as follows. In this case, the bottom line keeps as low as the beginning throughout the whole run.

Memory usage (using core memoize with threshold = 4096)

See here for the detailed data for the above charts.

@athos athos requested a review from alumi as a code owner February 12, 2020 09:12
@athos athos requested a review from a team February 12, 2020 09:12
@ghost ghost requested review from r6eve and removed request for a team February 12, 2020 09:12
@athos athos assigned athos and alumi and unassigned athos Feb 12, 2020
@codecov
Copy link

codecov bot commented Feb 12, 2020

Codecov Report

Merging #189 into master will decrease coverage by 0.01%.
The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #189      +/-   ##
==========================================
- Coverage   86.68%   86.67%   -0.02%     
==========================================
  Files          76       76              
  Lines        6055     6057       +2     
  Branches      501      502       +1     
==========================================
+ Hits         5249     5250       +1     
  Misses        305      305              
- Partials      501      502       +1
Impacted Files Coverage Δ
src/cljam/io/sam/util/cigar.clj 73.41% <66.66%> (-0.61%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c320af1...d1c7312. Read the comment docs.

Copy link
Member

@alumi alumi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Copy link
Contributor

@r6eve r6eve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@r6eve r6eve merged commit 0eb2542 into master Feb 13, 2020
@r6eve r6eve deleted the spike/cache-replacement-for-cigar branch February 13, 2020 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants