Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement buffered reading [10x speed increase] #33

Merged
merged 3 commits into from
Oct 16, 2018

Conversation

suyashkumar
Copy link
Contributor

After using pprof to run CPU profiles of the existing code (which I thought seemed slower than it should be), I noticed a lot of CPU time was going to syscalls. Given the file reading task that this code was performing this did not make sense (likey something I/O heavy going on here). After auditing the code I realized that the default readers in this codebase were unbuffered. Changing to buffered readers led to a 10x speed boost when reading a Native PixelData dicom from disk:

Before:

./build/dicomutil --extract-images-stream   13.86s user 9.69s system 106% cpu 22.204 total

After:

./build/dicomutil --extract-images-stream   2.91s user 0.38s system 159% cpu 2.061 total

pprof before:

(pprof) top5
Showing nodes accounting for 16.58s, 97.13% of 17.07s total
Dropped 69 nodes (cum <= 0.09s)
Showing top 5 nodes out of 43
      flat  flat%   sum%        cum   cum%
    15.46s 90.57% 90.57%     15.46s 90.57%  syscall.Syscall
     0.53s  3.10% 93.67%      0.58s  3.40%  runtime.mallocgc
     0.23s  1.35% 95.02%      0.23s  1.35%  runtime.greyobject
     0.20s  1.17% 96.19%      0.20s  1.17%  image.Rectangle.At
     0.16s  0.94% 97.13%      0.49s  2.87%  runtime.scanobject

pprof after change:

(pprof) top10
Showing nodes accounting for 2680ms, 92.10% of 2910ms total
Dropped 20 nodes (cum <= 14.55ms)
Showing top 10 nodes out of 82
      flat  flat%   sum%        cum   cum%
     930ms 31.96% 31.96%     1050ms 36.08%  runtime.mallocgc
     860ms 29.55% 61.51%     1510ms 51.89%  github.com/gradienthealth/dicom.readNativeFrames
     220ms  7.56% 69.07%      220ms  7.56%  image.(*Gray16).SetGray16
     210ms  7.22% 76.29%      210ms  7.22%  runtime.greyobject
     140ms  4.81% 81.10%      140ms  4.81%  syscall.Syscall
     110ms  3.78% 84.88%      360ms 12.37%  runtime.scanobject
      60ms  2.06% 86.94%      900ms 30.93%  main.generateNativeImage
      60ms  2.06% 89.00%       60ms  2.06%  runtime.memclrNoHeapPointers
      50ms  1.72% 90.72%       50ms  1.72%  runtime.memmove
      40ms  1.37% 92.10%       40ms  1.37%  runtime.heapBitsForObject

@suyashkumar suyashkumar self-assigned this Oct 16, 2018
@@ -16,7 +16,8 @@ import (
"github.com/gradienthealth/dicom"
"github.com/gradienthealth/dicom/dicomlog"
"github.com/gradienthealth/dicom/dicomtag"
)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dd

@suyashkumar suyashkumar merged commit 7c0f36e into master Oct 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant