Fix selection of VOLUME images

[hackermd]

We have been assuming that VOLUME images of a digital slides (images with the same Frame of Reference UID and Container Identifier) have the same Photometric Interpretation. The motivation for this has been to separate color and monochrome images. However, the value of Photometric Intpretation is dependent on the transfer syntax and color or monochrome images may have a different Photometric Interpretation when they have been compressed with a different codec (e.g., JPEG 2000 and JPEG).

[github-actions]

Visit the preview URL for this PR (updated for commit 9b43a2c):

https://idc-external-006--pr83-bugfix-volume-image-s89eob8w.web.app

_{(expires Sat, 05 Feb 2022 17:58:16 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

[Punzo]

LGTM!

[fedorov]

Comments from @hackermd made on slack, sharing here for the sake of facilitating conversation with @dclunie

In the current dev version of the dicom-microscopy-viewer (not yet publicly released), we specify many more acceptable media types for retrieval of frames via the Accept header field:
multipart/related; type="image/jpeg"; transfer-syntax=1.2.840.10008.1.2.4.50, multipart/related; type="image/jls"; transfer-syntax=1.2.840.10008.1.2.4.80, multipart/related; type="image/jls"; transfer-syntax=1.2.840.10008.1.2.4.81, multipart/related; type="image/jp2"; transfer-syntax=1.2.840.10008.1.2.4.90, multipart/related; type="image/jp2"; transfer-syntax=1.2.840.10008.1.2.4.91, multipart/related; type="image/jpx"; transfer-syntax=1.2.840.10008.1.2.4.92, multipart/related; type="image/jpx"; transfer-syntax=1.2.840.10008.1.2.4.93, multipart/related; type="application/octet-stream"; transfer-syntax=*
However, in that case we are getting another error:
generic::invalid_argument: image cannot be transcoded from 1.2.840.10008.1.2.4.91 to image/jpeg; transfer-syntax=1.2.840.10008.1.2.4.50, cause (internal only): generic::invalid_argument: com.google.apps.framework.request.StatusException:

generic::INVALID_ARGUMENT:
We usually don't get that error when retrieving JPEG 2000 encoded frames, so my guess is that this is an "exotic" JPEG 2000 compression that causes a server-side error.

1:09
When I retrieve the frames using Accept header field multipart/related; type="application/octet-stream"; transfer-syntax=* (a Google hack, which I am not sure is conformant with the standard), the server returns the frames, but the client-side decoder fails. Something is weird about those JPEG 2000 bitstreams. I didn't delve into this yet, but maybe @dclunie has further insight.

[fedorov]

All samples I tried to open in CPTAC-COAD failed to zoom.

A related bug in IDC portal is that OHIF Viewer, not Slim, is opened when attempting to open individual SM series as opposed to the study opening - see ImagingDataCommons/IDC-WebApp#872.

[hackermd]

@dclunie has anything changed in the JPEG 2000 encoding of frames when compared to the previous conversion? Since Healthcare API and client-side decoder both fail to decode the images, I suspect that it's due to the JPEG 2000 bitstreams.

[dclunie]

A couple of observations related to the "Slim won't zoom this series" problem, which may or may not be actually related to this pull request #83, as opposed to some other tracker or PR.

Re. PhotometricInterpretation

Since the beginning, I have been converting SVS images such that the first IFD, which is the highest resolution layer, is left in whatever tiled compressed encoding it was (JPEG or J2K), and the second IFD, which is stripped rather than tiled in the SVS, has always been decompressed, so there has always been (and is for all SVS converted CPTAC and TCGA images whether Slim zooms them or not) a difference in PhotometricInterpretation for at least these two (i.e., YBR_FULL_422 or RGB for JPEG (since Aperio sometimes color transforms for JPEG, and sometimes doesn't), YBR_ICT for J2K and RGB for uncompressed).

This is the case for one example we saw on today's call that fails to zoom:

CPTAC/01OV019-2f901c90-617e-46da-9fac-d68369] % dctable -describe -k PhotometricInterpretation -k TransferSyntaxUID -k ImageType DCM_*
Filename "Transfer Syntax UID" "Image Type" "Photometric Interpretation"
"DCM_0" "1.2.840.10008.1.2.4.91" "DERIVED\PRIMARY\VOLUME\NONE " "YBR_ICT "
"DCM_1" "1.2.840.10008.1.2.1" "DERIVED\PRIMARY\THUMBNAIL\RESAMPLED " "RGB "
"DCM_2" "1.2.840.10008.1.2.4.91" "DERIVED\PRIMARY\VOLUME\RESAMPLED" "YBR_ICT "
"DCM_3" "1.2.840.10008.1.2.4.91" "DERIVED\PRIMARY\VOLUME\RESAMPLED" "YBR_ICT "
"DCM_5" "1.2.840.10008.1.2.1" "DERIVED\PRIMARY\OVERVIEW\NONE " "RGB "

The original SVS for this one was "CPTAC-OV/01OV019-2f901c90-617e-46da-9fac-d68369.svs".

Re. JPEG 2000

I can confirm that, for this series at least, the encapsulated JPEG 2000 bitstream appears to be problematic, in that some codecs cannot decode it. The Java JJ2000 codec can't, for sure. Kakadu (kdu_expand) seems to be able to though (at least for the first frame that I extracted from DCM_3 and tested)

Is this perhaps related to:

https://docs.google.com/document/d/1hDj62kAZLYIzF_4-jnMd5RkpOFZaR684HPGw-S1Sjhg/edit
#7
OHIF/Viewers#2231

Is this just a J2K bitstream that Slim's codec cannot handle?

Interestingly, QuPath 0.3.1, which has the DICOM WSI BioFormats jar file, also fails to open the DICOM converted files (throws a Java codec-related exception), but it will load the SVS file, which should contain exactly the same J2K bitstreams. I mention this, because it might be worth asking Melissa why one loads with her code and not the other.

[hackermd]

Interesting! Thanks for the insight @dclunie.

I have included a check in the decode function that asserts that the decoded array buffer has the expected length. In this example, the check fails. My guess therefore is that the bitstream is truncated and some decoders may be able to handle it while others don't. Not sure whether this is compliant with the JPEG 2000 standard and just an edge feature that few bothered implementing or rather a compliance issue.

[hackermd]

If it turns out that the data is encoded incorrectly, it raises the questions what we do with such images. Shall we exclude them or attempt to fix them? The latter could result in lossy decompression.

[dclunie]

If Kakadu decodes them, then likely they are not "encoded incorrectly". Have you evidence of streams that do not begin, contain or end with the correct marker segments?

It is not inconceivable that I may be encapsulating some frames incorrectly in my conversion code, but I haven't seem evidence of that so far.

[hackermd]

Good idea! I do check for the start of file marker (also to determine which decoder to use), but not for the end of file marker. I will take a look at the end of the frames and report back to you.

[fedorov]

I did some exploration using this dashboard I shared with you a bit earlier.

If I look at the cptac_coad collection only, I see zoom failures both for 1.2.1 Explicit VR (example) and 1.2.4.91 JPEG2000 (example) transfer syntaxes. It seems that all samples that I tried to open that have one of those two transfer syntaxes in cptac_coad fail to zoom.

Those samples that I tried from cptac_coad that use 1.2.4.50 JPEG Baseline worked fine.

This is the error in the console:

[fedorov]

As aside, it is unfortunate that it is not possible to view the DICOM header in Slim - in this case it would be handy to confirm that the metadata shown in the dashboard matches what is in those specific series, as a way to double-check.

[hackermd]

As aside, it is unfortunate that it is not possible to view the DICOM header in Slim - in this case it would be handy to confirm that the metadata shown in the dashboard matches what is in those specific series, as a way to double-check.

Agreed, that would be a very useful feature.

[hackermd]

If I look at the cptac_coad collection only, I see zoom failures both for 1.2.1 Explicit VR (example) and 1.2.4.91 JPEG2000 (example) transfer syntaxes. It seems that all samples that I tried to open that have one of those two transfer syntaxes in cptac_coad fail to zoom.

Any series that contains images with different values for the Photometric Interpretation attribute is currently expected to fail. This behaviour should be fixed with this PR.

The other issue I reported via Slack regarding the decoding the JPEG 2000 bit streams is probably different. Let me draft a new release of Slim so that we can start untangling the two issues.