Skip to content
This repository has been archived by the owner on Sep 1, 2022. It is now read-only.

java.lang.UnsupportedOperationException:****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7 #27

Closed
RvanNieuwpoort opened this issue Aug 21, 2013 · 4 comments
Closed

java.lang.UnsupportedOperationException:****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7 #27

RvanNieuwpoort opened this issue Aug 21, 2013 · 4 comments

Comments

@RvanNieuwpoort
Copy link

Hi all,

I have an application that stores metadata in the HDF5 format, but the real "raw" data in a separate file. The HDF5 file refers to the raw file. I would like to use this library to parse the metadata. I don't have to read the external raw file, just the metadata in the HDF5. However, NetcdfFile.open bails out with this stack trace:

***_UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
java.io.IOException: java.lang.UnsupportedOperationException: ***_UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:425)
at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:699)
at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:560)
at ucar.nc2.NCdumpW.print(NCdumpW.java:168)
at ucar.nc2.NCdumpW.main(NCdumpW.java:982)
Caused by: java.lang.UnsupportedOperationException: ****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
at ucar.nc2.iosp.hdf5.H5header$HeaderMessage.read(H5header.java:2714)
at ucar.nc2.iosp.hdf5.H5header$DataObject.readMessagesVersion2(H5header.java:2420)
at ucar.nc2.iosp.hdf5.H5header$DataObject.(H5header.java:2306)
at ucar.nc2.iosp.hdf5.H5header$DataObject.(H5header.java:2180)
at ucar.nc2.iosp.hdf5.H5header.getDataObject(H5header.java:2027)
at ucar.nc2.iosp.hdf5.H5header.access$600(H5header.java:70)
at ucar.nc2.iosp.hdf5.H5header$DataObjectFacade.(H5header.java:2073)
at ucar.nc2.iosp.hdf5.H5header.readGroupNew(H5header.java:3920)
at ucar.nc2.iosp.hdf5.H5header.access$900(H5header.java:70)
at ucar.nc2.iosp.hdf5.H5header$H5Group.(H5header.java:2153)
at ucar.nc2.iosp.hdf5.H5header$H5Group.(H5header.java:2118)
at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:472)
at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:473)
at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:215)
at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:128)
at ucar.nc2.NetcdfFile.(NetcdfFile.java:1521)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:813)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:422)
... 4 more

Now, I don't need netcdf to understand about the external file, but it would be nice if it would just ignore it or maybe log a warning. Or, even better, support the ExternalDataFiles.

Is there any way to work around this? Or maybe it is relatively easy to patch?

Kind regards,

Rob van Nieuwpoort
Netherlands eScience center
[email protected]
@JohnLCaron
Copy link
Collaborator

Hi Rob:

I havent seen an example of this before. Can you send me both files?

What file format is the external file? Why do you do it that way?
Does the "metadata file" use an absolute path address or assumed
relative to the metadata file or ?

John

On 8/21/2013 9:04 AM, Rob V. van Nieuwpoort wrote:

Hi all,

I have an application that stores metadata in the HDF5 format, but the
real "raw" data in a separate file. The HDF5 file refers to the raw
file. I would like to use this library to parse the metadata. I don't
have to read the external raw file, just the metadata in the HDF5.
However, NetcdfFile.open bails out with this stack trace:

***_UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
java.io.IOException: java.lang.UnsupportedOperationException:
***_UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:425)
at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:699)
at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:560)
at ucar.nc2.NCdumpW.print(NCdumpW.java:168)
at ucar.nc2.NCdumpW.main(NCdumpW.java:982)
Caused by: java.lang.UnsupportedOperationException: ****UNPROCESSED
MESSAGE type = ExternalDataFiles(7) raw = 7
at ucar.nc2.iosp.hdf5.H5header$HeaderMessage.read(H5header.java:2714)
at
ucar.nc2.iosp.hdf5.H5header$DataObject.readMessagesVersion2(H5header.java:2420)
at ucar.nc2.iosp.hdf5.H5header$DataObject.(H5header.java:2306)
at ucar.nc2.iosp.hdf5.H5header$DataObject.(H5header.java:2180)
at ucar.nc2.iosp.hdf5.H5header.getDataObject(H5header.java:2027)
at ucar.nc2.iosp.hdf5.H5header.access$600(H5header.java:70)
at ucar.nc2.iosp.hdf5.H5header$DataObjectFacade.(H5header.java:2073)
at ucar.nc2.iosp.hdf5.H5header.readGroupNew(H5header.java:3920)
at ucar.nc2.iosp.hdf5.H5header.access$900(H5header.java:70)
at ucar.nc2.iosp.hdf5.H5header$H5Group.(H5header.java:2153)
at ucar.nc2.iosp.hdf5.H5header$H5Group.(H5header.java:2118)
at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:472)
at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:473)
at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:215)
at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:128)
at ucar.nc2.NetcdfFile.(NetcdfFile.java:1521)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:813)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:422)
... 4 more

Now, I don't need netcdf to understand about the external file, but it
would be nice if it would just ignore it or maybe log a warning. Or,
even better, support the ExternalDataFiles.

Is there any way to work around this? Or maybe it is relatively easy to
patch?

Kind regards,

Rob van Nieuwpoort
Netherlands eScience center
[email protected] mailto:[email protected]

—
Reply to this email directly or view it on GitHub
#27.

@RvanNieuwpoort
Copy link
Author

Hi John,

Thanks for your quick reply!
I attached the HDF5 file, I hope that works. The external file that is referenced is called “BeamFormedData.raw”, however it is too large to send by mail (for this small test it is 71MB). I could put it in a dropbox if you need it?
It is just raw 32-bit float data. We split it up in .h5 for the meta data, and .raw for the actual data, because the raw files can be large (think terabytes-petabytes ☺ ).
The path addressed is relative, so only “BeamFormedData.raw”. I don’t actually have to read that data, I just want to parse the meta data.
This is LOFAR telescope data by the way (http:https://www.lofar.org).

Cheers,

Rob

From: John Caron [mailto:[email protected]]
Sent: woensdag 21 augustus 2013 22:28
To: Unidata/thredds
Cc: Rob van Nieuwpoort
Subject: Re: [thredds] java.lang.UnsupportedOperationException:****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7 (#27)

Hi Rob:

I havent seen an example of this before. Can you send me both files?

What file format is the external file? Why do you do it that way?
Does the "metadata file" use an absolute path address or assumed
relative to the metadata file or ?

John

On 8/21/2013 9:04 AM, Rob V. van Nieuwpoort wrote:

Hi all,

I have an application that stores metadata in the HDF5 format, but the
real "raw" data in a separate file. The HDF5 file refers to the raw
file. I would like to use this library to parse the metadata. I don't
have to read the external raw file, just the metadata in the HDF5.
However, NetcdfFile.open bails out with this stack trace:

***_UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
java.io.IOException: java.lang.UnsupportedOperationException:
***_UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:425)
at ucar.nc2.dataset.NetcdfDataset.openOrAcquireFile(NetcdfDataset.java:699)
at ucar.nc2.dataset.NetcdfDataset.openFile(NetcdfDataset.java:560)
at ucar.nc2.NCdumpW.print(NCdumpW.java:168)
at ucar.nc2.NCdumpW.main(NCdumpW.java:982)
Caused by: java.lang.UnsupportedOperationException: ****UNPROCESSED
MESSAGE type = ExternalDataFiles(7) raw = 7
at ucar.nc2.iosp.hdf5.H5header$HeaderMessage.read(H5header.java:2714)
at
ucar.nc2.iosp.hdf5.H5header$DataObject.readMessagesVersion2(H5header.java:2420)
at ucar.nc2.iosp.hdf5.H5header$DataObject.(H5header.java:2306)
at ucar.nc2.iosp.hdf5.H5header$DataObject.(H5header.java:2180)
at ucar.nc2.iosp.hdf5.H5header.getDataObject(H5header.java:2027)
at ucar.nc2.iosp.hdf5.H5header.access$600(H5header.java:70)
at ucar.nc2.iosp.hdf5.H5header$DataObjectFacade.(H5header.java:2073)
at ucar.nc2.iosp.hdf5.H5header.readGroupNew(H5header.java:3920)
at ucar.nc2.iosp.hdf5.H5header.access$900(H5header.java:70)
at ucar.nc2.iosp.hdf5.H5header$H5Group.(H5header.java:2153)
at ucar.nc2.iosp.hdf5.H5header$H5Group.(H5header.java:2118)
at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:472)
at ucar.nc2.iosp.hdf5.H5header.makeNetcdfGroup(H5header.java:473)
at ucar.nc2.iosp.hdf5.H5header.read(H5header.java:215)
at ucar.nc2.iosp.hdf5.H5iosp.open(H5iosp.java:128)
at ucar.nc2.NetcdfFile.(NetcdfFile.java:1521)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:813)
at ucar.nc2.NetcdfFile.open(NetcdfFile.java:422)
... 4 more

Now, I don't need netcdf to understand about the external file, but it
would be nice if it would just ignore it or maybe log a warning. Or,
even better, support the ExternalDataFiles.

Is there any way to work around this? Or maybe it is relatively easy to
patch?

Kind regards,

Rob van Nieuwpoort
Netherlands eScience center
[email protected]:[email protected] mailto:[email protected]

—
Reply to this email directly or view it on GitHub
#27.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/27#issuecomment-23047543.

@JohnLCaron
Copy link
Collaborator

Hi Rob:

Im modifying the code to put out a warning but not fail when it sees a message type it doesnt understand.

This will be in the 4.5.0 release.

John

****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7

Hi John,

Thanks for your quick reply!
I attached the HDF5 file, I hope that works. The external file that is referenced is called “BeamFormedData.raw”, however it is too large to send by mail (for this small test it is 71MB). I could put it in a dropbox if you need it?
It is just raw 32-bit float data. We split it up in .h5 for the meta data, and .raw for the actual data, because the raw files can be large (think terabytes-petabytes ☺ ).
The path addressed is relative, so only “BeamFormedData.raw”. I don’t actually have to read that data, I just want to parse the meta data.
This is LOFAR telescope data by the way (http:https://www.lofar.org).

Cheers,

Rob

@RvanNieuwpoort
Copy link
Author

OK, great news, thanks John!

Rob

From: John Caron [mailto:[email protected]]
Sent: dinsdag 20 mei 2014 23:25
To: Unidata/thredds
Cc: Rob van Nieuwpoort
Subject: Re: [thredds] java.lang.UnsupportedOperationException:****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7 (#27)

Hi Rob:

Im modifying the code to put out a warning but not fail when it sees a message type it doesnt understand.

This will be in the 4.5.0 release.

John

****UNPROCESSED MESSAGE type = ExternalDataFiles(7) raw = 7

Hi John,

Thanks for your quick reply!
I attached the HDF5 file, I hope that works. The external file that is referenced is called “BeamFormedData.raw”, however it is too large to send by mail (for this small test it is 71MB). I could put it in a dropbox if you need it?
It is just raw 32-bit float data. We split it up in .h5 for the meta data, and .raw for the actual data, because the raw files can be large (think terabytes-petabytes ☺ ).
The path addressed is relative, so only “BeamFormedData.raw”. I don’t actually have to read that data, I just want to parse the meta data.
This is LOFAR telescope data by the way (http:https://www.lofar.org).

Cheers,

Rob

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/27#issuecomment-43685200.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JohnLCaron @RvanNieuwpoort