The bug for creating GRIB index file in AIX

[Yaqiang]

I am using netcdfAll-4.6+.jar to read GRIB2 data file. Two index files (.ncx3 and .gbx9) were created when reading GRIB2 data file. It's fine in Windows system. But it has problem in AIX unix system with following error message. I have tried to copy the .ncx3 and .gbx9 files from Windows computer to AIX computer, then the data reading process was correct. So I am thinking this is a bug of netcdf Java library in AIX system for creating GRIB index files, the size of them are little bigger than the files created in Windows system. BTW, if there is an attribute in NetcdfFile class to aviod the index file creating? Which can temporally solve this problem.

com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length.
at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:82)
at com.google.protobuf.CodedInputStream.pushLimit(CodedInputStream.java:985)
at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:495)
at ucar.nc2.grib.collection.GribCollectionProto$SparseArray.(GribCollectionProto.java:1038)
at ucar.nc2.grib.collection.GribCollectionProto$SparseArray.(GribCollectionProto.java:938)
at ucar.nc2.grib.collection.GribCollectionProto$SparseArray$1.parsePartialFrom(GribCollectionProto.java:1085)
at ucar.nc2.grib.collection.GribCollectionProto$SparseArray$1.parsePartialFrom(GribCollectionProto.java:1080)
at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:139)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:173)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:185)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:190)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
at ucar.nc2.grib.collection.GribCollectionProto$SparseArray.parseFrom(GribCollectionProto.java:1377)
at ucar.nc2.grib.collection.GribCollectionImmutable$VariableIndex.readRecords(GribCollectionImmutable.java:490)
at ucar.nc2.grib.collection.GribIosp.readDataFromCollection(GribIosp.java:998)
at ucar.nc2.grib.collection.GribIosp.readData(GribIosp.java:963)
at ucar.nc2.NetcdfFile.readData(NetcdfFile.java:2020)
at ucar.nc2.Variable.reallyRead(Variable.java:874)
at ucar.nc2.Variable._read(Variable.java:845)
at ucar.nc2.Variable.read(Variable.java:723)

[lesserwhirls]

This looks like a protobuf library issue on AIX. When netCDF-Java reads a collection of GRIB messages from a file, it first writes the index files (the ncx index files are actually encoded using protobuf messages). Everything except for accessing the actual data values on the grid is done from the index files, and so they are required and cannot be disabled.

The version of the protobuf library is newer in the next version of netCDF-Java, which is in beta. It might be worth trying that version to see if things work properly - the netcdfAll.jar for v5.0.0-beta6 can be found here. If that does not work, there is a newer version of protobuf. I can try to upgrade the latest snapshot to use it.

[Yaqiang]

@lesserwhirls Thanks for your quickly response and good suggestion! netcdfAll-5.0.0-beta6.jar with newer version of protobuf library solve this problem in AIX system.