GRIB scanMode stored in GDS, not data.

[JohnLCaron]

scanMode is needed to rearrange the points at the time we read the data. it is stored in the "Grid definition" section for both 1 and 2.

We keep unique GDS based on CRC of gds bytes. That should mean scanMode must be correct, ie different scanMode would generate different GDS. Which would be bad (spurious groups) but it doesnt seem to happen. So we are being overly cautious.

We have been storing the scanMode in the Record array in case it varies. Maybe not needed, could be stored in the Gds.

The case would be is its really the same GDS but scanMode differs. Im thinking this wont happen, since GRIB thinks of scanMode as part of the GDS.

Perhaps the safety factor would be to store the start of the record, not the drs position. That would allow us to read the entire record. This affects GRIB2.gbx9. But GRIB1.gbx9 stores start of record.

this would be worth fixing if we decide to redo the gbx9.

[JohnLCaron]

Currently we are merging groups, with only one GDS per group.

The ncx files should maintain the seperate GDS, and the variable should maintain link to its GDS.

The Groups may be merged when they are "close enough".

Related to #308.

[JohnLCaron]

Examining all of the gbx9 files in cdmUnitTest/formats/grib1 (new Grib1Report)

GDS (4)
0: count = 4 // have "predefined grids"
1: count = 62 // normal case
2: count = 11 // note 1
41: count = 1 // note 2

note 1:
case 1: spurious differences, eg encoding unneeded resolution flag
case 2: different projections

note 2:
one file from dutch ("de Bilt?) uses RotatedLatLon fields we dont understand.

anyway, putting these extra GDS sections in ncx is not a huge overhead, and guarentees that we have complete information, even if spurious.

[JohnLCaron]

Examining all of the gbx9 files in cdmUnitTest/formats/grib2 (new Grib2Report)

GDS (4)
0: count = 1
1: count = 64
2: count = 4
4: count = 1

GDShashes (4)
0: count = 1
1: count = 65
2: count = 3
4: count = 1

theres oly one case where seperate CRC != seperate GDS (RTMA_CONUS_2p5km_20111225_0000.grib2.gbx9)

storing seperate GDS definetly a go.

[JohnLCaron]

OK, im reversing myself on this. While the seperate GDS is simple enough, it adds more complexity and its not quite right anyway.

1. for GRIB1, when we read data, we read the entire record including GDS and use scanMode etc from there.
2. for GRIB2, we had to put DRS position instead of start of record in ncx, because of "repeat" records that share sections 0-3. so we cant easily read in entire record, and so we use scanMode from group. So we do have the message start pos in the gbx, but not in the ncx.

i think its fair to assume scanMode wont change, they put it in GDS because they think of it as part of grid def. Really we want access to whole record, since other things like center/subcenter/genProcess, etc might be needed.

So the right fix is probably to get the start of record into GRIB2 ncx, in case we need to read all the sections, eg the original gds.

One could also add the scanMode to the gds hash (see #308) which would force a new group, or simply report if the assumption is broken.

So

1. add drsOffset for grib1, make startPos be the start of record.
2. remove the scanMode field in the Record, for simplification.
3. put the GDS directly in the Group for simplification.

[JohnLCaron]

Changes are in the ncx4 version, so nothing more to be done.