Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bits: grib_encode_unsigned_long: increment pointer, not the pointed-to value #24

Merged

Conversation

mkrupcale
Copy link
Contributor

This fixes a long-standing (i.e. at least since 1.9.5 ca. Oct 2010) bug in grib_encode_unsigned_long where the first byte was incorrectly incremented instead of incrementing the pointer to the next byte. This would only happen if the bit position / offset was not aligned on a byte boundary (i.e. *bitp % 8 != 0), though. Fortunately, as far as I can tell, no grib_api/eccodes functions invoked grib_encode_unsigned_long such that this bug was manifested.

  • src/grib_bits_any_endian.c: increment pointer, not the pointed-to-value
  • src/grib_bits_ibmpow.c: likewise.

…o-value

This fixes a long-standing (i.e. at least since 1.9.5 ca. Oct 2010) bug in grib_encode_unsigned_long where the first byte was incorrectly incremented instead of incrementing the pointer to the next byte. This would only happen if the bit position / offset was not aligned on a byte boundary (i.e. *bitp % 8 != 0), though. Fortunately, as far as I can tell, no grib_api/eccodes functions invoked grib_encode_unsigned_long such that this bug was manifested.

 * src/grib_bits_any_endian.c: increment pointer, not the pointed-to-value
 * src/grib_bits_ibmpow.c: likewise.
@shahramn
Copy link
Collaborator

Many thanks. Do you have a test case for this? (So we can show the original problem and check your contribution actually fixes it)

@mkrupcale
Copy link
Contributor Author

No, I don't have a test case at the moment. I'm currently still trying to understand all of these low-level bit manipulation functions. In particular:

  1. Why do both grib_encode_unsigned_longb and grib_encode_unsigned_long exist when they appear to do the same thing but with different implementations?
  2. In contrast, the signed counterparts, grib_{decode,encode}_signed_long and grib_{decode,encode}_signed_longb appears to offer different interfaces: the variants without the b appear to take offsets and lengths in bytes, whereas those with the b take offsets and lengths in bits. Why is there this discrepancy?
  3. Why are grib_get_bit and grib_set_bit reversing the bit order? That is, bitp==0 gets or sets bit 7 in the byte, while bitp==7 gets or sets bit 0. Are GRIB and BUFR BE at the bit level?
  4. I have a few other related questions I posted on the ECMWF issues site[1].

I'd appreciate if you or someone else could help me understand these encoding/decoding issues so that I might try to improve it. It seems to me that the current situation is not entirely consistent and might need to be more formally specified. In particular, I think it might make sense to standardize on a LE encoding from a performance perspective these days.

[1] https://jira.ecmwf.int/browse/SUP-3097

@shahramn shahramn merged commit a2d3d57 into ecmwf:develop Mar 17, 2020
@mkrupcale
Copy link
Contributor Author

@shahramn Do you think you could try to answer questions 1-3 above, or should I open an issue on the ECMWF site for them? It looks like SUP-3097 has been escalated, so I suppose the developers are looking at those questions already (although I can't see the issue BUFR-65).

@mkrupcale mkrupcale deleted the encode-unsigned-long-pointer-increment branch March 29, 2020 12:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants