Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Read fails on PP file with extra data #241

Closed
davidhassell opened this issue Aug 6, 2021 · 0 comments · Fixed by #242
Closed

Read fails on PP file with extra data #241

davidhassell opened this issue Aug 6, 2021 · 0 comments · Fixed by #242
Assignees
@davidhassell
Labels
bug Something isn't working enhancement New feature or request um/pp Relating to UM or PP format files
Milestone
3.11.0

Comments

@davidhassell
Copy link
Collaborator

At versions 3.7.0 and 3.10.0 the following error appears when reading a PP file with extra data:

>>> tsta=cf.read('ts_t15.pp')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cfdm/decorators.py", line 186, in verbose_override_wrapper
    return method_with_verbose_kwarg(self, *args, **kwargs)
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/read_write/read.py", line 656, in read
    warn_valid=warn_valid,
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cfdm/decorators.py", line 186, in verbose_override_wrapper
    return method_with_verbose_kwarg(self, *args, **kwargs)
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/read_write/read.py", line 910, in _read_a_file
    chunk=chunk)  # , mask=mask, warn_valid=warn_valid)
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cfdm/decorators.py", line 186, in verbose_override_wrapper
    return method_with_verbose_kwarg(self, *args, **kwargs)
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2984, in read
    for var in f.vars]
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 2984, in <listcomp>
    for var in f.vars]
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/read_write/um/umread.py", line 440, in __init__
    groups = var.group_records_by_extra_data()
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/umread_lib/umfile.py", line 162, in group_records_by_extra_data
    if not compare(recs[0], recs[-1]):
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/umread_lib/umfile.py", line 137, in _compare_recs_by_extra_data
    return self._compare(a.get_extra_data(), b.get_extra_data())
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/umread_lib/umfile.py", line 258, in get_extra_data
    self._extra_data = self.read_extra_data()
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/umread_lib/umfile.py", line 250, in read_extra_data
    return edu.get_data()
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/umread_lib/extraData.py", line 136, in get_data
    rawvals = self.next_words(ia)
  File "/share/apps/NCAS/anaconda3/lib/python3.7/site-packages/cf/umread_lib/extraData.py", line 108, in next_words
    rv = self.rdata[:pos]
TypeError: slice indices must be integers or None or have an __index__ method

This is a result of some old pre-v3.0.0 Python2 code still existing in cf/umread_lib/extraData.py, relating to the handling of bytes and unicode strings.

With this fixed, it became apparent that the mapping of PP extra data to CF constructs for LCODE axis code 13 was deficient, and this has should also be fixed. This deficiency has not been seen in the post-v3.0.0 era, so no one will have seen it!

PR to follow.

Many thanks to @JonathanGregory for finding and reporting this bug.
@davidhassell davidhassell added bug Something isn't working enhancement New feature or request labels Aug 6, 2021
@davidhassell davidhassell self-assigned this Aug 6, 2021
@davidhassell davidhassell mentioned this issue Aug 6, 2021
Merged
@davidhassell davidhassell added this to the 3.11.0 milestone Jan 10, 2022
davidhassell added a commit that referenced this issue Jan 10, 2022
@davidhassell
Added #241 to 3.11.0
481264b
@davidhassell davidhassell added the um/pp Relating to UM or PP format files label Jun 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@davidhassell davidhassell

Labels
bug Something isn't working enhancement New feature or request um/pp Relating to UM or PP format files
Projects
None yet
Milestone
3.11.0
Development

Successfully merging a pull request may close this issue.

Fix reading and mapping of PP extra data davidhassell/cf-python
1 participant
@davidhassell