Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/modernisation_inheritance #209

Merged
merged 66 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
a78865c
Prepare migration
joobog Mar 15, 2024
6647a21
Create a todo list
joobog Mar 17, 2024
be71970
Abstract grib_accessor and grib_accessor_class
joobog Mar 17, 2024
35b689a
Accessor backend ready for running accessors
joobog Mar 17, 2024
c598a51
Add "accessor/" prefix to grib_accessor_class_*.cc
joobog Mar 18, 2024
f1d68aa
18% tests passed, 276 tests failed out of 337
joobog Mar 18, 2024
8529fc3
Enable disabled BUFR function calls
joobog Mar 19, 2024
78c1129
Fix offsets
joobog Mar 19, 2024
6fc7074
Fix initialisation
joobog Mar 19, 2024
05e8b45
99% tests passed, 4 tests failed out of 337
joobog Apr 4, 2024
8a7f8c8
Fix compilation
shahramn Apr 4, 2024
1184bcc
Make local functions static
shahramn Apr 4, 2024
77d358a
Fix memory issue
joobog Apr 9, 2024
6bdef3a
Fix compare function
joobog Apr 9, 2024
6f54b96
Merge branch 'feature/modernisation-inheritance' of github.com:ecmwf/…
joobog Apr 9, 2024
ea48de7
PNG fix
joobog Apr 9, 2024
78a9429
Add PNG_ANYBITS
shahramn Apr 9, 2024
8a58ba1
Merge branch 'feature/modernisation-inheritance' of github.com:ecmwf/…
joobog Apr 9, 2024
cec5b2f
Fix destroy() function
joobog Apr 9, 2024
cc40569
Fix memory leak
joobog Apr 9, 2024
eb8bdcf
ECC-1800: GRIB2: Failure setting stepUnits=s,paramId=210203
shahramn Apr 16, 2024
a43c798
C++ modernisation: Added Fortran
shahramn Apr 16, 2024
6f9a1b6
C++ modernisation: Fix Fortran
shahramn Apr 16, 2024
14648c6
Print dataset when concept match fails
shahramn Apr 25, 2024
f229bd5
Remove staticForceStepUnits
shahramn Apr 25, 2024
5dd62ff
Initialise return values
shahramn Apr 25, 2024
17e250b
Cleanup
shahramn Apr 27, 2024
82cb39c
C++: Move function from header to local static
shahramn Apr 27, 2024
aca85e1
C++: Move function from header to local static. Use pragma once
shahramn Apr 27, 2024
6018a0a
Fix the includes: Do not use ..
shahramn Apr 27, 2024
bbcf302
Copyright notices
shahramn Apr 27, 2024
7857878
Remove unnecessary header files
shahramn Apr 27, 2024
f6d4295
Initialize all data members
shahramn Apr 27, 2024
f55015e
Testing: Catch error messages (Setting integer values double array)
shahramn Apr 30, 2024
3630e3b
Merge branch 'develop' into feature/modernisation-inheritance
joobog Apr 30, 2024
437d6c9
Merge branch 'feature/modernisation-inheritance' of github.com:ecmwf/…
joobog May 2, 2024
41a7e62
Merge branch 'develop' into feature/modernisation-inheritance
joobog May 2, 2024
4735c01
Simplify generic pack_double function
joobog May 2, 2024
33dbaa5
Fix reflection
joobog May 2, 2024
ea6fca2
Initialisation of accessors lists
joobog May 3, 2024
7ddc88a
Fix py_bufr_ecc-869.py test
joobog May 7, 2024
5d63a54
Merge branch 'develop' into feature/modernisation-inheritance
joobog May 7, 2024
677f73f
ECC-1820: Fix Windows build
shahramn May 7, 2024
481c78c
ECC-1820: Fix CCSDS class when -DENABLE_AEC=OFF
shahramn May 7, 2024
867da99
Cleanup
joobog May 7, 2024
45b5d55
Merge branch 'feature/modernisation-inheritance' of github.com:ecmwf/…
joobog May 7, 2024
41c513e
ECC-1820: Remove duplicate entry
shahramn May 7, 2024
6b1dae3
Cleanup
joobog May 7, 2024
1a3e11e
Better commment
joobog May 7, 2024
6aa4be2
Remove unnecessary includes
joobog May 7, 2024
ce1aac2
Merge branch 'feature/modernisation-inheritance' of github.com:ecmwf/…
joobog May 7, 2024
57b2698
Compiler warnings re sprintf
shahramn May 8, 2024
cf72dca
Move function from header to local static. Use pragma once
shahramn May 8, 2024
0d802de
Fix grib_accessor_class_gen definition
joobog May 8, 2024
e0f2213
ECC-1820: Remove unnecessary header files
shahramn May 8, 2024
f0b68e4
ECC-1820: Accessor hash
shahramn May 8, 2024
915935d
ECC-1820: Remove unused files
shahramn May 8, 2024
d397f58
ECC-1820: Local static functions
shahramn May 8, 2024
e47f092
Move functions from header to local static. Use 'pragma once'
shahramn May 9, 2024
afb4271
Cosmetics
shahramn May 9, 2024
5661700
ECC-1820: Use 'pragma once' in all header files
shahramn May 10, 2024
98cf979
ECC-1820: grib_accessors_list.rank made private
shahramn May 10, 2024
80d3a68
Cosmetics
shahramn May 10, 2024
ac80478
Cosmetics
shahramn May 10, 2024
22bd62c
Cosmetics
shahramn May 10, 2024
1514b73
Merge branch 'develop' into feature/modernisation-inheritance
shahramn May 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
99% tests passed, 4 tests failed out of 337
  • Loading branch information
joobog committed Apr 4, 2024
commit 05e8b45d424ed6559a7d18482c249147ecdcaa14
12 changes: 0 additions & 12 deletions src/accessor/grib_accessor.cc
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,6 @@ GRIB_INLINE static int grib_inline_strcmp(const char* a, const char* b)
return (*a == 0 && *b == 0) ? 0 : 1;
}

template <>
int grib_accessor::unpack<double>(double* v, size_t* len)
{
return unpack_double(v, len);
}

template <>
int grib_accessor::unpack<float>(float* v, size_t* len)
{
return unpack_float(v, len);
}

int grib_accessor::compare_accessors(grib_accessor* a2, int compare_flags)
{
int ret = 0;
Expand Down
54 changes: 4 additions & 50 deletions src/accessor/grib_accessor.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,13 @@

#include "../grib_api_internal.h"
#include "../grib_value.h"
#include <stdexcept>

class grib_accessor
{
public:
grib_accessor() : name(nullptr), name_space(nullptr), context(nullptr), h(nullptr), creator(nullptr), length(0), offset(0), parent(nullptr), next(nullptr), previous(nullptr), cclass(nullptr), flags(0), sub_section(nullptr), dirty(0), same(nullptr), loop(0), vvalue(nullptr), set(nullptr), parent_as_attribute(nullptr) {}
grib_accessor() : name(nullptr), name_space(nullptr), context(nullptr), h(nullptr), creator(nullptr), length(0), offset(0), parent(nullptr), next_(nullptr), previous_(nullptr), cclass(nullptr), flags(0), sub_section(nullptr), dirty(0), same(nullptr), loop(0), vvalue(nullptr), set(nullptr), parent_as_attribute(nullptr) {}
virtual ~grib_accessor() {}

template <typename T> int unpack(T* v, size_t* len);

virtual void init_accessor(const long, grib_arguments*) = 0;
virtual void dump(grib_dumper* f) = 0;
virtual int pack_missing() = 0;
Expand Down Expand Up @@ -75,8 +72,8 @@ class grib_accessor
long length; /** < byte length of the accessor */
long offset; /** < offset of the data in the buffer */
grib_section* parent; /** < section to which the accessor is attached */
grib_accessor* next; /** < next accessor in list */
grib_accessor* previous; /** < next accessor in list */
grib_accessor* next_; /** < next accessor in list */
grib_accessor* previous_; /** < next accessor in list */
grib_accessor_class* cclass; /** < behaviour of the accessor */
unsigned long flags; /** < Various flags */
grib_section* sub_section;
Expand All @@ -93,6 +90,7 @@ class grib_accessor
grib_accessor* parent_as_attribute;
};


class grib_accessor_class
{
public:
Expand Down Expand Up @@ -145,47 +143,3 @@ class grib_accessor_class
};


template <typename T>
int unpack_helper(grib_accessor* a, T* v, size_t* len)
{
static_assert(std::is_floating_point<T>::value, "Requires floating point numbers");
int type = GRIB_TYPE_UNDEFINED;
const char* Tname = type_to_string<T>(*v);
//if (a->cclass->unpack_long && a->cclass->unpack_long != &unpack_long) {
try {
long val = 0;
size_t l = 1;
a->unpack_long(&val, &l);
*v = val;
grib_context_log(a->context, GRIB_LOG_DEBUG, "Casting long %s to %s", a->name, Tname);
return GRIB_SUCCESS;
}
catch (std::runtime_error& e) {
// TODO(maee): log error
}

//if (a->cclass->unpack_string && a->cclass->unpack_string != &unpack_string) {
try {
char val[1024];
size_t l = sizeof(val);
char* last = NULL;
a->unpack_string(val, &l);

*v = strtod(val, &last);
if (*last == 0) { /* conversion of string to double worked */
grib_context_log(a->context, GRIB_LOG_DEBUG, "Casting string %s to %s", a->name, Tname);
return GRIB_SUCCESS;
}
}
catch (std::runtime_error& e) {
// TODO(maee): log error
}

grib_context_log(a->context, GRIB_LOG_ERROR, "Cannot unpack key '%s' as %s", a->name, Tname);
if (grib_get_native_type(grib_handle_of_accessor(a), a->name, &type) == GRIB_SUCCESS) {
grib_context_log(a->context, GRIB_LOG_ERROR, "Hint: Try unpacking as %s", grib_get_type_name(type));
}

return GRIB_NOT_IMPLEMENTED;
}

1 change: 0 additions & 1 deletion src/accessor/grib_accessor_class_bits.cc
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,6 @@ int grib_accessor_class_bits_t::unpack_string(grib_accessor* a, char* v, size_t*
double dval = 0;
long lval = 0;
size_t llen = 1;
grib_accessor_class* super = NULL;

switch (get_native_type(a)) {
case GRIB_TYPE_LONG:
Expand Down
3 changes: 1 addition & 2 deletions src/accessor/grib_accessor_class_bufr_data_array.h
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,11 @@
#ifndef eccodes_accessor_bufr_data_array_h
#define eccodes_accessor_bufr_data_array_h

#include "../grib_api_internal.h"
#include "grib_accessor_class_gen.h"

class grib_accessor_bufr_data_array_t : public grib_accessor_gen_t
{
public:
/* Members defined in bufr_data_array */
const char* bufrDataEncodedName;
const char* numberOfSubsetsName;
const char* expandedDescriptorsName;
Expand Down Expand Up @@ -90,4 +88,5 @@ grib_vsarray* accessor_bufr_data_array_get_stringValues(grib_accessor* a);
grib_accessors_list* accessor_bufr_data_array_get_dataAccessors(grib_accessor* a);
grib_trie_with_rank* accessor_bufr_data_array_get_dataAccessorsTrie(grib_accessor* a);
void accessor_bufr_data_array_set_unpackMode(grib_accessor* a, int unpackMode);

#endif /* eccodes_accessor_bufr_data_array_h */
5 changes: 1 addition & 4 deletions src/accessor/grib_accessor_class_bufr_data_element.cc
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

/*
* (C) Copyright 2005- ECMWF.
*
Expand All @@ -9,14 +8,12 @@
* virtue of its status as an intergovernmental organisation nor does it submit to any jurisdiction.
*/

#include "grib_api_internal.h"
#include "grib_accessor_class_bufr_data_element.h"

grib_accessor_class_bufr_data_element_t _grib_accessor_class_bufr_data_element{"bufr_data_element"};
grib_accessor_class* grib_accessor_class_bufr_data_element = &_grib_accessor_class_bufr_data_element;


static grib_accessor* make_clone(grib_accessor* a, grib_section* s, int* err)
grib_accessor* grib_accessor_class_bufr_data_element_t::make_clone(grib_accessor* a, grib_section* s, int* err)
{
grib_accessor* the_clone = NULL;
grib_accessor* attribute = NULL;
Expand Down
1 change: 1 addition & 0 deletions src/accessor/grib_accessor_class_bufr_data_element.h
Original file line number Diff line number Diff line change
Expand Up @@ -52,5 +52,6 @@ class grib_accessor_class_bufr_data_element_t : public grib_accessor_class_gen_t
void dump(grib_accessor*, grib_dumper*) override;
void init(grib_accessor*, const long, grib_arguments*) override;
int unpack_double_element(grib_accessor*, size_t i, double* val) override;
grib_accessor* make_clone(grib_accessor*, grib_section*, int*) override;
};
#endif /* eccodes_accessor_bufr_data_element_h */
6 changes: 3 additions & 3 deletions src/accessor/grib_accessor_class_bufr_group.cc
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,16 @@ void grib_accessor_class_bufr_group_t::dump(grib_accessor* a, grib_dumper* dumpe
grib_dump_section(dumper, a, a->sub_section->block);
}

static grib_accessor* next(grib_accessor* a, int explore)
grib_accessor* grib_accessor_class_bufr_group_t::next(grib_accessor* a, int explore)
{
grib_accessor* next = NULL;
if (explore) {
next = a->sub_section->block->first;
if (!next)
next = a->next;
next = a->next_;
}
else {
next = a->next;
next = a->next_;
}
if (!next) {
if (a->parent->owner)
Expand Down
1 change: 1 addition & 0 deletions src/accessor/grib_accessor_class_bufr_group.h
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,6 @@ class grib_accessor_class_bufr_group_t : public grib_accessor_class_variable_t
grib_accessor_class_bufr_group_t(const char* name) : grib_accessor_class_variable_t(name) {}
grib_accessor* create_empty_accessor() override { return new grib_accessor_bufr_group_t{}; }
void dump(grib_accessor*, grib_dumper*) override;
grib_accessor* next(grib_accessor* a, int explore) override;
};
#endif /* eccodes_accessor_bufr_group_h */
80 changes: 0 additions & 80 deletions src/accessor/grib_accessor_class_data_apply_bitmap.cc
Original file line number Diff line number Diff line change
Expand Up @@ -231,86 +231,6 @@ int grib_accessor_class_data_apply_bitmap_t::pack_double(grib_accessor* a, const
return err;
}

template <typename T>
int unpack(grib_accessor* a, T* val, size_t* len){
static_assert(std::is_floating_point<T>::value, "Requires floating point numbers");
grib_accessor_data_apply_bitmap_t* self = (grib_accessor_data_apply_bitmap_t*)a;

size_t i = 0;
size_t j = 0;
size_t n_vals = 0;
long nn = 0;
int err = 0;
size_t coded_n_vals = 0;
T* coded_vals = NULL;
double missing_value = 0;

err = a->value_count(&nn); n_vals = nn;
if (err)
return err;

if (!grib_find_accessor(grib_handle_of_accessor(a), self->bitmap))
return grib_get_array<T>(grib_handle_of_accessor(a), self->coded_values, val, len);

if ((err = grib_get_size(grib_handle_of_accessor(a), self->coded_values, &coded_n_vals)) != GRIB_SUCCESS)
return err;

if ((err = grib_get_double_internal(grib_handle_of_accessor(a), self->missing_value, &missing_value)) != GRIB_SUCCESS)
return err;

if (*len < n_vals) {
*len = n_vals;
return GRIB_ARRAY_TOO_SMALL;
}

if (coded_n_vals == 0) {
for (i = 0; i < n_vals; i++)
val[i] = missing_value;

*len = n_vals;
return GRIB_SUCCESS;
}

if ((err = grib_get_array_internal<T>(grib_handle_of_accessor(a), self->bitmap, val, &n_vals)) != GRIB_SUCCESS)
return err;

coded_vals = (T*)grib_context_malloc(a->context, coded_n_vals * sizeof(T));
if (coded_vals == NULL)
return GRIB_OUT_OF_MEMORY;

if ((err = grib_get_array<T>(grib_handle_of_accessor(a), self->coded_values, coded_vals, &coded_n_vals)) != GRIB_SUCCESS) {
grib_context_free(a->context, coded_vals);
return err;
}

grib_context_log(a->context, GRIB_LOG_DEBUG,
"grib_accessor_class_data_apply_bitmap: %s : creating %s, %d values",
__func__,
a->name, n_vals);

for (i = 0; i < n_vals; i++) {
if (val[i] == 0) {
val[i] = missing_value;
}
else {
val[i] = coded_vals[j++];
if (j > coded_n_vals) {
grib_context_free(a->context, coded_vals);
grib_context_log(a->context, GRIB_LOG_ERROR,
"grib_accessor_class_data_apply_bitmap [%s]:"
" %s : number of coded values does not match bitmap %ld %ld",
a->name, __func__, coded_n_vals, n_vals);

return GRIB_ARRAY_TOO_SMALL;
}
}
}

*len = n_vals;

grib_context_free(a->context, coded_vals);
return err;
}

int grib_accessor_class_data_apply_bitmap_t::unpack_double(grib_accessor* a, double* val, size_t* len){
return unpack<double>(a, val, len);
Expand Down
86 changes: 86 additions & 0 deletions src/accessor/grib_accessor_class_data_apply_bitmap.h
Original file line number Diff line number Diff line change
Expand Up @@ -42,5 +42,91 @@ class grib_accessor_class_data_apply_bitmap_t : public grib_accessor_class_gen_t
int compare(grib_accessor*, grib_accessor*) override;
int unpack_double_element(grib_accessor*, size_t i, double* val) override;
int unpack_double_element_set(grib_accessor*, const size_t* index_array, size_t len, double* val_array) override;

private:
template <typename T> int unpack(grib_accessor*, T*, size_t*);
};


template <typename T>
int grib_accessor_class_data_apply_bitmap_t::unpack(grib_accessor* a, T* val, size_t* len){
static_assert(std::is_floating_point<T>::value, "Requires floating point numbers");
grib_accessor_data_apply_bitmap_t* self = (grib_accessor_data_apply_bitmap_t*)a;

size_t i = 0;
size_t j = 0;
size_t n_vals = 0;
long nn = 0;
int err = 0;
size_t coded_n_vals = 0;
T* coded_vals = NULL;
double missing_value = 0;

err = a->value_count(&nn); n_vals = nn;
if (err)
return err;

if (!grib_find_accessor(grib_handle_of_accessor(a), self->bitmap))
return grib_get_array<T>(grib_handle_of_accessor(a), self->coded_values, val, len);

if ((err = grib_get_size(grib_handle_of_accessor(a), self->coded_values, &coded_n_vals)) != GRIB_SUCCESS)
return err;

if ((err = grib_get_double_internal(grib_handle_of_accessor(a), self->missing_value, &missing_value)) != GRIB_SUCCESS)
return err;

if (*len < n_vals) {
*len = n_vals;
return GRIB_ARRAY_TOO_SMALL;
}

if (coded_n_vals == 0) {
for (i = 0; i < n_vals; i++)
val[i] = missing_value;

*len = n_vals;
return GRIB_SUCCESS;
}

if ((err = grib_get_array_internal<T>(grib_handle_of_accessor(a), self->bitmap, val, &n_vals)) != GRIB_SUCCESS)
return err;

coded_vals = (T*)grib_context_malloc(a->context, coded_n_vals * sizeof(T));
if (coded_vals == NULL)
return GRIB_OUT_OF_MEMORY;

if ((err = grib_get_array<T>(grib_handle_of_accessor(a), self->coded_values, coded_vals, &coded_n_vals)) != GRIB_SUCCESS) {
grib_context_free(a->context, coded_vals);
return err;
}

grib_context_log(a->context, GRIB_LOG_DEBUG,
"grib_accessor_class_data_apply_bitmap: %s : creating %s, %d values",
__func__,
a->name, n_vals);

for (i = 0; i < n_vals; i++) {
if (val[i] == 0) {
val[i] = missing_value;
}
else {
val[i] = coded_vals[j++];
if (j > coded_n_vals) {
grib_context_free(a->context, coded_vals);
grib_context_log(a->context, GRIB_LOG_ERROR,
"grib_accessor_class_data_apply_bitmap [%s]:"
" %s : number of coded values does not match bitmap %ld %ld",
a->name, __func__, coded_n_vals, n_vals);

return GRIB_ARRAY_TOO_SMALL;
}
}
}

*len = n_vals;

grib_context_free(a->context, coded_vals);
return err;
}

#endif /* eccodes_accessor_data_apply_bitmap_h */
Loading