Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ecc 1467 float data values #83

Merged
merged 49 commits into from
Mar 13, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
6a26e0f
ECC-1467: GRIB: Support data values array decoded as floats (Part 01)
shahramn Dec 28, 2022
78af61b
ECC-1467: GRIB: Support data values array decoded as floats (Part 02)
shahramn Dec 29, 2022
4dd790f
ECC-1467: Debugging
shahramn Dec 29, 2022
573f2f9
ECC-1467: First working version for simple packing
shahramn Dec 29, 2022
445798c
ECC-1467: Testing
shahramn Dec 29, 2022
8d55550
ECC-1467: GRIB: Support data values array decoded as floats (Part 03)
shahramn Dec 29, 2022
e0e4655
ECC-1467: GRIB: Support data values array decoded as floats (Part 04)
shahramn Dec 30, 2022
f16ba71
ECC-1467: GRIB: Support data values array decoded as floats (Part 04)
shahramn Dec 30, 2022
5fea28e
ECC-1467: Added test
shahramn Dec 30, 2022
ff77426
ECC-1467: Cleanup
shahramn Dec 30, 2022
4cb2e12
ECC-1467: Add grib_accessors_list_unpack_float
shahramn Dec 30, 2022
cc4ef5a
ECC-1467: Add codes_keys_iterator_get_float
shahramn Dec 30, 2022
c36ac36
ECC-1467: GRIB: Support data values array decoded as floats (Part 05)
shahramn Dec 30, 2022
6fcef54
ECC-1467: GRIB: Support data values array decoded as floats (Part 06)
shahramn Dec 31, 2022
ffc9f13
ECC-1467: cpplint warnings
shahramn Jan 2, 2023
d1275b0
ECC-1467: Placeholder for spherical harmonics (to be done)
shahramn Jan 2, 2023
dab6eef
ECC-1467: TODO items
shahramn Jan 2, 2023
e1beaac
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 4, 2023
7f076a3
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 6, 2023
19dbc00
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 22, 2023
b669abf
ECC-1508: Set project language to be C++
shahramn Jan 29, 2023
a555f55
ECC-1508: Set C++ standard to 11 and re-enable test
shahramn Jan 30, 2023
1d3b074
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 30, 2023
995ed22
ECC-1508: Change file extensions to '.cc'
shahramn Jan 30, 2023
2cad3be
ECC-1467: Complex packing single-precision
Jan 30, 2023
b8da4f1
ECC-1467: Minor changes
Jan 30, 2023
43f7faf
Merge branch 'feature/ECC-1467-FloatDataValues-Compression' into feat…
Jan 30, 2023
370f046
ECC-1467: Complex single-precision
Jan 30, 2023
213b250
ECC-1467: Cleanup debug output
Jan 30, 2023
c6717ff
ECC-1467: Basic shell constructs
Jan 30, 2023
0300878
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Feb 12, 2023
150c538
Remove unused variables
shahramn Feb 12, 2023
0a9d311
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
Feb 17, 2023
687e3f4
ECC-1467: Single-precision implementation with the use of templates
Feb 20, 2023
9d5d888
Merge develop
shahramn Feb 26, 2023
b1f5856
ECC-1467: template for second order packing
Mar 1, 2023
9371a32
ECC-1467: typename check in templates
Mar 1, 2023
3079588
ECC-1467: Simplification of templates
Mar 6, 2023
8486b71
ECC-1467: Missing header file
Mar 6, 2023
3a300d7
Pseudo code
Mar 7, 2023
58ee3be
ECC-1467: Revert changes in the unpack_double_element_set() function
Mar 8, 2023
8425eeb
ECC-1467: Minor changes
Mar 8, 2023
d300abb
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
joobog Mar 13, 2023
94f69da
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
joobog Mar 13, 2023
b892ff1
ECC-1467: Fixed data run length class
joobog Mar 13, 2023
ce053e1
ECC-1467: Fixed data run length class
joobog Mar 13, 2023
97a007a
ECC-1467: Use __func__ in log messages
joobog Mar 13, 2023
c41518d
Merge branch 'feature/ECC-1467-FloatDataValues' of github.com:joobog/…
joobog Mar 13, 2023
46b6736
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Mar 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ECC-1467: GRIB: Support data values array decoded as floats (Part 04)
  • Loading branch information
shahramn committed Dec 30, 2022
commit e0e4655d3e42b8a9c4e6683450f3c877436a313c
1 change: 1 addition & 0 deletions src/grib_accessor.c
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,7 @@ int grib_unpack_float(grib_accessor* a, float* v, size_t* len)
{
grib_accessor_class* c = a->cclass;
while (c) {
/* printf("grib_accessor.c grib_unpack_float:: c->name=%s\n",c->name); */
if (c->unpack_float) {
return c->unpack_float(a, v, len);
}
Expand Down
31 changes: 29 additions & 2 deletions src/grib_accessor_class_bitmap.c
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@

IMPLEMENTS = next_offset
IMPLEMENTS = unpack_double;unpack_double_element;unpack_double_element_set
IMPLEMENTS = unpack_float
IMPLEMENTS = unpack_long
IMPLEMENTS = unpack_string
IMPLEMENTS = init;dump;update_size
Expand All @@ -40,6 +41,7 @@ or edit "accessor.class" and rerun ./make_class.pl
*/

static int unpack_double(grib_accessor*, double* val, size_t* len);
static int unpack_float(grib_accessor*, float* val, size_t* len);
static int unpack_long(grib_accessor*, long* val, size_t* len);
static int unpack_string(grib_accessor*, char*, size_t* len);
static long next_offset(grib_accessor*);
Expand Down Expand Up @@ -88,7 +90,7 @@ static grib_accessor_class _grib_accessor_class_bitmap = {
0, /* grib_pack procedures double */
0, /* grib_pack procedures float */
&unpack_double, /* grib_unpack procedures double */
0, /* grib_unpack procedures float */
&unpack_float, /* grib_unpack procedures float */
0, /* grib_pack procedures string */
&unpack_string, /* grib_unpack procedures string */
0, /* grib_pack array procedures string */
Expand Down Expand Up @@ -127,7 +129,6 @@ static void init_class(grib_accessor_class* c)
c->pack_long = (*(c->super))->pack_long;
c->pack_double = (*(c->super))->pack_double;
c->pack_float = (*(c->super))->pack_float;
c->unpack_float = (*(c->super))->unpack_float;
c->pack_string = (*(c->super))->pack_string;
c->pack_string_array = (*(c->super))->pack_string_array;
c->unpack_string_array = (*(c->super))->unpack_string_array;
Expand Down Expand Up @@ -267,6 +268,32 @@ static int unpack_double(grib_accessor* a, double* val, size_t* len)
return GRIB_SUCCESS;
}

//TODO: ECC-1467: Copied the 'double' version and reused by copy/paste!
static int unpack_float(grib_accessor* a, float* val, size_t* len)
{
long pos = a->offset * 8;
long tlen;
long i;
int err = 0;
grib_handle* hand = grib_handle_of_accessor(a);

err = grib_value_count(a, &tlen);
if (err)
return err;

if (*len < tlen) {
grib_context_log(a->context, GRIB_LOG_ERROR, "Wrong size for %s it contains %ld values", a->name, tlen);
*len = 0;
return GRIB_ARRAY_TOO_SMALL;
}

for (i = 0; i < tlen; i++) {
val[i] = (float)grib_decode_unsigned_long(hand->buffer->data, &pos, 1);
}
*len = tlen;
return GRIB_SUCCESS;
}
//
static int unpack_double_element(grib_accessor* a, size_t idx, double* val)
{
long pos = a->offset * 8;
Expand Down
4 changes: 2 additions & 2 deletions src/grib_accessor_class_data_apply_bitmap.c
Original file line number Diff line number Diff line change
Expand Up @@ -265,8 +265,8 @@ static int unpack_double(grib_accessor* a, double* val, size_t* len)
return err;
}

// TODO: Not ideal!! First attempt at unpacking floats...
// Should be merged with unpack_double and refactored! Most probably using C++ templates
//TODO: ECC-1467: Copied the 'double' version and reused by copy/paste!
// Should be merged with unpack_double and refactored! Most probably using C++ templates
static int unpack_float(grib_accessor* a, float* val, size_t* len)
{
grib_accessor_data_apply_bitmap* self = (grib_accessor_data_apply_bitmap*)a;
Expand Down
4 changes: 2 additions & 2 deletions src/grib_accessor_class_data_ccsds_packing.c
Original file line number Diff line number Diff line change
Expand Up @@ -330,8 +330,8 @@ static int unpack_double(grib_accessor* a, double* val, size_t* len)
return err;
}

// TODO: Not ideal!! First attempt at unpacking floats...
// Should be merged with unpack_double and refactored! Most probably using C++ templates
//TODO: ECC-1467: Copied the 'double' version and reused by copy/paste!
// Should be merged with unpack_double and refactored! Most probably using C++ templates
static int unpack_float(grib_accessor* a, float* val, size_t* len)
{
grib_accessor_data_ccsds_packing* self = (grib_accessor_data_ccsds_packing*)a;
Expand Down
6 changes: 5 additions & 1 deletion src/grib_accessor_class_data_simple_packing.c
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,9 @@ static int unpack_double_element_set(grib_accessor* a, const size_t* index_array
return GRIB_SUCCESS;
}

// unpack an array of double or single precision real numbers. As doubles if dval!= NULL or floats if fval!=NULL
// unpack an array of double-precision (double) or single-precision (float) real numbers.
// As doubles if dval!=NULL
// As floats if fval!=NULL
static int _unpack_real(grib_accessor* a, double* dval, float* fval, size_t* len, unsigned char* buf, long pos, size_t n_vals)
{
grib_accessor_data_simple_packing* self = (grib_accessor_data_simple_packing*)a;
Expand All @@ -335,6 +337,8 @@ static int _unpack_real(grib_accessor* a, double* dval, float* fval, size_t* len
double d = 0;
double units_factor = 1.0;
double units_bias = 0.0;

// Either decode as double or float: cannot have both!
Assert( ! (fval && dval) );

if (*len < n_vals) {
Expand Down
21 changes: 1 addition & 20 deletions src/grib_accessor_class_gen.c
Original file line number Diff line number Diff line change
Expand Up @@ -341,28 +341,9 @@ static int unpack_double(grib_accessor* a, double* v, size_t* len)
return GRIB_NOT_IMPLEMENTED;
}

//TODO: ECC-1467: Copied the 'double' version and reused by copy/paste!
static int unpack_float(grib_accessor* a, float* v, size_t* len)
{
/* int type = GRIB_TYPE_UNDEFINED;
printf("grib_accessor_class_gen.c unpack_float a->name =%s\n",a->name);
printf("grib_accessor_class_gen.c unpack_float a->cclass->name=%s\n",a->cclass->name);
printf("grib_accessor_class_gen.c unpack_float alen=%zu\n",*len);
printf("DEBUG unpack_float:: a->cclass->unpack_double=%p\n", (void*)a->cclass->unpack_double);
printf("DEBUG unpack_float:: &unpack_double=%p\n", (void*)&unpack_double);
if (a->cclass->unpack_double && a->cclass->unpack_double != &unpack_double) {
double val = 0.0;
size_t l = 1;
grib_unpack_double(a, &val, &l);
*v = val;
grib_context_log(a->context, GRIB_LOG_DEBUG, "Casting double %s to float", a->name);
return GRIB_SUCCESS;
}

grib_context_log(a->context, GRIB_LOG_ERROR, "Cannot unpack %s as float", a->name);
if (grib_get_native_type(grib_handle_of_accessor(a), a->name, &type) == GRIB_SUCCESS) {
grib_context_log(a->context, GRIB_LOG_ERROR, "Hint: Try unpacking as %s", grib_get_type_name(type));
} */

return GRIB_NOT_IMPLEMENTED;
}

Expand Down
1 change: 1 addition & 0 deletions src/grib_value.c
Original file line number Diff line number Diff line change
Expand Up @@ -1318,6 +1318,7 @@ int grib_get_double_array(const grib_handle* h, const char* name, double* val, s
}
}

//TODO: ECC-1467: Copied the 'double' version and reused by copy/paste!
int grib_get_float_array(const grib_handle* h, const char* name, float* val, size_t *length)
{
size_t len = *length;
Expand Down