Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ecc 1467 float data values #83

Merged
merged 49 commits into from
Mar 13, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
6a26e0f
ECC-1467: GRIB: Support data values array decoded as floats (Part 01)
shahramn Dec 28, 2022
78af61b
ECC-1467: GRIB: Support data values array decoded as floats (Part 02)
shahramn Dec 29, 2022
4dd790f
ECC-1467: Debugging
shahramn Dec 29, 2022
573f2f9
ECC-1467: First working version for simple packing
shahramn Dec 29, 2022
445798c
ECC-1467: Testing
shahramn Dec 29, 2022
8d55550
ECC-1467: GRIB: Support data values array decoded as floats (Part 03)
shahramn Dec 29, 2022
e0e4655
ECC-1467: GRIB: Support data values array decoded as floats (Part 04)
shahramn Dec 30, 2022
f16ba71
ECC-1467: GRIB: Support data values array decoded as floats (Part 04)
shahramn Dec 30, 2022
5fea28e
ECC-1467: Added test
shahramn Dec 30, 2022
ff77426
ECC-1467: Cleanup
shahramn Dec 30, 2022
4cb2e12
ECC-1467: Add grib_accessors_list_unpack_float
shahramn Dec 30, 2022
cc4ef5a
ECC-1467: Add codes_keys_iterator_get_float
shahramn Dec 30, 2022
c36ac36
ECC-1467: GRIB: Support data values array decoded as floats (Part 05)
shahramn Dec 30, 2022
6fcef54
ECC-1467: GRIB: Support data values array decoded as floats (Part 06)
shahramn Dec 31, 2022
ffc9f13
ECC-1467: cpplint warnings
shahramn Jan 2, 2023
d1275b0
ECC-1467: Placeholder for spherical harmonics (to be done)
shahramn Jan 2, 2023
dab6eef
ECC-1467: TODO items
shahramn Jan 2, 2023
e1beaac
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 4, 2023
7f076a3
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 6, 2023
19dbc00
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 22, 2023
b669abf
ECC-1508: Set project language to be C++
shahramn Jan 29, 2023
a555f55
ECC-1508: Set C++ standard to 11 and re-enable test
shahramn Jan 30, 2023
1d3b074
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Jan 30, 2023
995ed22
ECC-1508: Change file extensions to '.cc'
shahramn Jan 30, 2023
2cad3be
ECC-1467: Complex packing single-precision
Jan 30, 2023
b8da4f1
ECC-1467: Minor changes
Jan 30, 2023
43f7faf
Merge branch 'feature/ECC-1467-FloatDataValues-Compression' into feat…
Jan 30, 2023
370f046
ECC-1467: Complex single-precision
Jan 30, 2023
213b250
ECC-1467: Cleanup debug output
Jan 30, 2023
c6717ff
ECC-1467: Basic shell constructs
Jan 30, 2023
0300878
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Feb 12, 2023
150c538
Remove unused variables
shahramn Feb 12, 2023
0a9d311
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
Feb 17, 2023
687e3f4
ECC-1467: Single-precision implementation with the use of templates
Feb 20, 2023
9d5d888
Merge develop
shahramn Feb 26, 2023
b1f5856
ECC-1467: template for second order packing
Mar 1, 2023
9371a32
ECC-1467: typename check in templates
Mar 1, 2023
3079588
ECC-1467: Simplification of templates
Mar 6, 2023
8486b71
ECC-1467: Missing header file
Mar 6, 2023
3a300d7
Pseudo code
Mar 7, 2023
58ee3be
ECC-1467: Revert changes in the unpack_double_element_set() function
Mar 8, 2023
8425eeb
ECC-1467: Minor changes
Mar 8, 2023
d300abb
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
joobog Mar 13, 2023
94f69da
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
joobog Mar 13, 2023
b892ff1
ECC-1467: Fixed data run length class
joobog Mar 13, 2023
ce053e1
ECC-1467: Fixed data run length class
joobog Mar 13, 2023
97a007a
ECC-1467: Use __func__ in log messages
joobog Mar 13, 2023
c41518d
Merge branch 'feature/ECC-1467-FloatDataValues' of github.com:joobog/…
joobog Mar 13, 2023
46b6736
Merge branch 'develop' into feature/ECC-1467-FloatDataValues
shahramn Mar 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
ECC-1467: GRIB: Support data values array decoded as floats (Part 02)
  • Loading branch information
shahramn committed Dec 29, 2022
commit 78af61b989fabfbd2b16a055407c80564a982404
4 changes: 4 additions & 0 deletions src/eccodes.c
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,10 @@ int codes_get_double_array(const grib_handle* h, const char* key, double* vals,
{
return grib_get_double_array(h, key, vals, length);
}
int codes_get_float_array(const grib_handle* h, const char* key, float* vals, size_t* length)
{
return grib_get_float_array(h, key, vals, length);
}
int codes_get_long_array(const grib_handle* h, const char* key, long* vals, size_t* length)
{
return grib_get_long_array(h, key, vals, length);
Expand Down
1 change: 1 addition & 0 deletions src/eccodes.h
Original file line number Diff line number Diff line change
Expand Up @@ -844,6 +844,7 @@ int codes_get_bytes(const codes_handle* h, const char* key, unsigned char* bytes
* @return 0 if OK, integer value on error
*/
int codes_get_double_array(const codes_handle* h, const char* key, double* vals, size_t* length);
int codes_get_float_array(const codes_handle* h, const char* key, float* vals, size_t* length);

/**
* Get long array values from a key. If several keys of the same name are present, the last one is returned
Expand Down
8 changes: 8 additions & 0 deletions src/eccodes_prototypes.h
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ int grib_pack_missing(grib_accessor* a);
int grib_pack_zero(grib_accessor* a);
int grib_is_missing_internal(grib_accessor* a);
int grib_pack_double(grib_accessor* a, const double* v, size_t* len);
int grib_pack_float(grib_accessor* a, const float* v, size_t* len);
int grib_pack_expression(grib_accessor* a, grib_expression* e);
int grib_pack_string(grib_accessor* a, const char* v, size_t* len);
int grib_pack_string_array(grib_accessor* a, const char** v, size_t* len);
Expand All @@ -123,8 +124,11 @@ int grib_pack_bytes(grib_accessor* a, const unsigned char* v, size_t* len);
int grib_unpack_bytes(grib_accessor* a, unsigned char* v, size_t* len);
int grib_unpack_double_subarray(grib_accessor* a, double* v, size_t start, size_t len);
int grib_unpack_double(grib_accessor* a, double* v, size_t* len);
int grib_unpack_float(grib_accessor* a, float* v, size_t* len);
int grib_unpack_double_element(grib_accessor* a, size_t i, double* v);
int grib_unpack_double_element_set(grib_accessor* a, const size_t* index_array, size_t len, double* val_array);
int grib_unpack_float_element(grib_accessor* a, size_t i, float* v);
int grib_unpack_float_element_set(grib_accessor* a, const size_t* index_array, size_t len, float* val_array);
int grib_unpack_string(grib_accessor* a, char* v, size_t* len);
int grib_unpack_string_array(grib_accessor* a, char** v, size_t* len);
int grib_accessors_list_unpack_long(grib_accessors_list* al, long* val, size_t* buffer_len);
Expand Down Expand Up @@ -1250,8 +1254,12 @@ int grib_get_bytes(const grib_handle* h, const char* name, unsigned char* val, s
int grib_get_native_type(const grib_handle* h, const char* name, int* type);
const char* grib_get_accessor_class_name(grib_handle* h, const char* name);
int ecc__grib_get_double_array_internal(const grib_handle* h, grib_accessor* a, double* val, size_t buffer_len, size_t* decoded_length);

int grib_get_double_array_internal(const grib_handle* h, const char* name, double* val, size_t* length);
int grib_get_double_array(const grib_handle* h, const char* name, double* val, size_t* length);
int grib_get_float_array_internal(const grib_handle* h, const char* name, float* val, size_t* length);
int grib_get_float_array(const grib_handle* h, const char* name, float* val, size_t* length);

int ecc__grib_get_string_length(grib_accessor* a, size_t* size);
int grib_get_string_length(const grib_handle* h, const char* name, size_t* size);
int ecc__grib_get_size(const grib_handle* h, grib_accessor* a, size_t* size);
Expand Down
52 changes: 52 additions & 0 deletions src/grib_accessor.c
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,19 @@ int grib_pack_double(grib_accessor* a, const double* v, size_t* len)
return 0;
}

int grib_pack_float(grib_accessor* a, const float* v, size_t* len)
{
grib_accessor_class* c = a->cclass;
while (c) {
if (c->pack_float) {
return c->pack_float(a, v, len);
}
c = c->super ? *(c->super) : NULL;
}
DebugAssert(0);
return 0;
}

int grib_pack_expression(grib_accessor* a, grib_expression* e)
{
grib_accessor_class* c = a->cclass;
Expand Down Expand Up @@ -209,6 +222,19 @@ int grib_unpack_double(grib_accessor* a, double* v, size_t* len)
return 0;
}

int grib_unpack_float(grib_accessor* a, float* v, size_t* len)
{
grib_accessor_class* c = a->cclass;
while (c) {
if (c->unpack_float) {
return c->unpack_float(a, v, len);
}
c = c->super ? *(c->super) : NULL;
}
DebugAssert(0);
return GRIB_NOT_IMPLEMENTED;
}

int grib_unpack_double_element(grib_accessor* a, size_t i, double* v)
{
grib_accessor_class* c = a->cclass;
Expand All @@ -220,6 +246,18 @@ int grib_unpack_double_element(grib_accessor* a, size_t i, double* v)
}
return GRIB_NOT_IMPLEMENTED;
}
int grib_unpack_float_element(grib_accessor* a, size_t i, float* v)
{
/* grib_accessor_class* c = a->cclass;
* while (c) {
* if (c->unpack_float_element) {
* return c->unpack_float_element(a, i, v);
* }
* c = c->super ? *(c->super) : NULL;
* }
*/
return GRIB_NOT_IMPLEMENTED;
}

/* Get the values for the indices passed in.
* The length of the 'index_array' argument should be 'len' and 'val_array' should also contain at least 'len' elements
Expand All @@ -236,6 +274,20 @@ int grib_unpack_double_element_set(grib_accessor* a, const size_t* index_array,
}
return GRIB_NOT_IMPLEMENTED;
}
int grib_unpack_float_element_set(grib_accessor* a, const size_t* index_array, size_t len, float* val_array)
{
/*
*grib_accessor_class* c = a->cclass;
*DebugAssert(len > 0);
*while (c) {
* if (c->unpack_float_element_set) {
* return c->unpack_float_element_set(a, index_array, len, val_array);
* }
* c = c->super ? *(c->super) : NULL;
*}
*/
return GRIB_NOT_IMPLEMENTED;
}

int grib_unpack_string(grib_accessor* a, char* v, size_t* len)
{
Expand Down
27 changes: 26 additions & 1 deletion src/grib_accessor_class_gen.c
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
CLASS = accessor
IMPLEMENTS = unpack_long;pack_long; clear
IMPLEMENTS = unpack_double;pack_double;unpack_double_element;unpack_double_element_set
IMPLEMENTS = unpack_float
IMPLEMENTS = unpack_string;pack_string
IMPLEMENTS = unpack_string_array;pack_string_array
IMPLEMENTS = unpack_bytes;pack_bytes
Expand Down Expand Up @@ -56,6 +57,7 @@ static int pack_string_array(grib_accessor*, const char**, size_t* len);
static int pack_expression(grib_accessor*, grib_expression*);
static int unpack_bytes(grib_accessor*, unsigned char*, size_t* len);
static int unpack_double(grib_accessor*, double* val, size_t* len);
static int unpack_float(grib_accessor*, float* val, size_t* len);
static int unpack_long(grib_accessor*, long* val, size_t* len);
static int unpack_string(grib_accessor*, char*, size_t* len);
static int unpack_string_array(grib_accessor*, char**, size_t* len);
Expand Down Expand Up @@ -110,7 +112,7 @@ static grib_accessor_class _grib_accessor_class_gen = {
&pack_double, /* grib_pack procedures double */
0, /* grib_pack procedures float */
&unpack_double, /* grib_unpack procedures double */
0, /* grib_unpack procedures float */
&unpack_float, /* grib_unpack procedures float */
&pack_string, /* grib_pack procedures string */
&unpack_string, /* grib_unpack procedures string */
&pack_string_array, /* grib_pack array procedures string */
Expand Down Expand Up @@ -339,6 +341,29 @@ static int unpack_double(grib_accessor* a, double* v, size_t* len)
return GRIB_NOT_IMPLEMENTED;
}

static int unpack_float(grib_accessor* a, float* v, size_t* len)
{
int type = GRIB_TYPE_UNDEFINED;
printf("====%s\n",a->name);
printf("DEBUG unpack_float:: a->cclass->unpack_double=%p\n", (void*)a->cclass->unpack_double);
printf("DEBUG unpack_float:: &unpack_double=%p\n", (void*)&unpack_double);
if (a->cclass->unpack_double && a->cclass->unpack_double != &unpack_double) {
double val = 0.0;
size_t l = 1;
grib_unpack_double(a, &val, &l);
*v = val;
grib_context_log(a->context, GRIB_LOG_DEBUG, "Casting double %s to float", a->name);
return GRIB_SUCCESS;
}

grib_context_log(a->context, GRIB_LOG_ERROR, "Cannot unpack %s as float", a->name);
if (grib_get_native_type(grib_handle_of_accessor(a), a->name, &type) == GRIB_SUCCESS) {
grib_context_log(a->context, GRIB_LOG_ERROR, "Hint: Try unpacking as %s", grib_get_type_name(type));
}

return GRIB_NOT_IMPLEMENTED;
}

static int unpack_string(grib_accessor* a, char* v, size_t* len)
{
if (a->cclass->unpack_double && a->cclass->unpack_double != &unpack_double) {
Expand Down
1 change: 1 addition & 0 deletions src/grib_api.h
Original file line number Diff line number Diff line change
Expand Up @@ -859,6 +859,7 @@ int grib_get_bytes(const grib_handle* h, const char* key, unsigned char* bytes,
* @return 0 if OK, integer value on error
*/
int grib_get_double_array(const grib_handle* h, const char* key, double* vals, size_t* length);
int grib_get_float_array(const grib_handle* h, const char* key, float* vals, size_t* length);

/**
* Get long array values from a key. If several keys of the same name are present, the last one is returned
Expand Down
42 changes: 42 additions & 0 deletions src/grib_value.c
Original file line number Diff line number Diff line change
Expand Up @@ -1248,6 +1248,24 @@ int _grib_get_double_array_internal(const grib_handle* h, grib_accessor* a, doub
}
}

int _grib_get_float_array_internal(const grib_handle* h, grib_accessor* a, float* val, size_t buffer_len, size_t* decoded_length)
{
if (a) {
int err = _grib_get_float_array_internal(h, a->same, val, buffer_len, decoded_length);

if (err == GRIB_SUCCESS) {
size_t len = buffer_len - *decoded_length;
err = grib_unpack_float(a, val + *decoded_length, &len);
*decoded_length += len;
}

return err;
}
else {
return GRIB_SUCCESS;
}
}

int grib_get_double_array_internal(const grib_handle* h, const char* name, double* val, size_t* length)
{
int ret = grib_get_double_array(h, name, val, length);
Expand All @@ -1259,6 +1277,17 @@ int grib_get_double_array_internal(const grib_handle* h, const char* name, doubl

return ret;
}
int grib_get_float_array_internal(const grib_handle* h, const char* name, float* val, size_t* length)
{
int ret = grib_get_float_array(h, name, val, length);

if (ret != GRIB_SUCCESS)
grib_context_log(h->context, GRIB_LOG_ERROR,
"unable to get %s as float array (%s)",
name, grib_get_error_message(ret));

return ret;
}

int grib_get_double_array(const grib_handle* h, const char* name, double* val, size_t* length)
{
Expand Down Expand Up @@ -1289,6 +1318,19 @@ int grib_get_double_array(const grib_handle* h, const char* name, double* val, s
}
}

int grib_get_float_array(const grib_handle* h, const char* name, float* val, size_t *length)
{
size_t len = *length;
grib_accessor* a = grib_find_accessor(h, name);
if(!a) return GRIB_NOT_FOUND;

/* TODO: For now only GRIB supported... no BUFR keys */
Assert(name[0]!='/');
Assert(name[0]!='#');
*length = 0;
return _grib_get_float_array_internal(h,a,val,len,length);
}

int ecc__grib_get_string_length(grib_accessor* a, size_t* size)
{
size_t s = 0;
Expand Down