Add integer, float, string, date, and datetime conversion functions #1437
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
refactor computed fn Use shared pointers for t_computed_expression add date() and datetime(), remove m_none, clean up computed fns datetime() takes POSIX timestamp
string() parse strings in integer() and float()
sc1f
force-pushed
the
more-expression-fixes
branch
from
ec35801
to
f705593
Compare
texodus
approved these changes
Jun 8, 2021
| @@ -230,7 +256,7 @@ t_computed_expression_parser::init() { | |||
| .disable_base_function(exprtk::parser<t_tscalar>::settings_store::e_bf_max); | |||
| } | |||
|
|
|||
| t_computed_expression | |||
| std::shared_ptr<t_computed_expression> | |||
There was a problem hiding this comment.
Why are these all shared_ptr now? Just curious
|
|
||
| #ifdef PSP_ENABLE_WASM | ||
| // check for overflow | ||
| if (number > std::numeric_limits<std::int32_t>::max() || number < std::numeric_limits<std::int32_t>::min()) { |
There was a problem hiding this comment.
I appreciate the attention to detail here, but it's worth noting we definitely don't handle this correctly at the JS boundaries and have frequent overflows .. though perhaps this doesn't need to be said #1346
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a set of type conversion functions to the expression language, which will allow users to quickly clean/coerce their data within Perspective if it is inferred as the wrong format:
integer(x): converts a column or scalar of any type to an integer. InPerspective.js, integers are 32-bit, thus values that overflow/underflow will be output asnull.float(x): converts a column or scalar of any type to a float.string(x): converts a column or scalar of any type into a string.datevalues are output using the formatYYYY-MM-DD, anddatetimevalues are output using the formatYYYY-MM-DD HH:MM:SS. Conversion of floats to strings can lose precision.date(year, month, day): given a year, month (1-12), and day, create a newdatevalue. This allows for parsing of numerical date formats that were previously impossible to cast to a date, such asYYYYMMDD. Parsing can be done with a simple expression:datetime(timestamp): given a POSIX timestamp of milliseconds since epoch, create a new datetime value. This allows for timestamp columns which would usually be inferred asfloat(unless the table is provided with a schema marking the column as adatetime) to be converted into datetimes. Because of the addition of scalars, converting from a seconds-since-epoch timestamp to a milliseconds timestamp is simple:Additionally, this PR removes a little debug cruft left over from #1431, and edits the Monaco autocomplete definitions to include the proper parameters for
bucket,if/else, andvar.