-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support user defined data types when describing table schema. #771
Conversation
@@ -4,6 +4,7 @@ import com.github.ghik.silencer.silent | |||
|
|||
import java.time._ | |||
import scala.language.implicitConversions | |||
import java.math.BigDecimal |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
doesn't seem to be needed
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately we don't support scala.math.BigDecimal until I add TypeId to Schema.Transform. scala.math.BigDecimal is encoded as Schema.transform of java.math.BigDecimal and without TypeId on Transform I am not able to derive TypeTag for scala.math.BigDecimal.
val productTable = ( | ||
string("id") ++ | ||
localDate("last_updated") ++ | ||
string("name") ++ | ||
int("base_amount") ++ | ||
int("final_amount") ++ | ||
boolean("deleted") | ||
).table("product") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should still have tests that are using the "old way".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We are not supporting ++ for definition of "source" tables. Let's discuss if there's a use case for having both.
builder.append( | ||
s"""TO_TIMESTAMP_TZ('${formatter.format( | ||
s"""TO_TIMESTAMP_TZ('${ISO_INSTANT.format( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
// TODO uncomment and fix java.sql.SQLDataException: ORA-01874: time zone hour must be between -15 and 15 | ||
// oracle won't run on m1 machine |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So this is a problem only on m1 machine?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, the test is failing everywhere but I am not able to reproduce on m1 because of oracle image not running at all.
val customers = | ||
(uuid("id") ++ localDate(“date_of_birth”) ++ string("first_name") ++ string("last_name") ++ boolean("verified_customer") ++ zonedDateTime("created")) | ||
.table("customers") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Later in section Subqueries & Correlated subqueries
of this document, there are still examples of this kind of definitions but without explanation.
I think we should have both ways well documented. The ++
operator is still useful for defining simple ad-hoc queries for which we don't need to define a model.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As I replaced ++ with data type definition, I now changed it in all places in docs.
This PR introduces new way to describe tables with the
defineTable[A]
method.Old way to define table schema was by using DSL and ++ operator.
While composable, it was a bit boilerplate-y and another DSL to learn. I think that for Scala devs its more natural to write a case class, which could be then reused when doing select or insert with the same table.
We can get table description just by calling
defineTable[Customer]
method. The case class data type also needs an implicit schema (which is required for inserts anyway)There are 2 overrides
defineTableSmart[Customer]
anddefineTable[Customer]("customers")
, where the first one smartly tries to make plural out of case class name, while taking into account also irregular english plural words.OrderOrigin -> order_origins
PersonAddress -> person_addresses
Foot -> feet
defineTable[Customer]("customers")
takes table name explicitly.Field names are also converted from camelCase to snake_case
firstName -> first_name
There is also a macro which verifies that data type can be used to describe table ( sql does not know sum types). For example:
defineTable[Bill]
will emit an compilation errorUnsupported types by SQL OrderDetails.PaymentMethod
@jdegoes @jczuchnowski let me know what you think. Docs are updated as well.