feat(db-postgres): fields enhancements

[r1tsuu]

Description

The PR adds:

Field config - dbStore

dbStore field property - removes schema creation for a field. Currently using Virtual Fields with Postgres bloats the db with unused columns / tables, as well may join empty arrays / blocks that are meant to be "Virtual". Realted discussion - #6270

Field config - dbJsonColumn

dbJsonColumn field property - stores a field, that should create additional tables (hasMany Text / Number / Select, Array / Blocks, Polymorphic / hasMany relationships) as a simple jsonb column.

Advantages

• Doesn't bloat a DB with too much tables for blocks (you can have them a lot).
• Migrations are simpler. therefore, chances to screw up them are less.
• 0 Joins and so the performance should be better. While originally you can have them a _lot
• Easier to work with the Payload nested data from DB driver, just SELECT blocks from table

Considerations:

• Querying on a field with dbJsonColumn. A solution would be is to implement a universal JSON querying that will work on any field type, that stored as a json column. Still you can use payload.db.drizzle.query to query on JSON with SQL.
• Foreign keys aren't created and so when relationship is deleted, setting to NULL isn't performed automatically. The same as with MongoDB.

My thoughts:
I haven't ever query on a blocks field, so i don't see a point of adding so much complexity just because of this.

Passing defaultValue into a DB schema

Non undefined and non function defaultValue is passed into a column builder. This 100% syncs defaultValue with a database and resolves a problem when adding a required field into a collection with existing docs, currently you can't do that unless you: lose data / modify migration manually.
Discussions: #6691 #6048

dbType for Number field

Accepts: 'integer' | 'bigint' | 'real' | 'numeric'
Simply stores number field in a specified column, by default numeric.

According to Postgres documentation calculations on numeric values are very slow compared to the integer types, or to the floating-point types.

Adds default validation with Number.isInteger.

• Chore (non-breaking change which does not add functionality)
• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Change to the templates directory (does not affect core functionality)
• Change to the examples directory (does not affect core functionality)
• This change requires a documentation update

Checklist:

• I have added tests that prove my fix is effective or that my feature works
• Existing test suite passes locally with my changes
• I have made corresponding changes to the documentation

[ryanleecode]

Goat

[DanRibbens]

Wow @r1tsuu, you're a boss!

I need some time to work through all of this. I like some of the ideas here a lot. Ideally you would be making smaller PRs covering one new feature or bugfix at a time. We move much slower digesting larger PRs like this as we have to plan to support all the features being added long term, it is harder to evaluate and risks not being able to be merged.

If you could make a PR that adds defaultValue to the DB directly, that would get merged in short order as it is higher priority for us.

Thanks, I'll have more feedback on the other changes when I have time to review.

[r1tsuu]

Wow @r1tsuu, you're a boss!

I need some time to work through all of this. I like some of the ideas here a lot. Ideally you would be making smaller PRs covering one new feature or bugfix at a time. We move much slower digesting larger PRs like this as we have to plan to support all the features being added long term, it is harder to evaluate and risks not being able to be merged.

If you could make a PR that adds defaultValue to the DB directly, that would get merged in short order as it is higher priority for us.

Thanks, I'll have more feedback on the other changes when I have time to review.

Hey Dan. I'll create separate PRs for each feature, initially it was easier for me to do these in 1, but just after I created the PR I realized that it would be probably hard to review, and it would be nice to track each feature as it's own.

But before doing this, I want to ask a question about defaultValue that I haven't thought while doing this PR.
Currently, actually, defaultValue works only on the admin panel level (via /api/get-form-state or something), not on the Local API level. As well, even if a field has defaultValue, you cannot just omit it in the Local API create operation.
So while it would be definetely helpful to have defaultValue synced in the DB, does it make sense now as we haven't done that on the Local API level?

But this won't be easy to do probably, because we as well want to ensure type-safetely. Currently, a required field is a required as well in the create operation, so we'll probably need to wrap these fields into some kind of generic WithDefaultValue<T> and then, create operation as well should receive instead of just T - CreateData<T>, where CreateData recursively removes "required" flag from fields that have WithDefaultValue..

What do you think? Sounds hard tbh for now. Maybe the best time for this can be with the other typescript improvements like depth and where and for now, just do that only on the db level.

Also, it looks like Mongoose has default values in their schema too https://mongoosejs.com/docs/defaults.html (I believe Payload does build the schema?) so maybe, it could be more generic PR instead of just with Postgres?

[r1tsuu]

Closing, will open a new, small ones like Dan said. defaultValue we already have