Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate Collection Schema #283

Merged
merged 3 commits into from
Mar 26, 2020
Merged

Validate Collection Schema #283

merged 3 commits into from
Mar 26, 2020

Conversation

asutula
Copy link
Member

@asutula asutula commented Mar 25, 2020

This one does a couple things...

Makes CollectionConfig.Schema a proper Schema struct. This is nice because it provided a programatic API to the schema we can use for validation. Also makes the API nicer to work with not having to pass in a schema string.

Performs validation of the provided schema when a new collection is created. Currently, that means checking that the schema defines an ID property of type string.

closes #281

@asutula asutula self-assigned this Mar 25, 2020
}
if !hasIDProperty(properties) {
return nil, ErrInvalidCollectionSchema
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Above is the schema validation. As far as I can tell, there are two ways an ID might be defined, and we have to handle them both. First is the "expanded" schema case where a type is defined in the top level of the schema. Second is when the top level schema specifies a Ref to type nested in the schema Definitions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still like the idea of validating the schema against another schema, but this was a more clear approach for now.

@@ -12,7 +12,7 @@ message NewDBRequest {

message CollectionConfig {
string name = 1;
string schema = 2;
bytes schema = 2;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema was always json data, so make it bytes like the rest of the json data we deal with

if _, err := db.NewCollection(cc); err != ErrInvalidCollectionSchema {
t.Fatal("the collection should be invalid")
}
})
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests reenabled to make sure schema validation is actually working

@@ -190,7 +197,7 @@ func (d *DB) reCreateCollections() error {
// CollectionConfig describes a new Collection.
type CollectionConfig struct {
Name string
Schema string
Schema *jsonschema.Schema
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not just a string anymore

schema := jsonschema.Reflect(i)
JSON, err := json.MarshalIndent(schema, "", " ")
if err != nil {
func SchemaFromInstance(i interface{}, expandedStruct bool) *jsonschema.Schema {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the noise in this PR is because of adding the expandedStruct option here. When set to true, the generated schema will use top level type definitions instead of using a ref to a nested definition. We need to be able to test both cases.

Signed-off-by: Aaron Sutula <[email protected]>
@asutula
Copy link
Member Author

asutula commented Mar 25, 2020

Ok, ready for review here.

@asutula
Copy link
Member Author

asutula commented Mar 25, 2020

Android failure is that rate limit related to some GH action thing... Should be fine when it re-runs soon.

Copy link
Contributor

@jsign jsign left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Member

@sanderpick sanderpick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@asutula asutula merged commit 613b81e into master Mar 26, 2020
@asutula asutula deleted the asutula/validate-schema branch March 26, 2020 15:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validate collection schema on creation
3 participants