keys in the top object

[gmabey]

I see that an asymmetry exists between .sigmf-meta files and .sigmf-collection files in that other keys besides "global", "captures", and "annotations" are allowed to be defined by the user (via an extension) for meta, but that collections may only have a single "collection" key.

Was this difference deliberate? Is there rationale for this besides my guess that the concept of a collection is very narrow in scope, whereas the metadata for recordings involves more unknowns?

For myself, I wish that only "global", "captures", and "annotations" were allowed at the top level for a recording, thereby forcing the user to decide whether it's a global concept or not.

[jacobagilbert]

SigMF .sigmf-meta files and .sigmf-collection files are really orthogonal in terms of metadata structure. Metadata schema have the usual three fields, which can all be extended via extensions, and collection files have specific objects which are applicable to the whole collection of individual SigMF recordings, and can also be extended by extensions.

There is some overlap in the key names from keys within sigmf-meta top level objects and sigmf-collection collection objects, but the values themselves are not interchangeable.

[gmabey]

Yes, but I think all of what you said is already in the spec, and you didn't address the asymmetry in allowance of top level fields like I'd hoped. I'm not hung up on this question, and if there's not a good answer then I can just close it I guess.

[jacobagilbert]

you didn't address the asymmetry in allowance of top level fields like I'd hoped

I must be missing something. Are you asking why collections files cannot have global, captures and annotations objects?

[gmabey]

Sorry let me back up. Both recordings and collections have a canonical set of top-level keys; in the case of recordings, that set is of size 3, whereas for collections it's 1.

The asymmetry I observe in the spec is that it's apparently allowed for there to exist other top-level keys (outside of their respective canonical set) in a .sigmf-meta file but not in a .sigmf-collection file.

[jacobagilbert]

Additional top level keys are not permitted in compliant metadata or collections.

For Metadata:

The top-level Object MUST contain three JSON Objects named global, captures, and annotations.

For Collections:

The Collection Object MUST be the only top-level object in the file.

Extensions are not permitted to create additional top level keys as far as i know though this could probably be more explicit in the spec. I think we can sharpen some of this to make it more clear "The top-level Object MUST contain exactly three JSON Objects"

It is possible to have noncompliant but still functional and useful SigMF schema though, just not something we really advocate users do.

[gmabey]

I agree that it could/should be sharpened, but here's the line that led me to believe that extensions could add new top-level, under Extension Namespaces:

4. An extension namespace MAY define new top-level SigMF Objects, key/value pairs, new files, new Dataset formats, or new datatypes.

Oh, does "SigMF Objects" refer to additional files?? Surely not ... at least now you know what wording let me to believe that other top-level key words were allowed.

And as I tried to originally state, I support the concept of only allowing those 3 key words at the top level.

[jacobagilbert]

Hm. This is not something I was aware of and is somewhat problematic. Will think on it.

[jacobagilbert]

@bhilburn

Is this still what we want? Specifically should new top level metadata keys or "new files" really be permitted by extensions?

[gmabey]

poke @bhilburn

[jacobagilbert]

Going to close this out.

Collections are intentionally very narrow in scope, while Recordings can be created to contain a wide range of data which may cause users to wish to have the freedom to create new top level objects. The spec is consistent in that metadata files must contain at least those three top level objects, and collections are restricted to the single top level object. On some level I would have probably liked to limit the metadata to those three top level keys but this is out there and its fine as far as i can tell.

If there are breaking use cases here [for collections], lets address them specifically.