Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] Allow usage of additional types in config #11

Open
Rizhiy opened this issue Jan 16, 2019 · 4 comments · May be fixed by #39
Open

[Feature request] Allow usage of additional types in config #11

Rizhiy opened this issue Jan 16, 2019 · 4 comments · May be fixed by #39

Comments

@Rizhiy
Copy link
Contributor

Rizhiy commented Jan 16, 2019

Currently only a few primitive types are allowed as values in config, this is a bit restrictive. For example, I would like to store pixel mean values in the config, currently I would need to use list or tuple. In most cases I would have to convert those values to np.array each time I want to use them, which is inconvenient.

It would be nice if I could use np.array inside config. I think any type which can be represented as a string or in terms of simple types should be available to be used in the config. In case of np.array it can be represented in terms of nested lists in yaml file and convert to array on load.

@rbgirshick
Copy link
Owner

This is something that I've thought about in the past and arrived at the conclusion of only allowing primitive types. Here are the learnings from Detectron, where we allow any type in the config, including user defined classes. Two issues arise:

  1. You cannot use the more security safe yaml.safe_load and yaml.safe_dump methods with custom types. Using the "danger" versions of these functions increases the risk of malicious config attacks.

  2. Having configs with serialized user classes can lead to annoying issues when refactoring. For example, in Detectron we had to introduce some hacks to deal with a renamed module,
    here: https://github.com/facebookresearch/Detectron/blob/master/detectron/core/config.py#L1063-L1065
    and here: https://github.com/facebookresearch/Detectron/blob/master/detectron/core/config.py#L1122-L1125

My conclusion from this is that it's probably best to restrict types to primitive ones. Though I do understand that there are tradeoffs and so it is a compromise position.

I have not checked if the numpy.array type can be used with yaml.safe_{load,dump}. If so, perhaps that could be one exception. On the other hand, the workaround of having np.array(cfg.SOME_ARRAY) is perhaps better than giving special status to some non-primitive types.

@ddkang
Copy link

ddkang commented Apr 16, 2019

What about the None type?

@Rizhiy
Copy link
Contributor Author

Rizhiy commented Mar 5, 2020

@rbgirshick, still having this problem.

I agree with the issues you raised and as a compromise I think yacs should support all types that are supported by yaml.safe_{load,dump}.

I currently have a problem with dates. When I try to use date-like string in yaml, pyyaml automatically converts it to datetime.date, e.g.:

DATE: 2020-03-05

Also, null is useful to indicate absence of value.

@Rizhiy Rizhiy linked a pull request Mar 5, 2020 that will close this issue
@Filos1992
Copy link

please add None as a a valid type :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants