-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trained models can be tied to specific backdoors #4
Comments
Thanks for writing up these options! I think saving the actual control grid is a better solution long-term than just using the same seed, for a few reasons:
Here's a potential way of implementing this, taking some inspiration from your choices:
Instead of One thing I don't like about this is that it assumes you always want to reuse transforms. Probably true most of the time, but this would make it completely impossible to e.g. test on a new control grid as an ablation. I suppose from simple_parsing.dataclasses import field
...
no_load: bool = field(action="store_true")
...
def load(path):
if self.no_load:
return Then by default you'd reuse the grid, but you could always add If you have ideas for making this more elegant, happy to discuss more, but otherwise I think this would be good enough from a practical perspective and doesn't require changing things too much. |
We could try to fix path specifications more thoroughly right away (there are a few cases where you need to specify paths multiple times). If that's something you want to tackle anyway, it probably makes sense to do both at once. Otherwise I'll probably look into a more general fix at some later point. |
Fixed in #5 |
An example of this issue as @ejnnr observed, is for the WaNet Backdoor. A trained model with this type of backdoor responds to a specific randomly generated warping field. Later you would typically like to evaluate a detector detector based on absence or presence of this warping field in question. As code currently looks this isn't possible.
I'd like to take this issue to consider some possible design choices.
Usage
This benchmark is about evaluating a Detector on how well anomalous behaviour can be detected in a Model for some input.
To make this possible there needs to be a known Task for which a dataset that is known to produce anomalous and normal behaviour in a model has to be known.
A Detector is then evaluated on a Task. Some Tasks need to initialized which in the case of the Backdoor task consists of:
When a Task has been initialized it should be possible to load this Task, at least in cases when initialization is costly (e.g. training a model). However, it is also of interest to load a modified Task. For example a different transform than the one used in training the model to see that these are not detected as anomalous.
Design choices
Main script is
so far this is the same set-up as before. Before this step there will be a
train_detector ... init_task ... # e.g. train_classifier
I don't see any need to change anything with the detector set-up.
The changes will have to be with how the task is handled both in initialization and the command line arguments to
eval detector
.Choice 1
Add a
--task from_run
similar as for the detector. This should preferably take as much as possible from the initialization of the task and then any additional settings.Most of the information already exist in the config.yml and model/ under
dir
. All that is missing is either a:For saving the backdoor I would probably suggest to pass the path to where to save the backdoor in a call to
train_loader.transforms.save_state(cfg.dir.path)
intrain_classifier.py
and add stuff in Transform and its children where needed to save what states are necessary for reproduction.Choice 2
Add an option
--task.my_task.run_path
,task.my_task.[...].run_path
or similar only when needed. In that case I would strongly prefer to have a reasonable default that uses an equivalent instance of a Task as was used when the Task was initialized.I can't think of a good way to set the default in e.g. WanetBackdoor such that this is the same path as was specified in
train_classifier.py
if this was called in that function. And then this something equivalent should also be the done when loading ineval_detector.py
.I don't think any of these alternatives is obviously good. The easiest one and the one I would probably pick in the short term is probably to just add something like
I'm not sure how this is/should be picked in
eval_detector.py
but the information should exist inconfig.yml
so shouldn't be too hard to retrieve.The text was updated successfully, but these errors were encountered: