add no_grad for inference runs

[Krovatkin]

add no_grad for inference runs

[wconstab]

I think this is a good idea, but is it potentially limiting and presumptive to put it here for all benchmarks? I can imagine one or two benchmarks might use grad in their forward, although I'm not aware of having one that does that now.

Also, i'd be wary of benchmarking the system in a way that is not how users use it. Do users generally set no_grad when doing inference?

Still I'm in favor of changing this either here or possibly inside the benchmark code themselves.

[asuhan]

@wconstab Why would our benchmarks use gradient in forward, are you thinking that might be the case because of the test failures we're seeing?

Would be interesting if we could run benchmarks both with and without no_grad set, it'd give us a sense of some pure overheads (since we wouldn't actually compute the gradients in either case) through comparison.

[wconstab]

@asuhan I don't think any of our current benchmarks do, but I think there are some types of less common models that compute a grad as part of their inference step. I need to look for an example.