Opportunistic GC collection

[maleadt]

As suggested by @gbaraldi: keep track of GPU memory pressure (using some atomic globals), and when allocating opportunistically call the GC to avoid only reclaiming memory when we run out of it.

[BioTurboNick]

Not sure how it impacts your decision here, but I ran into a lack-of-GC issue here (FluxML/Flux.jl#2414) which was completely solved by running GC.gc(false) every other iteration. Running the full GC helped over nothing at all, but was still much slower than the incremental sweep.

[maleadt]

Maybe you can try the linked PR? It needs tuning of the heuristic, so datapoints would be useful.