Simple and extensible hypergradient for PyTorch
First, install PyTorch appropriately. Then,
pip install hypergrad- conjugate gradient
- Neumann-series approsimation
- Nyström method
Implementation of these methods can be found in hypergrad/approximate_ihvp.py
@inproceedings{hataya2023nystrom,
author = {Ryuichiro Hataya and Makoto Yamada},
title = {{Nystr\"om Method for Accurate and Scalable Implicit Differentiation}},
booktitle = {AISTATS},
year = {2023}
}