How to Add Latest Distillation Techniques such as DIST or KD++ to this framework #377

Answered by yoshitomo-matsubara

nighting0le01 asked this question in Q&A

nighting0le01
Jul 24, 2023

How to Add Latest Distillation Techniques such as DIST or KD++ to this framework

Answered by yoshitomo-matsubara

Hi @arpitsahni04

If you explain how those methods should work precisely

what teacher and student models and datasets are used
how their loss functions are designed
which layers of the teacher and student models are used to compute the losses

then, I can explain or give it a try

View full answer

Replies: 1 comment

yoshitomo-matsubara
Jul 25, 2023
Maintainer

Hi @arpitsahni04

If you explain how those methods should work precisely

what teacher and student models and datasets are used
how their loss functions are designed
which layers of the teacher and student models are used to compute the losses

then, I can explain or give it a try

0 replies

Answer selected by nighting0le01

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment