Using Distillation on a differenta dataset using a trained teacher.

[Gharibw]

Thanks for this amazing work. The current command line example shows how to create a model using the kd process and a single dataset--ie.e, CIFAR10. However, I am trying to create a student model (using its own dataset) via distillation from a teacher model that was trained on a very similar but different dataset. Any guidance on how to accomplish this would be greatly appreciated.

[fruffy]

We mostly focussed on CIFAR10 and 100 so it might be fairly tricky to extend our scripts to allow arbitrary data sets.
You can hook up your own dataset here by replacing the get_cifar function call with your own here.

But there is no guarantee it will work since you will also likely have to customize how a teacher and student is loaded here and here.

I think at this point it might be easier to take the kd trainer here and use it with your own scripts.

[Gharibw]

Thanks for your response!