knowledge-distillation-a-survey

Choose what kind of knowledge to transfer between teacher and student models: response, feature, or relational, and decide among offline, online, self, or cross-modal distillation schemes. Best for distillation strategy selection, capacity-gap diagnosis, and transfer planning. Activate on "knowledge distillation", "teacher-student", "soft labels", "feature distillation", "online distillation", or "cross-modal transfer". NOT for generic compression checklists or unrelated training work.