Model and Inference / TOKEN WIKI
The capabilities of the
Distillation / Knowledge Distillation
In one sentence
model distillation is to use the output, probability distribution or intermediate information of the teacher model to train the student model so that it can learn part of the teacher's capabilities on the target task.
Why it matters
can reduce deployment resources and inference costs in appropriate tasks, but the effect is affected by teacher quality, training samples and evaluation methods.
Practical considerations
first clarifies the target tasks and indicators, prepares an independent evaluation set, and verifies data and model authorization; at the same time, it compares quality, delay and unit task cost.
FAQ
Does distillation equal quantification or replication of model parameters?
No. Distillation usually involves student model training; quantization mainly changes the numerical representation, and the two can be used in combination.
