How does pruning reduce model size for offline deployment?
Pruning removes redundant or less important parameters from neural networks, reducing model size and computation. Unstructured pruning zeros out individual weights based on magnitude or sensitivity analysis, achieving high sparsity but requiring sparse computation support for actual speedup. Structured pruning removes entire neurons, attention heads, or layers, producing naturally smaller dense models that run faster on standard hardware. The process typically involves training, pruning according to an importance criterion, then fine-tuning to recover accuracy. Modern approaches like SparseGPT can prune large language models to 50-60% sparsity in one shot without retraining. Wanda method uses weight and activation norms for efficient pruning. The combination of pruning followed by quantization can reduce model size by 10-20x while maintaining 90%+ of original quality.