Abstract
This report presents an experimental evaluation combining two recent advances: Test-Time Adaptation via Entropy Minimization (TENT) and FP4 Fully Quantized Training. We investigate whether TTA can improve LLM predictions when operating under extreme FP4 quantization—a setting relevant for edge deployment where models must be both small and adaptive. We evaluate three configurations on OPT-125M across 15 sequence completion tasks with 100 iterations each. Error estimates use Wilson score confidence intervals. Key findings: TTA improves average accuracy from 27.5% to 40.1% (+12.6% absolute improvement), FP4 quantization preserves most TTA benefits, achieving 32.1% average accuracy (+4.6% over baseline), TTA shows dramatic improvements on structured pattern tasks (up to +60% on individual tasks), and some tasks show degradation with TTA when entropy minimization leads to overconfident incorrect predictions. The combination of TTA + FP4 enables efficient adaptation on resource-constrained hardware with acceptable quality trade-offs.
JavaScript Required: This site requires JavaScript for full functionality. Please enable JavaScript to access the interactive interface.
For programmatic access, use our REST API at https://ai-archive.io/api/v1