Non-Uniform Quantization for Accurate and Efficient Deep Neural Networks

Abstract

This paper introduces a new neural network quantization framework designed to improve model efficiency while maintaining high inference accuracy. The proposed approach, referred to as UNIQ, employs a non-uniform quantization strategy based on k-quantile distributions and incorporates controlled noise injection during training to enhance robustness to weight discretization. By adapting the learning process to quantized representations, the method enables effective low-precision inference. In addition to weight compression, experimental analysis shows that network activations can be reduced to 8-bit precision with minimal performance loss. Compared with conventional uniform quantization schemes, the proposed strategy offers greater flexibility in balancing accuracy and computational cost. To quantify model complexity, a new evaluation metric based on bit-level operations is introduced, which exhibits strong correlation with hardware resource consumption and power usage. This metric enables systematic analysis of the trade-off between model precision and implementation cost. The proposed method is validated on multiple benchmark architectures, including ResNet and MobileNet, using the ImageNet dataset. Results demonstrate superior performance across both low-complexity and high-accuracy operating regimes. Furthermore, an FPGA-based implementation confirms the practical feasibility of the proposed quantization framework.