This repository hosts the code and resources for Quantized Model, a project focused on optimizing machine learning models through quantization techniques. Quantization reduces the computational requirements of models, making them more efficient for deployment on edge devices with limited hardware resources.
The goal of this project is to experiment with various quantization techniques to compress and optimize machine learning models while maintaining accuracy. This process makes it easier to deploy these models on low-power devices or systems with limited computational capacity.
The project explores:
- Post-training quantization and quantization-aware training.
- Model size reduction through parameter quantization.
- Performance benchmarks comparing quantized and non-quantized models.
- Quantization Techniques: Implements common quantization strategies including 8-bit and dynamic range quantization.
- Performance Comparison: Tools to measure and compare the accuracy, inference speed, and size of models before and after quantization.
- Deployment-Friendly: Optimized models are suitable for deployment on edge devices like mobile phones, Raspberry Pi, and embedded systems.
To use the project, you’ll need the following installed:
- Python 3.x
- TensorFlow, PyTorch, or the relevant ML framework
- Numpy
You can install the dependencies using:
pip install -r requirements.txt-
Clone the repository:
git clone https://github.com/DerekW00/quantized-model.git cd quantized-model -
Install required Python packages:
pip install -r requirements.txt
-
Run the quantization script (replace with the specific script name):
python quantize_model.py --model model.h5 --output quantized_model.tflite
The main functionality of the repository is to quantize models. Use the quantize_model.py script to apply quantization:
python quantize_model.py --model <model_path> --output <output_path> --method <quantization_method>Parameters:
--model: Path to the original model.--output: Output path for the quantized model.--method: Choose the quantization method (post_training, quantization_aware).
To compare the performance of the quantized model to the original:
python benchmark.py --model <quantized_model_path>This will output the inference time, model size, and accuracy metrics.
Example 1: Post-Training Quantization
python quantize_model.py --model ./models/mnist.h5 --output ./models/mnist_quantized.tflite --method post_trainingExample 2: Quantization-Aware Training
python quantize_model.py --model ./models/cifar10.h5 --output ./models/cifar10_quant_aware.tflite --method quantization_aware- Model Size: Quantized models show a significant reduction in size (up to 4x smaller).
- Inference Speed: Quantized models perform inference faster on edge devices.
- Accuracy Impact: Quantization introduces minimal accuracy loss while optimizing performance.
- Some models may experience more accuracy degradation than others, especially when using aggressive quantization strategies.
- Compatibility varies depending on the model architecture and framework.
- Explore further optimizations for convolutional neural networks (CNNs).
- Integrate additional frameworks (e.g., ONNX).
- Experiment with mixed precision and hardware-specific quantization optimizations.
Contributions are welcome! Please feel free to submit a pull request or open an issue to discuss potential changes or improvements.
This project is licensed under the MIT License.