I have use PEFT lora technique to fine tune a mistral model i used bit and byte for my quantization so wanted to where its saving ,mu quantize model and how i can use quantize model with my adapter layer. if any reference please share article
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Finetuned LLM model conversion to GGUF - performance drop | 4 | 2073 | July 31, 2024 | |
| How to load a model fine-tuned with QLoRA | 2 | 7094 | July 29, 2024 | |
| Peft model from pretrained load in 8/4 bit | 6 | 17883 | October 12, 2023 | |
| Quantizing a model on M1 Mac for qlora | 0 | 1777 | March 14, 2024 | |
| Using Trainer class + 4/8 bit quantised model for prediction | 1 | 272 | August 22, 2025 |