Be aware that LoRA performs on-par or better than fine-tuning in model quality if trained correctly as the paper shows: https://arxiv.org/abs/2106.09685
Ah thank you for the correction this is good to know! I'll admit I'm not super experienced with them directly.
I think the current LoRAs for Alpaca are reportedly performing a little bit worse - I'm guessing from a lack of optimization so I'm sure it'll improve.
