Done for today. Have a great day!
Punchy
Wow, 17 animes today. You have deserved your break very well!
Looking forward to hear (see) about you tomorrow - and, if your time allows, perhaps you can reveal what LoRA stands for?
The two interpretations I've found in the internet do not make sense for me:
LoRa = Long range Radio
https://www.thethingsnetwork.org/docs/lorawan/what-is-lorawan/
and
LoRa = Low (power) Radio
https://en.wikipedia.org/wiki/LoRa?
Put on your nerd glasses...
LoRA stands for Low-Rank Adaptation, a technique used to efficiently fine-tune large machine learning models, such as large language models (LLMs), for specific tasks without retraining the entire model.
It works by adding small, lightweight matrices (low-rank matrices) to the original model's frozen weights, which are then trained to adapt the model's outputs for a new context.
This method drastically reduces the number of trainable parameters and computational resources required compared to full fine-tuning, making it a popular form of parameter-efficient fine-tuning (PEFT).
For example, LoRA can reduce the trainable parameters for a model like GPT-3 from 175 billion to around 18 million, significantly lowering GPU memory usage.
TL/DR, a LoRA is a tiny ai model that does a specific task that a larger model isn't trained on or does poorly. Make boobs bigger, make boobs bounce, make women fondle their breasts, etc. You add LoRAs onto the main model to fine tune it.
Punchy