Soup CLI

Soup CLI is an innovative open-source tool designed for fine-tuning large language models (LLMs) on modest hardware, specifically laptops with 4 GB VRAM. It leverages a unique approach by keeping the base model frozen with LoRA (Low-Rank Adaptation) techniques, streaming decoder layers into the GPU one at a time, which significantly reduces VRAM requirements. This enables developers and AI enthusiasts to train and customize 8-billion-parameter models like Llama-3.1-8B on accessible hardware, making advanced AI model fine-tuning more approachable and cost-effective. With a simple YAML configuration and a single command, users can perform tasks such as supervised fine-tuning (SFT), direct policy optimization (DPO), and evaluation, all while maintaining transparency, as every performance metric is published. Its open-source nature and lightweight design make it ideal for researchers, hobbyists, and small teams looking to experiment with large models without high-end infrastructure.