Calculation Methodology
1. Model VRAM Estimation
The base VRAM requirement for loading a model depends on its size and quantization level:
base_vram_fp16 = model_size_billions × 2 GB
model_vram = base_vram_fp16 × quantization_factor
quantization_factor = 1.0 for FP16, 0.5 for 8-bit, 0.25 for 4-bit- Each billion parameters requires approximately 2 GB in FP16 precision
- LoRA adapters add ~1% additional VRAM (negligible for estimation)
2. Batch Size Calculation
The maximum batch size is determined by available VRAM after loading the model:
available_vram = total_vram - model_vram - lora_vram - overhead
vram_per_batch = seq_length × 0.00002 GB
max_batch_size = floor(available_vram / vram_per_batch)
overhead = 10% of total VRAM reserved for system operations- Each token in a batch requires approximately 0.00002 GB (empirical)
- Batch size is automatically reduced to keep total usage below 95%
3. Gradient Checkpointing
When VRAM is insufficient for a batch size of 1, gradient checkpointing is enabled:
- Trades computation time for memory by recomputing activations during backward pass
- Reduces activation memory by approximately 30-50%
- Automatically enabled when
max_batch_size < 1
- May increase training time by 10-20%
4. Gradient Accumulation
Simulates larger batch sizes by accumulating gradients across multiple forward passes:
target_batch = 64 (for models ≤ 3B) or 32 (for larger models)
accumulation_steps = ceil(target_batch / max_batch_size)
effective_batch = max_batch_size × accumulation_steps
- Enables training with effective batch sizes larger than VRAM allows
- No memory overhead, only increases training steps
- Gradients are accumulated and applied every N steps
5. Training Steps & Epochs
Total training steps are calculated based on dataset size and epochs:
steps_per_epoch = ceil(dataset_tokens / (effective_batch × seq_length))
total_steps = steps_per_epoch × num_epochs
- Each step processes
effective_batch_size × seq_length tokens
- Epoch recommendations: 6 (<100K tokens), 4 (100K-500K), 3 (500K-1M), 2 (>1M)
- More epochs on small datasets help the model learn patterns effectively
6. Time Estimation
Training time is estimated based on GPU throughput and total computational load:
tokens_per_step = effective_batch_size × seq_length
time_per_step = (tokens_per_step × 3) / gpu_speed
total_time = total_steps × time_per_step
gpu_speed is measured in tokens processed per second during training- Factor of 3 accounts for forward pass, backward pass, and optimizer step
- Actual times vary based on model architecture, optimizer settings, and I/O overhead
- These are conservative estimates; real performance may differ by ±20-30%
7. LoRA Parameter Selection
When set to auto (0), LoRA rank is chosen based on model size:
- Models ≤ 1B parameters: rank = 32, alpha = 64
- Models 1B-7B parameters: rank = 16, alpha = 32
- Models > 7B parameters: rank = 8, alpha = 16
- Lower ranks reduce trainable parameters and VRAM usage
- Alpha is typically set to 2× rank for stable training