Best for:

• Initial prototyping and experimentation
• Limited GPU resources (single consumer GPU)
• Fast iteration during development
• Simple tasks with clear visual patterns
• Real-time inference requirements

Tradeoffs:

• Lower capacity for complex reasoning
• May struggle with subtle distinctions
• Faster training and inference
• Lower memory and compute costs

Example use cases:

• Binary classification (good/defective)
• Single object detection
• Simple quality control

Best for:

• Production deployments
• Most standard computer vision tasks
• Multi-class detection and classification
• Balanced performance and resource usage

Tradeoffs:

• Good balance of accuracy and efficiency
• Reasonable training times
• Moderate GPU requirements
• Suitable for most use cases

Example use cases:

• Multi-object detection
• Complex defect classification
• Retail product recognition
• General-purpose visual understanding

Best for:

• Maximum accuracy requirements
• Complex reasoning tasks
• Fine-grained distinctions
• Production systems with ample resources

Tradeoffs:

• Highest accuracy potential
• Significantly longer training times
• High GPU memory requirements
• May require distributed training

Example use cases:

• Medical image analysis
• Detailed inspection tasks
• Open-ended visual reasoning
• Challenges requiring nuanced understanding

Use for most production applications:

• Standard computer vision tasks (detection, classification)
• Limited GPU resources
• Faster iteration during development
• Domain adaptation (e.g., retail → manufacturing)
• Cost-effective training

Use when your task is significantly different from the base model's training:

• Highly specialized domain (e.g., microscopy, satellite imagery)
• Novel visual patterns not seen in general training
• Maximum accuracy is critical and resources are available
• Task requires fundamental changes to feature extraction

Use for most scenarios:

• Standard VLM training workflows
• Production model training
• When you want best quality with quantization
• Recommended default choice

Advantages:

• Optimized value distribution for neural network weights
• Better preservation of model quality than FP4
• Specifically designed for transformer architectures
• Minimal accuracy loss compared to full precision
• Industry best practice for VLM training

Use when:

• Compatibility issues with NF4
• Debugging quantization-related issues
• Specific requirements for standard floating point format
• Legacy configurations

Note: For most use cases, NF4 is preferred as it provides better quality with the same memory savings.

Best for most scenarios:

• Modern GPUs (NVIDIA A100, H100, RTX 30/40 series)
• Training large models (7B+ parameters)
• Balanced speed and stability
• Production training workflows

Why it's better:

• Preserves Float32's exponent range (better for extreme values)
• Reduces gradient underflow/overflow issues
• Widely supported in modern ML frameworks
• Minimal accuracy loss compared to Float32

Use when:

• Older GPUs without BFloat16 support
• Maximum speed is critical
• Working with smaller models (<7B parameters)

Considerations:

• May encounter numerical instability with very large or small gradients
• Requires gradient scaling for stability
• Slightly more prone to training issues than BFloat16

Use when:

• Debugging numerical stability issues
• Research requiring maximum precision
• Training is unstable with lower precision
• GPU memory is not a constraint

Tradeoffs:

• 2x slower than Float16/BFloat16
• 2x more memory usage
• Rarely necessary for production training

Small datasets (<100 images):

• Recommended: 100-300 epochs
• Reasoning: More passes needed to learn from limited data
• Watch for: Overfitting after ~200 epochs

Medium datasets (100-1000 images):

• Recommended: 50-150 epochs
• Reasoning: Balanced learning with sufficient data
• Watch for: Convergence plateau around 100 epochs

Large datasets (1000+ images):

• Recommended: 20-100 epochs
• Reasoning: Fewer passes needed with abundant data
• Watch for: Diminishing returns after 50-75 epochs

Increase epochs when:

• Training loss is still decreasing steadily
• Validation metrics improving each epoch
• Model hasn't converged yet
• Early in experimentation phase

Example: Training loss: epoch 50 = 0.45, epoch 100 = 0.32, epoch 150 = 0.28

• Still improving → continue training

Reduce epochs when:

• Training loss plateaus early
• Validation performance stops improving or degrades (overfitting)
• Training time is excessive
• Model converges quickly

Example: Training loss: epoch 30 = 0.25, epoch 60 = 0.24, epoch 90 = 0.24

• Converged at epoch 30 → reduce to 50 epochs

Signs:

• Training loss increases or oscillates wildly
• Loss suddenly spikes to very large values
• Model predictions become nonsensical
• Training diverges or produces NaN values

Example:

Epoch 1: loss = 2.3
Epoch 2: loss = 1.8
Epoch 3: loss = 5.7 ← Spike indicates too high
Epoch 4: loss = NaN ← Training diverged

Fix:

• Reduce learning rate by 2-5x (e.g., 0.0001 → 0.00002)
• Start a new training run with adjusted rate
• Consider using smaller batch size for more stable gradients

Signs:

• Training loss decreases very slowly
• Progress stalls at high loss values
• Training takes excessively long
• Model underfits the data

Example:

Epoch 10: loss = 2.1
Epoch 20: loss = 2.08
Epoch 30: loss = 2.06 ← Very slow improvement
Epoch 40: loss = 2.04

Fix:

• Increase learning rate by 2-3x (e.g., 0.00001 → 0.00003)
• Start a new training run with adjusted rate
• Monitor closely to ensure stability

Good training behavior:

• Loss decreases steadily without large spikes
• Occasional small fluctuations are normal
• Converges within expected number of epochs
• Validation metrics improve consistently

Example:

Epoch 5:  loss = 2.1
Epoch 10: loss = 1.6
Epoch 15: loss = 1.3 ← Steady improvement
Epoch 20: loss = 1.1
Epoch 25: loss = 0.95

Characteristics:

• Smooth loss curve with minor noise
• Clear downward trend
• No sudden spikes or divergence
• Validation performance tracks training improvement

Learning rate schedules automatically adjust the learning rate during training:

Common schedules:

1. Constant (default):

   • Same rate throughout training
   • Simple and predictable
   • Good for most use cases

2. Linear decay:

   • Gradually reduces rate over training
   • Helps fine-tune convergence at the end
   • Useful for long training runs

3. Cosine annealing:

   • Reduces rate following cosine curve
   • Smoother decay than linear
   • Popular for large model training

When to use schedules:

• Long training runs (100+ epochs)
• Fine-tuning pre-trained models
• Seeking optimal convergence
• Advanced optimization scenarios

Note: Most training workflows work well with constant learning rate. Schedules are an advanced optimization technique.

When GPU memory is limited:

Start with smallest batch size that trains successfully:

1. Try batch size 4
2. If out of memory, reduce to 2
3. If still issues, try batch size 1
4. Consider enabling quantization or using gradient accumulation

Memory optimization strategies:

• Enable quantization (NF4)
• Use gradient accumulation to simulate larger batches
• Choose smaller model size
• Reduce precision type to FP16/BF16

For optimal training:

Use the largest batch size that:

• Fits in GPU memory comfortably (~80% utilization)
• Maintains stable training (no memory errors)
• Provides reasonable training speed

Practical approach:

1. Start with recommended value for your GPU
2. Increase until you encounter memory issues
3. Reduce by 25-50% for safety margin
4. Monitor training stability

Example:

• GPU: 24GB VRAM
• Model: 13B parameters with LoRA
• Test batch sizes: 4 → 8 → 16
• Batch size 16 causes OOM errors
• Final choice: batch size 8 (safe maximum)

Batch size is less critical when:

• Using gradient accumulation (can simulate larger batches)
• Training small models with ample GPU memory
• Dataset is large (1000+ images)

Batch size is more critical when:

• GPU memory is constrained
• Training very large models
• Dataset is small (gradient noise matters more)

Problem: Want larger batch size but GPU memory is insufficient

Solution: Use gradient accumulation

Example:

• Target effective batch size: 16
• GPU can only handle batch size 4
• Set gradient accumulation steps: 4
• Effective batch size: 4 × 4 = 16 ✓

Tradeoff:

• Training takes longer (4x more forward passes before each update)
• Same memory usage as batch size 4
• Training stability of batch size 16

Higher effective batch sizes provide:

• More stable gradient estimates
• Smoother loss curves
• Better convergence for large models
• Reduced gradient noise

Recommended combinations:

All achieve same effective batch size with different memory/speed tradeoffs.

Avoid gradient accumulation when:

• GPU memory is sufficient for desired batch size
• Small datasets where gradient noise aids generalization
• Training is already slow and speed is critical
• Batch size 1-2 is sufficient for your model size

Why avoid unnecessary accumulation:

• Slower training (more forward passes per update)
• Adds complexity without benefit
• Default (accumulation steps = 1) works fine for most cases

16GB GPU:

• Batch size 2, accumulation steps 4 (effective: 8)
• Batch size 1, accumulation steps 8 (effective: 8)

24GB GPU:

• Batch size 4, accumulation steps 2 (effective: 8)
• Batch size 2, accumulation steps 4 (effective: 8)

40GB+ GPU:

• Batch size 8, accumulation steps 1 (effective: 8)
• Usually no accumulation needed

Goal: Effective batch size of 8-16 for most VLM training

Use for:

• All standard VLM training (recommended for 95%+ of cases)
• Production models
• Fine-tuning transformer models
• When you want best practices defaults

Advantages:

• Decoupled weight decay improves regularization
• Adaptive learning rates per parameter
• Proven effective for large language and vision models
• Industry standard for transformer training
• Better generalization than Adam

Why it's better than Adam:

• Separates weight decay from gradient-based updates
• More effective regularization
• Better final model performance
• Widely adopted as best practice

Use for:

• Reproducing older experiments that used Adam
• Compatibility with existing configurations
• Specific research requirements

Note: AdamW is generally preferred over Adam. The main reason to use Adam is backward compatibility with older experiments or when reproducing specific published results that used Adam.

Difference from AdamW:

• Couples weight decay with gradient updates
• Slightly less effective regularization
• May lead to marginally lower performance

Recommended: 512 tokens

Phrase grounding outputs include:

• Descriptive caption
• Multiple grounded phrases
• Bounding box coordinates
• JSON structure

Example output size:

{
  "phrase_grounding": {
    "sentence": "A red car parked next to two people wearing blue shirts on the sidewalk near a green tree",
    "groundings": [
      {"phrase": "A red car", "grounding": [[120,340,580,670]]},
      {"phrase": "two people", "grounding": [[600,280,750,720],[780,290,920,710]]},
      {"phrase": "the sidewalk", "grounding": [[0,650,1024,900]]},
      ...
    ]
  }
}

Typical length: 200-400 tokens

Setting 512 provides comfortable margin without waste.

Recommended: 128-256 tokens

VQA responses are typically:

• 1-3 sentences
• 20-80 tokens
• Focused and concise

Example responses:

• Short: "There are three dogs in the image" (~8 tokens)
• Medium: "The image shows three golden retrievers playing in a park on a sunny day" (~15 tokens)
• Long: "There are three dogs visible: two golden retrievers playing with a ball in the foreground and one black labrador resting under a tree in the background" (~30 tokens)

Setting 256 allows detailed answers without encouraging verbosity.

Consider:

1. Measure typical output length:

   • Generate sample outputs
   • Count tokens (roughly 1 token per word)
   • Use 1.5-2x the typical length as max

2. Balance quality and cost:

   • Longer limits → more flexibility but slower
   • Shorter limits → faster but may truncate
   • Start conservative, increase if needed

3. Monitor for truncation:

   • If outputs frequently hit the limit, increase
   • If outputs are always much shorter, decrease

Recommended: 50

For tasks requiring factual accuracy:

• Object detection
• Bounding box generation
• Answering specific questions
• Structured output generation

Why 50 works well:

• Enough diversity to avoid repetition
• Focused enough for accurate outputs
• Balances creativity and precision

Example: "How many cars are in the image?"

• Lower K: "There are 3 cars" (consistent)
• Higher K: Various phrasings but same count

Recommended: 70-100

For tasks benefiting from diversity:

• Detailed scene descriptions
• Creative captions
• Varied phrasing

Why higher K helps:

• More vocabulary variety
• Diverse expression styles
• Less repetitive phrasing

Example: "Describe this scene"

• Lower K: More standardized descriptions
• Higher K: More varied and creative language

Top K and Top P work together:

Both limit the sampling pool:

• Top K: "Keep top 50 tokens"
• Top P: "Keep tokens until cumulative probability reaches 0.95"

In practice:

• Set Top K as hard upper limit
• Top P provides dynamic threshold
• Both constrain sampling for quality

Recommended combination:

• Top K: 50
• Top P: 0.95
• Works well for most scenarios

More diverse outputs:

• Includes less probable tokens
• Greater output variety
• More creative and unpredictable
• Risk of lower coherence

Use when:

• Creative description tasks
• Varied phrasing desired
• Output diversity important
• Repetition is a problem

Example: Top P = 0.95

• Allows more vocabulary choices
• Includes less common but valid alternatives
• Increases expression variety

Balanced outputs (recommended for most tasks):

• Moderate diversity
• Maintains coherence
• Focused but not repetitive
• Good default choice

Use when:

• Standard VLM tasks
• Balance needed between quality and diversity
• First-time configuration
• Phrase grounding and VQA

Example: Top P = 0.90

• Focuses on high-probability tokens
• Allows some variation
• Prevents most low-quality choices

Focused outputs:

• Very deterministic
• Minimal diversity
• Highly consistent
• Risk of repetition

Use when:

• Maximum consistency required
• Factual accuracy critical
• Template-like outputs desired
• Debugging generation issues

Example: Top P = 0.80

• Very focused token selection
• Predictable outputs
• May become repetitive

Configuration:

• Top K: 50
• Top P: 0.95
• Temperature: 1.0

Why this works:

• Allows diverse phrasing for captions
• Maintains focus for accurate groundings
• Balances creativity and precision

Configuration:

• Top K: 50
• Top P: 0.90
• Temperature: 0.7

Why this works:

• Focuses on most probable (accurate) answers
• Reduces creative but potentially incorrect responses
• Maintains consistency across similar questions

Configuration:

• Top K: 70
• Top P: 0.95
• Temperature: 1.2

Why this works:

• High diversity in vocabulary
• Varied expression styles
• Creative but coherent outputs

Focused, deterministic outputs:

Use for:

• Factual question answering
• Structured output generation
• Consistency critical tasks
• Bounding box generation

Characteristics:

• Very predictable outputs
• High consistency
• Low diversity
• Risk of repetition with very low values

Example use case: Counting objects in images:

• Temperature 0.5
• Consistent "There are X objects" format
• Minimal phrasing variation
• Focus on accuracy

Recommended values:

• VQA (factual): 0.7
• Object counting: 0.5
• Structured outputs: 0.6

Balanced outputs (recommended for most tasks):

Use for:

• Standard VLM tasks
• Phrase grounding
• General image understanding
• Most production scenarios

Characteristics:

• Good balance of quality and diversity
• Natural-sounding outputs
• Appropriate variation
• Reliable performance

Example use case: Phrase grounding captions:

• Temperature 1.0
• Varied but accurate descriptions
• Natural language flow
• Consistent quality

Recommended values:

• Phrase grounding: 1.0
• VQA (descriptive): 1.0
• General tasks: 0.9-1.1

Creative, diverse outputs:

Use for:

• Creative image descriptions
• Multiple phrasing alternatives
• Exploring diverse generations
• Reducing repetition

Characteristics:

• High output diversity
• Creative language use
• Less predictable
• Risk of incoherence if too high

Example use case: Creative scene descriptions:

• Temperature 1.5
• Varied vocabulary and phrasing
• Multiple perspectives
• Interesting but potentially less consistent

Recommended values:

• Creative descriptions: 1.3-1.5
• Diverse generations: 1.4
• Maximum diversity: 1.6-1.8

Caution: Values above 1.5 may reduce quality

Temperature works with Top K and Top P:

Processing order:

1. Temperature: Reshapes probability distribution
2. Top K: Limits to K most probable tokens
3. Top P: Limits to cumulative probability threshold
4. Sample: Choose next token from remaining options

Recommended combinations:

For factual tasks:

Temperature: 0.7 (focused)
Top K: 50 (moderate limit)
Top P: 0.90 (focused sampling)
→ Deterministic, accurate outputs

For balanced tasks:

Temperature: 1.0 (neutral)
Top K: 50 (moderate limit)
Top P: 0.95 (balanced sampling)
→ Natural, reliable outputs

For creative tasks:

Temperature: 1.3 (creative)
Top K: 70 (wider limit)
Top P: 0.95 (diverse sampling)
→ Varied, interesting outputs

Minimal repetition discouragement:

Use for:

• Natural language where repetition is acceptable
• Technical descriptions requiring specific terminology
• Structured outputs with repeated elements
• Default starting point

Value 1.0 (no penalty):

• Natural repetition patterns
• May repeat common words naturally
• Good for most standard tasks

Value 1.05 (gentle penalty):

• Slight preference for variety
• Maintains natural language flow
• Recommended default for most VLM tasks

Example: Phrase grounding captions naturally repeat:

• "The red car next to a blue car" (car repeated appropriately)
• Penalty 1.05 allows this natural repetition

Balanced repetition control (recommended):

Use for:

• Long-form descriptions
• Reducing noticeable repetition
• Creative text generation
• When repetition becomes problematic

Value 1.1:

• Good balance for most use cases
• Reduces obvious repetition
• Maintains coherence

Value 1.2:

• Stronger variety encouragement
• For longer outputs
• When 1.1 shows too much repetition

Example: Without penalty: "The image shows a person wearing a hat. The person is standing next to another person. Each person has a bag."

With penalty 1.2: "The image shows a person wearing a hat, standing next to someone else. Both individuals carry bags." → More varied vocabulary

Strong repetition avoidance:

Use for:

• Creative writing scenarios
• Extreme repetition problems
• Experimental settings

Caution:

• May force unnatural word choices
• Can reduce coherence
• Might avoid necessary repetitions
• Use only when repetition is severe

Value 1.5:

• Very strong penalty
• Significantly alters word choice
• Risk of awkward phrasing

Example: With very high penalty: "The car is red" → hard to repeat "car" → forced to use "vehicle", "automobile", "auto" even when "car" is most natural

Generally not recommended unless repetition is extreme.

Recommended: 1.05 (gentle)

Phrase grounding naturally involves:

• Repeating object names in groundings
• Similar phrasing across detections
• Structured JSON format

Why gentle penalty:

• Allow natural terminology repetition
• Maintain accurate object references
• Preserve structured output format

Example:

{
  "groundings": [
    {"phrase": "a red car", ...},
    {"phrase": "a blue car", ...},
    {"phrase": "the parked cars", ...}
  ]
}

→ "car" repeated appropriately

Recommended: 1.05-1.1

VQA responses are typically:

• Short (1-3 sentences)
• Focused answers
• Limited repetition risk

Why light penalty:

• Short outputs have less repetition
• Focus on accuracy over variety
• Natural answer patterns acceptable

Example question: "What color are the cars?"

Answer with 1.05: "There are two cars: a red car and a blue car." → Natural repetition of "car" acceptable

Recommended: 1.1-1.2

Longer outputs risk more repetition:

• Multiple sentences
• Describing many objects
• Detailed scene understanding

Why moderate penalty:

• Encourages vocabulary variety
• Maintains natural flow
• Prevents monotonous phrasing

Example without penalty: "The scene shows a person in a red shirt. Next to the person is another person in a blue shirt. Behind these two people is a third person."

Example with penalty 1.15: "The scene shows a person in a red shirt, beside someone wearing blue. A third individual stands behind them." → More varied phrasing

Recommended starting configuration:

Model Options:

• Architecture size: Medium (7-13B) or small (1-3B) for testing
• Training mode: LoRA Training
• Quantization: NF4
• Precision type: BFloat16

Hyperparameters:

• Epochs: 100 (adjust based on dataset size)
• Learning rate: 0.0001 (for LoRA Training)
• Batch size: 4 (adjust for your GPU)
• Gradient accumulation: 1
• Optimizer: AdamW

Evaluation:

• Max new tokens: 512
• Top K: 50
• Top P: 0.95
• Temperature: 1.0
• Repetition penalty: 1.05

Why these defaults:

• Balanced across speed, quality, and resource usage
• Work well for most VLM tasks
• Safe starting point for experimentation
• Proven effective in production

Systematic tuning approach:

1. Start with defaults (above configuration)
2. Train initial model and evaluate performance
3. Identify issues from training behavior:
   • Slow convergence → adjust learning rate
   • Memory errors → reduce batch size or enable quantization
   • Poor quality → try larger model
   • Overfitting → reduce epochs or add regularization
4. Change one thing at a time for clear cause-effect
5. Compare results across runs using evaluation metrics
6. Refine iteratively until satisfactory performance

Example iteration:

Run 1 (defaults):

• Result: Good quality but slow convergence

Run 2 (adjust learning rate):

• Learning rate: 0.0001 → 0.0002
• Result: Faster convergence, similar quality

Run 3 (adjust batch size):

• Batch size: 4 → 8
• Result: Stable training, faster per-epoch time

Run 4 (final tuning):

• Epochs: 100 → 75 (converged earlier)
• Result: Optimal configuration found

For limited GPU memory (8-16GB):

Architecture: Small (1-3B)
Training mode: LoRA Training
Quantization: NF4
Precision: BFloat16
Batch size: 1-2
Gradient accumulation: 4-8

For standard GPU (24-32GB):

Architecture: Medium (7-13B)
Training mode: LoRA Training
Quantization: NF4
Precision: BFloat16
Batch size: 4-8
Gradient accumulation: 1-2

For high-end GPU (40-80GB):

Architecture: Medium to Large
Training mode: LoRA Training or Full Finetuning (SFT)
Quantization: NF4 or FP4
Precision: BFloat16
Batch size: 8-16
Gradient accumulation: 1

Watch for these patterns:

Loss curves:

• Smooth decrease: Good configuration ✓
• Erratic spikes: Learning rate too high
• Plateau early: Learning rate too low or model capacity insufficient
• Overfitting: Training loss decreases but validation increases

Memory usage:

• Consistent 70-80% GPU utilization: Optimal ✓
• Frequent OOM errors: Reduce batch size or enable quantization
• Low utilization (<50%): Can increase batch size

Training speed:

• Consistent step times: Good ✓
• Increasing step times: Memory/swap issues
• Very slow: Consider larger batch size or better GPU

Learn more about monitoring runs →

Track successful configurations:

Keep records of:

• Which settings worked for which tasks
• Resource requirements (GPU memory, training time)
• Performance metrics achieved
• Issues encountered and solutions

Naming convention for workflows:

[Task]-[Model Size]-[Key Settings]-v[Number]

Examples:
- "DefectDetection-7B-LoRA-NF4-v1"
- "ProductRecog-13B-LoRA-FastConv-v2"
- "QualityControl-3B-FullFT-HighAcc-v1"

Benefits:

• Quickly identify successful configurations
• Share settings with team members
• Reproduce results reliably
• Track iteration progress

Most critical settings for quality:

1. Model architecture size (35% impact)

   • Larger models generally achieve higher quality
   • Most important factor for capacity

2. Learning rate (25% impact)

   • Correct learning rate ensures convergence
   • Too high/low severely impacts results

3. Epochs (20% impact)

   • Sufficient epochs required to converge
   • More isn't always better (overfitting risk)

4. Dataset quality and size (not a setting, but 40% impact)

   • High-quality annotations critical
   • More data generally helps

Less critical for quality (but affect speed/resources):

• Training mode (LoRA Training vs Full Finetuning): Similar quality
• Quantization: Minimal quality impact
• Batch size: Affects stability more than final quality
• Evaluation settings: Only affect inference, not training

Speed optimization strategies:

1. Reduce model size:

• Large (20-34B) → Medium (7-13B): 2-3x faster
• Medium (7-13B) → Small (1-3B): 2-4x faster

2. Enable optimizations:

• Use LoRA Training instead of Full Finetuning (SFT): 2-3x faster
• Use NF4 quantization: 1.5-2x faster
• Use BFloat16 precision: 2x faster vs Float32

3. Increase batch size:

• Batch 2 → Batch 4: ~1.5x faster
• Batch 4 → Batch 8: ~1.3x faster
• Limited by GPU memory

4. Reduce epochs:

• Monitor for early convergence
• Stop when validation metrics plateau
• May be training longer than needed

5. Reduce gradient accumulation:

• Only if memory allows
• Accumulation adds overhead

Typical speedup example:

• Before: Large model, Full Finetuning (SFT), batch 2 → 10 hours
• After: Medium model, LoRA Training, NF4, batch 4 → 2-3 hours
• ~3-4x total speedup with minimal quality impact

Indicators of good configuration:

Training behavior:

• Loss decreases smoothly
• No frequent spikes or NaN values
• Reasonable training speed
• GPU utilization 70-90%
• No out-of-memory errors

Results quality:

• Validation metrics improve over training
• Test set performance meets requirements
• Generated outputs are coherent and accurate
• Model generalizes to new examples

Resource usage:

• Training completes in acceptable time
• GPU memory usage stable
• Costs within budget

Comparison approach:

1. Train with default settings (baseline)
2. Evaluate quality metrics
3. Adjust one setting at a time
4. Compare metrics to baseline
5. Keep changes that improve results

Learn about model evaluation →

Start similar, then customize:

Reusable baseline:

• Core settings (LoRA Training, NF4, AdamW) work across projects
• Training approach generalizes well
• Resource optimizations apply universally

Project-specific tuning:

• Epochs: Depends on dataset size
• Learning rate: May need adjustment per task
• Batch size: Constrained by your GPU
• Model size: Depends on task complexity

Best practice:

1. Create a "baseline" workflow with proven settings
2. Clone it for new projects
3. Adjust task-specific parameters:
   • System prompt
   • Dataset
   • Epochs based on data size
4. Fine-tune if baseline doesn't perform well

Example:

Baseline workflow:

• 7B model, LoRA Training, NF4, BFloat16
• Batch 4, AdamW, LR 0.0001
• Works for most detection tasks

Project A (100 images):

• Clone baseline
• Adjust: Epochs 150 (small dataset)

Project B (1000 images):

• Clone baseline
• Adjust: Epochs 75 (large dataset)
• Adjust: Batch 8 (more memory available)

Use Full Finetuning (SFT) when:

1. Domain is drastically different:

   • Medical/microscopy images (if base model trained on natural images)
   • Satellite/aerial imagery
   • Specialized visual domains

2. Maximum quality is critical:

   • Production systems with strict accuracy requirements
   • Research requiring state-of-the-art results
   • When LoRA Training results are insufficient

3. Resources are abundant:

   • Access to high-end GPUs (A100, H100)
   • Training time is not a constraint
   • Budget allows for higher compute costs

Stick with LoRA Training when:

1. Standard computer vision tasks:

   • Object detection
   • Classification
   • Quality control
   • Most production applications

2. Limited resources:

   • Consumer GPUs
   • Time constraints
   • Budget limitations

3. Iterating quickly:

   • Development phase
   • Prototyping
   • A/B testing different approaches

Reality: 90%+ of use cases work excellently with LoRA Training. Try LoRA Training first, only switch to Full Finetuning (SFT) if results are insufficient and you have the resources.

Solutions ranked by effectiveness:

1. Increase repetition penalty (first try):

• Current: 1.05 → Try: 1.1 or 1.15
• Directly addresses repetition
• Usually most effective solution

2. Adjust temperature:

• Current: 1.0 → Try: 1.2
• Increases output diversity
• More creative word choices

3. Increase Top P:

• Current: 0.90 → Try: 0.95
• Allows more token variety
• Broader vocabulary usage

4. Increase Top K:

• Current: 50 → Try: 70-100
• Widens sampling pool
• More diverse token selection

5. Training-level solutions (if inference changes don't help):

• Increase dataset diversity
• Add more varied training examples
• Adjust system prompt to encourage variety

Typical fix:

Before:
- Temperature: 1.0
- Repetition penalty: 1.05
- Result: "The car is red. The car is large. The car is parked."

After:
- Temperature: 1.1
- Repetition penalty: 1.15
- Result: "The vehicle is red and large, parked near the building."

Try adjustments incrementally—don't change all at once.

Both affect effective batch size, but differently:

Batch size:

• Number of images processed simultaneously
• Limited by GPU memory
• Higher = faster training (better GPU utilization)
• Each batch computes gradients, model updates immediately

Gradient accumulation:

• Number of batches before updating model
• Not limited by GPU memory
• Higher = simulates larger batches without extra memory
• Gradients accumulated across batches, then single update

Example comparison:

Configuration A:

• Batch size: 8
• Gradient accumulation: 1
• Effective batch: 8
• Updates per epoch: dataset_size / 8
• Speed: Fast (8 images at once)
• Memory: High (8 images in GPU)

Configuration B:

• Batch size: 2
• Gradient accumulation: 4
• Effective batch: 8 (same as A)
• Updates per epoch: dataset_size / 8 (same as A)
• Speed: Slower (only 2 images at once, but 4x more forward passes)
• Memory: Low (only 2 images in GPU)

When to use which:

Prefer higher batch size (less accumulation):

• When GPU memory allows
• For faster training
• Simpler configuration

Use gradient accumulation:

• When GPU memory is limited
• To simulate larger batches than GPU allows
• To improve training stability without upgrading GPU

Best practice: Use largest batch size that fits in memory, then add accumulation if you need larger effective batch sizes.