Fine-Tuning Assistant

The Fine-Tuning Assistant skill guides you through the process of adapting pre-trained models to your specific use case. Fine-tuning can dramatically improve model performance on specialized tasks, teach models your preferred style, and add capabilities that prompting alone cannot achieve.

This skill covers when to fine-tune versus prompt engineer, preparing training data, selecting base models, configuring training parameters, evaluating results, and deploying fine-tuned models. It applies modern techniques including LoRA, QLoRA, and instruction tuning to make fine-tuning practical and cost-effective.

Whether you are fine-tuning GPT models via API, running local training with open-source models, or using platforms like Hugging Face, this skill ensures you approach fine-tuning strategically and effectively.

Core Workflows

Workflow 1: Decide Whether to Fine-Tune

• Assess the problem:

• Can prompting achieve the goal?

• Is the task format or style consistent?

• Do you have quality training data?

• Is this worth the investment?

• Compare approaches:

Approach When to Use Investment

Better prompts First attempt, variable tasks Low

Few-shot examples Consistent format, limited data Low

RAG Knowledge-intensive, dynamic data Medium

Fine-tuning Consistent style, specialized task High

• Evaluate requirements:

• Minimum 100-1000 quality examples

• Clear evaluation criteria

• Budget for training and hosting

• Decision: Fine-tune only if prompting/RAG insufficient

Workflow 2: Prepare Fine-Tuning Dataset

• Collect training examples:

• Representative of target use case

• High quality (no errors in outputs)

• Diverse coverage of task variations

• Format for training: {"messages": [ {"role": "system", "content": "You are a helpful assistant..."}, {"role": "user", "content": "User input here"}, {"role": "assistant", "content": "Ideal response here"} ]}

• Quality assurance:

• Review sample of examples manually

• Check for consistency in style/format

• Remove duplicates and low-quality entries

• Split train/validation/test sets

• Validate dataset format

Workflow 3: Execute Fine-Tuning

• Select base model:

• Consider size vs capability tradeoff

• Match model to task complexity

• Check licensing for your use case

• Configure training:

OpenAI fine-tuning

training_config = { "model": "gpt-4o-mini-2024-07-18", "training_file": "file-xxx", "hyperparameters": { "n_epochs": 3, "batch_size": "auto", "learning_rate_multiplier": "auto" } }

LoRA fine-tuning (local)

lora_config = { "r": 16, # Rank "lora_alpha": 32, "lora_dropout": 0.05, "target_modules": ["q_proj", "v_proj"] }

• Monitor training:

• Watch loss curves

• Check for overfitting

• Validate on held-out set

• Evaluate results:

• Compare to baseline model

• Test on diverse inputs

• Check for regressions

Quick Reference

Action Command/Trigger

Decide approach "Should I fine-tune for [task]"

Prepare data "Format data for fine-tuning"

Choose model "Which model to fine-tune for [task]"

Configure training "Fine-tuning parameters for [goal]"

Evaluate results "Evaluate fine-tuned model"

Debug training "Fine-tuning loss not decreasing"

Best Practices

Start with Prompting: Fine-tuning is expensive; exhaust cheaper options first

• Can better prompts achieve 80% of the goal?

• Try few-shot examples in the prompt

• Consider RAG for knowledge tasks

Quality Over Quantity: 100 excellent examples beat 10,000 mediocre ones

• Each example should be a gold standard

• Better to have humans verify examples

• Remove anything you wouldn't want the model to learn

Match Format to Use Case: Training examples should mirror real usage

• Same prompt structure as production

• Realistic input variations

• Cover edge cases explicitly

Don't Over-Train: More epochs isn't always better

• Watch validation loss for overfitting

• Start with 1-3 epochs

• Early stopping when validation plateaus

Evaluate Properly: Training loss isn't the goal

• Use held-out test set

• Compare to baseline on same tests

• Check for capability regressions

• Test on edge cases explicitly

Version Everything: Fine-tuning is iterative

• Version your training data

• Track experiment configurations

• Document what worked and what didn't

Advanced Techniques

LoRA (Low-Rank Adaptation)

Efficient fine-tuning for large models:

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig( r=16, # Rank of update matrices lora_alpha=32, # Scaling factor target_modules=["q_proj", "v_proj", "k_proj", "o_proj"], lora_dropout=0.05, bias="none", task_type="CAUSAL_LM" )

Apply LoRA to base model

model = get_peft_model(base_model, lora_config)

Only ~0.1% of parameters are trainable

trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)

QLoRA (Quantized LoRA)

Fine-tune large models on consumer hardware:

from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True )

Load model in 4-bit

model = AutoModelForCausalLM.from_pretrained( "meta-llama/Llama-2-7b-hf", quantization_config=bnb_config )

Apply LoRA on top

model = get_peft_model(model, lora_config)

Instruction Tuning Dataset Creation

Convert raw data to instruction format:

def create_instruction_example(raw_data): return { "messages": [ { "role": "system", "content": "You are a customer service agent for TechCorp..." }, { "role": "user", "content": f"Customer inquiry: {raw_data['inquiry']}" }, { "role": "assistant", "content": raw_data['ideal_response'] } ] }

Apply to dataset

instruction_dataset = [create_instruction_example(d) for d in raw_dataset]

Evaluation Framework

Comprehensive assessment of fine-tuned models:

def evaluate_fine_tuned_model(model, test_set, baseline_model=None): results = { "task_accuracy": [], "format_compliance": [], "style_match": [], "regression_check": [] }

for example in test_set:
    output = model.generate(example.input)

    # Task-specific accuracy
    results["task_accuracy"].append(
        check_correctness(output, example.expected)
    )

    # Format compliance
    results["format_compliance"].append(
        matches_expected_format(output)
    )

    # Style matching (for style transfer tasks)
    results["style_match"].append(
        style_similarity(output, example.expected)
    )

    # Regression on general capabilities
    if baseline_model:
        results["regression_check"].append(
            compare_general_capability(model, baseline_model, example)
        )

return {k: np.mean(v) for k, v in results.items()}

Curriculum Learning

Order training data by difficulty:

def create_curriculum(dataset): # Score examples by complexity scored = [(score_complexity(ex), ex) for ex in dataset] scored.sort(key=lambda x: x[0])

# Create epochs with increasing difficulty
n = len(scored)
curriculum = {
    "epoch_1": [ex for _, ex in scored[:n//3]],           # Easy
    "epoch_2": [ex for _, ex in scored[:2*n//3]],         # Easy + Medium
    "epoch_3": [ex for _, ex in scored],                   # All
}
return curriculum

Common Pitfalls to Avoid

• Fine-tuning when better prompting would suffice

• Using low-quality or inconsistent training examples

• Not holding out a proper test set

• Training for too many epochs (overfitting)

• Ignoring capability regressions from fine-tuning

• Not versioning training data and configurations

• Expecting fine-tuning to add factual knowledge (use RAG instead)

• Fine-tuning on data that doesn't match production use