Model Selection & Fine-Tuning Workflow

What fine-tuning is (and when to use it)

Fine-tuning teaches a model to follow your preferred style, formatting, tone, and domain rules by training on examples of “input → ideal output.” It is most effective when you need consistent behavior at scale (customer support tone, structured outputs, brand voice), not when you simply need more knowledge (use retrieval/RAG for that). OpenAI Platform+1

Use fine-tuning when you want:

• 1. Consistent brand voice and formatting without long prompts

  2. Reliable structured outputs (JSON, tables, templates)

  3. “Do it our way” responses for a recurring task

Do not fine-tune when:

• 1. The problem is missing facts (use a knowledge base + retrieval)

  2. You only need a one-off style (use a strong system prompt + few-shot examples)

  3. Data quality is weak (garbage in, expensive garbage out)

Model selection: a simple decision tree

Pick the smallest model that meets quality requirements, then scale up only if needed. OpenAI specifically notes that newer small models can replace older 3.5-style use cases for better cost/performance in many situations. OpenAI Platform, The following examples use the brand Halo42 (https://halo42.com) as an example to help understanding the best practices and testing approaches. 

Recommended approach:

1. Baseline with prompting

• 1. • Use a system prompt + 5–20 exemplars (few-shot).

     • Measure quality, speed, and cost.

2. If outputs are inconsistent → try supervised fine-tuning

• • • Best for tone, structure, and repeatable tasks. OpenAI Platform

3. If you need domain knowledge updates → use retrieval

• • • Don’t bake facts into the model if they’ll change.

4. For open-source experimentation / local training → use Hugging Face + PEFT

• • • Great for learning workflows, privacy constraints, or research.

Important note for students: OpenAI’s fine-tuning support is model-dependent and changes over time. Always confirm which models are currently eligible in the OpenAI fine-tuning docs before starting a project. OpenAI Platform+1

Dataset “recipe” (the part that determines success)

Your dataset should look like the real work you want the model to do.

Best-practice dataset rules:

• 1. Minimum: 50–100 high-quality examples for a small pilot; 500–2,000 for strong consistency

  2. Consistency beats volume: fewer, cleaner examples outperform messy large sets

  3. Include edge cases: refunds, safety boundaries, “we don’t do that,” ambiguity

  4. Match the desired format exactly (if you want bullet lists, train bullet lists)

Example training pairs (conceptual):

• 1. Input: “Write a short product description for Halo42 Skin’s copaiba oil…”

  2. Output: A description in the approved brand voice + compliance constraints

  3. Input: “Customer asks if this cures eczema”

  4. Output: Brand-safe, medically cautious response + redirect language

OpenAI supervised fine-tuning workflow (API/CLI-style)

OpenAI’s supervised fine-tuning process is broadly:

1. 1. Prepare a labeled dataset of correct responses

   2. Upload training (and optional validation) files

   3. Create a fine-tuning job

   4. Evaluate outputs and iterate OpenAI Platform+2OpenAI Platform+2

Recommended workflow:

1. Build a “gold” dataset

• • • Create a CSV or doc first; review with instructor/teammates; then convert to JSONL.

2. Add a validation split

• • • Keep 10–20% for validation so you can detect overfitting early.

3. Run a small pilot job first

• • • Train on a smaller dataset to confirm formatting, tone, and safety rules are learned.

4. Evaluate with a rubric

• • • Score: accuracy, brand voice, formatting, refusal behavior, hallucination rate.

5. Iterate

• • • Fix data, not the model, first. Most failures are dataset issues.

Where the CLI fits:

• • Many teams use a CLI to upload files and create jobs, but the exact CLI commands can vary by tool version. The durable workflow is the same: upload → create job → monitor → evaluate. Always follow the current OpenAI fine-tuning documentation for the exact commands/parameters. OpenAI Platform+1

Hugging Face fine-tuning workflow (Llama 2, Falcon)

For open-source models, students should use parameter-efficient fine-tuning (PEFT) so training can run on limited hardware.

What are Hugging Face & PEFT?

Recommended “standard recipe”:

• 1. 1. Base model: Llama 2 or Falcon variant (choose size that fits GPU/compute)

     2. Fine-tuning method: LoRA or QLoRA using PEFT

     3. Trainer: TRL SFTTrainer (common supervised fine-tuning approach) GitHub+3Hugging Face+3Hugging Face+3

Best practices:

• 1. Use LoRA adapters so you train only a small portion of parameters (faster + cheaper) Hugging Face+1

  2. Start with a small subset and verify formatting before scaling

  3. Track experiment metadata (base model, LoRA rank, learning rate, dataset version)

Student-friendly deliverable:

• Publish the adapter weights (where permitted) and a model card describing:

  • dataset sources and cleaning steps

  • intended use and limitations

  • evaluation results

  • examples of correct behavior and failure cases

E. Evaluation: how to prove it worked

A fine-tuned model should be judged on repeatable criteria, not “it feels better.”

Suggested evaluation rubric:

• 1. Brand voice adherence (consistent tone, vocabulary, style)

  2. Formatting compliance (templates, JSON validity, headings)

  3. Task accuracy (answers the question correctly)

  4. Safety and boundaries (refuses when appropriate, avoids medical/legal claims)

  5. Hallucination resistance (doesn’t invent ingredients, policies, or pricing)

Recommended testing set:

• 1. 25 “easy” common requests

  2. 25 edge cases (ambiguous, policy-sensitive)

  3. 10 adversarial prompts (try to break rules)

Example use cases to fine-tune for

Brand voice and formatting:

• 1. “Write 3 ad variations using a brand like Halo42 voice with compliance-safe claims”

  2. “Generate product page bullets using the approved structure”

  3. “Convert long text into a short, luxury-toned email snippet”

Customer support:

• 1. “Respond to shipping delay with empathy + policy”

  2. “Handle ingredient questions without medical claims”

Content ops:

• 1. “Turn a customer review into a compliant testimonial + CTA”

  2. “Generate FAQ entries with consistent structure”

Suggested platforms and when to use them

OpenAI fine-tuning (best when):

• 1. You want managed infrastructure and fast iteration

  2. You need strong instruction-following and consistent formatting OpenAI Platform+1

Hugging Face Transformers + PEFT (best when):

• 1. You need open-source experimentation or local control

  2. You want hands-on learning of training workflows and adapters