Fine-Tuning Open-Weight Models

Lesson 2 produced a clean JSONL file of real training examples. This lesson turns that file into an actual fine-tuned model, and it happens entirely outside WordPress and PHP. There is no WordPress-specific fine-tuning tool, no plugin that trains a model for you, and this lesson doesn’t invent one. What follows is real, standard, widely-used machine learning practice: Hugging Face’s transformers library to load and train an open-weight model, and peft for LoRA, a technique that fine-tunes a small fraction of a model’s parameters instead of all of them, making the whole process feasible on a single GPU instead of a data center.

Step 1: Install the real tooling

# File: terminal, on a GPU-equipped machine, not your WordPress host
pip install transformers peft datasets accelerate bitsandbytes

transformers loads and trains the model, peft provides LoRA, datasets reads Lesson 2’s JSONL file in the shape training tooling expects, accelerate handles device placement, and bitsandbytes enables 4-bit quantization so a large model fits in less GPU memory during training, a technique commonly called QLoRA when combined with LoRA.

Step 2: Load the base model and tokenizer

# File: finetune.py
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

model_id = "meta-llama/Llama-3.1-8B-Instruct"

quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=quant_config,
    device_map="auto",
)

Choosing an open-weight model matters specifically because fine-tuning requires access to the weights themselves, a closed API-only model like a cloud provider’s hosted GPT or Claude model can’t be fine-tuned this way at all. Llama 3.1 is one real, well-known open-weight option; Mistral and Qwen are other genuine alternatives with the same transformers loading pattern.

Step 3: Load your WordPress export as a dataset

# File: finetune.py
from datasets import load_dataset

dataset = load_dataset("json", data_files="training-data.jsonl", split="train")

def format_example(example):
    text = f"### Instruction:\n{example['prompt']}\n\n### Response:\n{example['completion']}"
    return {"text": text}

dataset = dataset.map(format_example)

This is the file Lesson 2 wrote with my_plugin_export_training_data(). The instruction and response format shown here is a common, simple template, real fine-tuning setups frequently follow the specific chat template a given base model was itself trained with, which tokenizer.apply_chat_template() can produce directly from the same prompt and completion fields.

Step 4: Configure LoRA with PEFT

Full fine-tuning updates every parameter in the model, billions of them, and requires enormous GPU memory. LoRA instead freezes the base model and trains a small pair of low-rank matrices injected into specific layers, a small fraction of the parameter count, with results that are surprisingly close to full fine-tuning for adaptation tasks like this one:

# File: finetune.py
from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

print_trainable_parameters() reports the real, concrete payoff here, typically well under 1% of the base model’s total parameters are actually being trained, which is what makes this feasible on a single consumer or rented cloud GPU rather than a training cluster.

Step 5: Run the training loop

# File: finetune.py
from transformers import TrainingArguments, Trainer

training_args = TrainingArguments(
    output_dir="./site-model-adapter",
    per_device_train_batch_size=4,
    gradient_accumulation_steps=4,
    num_train_epochs=3,
    learning_rate=2e-4,
    logging_steps=10,
    save_strategy="epoch",
    bf16=True,
)

def tokenize(example):
    return tokenizer(example["text"], truncation=True, max_length=1024)

tokenized_dataset = dataset.map(tokenize)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset,
)

trainer.train()
model.save_pretrained("./site-model-adapter")

Three epochs is a reasonable starting point for a dataset in the hundreds to low thousands of examples, not a rule. Watch the logged training loss, if it stops decreasing or starts climbing, that’s the training run telling you it’s overfitting your held-out data before Lesson 7 even measures it.

Step 6: Merge the adapter for serving

save_pretrained() above saves only the small LoRA adapter, not a full standalone model. Lesson 4’s serving setup needs a complete model directory, so merge the adapter into the base weights:

# File: merge_adapter.py
from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    torch_dtype="bfloat16",
)
merged_model = PeftModel.from_pretrained(base_model, "./site-model-adapter")
merged_model = merged_model.merge_and_unload()

merged_model.save_pretrained("./site-model-merged")
tokenizer.save_pretrained("./site-model-merged")

./site-model-merged is now a complete, standalone model directory, no different in shape from any Hugging Face model you’d download directly, ready for Lesson 4 to serve.

Test it: generate from the fine-tuned model directly

# File: test_generation.py
from transformers import pipeline

pipe = pipeline("text-generation", model="./site-model-merged", device_map="auto")
output = pipe("### Instruction:\nWrite a product description for a wireless keyboard, a peripheral.\n\n### Response:\n", max_new_tokens=200)

print(output[0]["generated_text"])

You should see output that reads closer to your training examples’ style than the base model’s default output would, that’s the real, informal signal to check before Lesson 7’s rigorous held-out comparison. This isn’t the evaluation itself, just a sanity check that training actually did something.

Recap

Fine-tuning happens entirely in Python, entirely outside WordPress: load an open-weight model with transformers, wrap it with a LoRA adapter using peft so only a small fraction of parameters actually train, run Hugging Face’s Trainer against Lesson 2’s exported data, and merge the adapter into a standalone model directory. The result is a real fine-tuned model sitting on disk, with no connection to WordPress yet, that’s exactly what the next lesson serves back to it.

Resources & further reading

• Hugging Face Transformers documentation, huggingface.co
• Hugging Face PEFT documentation, huggingface.co
• Meta Llama 3.1 model card, huggingface.co