Preparing Training Data From Your Site

Lesson 1 drew the line: fine-tuning is worth it only for a genuine behavior or vocabulary gap, and once you’ve decided that’s your actual problem, the fine-tuning run itself is only as good as the data you feed it. This lesson stays entirely inside WordPress and PHP, since pulling and shaping your own content is the one part of this whole process that’s genuinely a WordPress job. The output is a clean, structured file that Lesson 3’s external Python tooling reads as input, and everything past that point leaves PHP behind.

Step 1: Decide what “a training example” means for your case

Before writing any export code, be specific about what one training row represents, since this shapes the whole export:

Most WordPress use cases, adapting a model to write in your voice for a specific content type, fit the first shape better, since it teaches the model both the trigger and the expected response, not just a style in the abstract.

Step 2: Query real content with WP_Query

Pull the specific slice of content that actually reflects the style or domain you’re adapting toward, not your entire database indiscriminately:

// File: my-plugin/class-training-data-export.php
function my_plugin_get_training_source_posts(): array {
	$query = new WP_Query(
		array(
			'post_type'      => 'product',
			'post_status'    => 'publish',
			'posts_per_page' => -1,
			'date_query'     => array(
				array( 'after' => '2024-01-01' ), // Recent content, closer to current voice.
			),
		)
	);

	return $query->posts;
}

Scoping by post type and a recent date range matters here in a way it didn’t for Course 10’s RAG indexing. RAG benefits from indexing everything, since irrelevant content just scores low at retrieval time. A fine-tuning dataset actively teaches the model whatever you feed it, including old, off-brand, or low-quality writing if you don’t filter it out first.

Step 3: Strip WordPress markup down to real prose

Post content in the database is Gutenberg block comments, shortcodes, and HTML, none of which belongs in training text a language model should learn writing style from:

// File: my-plugin/class-training-data-export.php
function my_plugin_clean_post_content( string $raw_content ): string {
	$content = do_blocks( $raw_content );          // Render blocks to plain HTML first.
	$content = strip_shortcodes( $content );        // Remove any remaining shortcode tags.
	$content = wp_strip_all_tags( $content );       // Strip HTML tags, keep the text.
	$content = html_entity_decode( $content );       // & back to &, etc.
	$content = preg_replace( '/\s+/', ' ', $content ); // Collapse whitespace.

	return trim( $content );
}

This is deliberately simpler than Course 10’s chunking, since fine-tuning examples don’t need to be split into retrieval-sized pieces, most tooling handles a full, reasonably-sized post as a single training example directly.

Step 4: Build prompt/completion pairs from real post fields

Derive the “prompt” side from fields that already exist on the post, so each example reflects something a real user would actually ask for:

// File: my-plugin/class-training-data-export.php
function my_plugin_build_training_example( WP_Post $post ): ?array {
	$completion = my_plugin_clean_post_content( $post->post_content );

	if ( strlen( $completion ) < 200 ) {
		return null; // Too short to be a useful style example.
	}

	$product_name = get_the_title( $post );
	$category     = get_the_terms( $post->ID, 'product_cat' );
	$category_name = ( $category && ! is_wp_error( $category ) ) ? $category[0]->name : 'product';

	return array(
		'prompt'     => "Write a product description for {$product_name}, a {$category_name}.",
		'completion' => $completion,
	);
}

Discarding anything under a minimum length matters, a thin, low-effort post teaches the model that thin, low-effort output is what you want, which is the opposite of the goal.

Step 5: Export the full dataset as JSONL

Fine-tuning tooling, Hugging Face’s datasets library included, near-universally reads JSON Lines, one complete JSON object per line, rather than a single large JSON array:

// File: my-plugin/class-training-data-export.php
function my_plugin_export_training_data( string $file_path ): int {
	$posts   = my_plugin_get_training_source_posts();
	$written = 0;
	$handle  = fopen( $file_path, 'w' );

	foreach ( $posts as $post ) {
		$example = my_plugin_build_training_example( $post );

		if ( null === $example ) {
			continue;
		}

		fwrite( $handle, wp_json_encode( $example ) . "\n" );
		++$written;
	}

	fclose( $handle );

	return $written;
}

Test it: export and inspect the result

wp-env run cli wp eval '
$count = my_plugin_export_training_data( "/tmp/training-data.jsonl" );
echo "Wrote {$count} training examples\n";
'

head -n 2 /tmp/training-data.jsonl

You should see a real count in the hundreds, not a handful, and each line should be a complete, valid JSON object with non-empty prompt and completion fields, not truncated markup or an empty string.

Recap

Training data preparation is the one step in this whole process that’s genuinely a WordPress job: WP_Query or get_posts() pulls the right slice of real content, stripping blocks, shortcodes, and HTML down to clean prose keeps markup out of what the model learns to write, and structuring the result as prompt/completion pairs in JSONL produces a file any standard fine-tuning tool can read directly. Set aside a held-out slice now, before training starts, Lesson 7 needs it later. Everything from here forward happens outside WordPress entirely.

Resources & further reading

• WP_Query class reference, developer.wordpress.org
• get_posts() function reference, developer.wordpress.org
• Hugging Face Datasets documentation, huggingface.co