Handling Ambiguous or Bad Model Output

Course 5’s error-handling lesson covers what happens when a call fails outright, a WP_Error, a network timeout, a missing key. This lesson covers a different problem: the call succeeds, the model returns something, and that something doesn’t hold up against what your feature actually needs, a missing field, a category that isn’t in your allowed list, a title that’s empty. That’s not a network failure, it’s bad content wearing the shape of a successful response, and it needs its own handling: validate, then retry with a clarification if it’s worth retrying, then fall back safely if it still isn’t right.

Step 1: A schema request lowers bad output, it doesn’t remove it

Lesson 3’s as_json_response( $schema ) call constrains what the model is asked to return. It doesn’t guarantee every response satisfies every constraint on every call, providers vary in how strictly they enforce a schema, and a model can still return valid JSON that’s missing a required field or contains a value outside your intended set. The validation step is where you check for that, in your own PHP, after the decode, every time.

// File: my-plugin/class-bulk-tagger.php
function validate_tag_response( array $data, array $allowed_categories ): bool {
	if ( empty( $data['title'] ) || ! is_string( $data['title'] ) ) {
		return false;
	}
	if ( empty( $data['tags'] ) || ! is_array( $data['tags'] ) ) {
		return false;
	}
	if (
		empty( $data['category'] )
		|| ! in_array( $data['category'], $allowed_categories, true )
	) {
		return false;
	}
	return true;
}

Step 2: Retrying with a clarification, not a repeat

Calling the exact same prompt again after a validation failure rarely fixes anything, you’re re-rolling the same dice, not correcting a mistake. A retry that actually helps tells the model specifically what was wrong with its last answer:

// File: my-plugin/class-bulk-tagger.php
function retry_with_clarification(
	string $post_content,
	array $schema,
	array $data,
	array $allowed_categories
) {
	$problem = empty( $data['category'] ) || ! in_array( $data['category'], $allowed_categories, true )
		? 'Your last response was missing a valid category. category must be exactly ' .
		  'one of: ' . implode( ', ', $allowed_categories ) . '.'
		: 'Your last response was missing a required field.';

	$raw = wp_ai_client_prompt()
		->with_text( "Post content:\n" . $post_content )
		->using_system_instruction(
			'Suggest a concise title, up to five relevant tags, and the single ' .
			'best-fit category. ' . $problem
		)
		->as_json_response( $schema )
		->using_temperature( 0.2 )
		->generate_text();

	return json_decode( $raw, true );
}

Step 3: Capping the retry count

A retry loop with no limit is its own reliability risk, every retry is a full additional API call, with its own cost and its own chance of a different kind of failure. Two attempts total, the original call plus one retry, is a reasonable default for most WordPress content features, more than that rarely improves the odds enough to justify the added cost and latency:

// File: my-plugin/class-bulk-tagger.php
function tag_post_with_retry( int $post_id ): ?array {
	$post_content = wp_strip_all_tags( get_post_field( 'post_content', $post_id ) );
	$categories   = wp_list_pluck( get_categories( array( 'hide_empty' => false ) ), 'name' );
	$schema       = build_tag_schema( $categories ); // Same schema shape as Lesson 3.

	$raw  = wp_ai_client_prompt()
		->with_text( "Post content:\n" . $post_content )
		->using_system_instruction( 'Suggest a concise title, up to five relevant tags, and the single best-fit category.' )
		->as_json_response( $schema )
		->using_temperature( 0.2 )
		->generate_text();

	$data = json_decode( $raw, true );

	if ( ! is_array( $data ) || ! validate_tag_response( $data, $categories ) ) {
		$data = retry_with_clarification( $post_content, $schema, (array) $data, $categories );
	}

	if ( ! is_array( $data ) || ! validate_tag_response( $data, $categories ) ) {
		return null; // Step 4 handles this.
	}

	return $data;
}

Step 4: Falling back safely once retries are exhausted

What happens when tag_post_with_retry() returns null matters as much as the retry logic itself. The safe default is to change nothing and make the failure visible, not to guess at a value and write it anyway:

// File: my-plugin/class-bulk-tagger.php
$result = tag_post_with_retry( $post_id );

if ( null === $result ) {
	update_post_meta( $post_id, '_ai_tagging_needs_review', true );
	error_log( "AI bulk-tagging failed validation twice for post {$post_id}." );
	return; // Leave the post's existing title and terms untouched.
}

wp_update_post( array( 'ID' => $post_id, 'post_title' => sanitize_text_field( $result['title'] ) ) );
wp_set_post_terms( $post_id, $result['tags'], 'post_tag' );
wp_set_post_terms( $post_id, array( $result['category'] ), 'category' );

Flagging the post with _ai_tagging_needs_review gives a human editor a real signal to follow up on, rather than the post silently keeping stale data with no indication anything went wrong.

Test it: force a validation failure

wp-env run cli wp eval '
// Simulate a response missing category to exercise the fallback path.
$fake_response = array( "title" => "Example Post", "tags" => array( "example" ) );
$categories    = array( "Tutorials", "News", "Reviews" );

var_dump( validate_tag_response( $fake_response, $categories ) );
'

You should see bool(false), confirming validation correctly rejects a response missing category before your code ever tries to write it to the database.

Recap

A schema request reduces bad structured output, it doesn’t eliminate it, so validate every decoded response against the fields your feature actually needs before trusting it. When validation fails, one retry with a specific clarification about what was wrong is far more useful than repeating the identical call, and a hard cap on retry attempts keeps a bad response from turning into an unbounded cost. When retries are exhausted, fall back to leaving existing data untouched and flagging the item for review, never to guessing.

Resources & further reading

• PHP AI Client SDK repository, GitHub
• WP_Error class reference, developer.wordpress.org