Human-in-the-Loop Editing and Fact-Checking

Lesson 3 named the practices E-E-A-T requires, a real byline, real citations, a real subject-matter review. This lesson turns that into an actual editorial pipeline: a sequence where AI produces a draft, and a human does specific, well-defined work on top of it before anything gets published. The goal isn’t “have a human skim it,” it’s a workflow with a clear job at each stage, and a small WordPress ability that helps a human fact-checker by surfacing the claims in a draft that actually need checking, rather than leaving them to reread the whole thing and hope they notice.

Step 1: The five stages, and who’s actually responsible for each

The critical separation is between stage 3 and stage 4. “A human read this and it sounds fine” is not fact-checking, it’s a fluency check, and fluent, wrong content is exactly what an unreviewed AI draft can produce. Fact-checking means confirming specific claims against sources outside the draft itself.

Step 2: Why claim extraction earns its place as a separate ability

A human fact-checker reading a full draft for the first time has to do two things at once: read for meaning, and simultaneously notice every specific, checkable assertion buried in normal prose. That’s a lot to hold in attention at once, and specific numbers, dates, and attributions are easy to skim past when they’re embedded in otherwise-fluent sentences. Pulling every checkable claim into its own list first turns fact-checking into a task with a visible, completable checklist instead of an open-ended “read carefully” instruction.

Step 3: Build the claim-extraction ability

The ability takes a draft’s post ID, reads its content, and asks a model to extract factual claims worth checking, specifically excluding opinion and stylistic statements that have nothing to verify.

// File: content-review/content-review.php
add_action( 'wp_abilities_api_init', function () {
	wp_register_ability(
		'content-review/extract-claims',
		array(
			'label'               => __( 'Extract checkable claims from a draft', 'content-review' ),
			'description'         => __( 'Reads a draft post and returns a list of specific, checkable factual claims (statistics, dates, names, attributions) for a human fact-checker to verify. Does not verify anything itself.', 'content-review' ),
			'category'            => 'content',
			'input_schema'        => array(
				'type'       => 'object',
				'properties' => array(
					'post_id' => array( 'type' => 'integer' ),
				),
				'required'   => array( 'post_id' ),
			),
			'output_schema'       => array(
				'type'       => 'object',
				'properties' => array(
					'claims' => array(
						'type'  => 'array',
						'items' => array(
							'type'       => 'object',
							'properties' => array(
								'claim'   => array( 'type' => 'string' ),
								'context' => array( 'type' => 'string' ),
							),
						),
					),
				),
			),
			'permission_callback' => function ( $input ) {
				return current_user_can( 'edit_post', $input['post_id'] );
			},
			'execute_callback'    => 'content_review_extract_claims',
			'meta' => array(
				'annotations' => array( 'readonly' => true ),
				'mcp'         => array( 'public' => true ),
			),
		)
	);
} );

function content_review_extract_claims( $input ) {
	$post = get_post( $input['post_id'] );
	if ( ! $post ) {
		return new WP_Error( 'not_found', 'No post found with that ID.' );
	}

	$body = wp_strip_all_tags( $post->post_content );

	$prompt = <<<PROMPT
Read this draft and list every specific, checkable factual claim it makes: statistics,
dates, named studies, attributed quotes, named people or organizations, and specific
numeric assertions. Exclude opinions, stylistic statements, and generic advice that has
nothing concrete to verify.

For each claim, return the claim itself and a short snippet of surrounding context so a
human fact-checker can find it in the draft.

Draft:
{$body}
PROMPT;

	$response = wp_ai_client_prompt( $prompt )
		->using_model_preference( 'anthropic/claude', 'openai/gpt', 'google/gemini' )
		->generate_text();

	if ( empty( $response ) ) {
		return new WP_Error( 'empty_response', 'The AI provider returned no claims.' );
	}

	return array( 'claims' => content_review_parse_claims( $response ) );
}

function content_review_parse_claims( $response ) {
	$lines  = array_filter( array_map( 'trim', explode( "\n", $response ) ) );
	$claims = array();

	foreach ( $lines as $line ) {
		if ( preg_match( '/^[-*\d.]+\s*(.+)/', $line, $m ) ) {
			$claims[] = array(
				'claim'   => $m[1],
				'context' => $m[1],
			);
		}
	}

	return $claims;
}

This ability never touches the post. It reads content and returns a list, which is why meta.annotations.readonly is true, the same pattern the SEO Content Optimizer used in Course 6. The human fact-checker takes that list and does the actual verification against real sources, the ability’s job ends at surfacing what needs checking.

Step 4: Wiring it into the workflow

Recap

A human-in-the-loop workflow needs a clear job at each stage, not just “a human looked at it.” AI produces the draft, a claim-extraction ability surfaces every specific factual assertion worth checking, a human verifies those claims against real sources, a qualified person adds or confirms genuine expertise and experience, and only then does a named byline go on the piece. The content-review/extract-claims ability built here stays strictly read-only. It supports the fact-checking stage, it never replaces the human judgment that stage actually requires.

Resources & further reading

• wp_register_ability() function reference, developer.wordpress.org
• PHP AI Client SDK repository, GitHub
• Project: Automated Blog Post Generator, this track, Course 6
• Creating helpful, reliable, people-first content, Google Search Central