Answering With Real URLs

Course 10’s grounding lesson returned a list of source post IDs alongside an answer, good enough to show “based on: Refund Policy, Shipping FAQ” in a UI. It’s not good enough for a visitor who wants to click through and read the actual policy, and it leaves an unchecked gap: nothing in that lesson’s code stops a model from citing a page it never actually saw. This lesson closes both problems by putting real permalinks directly in the prompt and validating every citation the model returns against the sources it was actually given.

Step 1: attach a real permalink to every retrieved chunk

Course 10’s search function already returns each result’s post_id. get_permalink() turns that into a real, clickable URL, no different for a product or a custom post type than for a blog post:

// File: my-plugin/class-rag-assistant.php
function my_plugin_attach_urls( array $retrieved_chunks ): array {
	foreach ( $retrieved_chunks as &$chunk ) {
		$chunk['url'] = get_permalink( $chunk['post_id'] );
	}
	return $retrieved_chunks;
}

Step 2: build a prompt that carries the URL, not just the text

Each source block in the context now includes its own URL, so the model has something concrete to cite rather than a bare number it has to invent a link around:

// File: my-plugin/class-rag-assistant.php
function my_plugin_build_cited_prompt( string $question, array $retrieved_chunks ): string {
	$context = '';

	foreach ( $retrieved_chunks as $i => $chunk ) {
		$context .= "[Source " . ( $i + 1 ) . "]\n" .
			"URL: {$chunk['url']}\n" .
			"Content: {$chunk['chunk_text']}\n\n";
	}

	return <<<PROMPT
You are a support assistant answering questions using only the sources below. Each
source has a URL. Respond with a JSON object with two fields: "answer", your response as
plain text, and "citations", an array of the exact URLs (copied character-for-character
from the sources below) that your answer actually relies on. If none of the sources
answer the question, say so in "answer" and return an empty "citations" array. Never
invent a URL that isn't listed below.

Sources:
{$context}
Question: {$question}

Respond with JSON only.
PROMPT;
}

Asking for exact, copied URLs instead of “cite your sources” in prose gives Step 3 something precise to check. A model can still get this wrong, which is exactly why validation is a separate step and not just a stronger-worded instruction.

Step 3: validate every citation against what was actually retrieved

This is the step Course 10 didn’t have, because Course 10 never gave the model a URL to invent in the first place. Now that it can, check every returned citation against the real list before it reaches a visitor:

// File: my-plugin/class-rag-assistant.php
function my_plugin_answer_with_verified_citations( string $question ): array {
	$retrieved = my_plugin_attach_urls( my_plugin_semantic_search( $question, 5 ) );
	$prompt    = my_plugin_build_cited_prompt( $question, $retrieved );

	$result = wp_ai_client_prompt( $prompt )->as_json_response()->generate_text();

	if ( is_wp_error( $result ) ) {
		return array( 'answer' => 'Something went wrong answering that.', 'citations' => array() );
	}

	$parsed = json_decode( $result, true );
	$answer = is_array( $parsed ) && isset( $parsed['answer'] ) ? $parsed['answer'] : $result;
	$raw_citations = is_array( $parsed ) && isset( $parsed['citations'] ) ? (array) $parsed['citations'] : array();

	$valid_urls = wp_list_pluck( $retrieved, 'url' );
	$verified   = array_values( array_intersect( $raw_citations, $valid_urls ) );

	return array(
		'answer'    => $answer,
		'citations' => $verified,
	);
}

array_intersect() against $valid_urls, the real list of URLs actually handed to the model in Step 2, is the entire validation. Anything the model returns that doesn’t exactly match one of those URLs is silently dropped, not shown to the visitor and not trusted.

Step 4: rendering verified citations in the widget

// File: assets/js/chat-widget.js
function renderCitations( citations ) {
	if ( ! citations || ! citations.length ) {
		return '';
	}

	var links = citations.map( function ( url, i ) {
		return '<a href="' + url + '" target="_blank" rel="noopener">Source ' + ( i + 1 ) + '</a>';
	} );

	return '<div class="site-chat-citations">' + links.join( ' &middot; ' ) + '</div>';
}

Every link rendered here came from get_permalink() on the server side, filtered through array_intersect() against real retrieved sources, so a visitor clicking “Source 1” lands on an actual page on the actual site, not a broken or fabricated address.

Test it: force a citation mismatch and confirm it gets stripped

Ask a question your knowledge base genuinely doesn’t cover, then check the response directly:

wp-env run cli wp eval '
$r = my_plugin_answer_with_verified_citations( "Do you offer same-day drone delivery?" );
echo $r["answer"] . "\n";
print_r( $r["citations"] );
'

The citations array should come back empty here, since nothing retrieved actually covers drone delivery. Then ask a question you know is well covered by an indexed page or product, and confirm the returned citations are real URLs that load and match the content the answer describes, not just plausible-looking links.

Recap

Giving the model a real permalink for every retrieved chunk, and asking it to cite by exact URL instead of a source number, turns Course 10’s plain source-ID list into clickable proof a visitor can actually verify. The step that makes this trustworthy isn’t the prompt wording, it’s validating every returned citation against the real list of URLs handed to the model with array_intersect(), and dropping anything that doesn’t match exactly. A citation that survives that check is guaranteed to point somewhere real.

Resources & further reading

• Grounding an AI Assistant’s Answers in Retrieved Content, Course 10
• get_permalink() function reference, developer.wordpress.org
• PHP AI Client SDK repository, GitHub