Grounding an AI Assistant's Answers in Retrieved Content

Retrieval, the last two lessons, finds the right material. This lesson is the other half of RAG: turning those retrieved chunks plus the user’s question into a single prompt that makes the model answer from what it was actually given, and say so plainly when the retrieved content doesn’t cover the question at all.

Step 1: The shape of a grounded prompt

A RAG prompt has three distinct parts, assembled in a fixed order every time:

Step 2: Building the prompt string

// File: my-plugin/class-rag-assistant.php
function my_plugin_build_grounded_prompt( string $question, array $retrieved_chunks ): string {
	$context = '';

	foreach ( $retrieved_chunks as $i => $chunk ) {
		$context .= "[Source " . ( $i + 1 ) . ", post #{$chunk['post_id']}]\n{$chunk['chunk_text']}\n\n";
	}

	return <<<PROMPT
You are a support assistant answering questions using only the context provided below.
If the context does not contain the answer, say plainly that you don't have information
on that topic. Do not use outside knowledge, and do not guess.

Context:
{$context}
Question: {$question}

Answer:
PROMPT;
}

That second sentence in the system instruction, telling the model explicitly what to do when the context doesn’t cover the question, is the single most important line in this whole lesson. Without it, a model asked a question its retrieved chunks don’t actually answer will often fall back to its general training knowledge or invent a plausible answer rather than admitting the gap, quietly reintroducing the exact hallucination problem Lesson 1 described.

Step 3: Calling generate_text() with the assembled prompt

// File: my-plugin/class-rag-assistant.php
function my_plugin_answer_from_knowledge_base( string $question ): array {
	$retrieved = my_plugin_semantic_search( $question, 5 );

	$prompt = my_plugin_build_grounded_prompt( $question, $retrieved );

	$answer = wp_ai_client_prompt( $prompt )->generate_text();

	return array(
		'answer'  => $answer,
		'sources' => array_unique( wp_list_pluck( $retrieved, 'post_id' ) ),
	);
}

This is the whole pipeline meeting in one function: Lesson 6’s retrieval feeds Step 2’s prompt construction, and generate_text(), the same call Course 5 introduced for plain text generation, produces the final answer, only now with real, relevant content behind it instead of nothing.

Step 4: Returning sources, not just an answer

Returning sources, the distinct post IDs behind the retrieved chunks, alongside the answer lets your UI show “based on: [Refund Policy], [Shipping FAQ]” next to the response. This does two real things: it lets a user verify the answer against the original page, and it makes it immediately obvious when the assistant answered from weak or irrelevant material, since an oddly-unrelated source list is a visible signal that retrieval didn’t find anything genuinely relevant.

// File: my-plugin/class-rag-assistant.php
$result = my_plugin_answer_from_knowledge_base( 'How do I request a refund?' );

echo $result['answer'] . "\n\n";
echo "Sources: " . implode( ', ', array_map( 'get_the_title', $result['sources'] ) ) . "\n";

Test it: ask a question your content actually covers, and one it doesn’t

wp-env run cli wp eval '
$r1 = my_plugin_answer_from_knowledge_base( "How do I request a refund?" );
echo "Q1: " . $r1["answer"] . "\n\n";

$r2 = my_plugin_answer_from_knowledge_base( "What is the capital of France?" );
echo "Q2: " . $r2["answer"] . "\n";
'

The first answer should reflect your actual indexed content. The second, assuming your knowledge base has nothing about geography, should be a plain statement that the assistant doesn’t have information on that, not a confident, hallucinated capital city. That second result is the real test of whether grounding is actually working.

Recap

A grounded RAG prompt has three parts in a fixed order: a system instruction that explicitly tells the model to answer only from context and to admit when it can’t, the actual retrieved chunk text from Lesson 6, and the user’s question last. Feed that assembled prompt to the same wp_ai_client_prompt()->generate_text() call from Course 5, and return the source post IDs alongside the answer so users can verify it. This is the complete retrieve-then-generate loop Lesson 1 described, now real and running end to end. The remaining lessons cover keeping it accurate as your content changes.

Resources & further reading

• Generating Text With wp_ai_client_prompt(), Course 5
• PHP AI Client SDK repository, GitHub