Local RAG With a Self-Hosted Model

Course 10 built a complete RAG pipeline: embed your content, store the vectors, retrieve the closest chunks for a question, and generate a grounded answer. Every step in that pipeline called AiClient or wp_ai_client_prompt() without caring which provider answered. That’s exactly what makes this lesson possible: Lesson 4’s ollama-local provider is now just another provider those calls can resolve to, which means a complete RAG pipeline, embeddings and generation both, can run entirely on infrastructure you control, with no request ever reaching a cloud AI vendor.

Step 1: Forcing the generation step to your local model

Course 10’s grounding lesson called wp_ai_client_prompt( $prompt )->generate_text() without specifying a model, letting the SDK pick whatever’s configured. For a genuinely local pipeline, be explicit and force it to the model Lesson 4 registered:

// File: my-plugin/class-rag-assistant.php
function my_plugin_answer_from_knowledge_base( string $question ): array {
	$retrieved = my_plugin_semantic_search( $question, 5 );
	$prompt    = my_plugin_build_grounded_prompt( $question, $retrieved );

	$answer = wp_ai_client_prompt( $prompt )
		->using_model_preference( 'llama3.1' )
		->generate_text();

	return array(
		'answer'  => $answer,
		'sources' => array_unique( wp_list_pluck( $retrieved, 'post_id' ) ),
	);
}

Nothing about using_model_preference() changed, this is the exact call Course 5, Lesson 6 taught. What changed is that llama3.1 now resolves through the ollama-local provider Lesson 4 registered, instead of falling through to a cloud provider, because that’s the only registered provider whose model metadata directory lists it.

Step 2: The one real gap, embeddings

Course 10’s indexing lesson used AiClient::input()->generateEmbedding(). Lesson 4’s custom provider, though, only built a text-generation model, because the SDK ships AbstractOpenAiCompatibleTextGenerationModel and AbstractOpenAiCompatibleModelMetadataDirectory, but no equivalent OpenAI-compatible abstract class for embeddings, as of this SDK version. Ollama itself does expose an OpenAI-compatible /v1/embeddings route for embedding models like nomic-embed-text, so the gap is in the SDK’s built-in base classes, not in Ollama. A few more lines close it, implementing the real EmbeddingGenerationModelInterface directly:

// File: my-plugin/includes/ollama-provider/class-ollama-embedding-model.php
namespace MyPlugin\OllamaProvider;

use WordPress\AiClient\Messages\DTO\MessagePart;
use WordPress\AiClient\Providers\ApiBasedImplementation\AbstractApiBasedModel;
use WordPress\AiClient\Providers\Http\DTO\Request;
use WordPress\AiClient\Providers\Http\Enums\HttpMethodEnum;
use WordPress\AiClient\Providers\Models\EmbeddingGeneration\Contracts\EmbeddingGenerationModelInterface;
use WordPress\AiClient\Results\DTO\EmbeddingResult;
use WordPress\AiClient\Results\DTO\TokenUsage;

class Ollama_Embedding_Model extends AbstractApiBasedModel implements EmbeddingGenerationModelInterface {

	public function generateEmbeddingResult( array $inputs ): EmbeddingResult {
		$texts = array_map(
			static fn( MessagePart $part ): string => (string) $part->getText(),
			$inputs
		);

		$request = new Request(
			HttpMethodEnum::POST(),
			Ollama_Provider::url( 'embeddings' ),
			array( 'Content-Type' => 'application/json' ),
			array(
				'model' => $this->metadata()->getId(),
				'input' => $texts,
			)
		);
		$request = $this->getRequestAuthentication()->authenticateRequest( $request );

		$response = $this->getHttpTransporter()->send( $request );
		$data     = $response->getData();

		$embeddings = array_map(
			static fn( array $item ): array => $item['embedding'],
			$data['data']
		);

		return new EmbeddingResult(
			$data['id'] ?? '',
			$embeddings,
			count( $embeddings[0] ?? array() ),
			new TokenUsage( $data['usage']['prompt_tokens'] ?? 0, 0, $data['usage']['total_tokens'] ?? 0 ),
			$this->providerMetadata(),
			$this->metadata()
		);
	}
}

Update Ollama_Provider::createModel() to hand back this class for embedding-capable models, and the metadata directory to mark them as such (a simple naming heuristic, since Ollama’s models listing doesn’t separately flag a model’s capability):

// File: my-plugin/includes/ollama-provider/class-ollama-provider.php - createModel() only
protected static function createModel( ModelMetadata $modelMetadata, ProviderMetadata $providerMetadata ): ModelInterface {
	foreach ( $modelMetadata->getSupportedCapabilities() as $capability ) {
		if ( $capability->isEmbeddingGeneration() ) {
			return new Ollama_Embedding_Model( $modelMetadata, $providerMetadata );
		}
	}

	return new Ollama_Text_Generation_Model( $modelMetadata, $providerMetadata );
}

// File: my-plugin/includes/ollama-provider/class-ollama-model-metadata-directory.php - inside parseResponseToModelMetadataList()
$is_embedding_model = false !== stripos( $model_data['id'], 'embed' );

$models[] = new ModelMetadata(
	$model_data['id'],
	$model_data['id'],
	array( $is_embedding_model ? CapabilityEnum::embeddingGeneration() : CapabilityEnum::textGeneration() ),
	array()
);

Step 3: Pull an embedding model and re-index locally

# File: terminal
ollama pull nomic-embed-text

Then re-run Course 10’s indexing pass, forcing embeddings to the local model the same way Step 1 forced generation:

// File: my-plugin/class-rag-indexer.php - the one line that changes from Course 10
$model     = \WordPress\AiClient\AiClient::defaultRegistry()->getProviderModel( 'ollama-local', 'nomic-embed-text' );
$embedding = \WordPress\AiClient\AiClient::generateEmbedding( $post_content, $model );

Everything downstream, chunking, the custom storage table, and cosine similarity search from Course 10, is unchanged. Only the source of the vectors moved from a cloud provider to your own machine.

Test it: confirm the pipeline stayed local end to end

wp-env run cli wp eval '
$result = my_plugin_answer_from_knowledge_base( "How do I request a refund?" );
echo $result["answer"] . "\n";
echo "Sources: " . implode( ", ", $result["sources"] ) . "\n";
'

To confirm nothing left your network, stop your machine’s internet connection entirely (or block outbound traffic at a firewall level) and re-run the same command. A genuinely local pipeline answers identically; any change in behavior means some step, embeddings, generation, or both, was quietly still reaching out to a cloud provider.

Recap

Course 10’s RAG pipeline needed no structural changes to run entirely locally, only explicit model targeting at its two AI-calling points: using_model_preference() for generation, and a specific ollama-local model instance for embeddings. The embeddings half needed one additional small class, since the SDK’s OpenAI-compatible base classes don’t yet cover that capability the way they cover text generation. Cutting network access and re-testing is the honest way to confirm the whole retrieve-then-generate loop never left your own infrastructure.

Resources & further reading

• RAG Knowledge Base for WordPress, Course 10
• Ollama embedding models blog post, ollama.com
• PHP AI Client SDK repository, GitHub