Jaccard Similarity for AI Agent Lesson Retrieval

Why Jaccard Similarity Beats Keyword Matching for Lesson Retrieval

When building agentic systems, you need your AI to pull the right lessons from a knowledge base to inform its next action. Keyword matching fails because it ignores semantic relationships. Embedding similarity works but adds latency and cost.

Jaccard Similarity—the ratio of intersection to union of two sets—bridges this gap. It's set-based, so it naturally captures what lessons and contexts have in common, requires zero ML infrastructure, and runs in milliseconds on Supabase with a simple SQL query.

The Math: How Jaccard Works for Your Agent

Jaccard(A, B) = |A ∩ B| / |A ∪ B|. For lesson retrieval, convert your agent's current context into a set of tokens or concepts, then compare it against stored lesson prerequisite sets.

A score of 1.0 means perfect overlap. A score of 0 means no common ground. In practice, lessons scoring 0.3–0.7 are often your signal that an agent should retrieve and apply them. You control the threshold based on your domain's specificity.

Building the Retrieval Pipeline in Next.js and Supabase

Start by storing lessons in Supabase with a `lesson_concepts` column (array of strings: tags, skills, or domain tokens). When your Claude-powered agent hits a decision point, send its current state as a set of relevant concepts.

Query Supabase to compute Jaccard scores server-side, then rank and return the top N lessons. The agent can then use those lessons as context in its next Claude call to make a more informed decision.

-- Supabase SQL: compute Jaccard similarity
SELECT 
  id, title,
  (array_length(array_intersect(lesson_concepts, $1::text[]), 1)::float / 
   array_length(array_union(lesson_concepts, $1::text[]), 1)::float) as jaccard_score
FROM lessons
WHERE array_length(array_intersect(lesson_concepts, $1::text[]), 1) > 0
ORDER BY jaccard_score DESC
LIMIT 5;

Tuning Concept Sets for Your Domain

The quality of your retrieval depends on how you define concepts. For a coding tutor, use function names, error types, and algorithms. For customer support, use intent categories and issue patterns.

Start coarse, then refine based on what lessons your agent actually needs. A lesson with concepts ['array-iteration', 'performance', 'javascript'] will only match agents working in that space—which is exactly what you want.

When to Combine Jaccard with Embeddings

Jaccard alone excels when your lesson concepts are well-defined and your agent's context is structured. If you're working with free-form text queries or need fuzzy matching, layer a two-stage retrieval: use Jaccard for fast pre-filtering, then re-rank with embeddings.

This hybrid approach keeps latency low while improving recall. For most indie founders, Jaccard alone is the right starting point—simpler, cheaper, and easier to debug.

Open-Source Implementation: Pantheon

The Pantheon repository (github.com/lewisallena17/pantheon) provides a production-ready reference implementation of Jaccard-based lesson retrieval for Claude agents. It includes Next.js API routes, Supabase schema and queries, and a lesson storage pattern that scales.

Fork it, adapt the concept taxonomy to your domain, and integrate it into your agent loop. It's built specifically for indie developers who want proven patterns without the overhead of a full LMS framework.

Open-source implementation

Everything in this article runs in pantheon — a production-ready Next.js + Supabase + Claude starter. Clone it, deploy to Vercel, run PM2. The dashboard auto-commits every agent edit and reverts itself if TypeScript breaks.