AI Engineering Blog

Cole Hoffer

Making AI do the thing.

Practical notes on building production RAG systems, LLM evaluations, and retrieval optimization.

Read ArticlesBrowse Guides

Evaluating LLM Chat Agents with Real World Signals

How human follow-up behavior reveals response quality, and why specific instruction adherence outperforms vague relevance scoring.

FEB 19, 2026evals
→

Mining Reranker Training Data from RAG Citations

Using citation behavior in production RAG systems to generate labeled training data for domain-specific reranking models.

FEB 7, 2026rag
→

Relevancy Filtering in Retrieval Pipelines

Using LLM-based classification as a second pass to filter retrieval candidates when similarity thresholds fail to generalize.

JAN 24, 2026rag
→

About me

Staff AI Engineer at Reforge.

My boss told me the path to immortality is to write a niche blog that I'm too socially awkward to advertise. So here we are, a collection of notes on lessons learned from yelling at LLMs, professionally.

Cole Hoffer·TwitterLinkedIn

Latest

5 items

Reciprocal Rank Fusion for Hybrid Search

A comprehensive guide to using Reciprocal Rank Fusion (RRF) to combine BM25 and semantic search results for production RAG systems.

FEB 14, 2026Guide
→

Applying HyDE to Domain-Specific Search

Using Hypothetical Document Embeddings to bridge vocabulary gaps between user queries and specialized document corpora.

JAN 16, 2026rag
→

Structured Pre-Filtering for RAG Systems

Why decomposing queries into structured filters before semantic search improves retrieval precision and performance.

JAN 9, 2026rag
→

BM25 Search in LLM Systems

A comprehensive, interactive guide to understanding the BM25 ranking algorithm for AI engineers building RAG systems and search applications.

DEC 22, 2025Guide
→

Structured Logprobs for Better Classification

Combining chain-of-thought reasoning with logprob extraction improves LLM classification accuracy while giving you real confidence scores.

DEC 18, 2025classification
→

Stay in the loop

Get notified when I publish new articles on RAG, search, and AI systems.

 

Cole Hoffer·TwitterLinkedIn