AI Engineering Blog

Cole Hoffer

Making AI do the thing.

Practical notes on building production RAG systems, LLM evaluations, and retrieval optimization.

Read ArticlesBrowse Guides

Let the Judge Choose, Not Score

Build continuous ratings from pairwise LLM judgments, without comparing every candidate pair on every task.

SEP 24, 2026evals
→

Evaluating LLM Chat Agents with Real World Signals

How human follow-up behavior reveals response quality, and why specific instruction adherence outperforms vague relevance scoring.

FEB 19, 2026evals
→

Mining Reranker Training Data from RAG Citations

Using citation behavior in production RAG systems to generate labeled training data for domain-specific reranking models.

FEB 7, 2026rag
→

About me

Senior Staff AI Engineer at Miro, previously Reforge + Monterey.ai via acquisitions.

My boss told me the path to immortality is to write a niche blog that I'm too socially awkward to advertise. So here we are, a collection of notes on lessons learned from yelling at LLMs, professionally.

Cole Hoffer·TwitterLinkedIn

Latest

6 items

Let the Judge Choose, Not Score

Build continuous ratings from pairwise LLM judgments, without comparing every candidate pair on every task.

SEP 24, 2026evals
→

Evaluating LLM Chat Agents with Real World Signals

How human follow-up behavior reveals response quality, and why specific instruction adherence outperforms vague relevance scoring.

FEB 19, 2026evals
→

Reciprocal Rank Fusion for Hybrid Search

A comprehensive guide to using Reciprocal Rank Fusion (RRF) to combine BM25 and semantic search results for production RAG systems.

FEB 14, 2026Guide
→

Mining Reranker Training Data from RAG Citations

Using citation behavior in production RAG systems to generate labeled training data for domain-specific reranking models.

FEB 7, 2026rag
→

Relevancy Filtering in Retrieval Pipelines

Using LLM-based classification as a second pass to filter retrieval candidates when similarity thresholds fail to generalize.

JAN 24, 2026rag
→

Applying HyDE to Domain-Specific Search

Using Hypothetical Document Embeddings to bridge vocabulary gaps between user queries and specialized document corpora.

JAN 16, 2026rag
→

Stay in the loop

New posts when I have something to say. A radical email strategy.

 

Cole HofferTwitterLinkedIn