Fast Embeddings on GPUs
Perplexity, Friday, September 4th, 2026
Perplexity engineering details the GPU serving infrastructure behind its embedding and ranking models for AI-native search.
Perplexity's search quality depends on embedding and ranking models that it trains and serves itself, including pplx-embed.
This engineering post opens up the inference infrastructure that serves those models across the company's exabyte-scale search index.
It distinguishes batch embedding, which is compute-bound and optimized for throughput during indexing, from online embedding, where a short query must be embedded with minimum latency.
Perplexity shares the common components it reuses between embedding serving and LLM inference, and the techniques that let it push the quality-versus-cost frontier for search.