When Vector Databases Matter: RAG, Agents and Retrieval Trade-offs

IBM Technology's video with Martin Keen examines whether vector databases remain essential in modern AI systems. The explainer covers embeddings, semantic search, and the retrieval pyramid—a framework for choosing between vector search, keyword search, and agentic retrieval based on workload needs.

Key topics include whether long context windows reduce the need for retrieval altogether, the hidden costs of embedding generation and vector indexing, and when vector search outperforms traditional keyword approaches. The video also clarifies the distinction between vector indexes and full vector databases.

The aim is to help engineers and architects select the right retrieval architecture rather than assuming vector databases are always necessary.