In the RAG (Retrieval-Augmented Generation) pipeline, once text is segmented into chunks, the next critical process is embedding. This step converts each text chunk into vectors—represented as points in a multi-dimensional vector space. In vector-based RAG systems, this transformation is essential for performing efficient and accurate semantic searches.
The primary driver for converting chunks into vectors is to achieve semantic understanding. For instance, the term "feline" is conceptually linked to "cat" despite the words being different. Semantic similarity combines intent, context, and meaning to establish relationships between a user query and stored documents. By storing semantically related words closer together in a vector space, the system can retrieve the most relevant information from the database and provide it to the LLM for refined processing.
To determine the proximity of vectors, Cosine Similarity is the standard methodology. When a query arrives, it is converted into a vector, and the cosine similarity is calculated against all stored vectors. The closer the vectors are, the closer the value is to 1. Interestingly, cosine is preferred over sine or tangent because, for small angles, sine values remain near zero and tangent values can fluctuate significantly. These measurements are not stable for semantic comparison, whereas cosine similarity provides a reliable way to measure semantic closeness.
Retrieval methodologies generally fall into two categories: KNN (K-Nearest Neighbors) and ANN (Approximate Nearest Neighbors). KNN compares the query vector with every single stored vector one by one, ensuring high accuracy but resulting in slower performance for massive datasets. Conversely, ANN is designed for large-scale applications where speed is paramount; it finds approximate nearest vectors rather than checking every point, significantly improving retrieval speed while sacrificing a negligible amount of accuracy.
The effectiveness of these vectors is also defined by their dimensions, which typically range from 256 to over 3000. The size of the dimension depends on the embedding model and the depth of contextual information it captures. Generally, higher dimensions allow the model to capture richer semantic information and more complex relationships within the data.
