Semantic Chunking & Vector Similarity Playground

Understanding RAG, Text Chunking, and Vector Search

In Large Language Model applications, Retrieval-Augmented Generation (RAG) is an architectural pattern that retrieves relevant information from a custom document database and appends it to the LLM prompt context. This enables the model to answer queries using updated corporate facts, bypassing static training cutoff limitations and preventing hallucinations.

To perform vector search, source documents must be split into readable segments called Chunks, which are then compiled into high-dimensional numerical vectors (Embeddings) by an embedding model. Finding the most relevant chunk for a user query is accomplished by calculating the geometric distance between the query vector and document chunk vectors in vector space.

Why Chunk Size and Overlap Matter

When indexing documents, choosing the correct chunk size and overlap parameters is critical to retrieval accuracy:

• Chunk Size: Determines the maximum size of each text block. Chunks that are too small may lose critical context surrounding a sentence. Chunks that are too large dilute semantic focus, introducing noisy text that reduces retrieval precision.
• Chunk Overlap: Specifies the number of characters, words, or tokens shared between consecutive chunks. Overlap acts as a buffer zone, preventing facts from being sliced in half at chunk boundaries. Without overlap, a statement split across two chunks can become unretrievable.

Cosine Similarity vs. Dot Product vs. Euclidean Distance

Vector databases rank relevant document chunks by computing metric distances. The three most common mathematical distance metrics are:

1. Cosine Similarity

Measures the cosine of the angle between two vectors, regardless of their magnitude or text length. It evaluates directional alignment, making it the default choice for document text retrieval. It scales between -1.0 and 1.0 (or 0.0 and 1.0 for positive term counts):

$$\text{Similarity}(A, B) = \cos(\theta) = \frac{A \cdot B}{\|A\| \|B\|}$$

2. Dot Product (Inner Product)

Multiplies matching coordinate values directly. If vectors are normalized to unit length, Dot Product is mathematically identical to Cosine Similarity. If vectors are unnormalized, Dot Product is heavily skewed by document lengths, favoring longer text blocks.

3. Euclidean Distance (L2)

Measures the straight-line distance between two points in vector space. It is highly sensitive to vector magnitudes. A smaller distance represents greater similarity.

How to Integrate Vector Search in Code

Once you establish your chunking parameters using our simulator, you can implement RAG vector databases using popular frameworks:

Python Integration (Using ChromaDB)

import chromadb
from chromadb.utils import embedding_functions

# Initialize local database client
client = chromadb.Client()

# Set up default sentence embedding model
emb_fn = embedding_functions.DefaultEmbeddingFunction()
collection = client.create_collection(name="policy_documents", embedding_function=emb_fn)

# Add chunks to collection
collection.add(
    documents=[
        "Licenses purchased within 14 days are eligible for a full 100% refund.",
        "Outages exceeding 4 hours receive a 10% monthly billing credit."
    ],
    ids=["chunk_1", "chunk_2"]
)

# Run semantic query search
results = collection.query(
    query_texts=["How do I get a refund?"],
    n_results=1
)
print(results["documents"])

Frequently Asked Questions (FAQ)

Q: How does this playground calculate vectors locally?
A: To run 100% client-side without network requests, this tool compiles local TF-IDF (Term Frequency-Inverse Document Frequency) vectors. It counts unique word occurrences across all document chunks, normalizes them, and runs Cosine Similarity. This demonstrates the exact mathematical scoring and ranking mechanics used by commercial vector databases.

Q: What is a typical chunking configuration for RAG?
A: Standard starting parameters for general text documents are a chunk size of 500 to 1000 characters (or 150 to 250 tokens) with a 10% to 20% overlap (e.g. 100 characters or 25 tokens). Adjust these values in our simulator to check how text blocks split.

Q: Is my pasted text safe?
A: Yes. All text splitting, token mapping, vector constructions, and cosine similarity comparisons are executed locally in your browser sandbox. RTSALL does not upload, log, or transmit any document data.