Creating RAG Experiments with Dataworkz

Create experimental versions of a RAG application with different chunking and embedding strategies to compare retrieval performance.

Experiments let you create alternate versions of an existing RAG application with a different vectorization strategy — changing chunking size, chunking strategy, or embedding model — without modifying the original. You can then compare retrieval quality between the original and the experiment to find the optimal configuration.

Prerequisites

You need an existing RAG application. If you have not created one yet, see RAG Quickstart: Default Settings or RAG Quickstart: Custom Settings.

The RAG Applications dashboard lists all your applications with their ingestion, vectorization, and retrieval status.

The RAG Applications dashboard — locate the app you want to experiment with

---

Step 1: Open Create Experiment

Locate the RAG application you want to experiment with. In the Actions column, click the Create Experiment icon (the flask icon with a plus).

Hover over the Actions icons to reveal Create Experimentation — click it to open the configuration panel

---

Step 2: Configure the Experiment

The Create experiment panel opens. Set the vectorization parameters for this experimental version:

Set the chunking strategy, size, overlap, and embedding model — these differ from the original app's configuration

Parameter	Description
Chunking strategy	How documents are split — e.g., WordSplitter, MarkdownSplitter. Try different strategies to match your document structure.
Chunking size	Target size of each chunk (in tokens or characters). Smaller chunks improve precision; larger chunks preserve more context.
Chunk overlap	Tokens shared between adjacent chunks. Overlap preserves context across split boundaries — too much introduces redundancy.
Embedding model	The model used to convert chunks into vectors. Different models produce different semantic representations.

Click Create experiment to save. A copy of the original app is created with the new vectorization settings applied.

---

Step 3: Monitor and Compare

The experiment appears as an indented row under the parent application in the dashboard.

A newly created experiment starts with Not started vectorization — the pipeline runs automatically after creation

Once the experiment finishes ingestion and vectorization, switch to Compare mode using the button in the top-right of the dashboard to view the original and experimental versions side by side.

Compare mode — view the original app and its experiments side by side to evaluate configuration differences

To run a quantitative comparison, use the Dataworkz RAG Evaluation repo to benchmark the experimental version against the original.