Creating RAG Experiments with Dataworkz

Quick overview of Experiments, how to create a RAG experiment and how to compare the response accuracy

Overview: This guide walks you through the process of creating experiments with your RAG (Retrieval-Augmented Generation) applications using Dataworkz. You’ll learn how to create a new RAG application, then experiment with different chunking and embedding strategies to optimize the performance of your RAG-based models.

Prerequisites

Before you start creating RAG experiments, ensure you’ve set up a RAG application in Dataworkz. For instructions on creating your RAG application, refer to Create a RAG Application for detailed steps.

---

Step 1: Create a RAG Application

To start creating experiments, you first need to create a RAG application. If you haven't done so already, follow the instructions below:

1. Navigate to the RAG Applications tab in Dataworkz.

2. Click on Create New Application and follow the prompts to set up your RAG application. For detailed instructions on creating a RAG application, refer to RAG Application Quickstart.

---

Step 2: Create a RAG Experiment

Once your RAG application is set up, you can begin experimenting with different configurations. Here’s how:

1. Go to the RAG Applications tab in Dataworkz.

2. Locate the RAG application you want to experiment with and click on the 'Create Experiment' button next to it.

   This will allow you to create a new experiment based on the application and modify key parameters.

---

Step 3: Select Your Experiment Parameters

After clicking the 'Create Experiment' button, you’ll be prompted to configure several parameters for your experiment:

1. Chunking Strategy

Choose a chunking strategy that suits your application’s needs. The chunking strategy defines how your documents are split into smaller pieces for efficient retrieval and processing. Options may include:

• Fixed-size chunks

• Semantic-based chunking

• Custom strategies

2. Chunking Size

Specify the size of the chunks you want to use. This value determines the length of each chunk (in terms of tokens or characters). Larger chunks may reduce retrieval overhead, while smaller chunks could improve accuracy.

3. Chunk Overlap

Set the overlap between adjacent chunks. Overlapping chunks can help ensure that relevant information is retained across splits, but too much overlap may lead to redundancy. Adjust the overlap to strike a balance between performance and precision.

4. Embedding Model

Select an embedding model

Step 4: Save and Run the Experiment

Once you've configured the settings, click Save to create a copy of your original RAG application, now with the new chunking and embedding strategies applied. There will be a new experimental RAG application under the drop down below the original with these parameters, and you can compare the results to the original RAG application for performance improvements.

---

Next Steps:

Once the experiment is created, you can monitor its progress and analyze the results to see how the new chunking and embedding strategies impact your RAG application’s performance. If needed, you can create additional experiments to further refine the setup. If you want to use the RAG Evaluation repo you can follow the steps and compare the experimental version of your RAG app.