Product Docs
  • What is Dataworkz?
  • Getting Started
    • What You Will Need
    • Advanced RAG Quickstart
      • Configure LLM's
        • AWS Bedrock
      • Connecting Your Data
        • Finding Your Data Storage: Collections
      • Ingesting Unstructured Data
      • Create Vector Embeddings
        • How to Build the Vector embeddings from Scratch
        • How do Modify Existing Chunking/Embedding Dataflows
      • Response History
    • Embed RAG into your App
    • Data Lake to Salesforce
  • Concepts
    • Datasets
    • No-code Transformations
    • Lineage
      • Features of Lineage:
      • Viewing a dataset's lineage:
    • Discovery
      • How to: Discovery
    • Dataflows
      • Single Dataflows:
      • Composite dataflows:
      • Benefits of Dataflows:
  • Guides
    • Working With Rag Applications
      • High Precision RAG
      • Advanced RAG - Access Control for your data corpus
      • Creating RAG Experiments with Dataworkz
      • Embedding Models
        • Privately Hosted Embedding Models
        • Amazon Bedrock Hosted Embedding Model
        • OpenAI Embedding Model
      • Unstructured Data Ingestion
        • File Ingestion
        • Html/Sharepoint Ingestion
    • Transformation Functions
      • Column Transformations
        • String Operations
          • Format Operations
          • String Calculation Operations
          • Remove Stop Words Operation
          • Fuzzy Match Operation
          • Masking Operations
          • 1-way Hash Operation
          • Copy Operation
          • Unnest Operation
          • Convert Operation
          • Vlookup Operation
        • Numeric Operations
          • Tiles Operation
          • Numeric Calculation Operations
          • Custom Calculation Operation
          • Numeric Encode Operation
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
          • Convert Operation
          • VLookup Operation
        • Boolean Operations
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
        • Date Operations
          • Date Format Operations
          • Date Calculation Operations
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
          • Encode Operation
          • Convert Operation
        • Datetime/Timestamp Operations
          • Datetime Format Operations
          • Datetime Calculation Operations
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
          • Encode Operation
          • Page 1
      • Dataset Transformations
        • Utility Functions
          • Area Under the Curve
          • Page Rank Utility Function
          • Transpose Utility Function
          • Semantic Search Template Utility Function
          • New Header Utility Function
          • Transform to JSON Utility Function
          • Text Utility Function
          • UI Utility Function
        • Window Functions
        • Case Statement
          • Editor Query
          • UI Query
        • Filter
          • Editor Query
          • UI Query
    • Working with CSV files
    • Connect to data source(s)
      • Cloud Data Platforms
        • AWS S3
        • BigQuery
        • Google Cloud Storage
        • Azure
        • Snowflake
        • Redshift
        • Databricks
      • Databases
        • MySQL
        • Microsoft SQL Server
        • Oracle
        • MariaDB
        • Postgres
        • DB2
        • MongoDB
        • Couchbase
        • Aerospike
        • Pinecone
      • SaaS Applications
        • Google Ads
        • Google Analytics
        • Marketo
        • Zoom
        • JIRA
        • Salesforce
        • Zendesk
        • Hubspot
        • Outreach
        • Fullstory
        • Pendo
        • Box
        • Google Sheets
        • Slack
        • OneDrive / Sharepoint
    • IP safelist
    • Data Prep
      • Joins
        • Configuring a Join
      • Union
        • Configuring a Union
    • Job Monitoring
    • Authentication
    • User Management
  • API
    • Generate API Key in Dataworkz
  • Open Source License Types
Powered by GitBook
On this page
  • What You Will Need:
  • Step 1: Configure LLM API
  • Step 2: Create Your APP
  • Step 3: Creating Vector Embeddings from Ingested Data:
  • Step 4: Create the Rag App
  • Step 5: Using the Q&A System
  1. Guides
  2. Working With Rag Applications

High Precision RAG

This page will show an end-to-end creation of an Advanced RAG application using Meta's 2024 Second Quarter Financial Report PDF. Highlighted steps; Ingestion, Chunking/Embedding, and App Creation

PreviousWorking With Rag ApplicationsNextAdvanced RAG - Access Control for your data corpus

Last updated 5 months ago

What You Will Need:

  • API Key and Credentials: Ensure you have an API key and any other necessary credentials for your Large Language Model (LLM).

Step 1: Configure LLM API

Before creating the Q&A system, configure the OpenAI GPT-4 model via the AI Applications page.

  • Add API Key: Enter your API key into the LLM Configuration box.

  • Set Parameters: Copy a response length of 1200 characters and set the temperature to 0.5.

Step 2: Create Your APP

Click the “Create AI App” Button to begin

Enter your application name:

  • Make sure to select the default language model for your application. This can be changed in the editor screen.

Configure Sources:

  • PDF (Upload from Local System): Select PDF as the file type, upload from your desktop, and locate your file of interest.

Already Existing Dataset(Skip if you are ingesting new data)

  • Select the already existing dataset option once you have selected the configured LLM, then click continue to select the correct dataset

  • Select the dataset that has already been processed from pdf, html, word, etc...

  • Hit the next button, and continue with the following steps

Submit Ingest:

  • After configuring your source data, the default target will be created. If you would like to change this you can hit the edit button above the green box.

  • Hit Next

Step 3: Creating Vector Embeddings from Ingested Data:

Run the Process:

  • Hit Next, Select the dataflow to use for this app.

Step 4: Create the Rag App

Hit Review and Submit:

  • Make sure that you are happy with the pre-set rules, if there are any components(retrieval strategies, custom concepts to add, want to select specific LLM for text generation, and adding a custom prompt) you would like to change, you can do so before submitting.

Option to Change Language Model and Prompt:

  • Choose the language models (e.g., LLaMA 2, ChatGPT, Dolly).

  • Provide a custom prompt for the system.

Save and Submit:

  • Save your configurations and submit to create the Q&A system.

  • MongoDB will automatically index the data.

Step 5: Using the Q&A System

Hit the search icon:

Testing the System:

  • Ask a question in the chat box, the system will return an answer along with the data location. For Example: What is the 2024 GAAP advertising Revenue in the three months ended June 30th? What about net cash by operating activities?

Review Results:

  • Users can check the source links to verify the origin of the generated information.