RAG Apps API

Dataworkz API to RAG Apps - to perform QnA, Search and Chat.

Dataworkz RAG Applications can be created over multi-modal data (images, text, etc) with different formats (pdf, powerpoints, etc) from different sources (Google Drive, Sharepoint, etc). RAG Applications can be utilized for multiple use-cases

• QnA: Allow users to ask questions in natural language and leverage the power of RAG to respond with answers from the underlying data corpus

• Enterprise Search: Allow users to perform semantic search over the underlying data corpus

• Chat with the data: Allow users to have a conversation with the underlying data corpus. (Note: this is not the Agents API)

Note: All API calls require an API Token. Please follow these steps to generate an API Key. The API Key has to be provided in the Authorizationheader in the format SSWS <your api key>

Get all QnA systems configured in this account

get

Returns a map containing unique identifier and name for all QnA systems that have been set up in Dataworkz

Authorizations

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "f9c5aa7e-57a9-4f77-9e91-e8b37eb46": "Security Policies QnA"
}

Get details of a specific QnA system

get

Get details of a specific QnA system

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId} HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "uuid": "f9c5aa7e-57a9-4f77-9e91-e8b37eb46c18",
  "name": "mdb",
  "mode": "qa",
  "prompt": "this is a prompt",
  "dimensions": "0",
  "database": "mongodb",
  "params": "{}"
}

Get LLM providers configured for a given QnA system

get

Returns a map containing unique identifier and LLM models that have been configured for a given QnA system

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId}/llm-providers HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "47b51bb5-bdfa-413d-8bfa-ff040ab1c901": "llama-2-13b-chat"
}

Get answer to the question asked

get

Ask a question to a specific QnA system and leverage RAG techniques to get the question answered based on the data corpus configured for that QnA system. You should specify the LLM to use from the ones configured for this QnA system. You can pass filter expressions to control the data retrieval process. Optionally, flags can be passed to configure the RAG engine behaviour. Each request is associated with an id to perform additional actions later.

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Query parameters

questionTextstringRequired

Text of the question asked to the QnA system. Must be url encoded.

Example: Example to join 2 datasets in S3 and write the output to MongoDB

llmProviderIdstringRequired

Id of the LLM to use. This has to be from the list configured in the pipeline. Id can be retrieved using /systems/{systemId}/llm-providers api or copied from the LLM Configuration page.

Example: 774c246b-a079-4fa5-b3dc-4a82b35aa258

filterstringOptional

Filter expression to apply to data retrieval

Example: version in ('1.2', '2.0')

propertiesstringOptional

Flags to control behaviour. Format is Key=value pairs separated by ; Supported flags - include_probe : Default is true. Set to false to disable returning probe data with request

Example: include_probe=false;context=3

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId}/answer HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "answer": "This is the answer based on system id and question text",
  "type": "Dataworkz",
  "questionText": "question Text",
  "questionUuid": "6b807b37-0c41-4cd9-a420-7d2dc9170fe9",
  "context": [
    {
      "source": "",
      "type": "",
      "data": "",
      "link": ""
    }
  ]
}

Get answer to the question asked or use as chat api

post

Ask a simple question to a specific QnA system or ask a question in the context of a conversation and leverage RAG techniques to get the question answered based on the data corpus configured for that QnA system. You should specify the LLM to use from the ones configured for this QnA system. Note that for chat type conversations (i.e. with conversation history) you are limited to gpt-4 and llama3-70B models. You can pass filter expressions to control the data retrieval process. Optionally, flags can be passed to configure the RAG engine behaviour. Each request is associated with an id to perform additional actions later.

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Query parameters

questionTextstringRequired

Text of the question asked to the QnA system. Must be url encoded.

Example: Example to join 2 datasets in S3 and write the output to MongoDB

llmProviderIdstringRequired

Id of the LLM to use. This has to be from the list configured in the pipeline. Id can be retrieved using /systems/{systemId}/llm-providers api or copied from the LLM Configuration page.

Example: 774c246b-a079-4fa5-b3dc-4a82b35aa258

filterstringOptional

Filter expression to apply to data retrieval

Example: version in ('1.2', '2.0')

propertiesstringOptional

Flags to control behaviour. Format is Key=value pairs separated by ; Supported flags - include_probe : Default is true. Set to false to disable returning probe data with request

Example: include_probe=false;context=3

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Body

objectOptional

Responses

200

successful operation

application/json

Responseobject

post

HTTPcURLJavaScriptPython

POST /api/qna/v1/systems/{systemId}/answer HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Content-Type: application/json
Accept: */*
Content-Length: 231

{
  "conversationHistory": [
    {
      "by": "USER",
      "text": "Hi"
    },
    {
      "by": "AI",
      "text": "Hi! How can I assist you today?"
    },
    {
      "by": "USER",
      "text": "I would like status for order 33"
    },
    {
      "by": "AI",
      "text": "Order 33 is delayed. Would you like more details?"
    }
  ]
}

200

successful operation

{
  "answer": "This is the answer based on system id and question text",
  "type": "Dataworkz",
  "questionText": "question Text",
  "questionUuid": "6b807b37-0c41-4cd9-a420-7d2dc9170fe9",
  "context": [
    {
      "source": "",
      "type": "",
      "data": "",
      "link": ""
    }
  ]
}

Semantic Enterprise Search for the provided query

get

Get a list of search results from the configured system that semantically match the query asked based on similarity with the data corpus configured for that QnA system. You can pass filter expressions to control the data retrieval process. Optionally, flags can be passed to configure the RAG engine behaviour. Each request is associated with an id to perform additional actions later.

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Query parameters

querystringRequired

search query terms

Example: Example to join 2 datasets in S3 and write the output to MongoDB

llmProviderIdstringOptional

Id of the LLM to use. This has to be from the list configured in the pipeline. Id can be retrieved using /systems/{systemId}/llm-providers api or copied from the LLM Configuration page. Will only be used if required.

Example: 774c246b-a079-4fa5-b3dc-4a82b35aa258

filterstringOptional

Filter expression to apply to data retrieval

Example: version in ('1.2', '2.0')

propertiesstringOptional

Flags to control behaviour. Format is Key=value pairs separated by ; Supported flags - include_probe : Default is true. Set to false to disable returning probe data with request

Example: include_probe=false;context=3

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId}/search HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "answer": "This is the answer based on system id and question text",
  "type": "Dataworkz",
  "query": "question Text",
  "questionUuid": "6b807b37-0c41-4cd9-a420-7d2dc9170fe9",
  "context": [
    {
      "source": "",
      "type": "",
      "data": "",
      "link": ""
    }
  ]
}

Get list of all questions asked by a user in last 6 months

get

Get list of all questions that have been asked by a user in last 6 months from a QnA system

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId}/questionshistory HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "llm_response": {
    "answer": "This is the answer based on system id and question text",
    "type": "Dataworkz",
    "questionText": "question Text",
    "questionUuid": "6b807b37-0c41-4cd9-a420-7d2dc9170fe9",
    "context": [
      {
        "source": "",
        "type": "",
        "data": "",
        "link": ""
      }
    ]
  },
  "activity_response_date": "2024-01-04T05:49:01.000+00:00",
  "comments": "",
  "llm_model": "",
  "retrival_relevance": "",
  "retrival_context_recall": "",
  "generation_groundedness": "",
  "generation_correctness": "",
  "generation_relevance": "",
  "generation_confidence_score": "",
  "response_sentiment": "",
  "expert_comments": "",
  "expert_sentiment": ""
}

Get details about previously asked question

get

Get details about any question that had been asked previously by a user from a QnA system

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

questionIdstringRequired

Id of the answer request

Example: a647381-1087-6e98-b3f0-5o9876a03a54

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId}/questions/{questionId} HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "llm_response": {
    "answer": "This is the answer based on system id and question text",
    "type": "Dataworkz",
    "questionText": "question Text",
    "questionUuid": "6b807b37-0c41-4cd9-a420-7d2dc9170fe9",
    "context": [
      {
        "source": "",
        "type": "",
        "data": "",
        "link": ""
      }
    ]
  },
  "activity_response_date": "2024-01-04T05:49:01.000+00:00",
  "comments": "",
  "llm_model": "",
  "retrival_relevance": "",
  "retrival_context_recall": "",
  "generation_groundedness": "",
  "generation_correctness": "",
  "generation_relevance": "",
  "generation_confidence_score": "",
  "response_sentiment": "",
  "expert_comments": "",
  "expert_sentiment": ""
}

Get list of all filters with metadata

get

Get a list of all filters associated with this system and their metadata which includes their datatype, cardinality and values (for low cardinality filters)

Authorizations

Path parameters

systemIdstringRequired

Id of the QnA system

Example: 3756809c-4032-4e48-b3f0-2a6d30a03a54

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/systems/{systemId}/filters HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "filters": [
    {
      "name": "year",
      "type": "integer",
      "description": "The year that the data was published",
      "cardinality": 3,
      "values": [
        2021,
        2022,
        2023
      ]
    }
  ]
}

Get LLM providers configured in Dataworkz

get

Returns a map containing unique identifier and LLM models that have been configured

Authorizations

Header parameters

AuthorizationanyOptional

For Dataworkz API Token use the format 'SSWS ' and for OAuth2 generated JWT token use 'Bearer '

Responses

200

successful operation

application/json

Responseobject

get

HTTPcURLJavaScriptPython

GET /api/qna/v1/llm-providers HTTP/1.1
Host: ragapps.dataworkz.com
Authorization: YOUR_API_KEY
Accept: */*

200

successful operation

{
  "47b51bb5-bdfa-413d-8bfa-ff040ab1c901": "llama-2-13b-chat"
}