Product Docs
  • What is Dataworkz?
  • Getting Started
    • What You Will Need
    • Advanced RAG Quickstart
      • Configure LLM's
        • AWS Bedrock
      • Connecting Your Data
        • Finding Your Data Storage: Collections
      • Ingesting Unstructured Data
      • Create Vector Embeddings
        • How to Build the Vector embeddings from Scratch
        • How do Modify Existing Chunking/Embedding Dataflows
      • Response History
    • Embed RAG into your App
    • Data Lake to Salesforce
  • Concepts
    • Datasets
    • No-code Transformations
    • Lineage
      • Features of Lineage:
      • Viewing a dataset's lineage:
    • Discovery
      • How to: Discovery
    • Dataflows
      • Single Dataflows:
      • Composite dataflows:
      • Benefits of Dataflows:
  • Guides
    • Working With Rag Applications
      • High Precision RAG
      • Advanced RAG - Access Control for your data corpus
      • Creating RAG Experiments with Dataworkz
      • Embedding Models
        • Privately Hosted Embedding Models
        • Amazon Bedrock Hosted Embedding Model
        • OpenAI Embedding Model
      • Unstructured Data Ingestion
        • File Ingestion
        • Html/Sharepoint Ingestion
    • Transformation Functions
      • Column Transformations
        • String Operations
          • Format Operations
          • String Calculation Operations
          • Remove Stop Words Operation
          • Fuzzy Match Operation
          • Masking Operations
          • 1-way Hash Operation
          • Copy Operation
          • Unnest Operation
          • Convert Operation
          • Vlookup Operation
        • Numeric Operations
          • Tiles Operation
          • Numeric Calculation Operations
          • Custom Calculation Operation
          • Numeric Encode Operation
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
          • Convert Operation
          • VLookup Operation
        • Boolean Operations
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
        • Date Operations
          • Date Format Operations
          • Date Calculation Operations
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
          • Encode Operation
          • Convert Operation
        • Datetime/Timestamp Operations
          • Datetime Format Operations
          • Datetime Calculation Operations
          • Mask Operation
          • 1-way Hash Operation
          • Copy Operation
          • Encode Operation
          • Page 1
      • Dataset Transformations
        • Utility Functions
          • Area Under the Curve
          • Page Rank Utility Function
          • Transpose Utility Function
          • Semantic Search Template Utility Function
          • New Header Utility Function
          • Transform to JSON Utility Function
          • Text Utility Function
          • UI Utility Function
        • Window Functions
        • Case Statement
          • Editor Query
          • UI Query
        • Filter
          • Editor Query
          • UI Query
    • Working with CSV files
    • Connect to data source(s)
      • Cloud Data Platforms
        • AWS S3
        • BigQuery
        • Google Cloud Storage
        • Azure
        • Snowflake
        • Redshift
        • Databricks
      • Databases
        • MySQL
        • Microsoft SQL Server
        • Oracle
        • MariaDB
        • Postgres
        • DB2
        • MongoDB
        • Couchbase
        • Aerospike
        • Pinecone
      • SaaS Applications
        • Google Ads
        • Google Analytics
        • Marketo
        • Zoom
        • JIRA
        • Salesforce
        • Zendesk
        • Hubspot
        • Outreach
        • Fullstory
        • Pendo
        • Box
        • Google Sheets
        • Slack
        • OneDrive / Sharepoint
    • IP safelist
    • Data Prep
      • Joins
        • Configuring a Join
      • Union
        • Configuring a Union
    • Job Monitoring
    • Authentication
    • User Management
  • API
    • Generate API Key in Dataworkz
  • Open Source License Types
Powered by GitBook
On this page
  1. Guides
  2. Connect to data source(s)
  3. Cloud Data Platforms

Google Cloud Storage

PreviousBigQueryNextAzure

Last updated 1 year ago

This document describes the Dataworkz connector configuration required to access Google Cloud Storage. Dataworkz requires authorizing a Service Account to access GCS storage. Follow the Setup Service Account instructions for the purpose of setting up a Service Account. Ensure that service account that has been created has access to the Google Storage bucket. The service account used by Spark/Dataworkz to write to the GCS bucket must have the roles/storage.objectCreator and roles/storage.objectViewer roles on the bucket. For example, if Dataworkz is running as service account datXXXXXXXXX@datXXXXXXXXXX.iam.gserviceaccount.com, one has to explicitly add the permissions storage.objectCreator and storage.objectViewer for datXXXXXXXXX@datXXXXXXXXXX.iam.gserviceaccount.com account in customer's Google Cloud Storage console (this can also be done via cli tool).

Create Connector for Google Cloud Storage

  1. Login to Dataworkz Application

  2. Go to Configuration -> Cloud Data Platforms -> GCS Storage

  3. Click the + icon to add a new configuration

  1. Enter name for the configuration in the above screen

  2. Enter the Project ID

  3. Enter the Client ID

  4. Enter the Client Email ID

  5. Enter the Private Key ID

  6. Enter the Private Key

  7. Provide the storage base path

  8. Test the connection

  9. If successful click Save

Newly created connector would show up in the list of Google Cloud Storage configurations