ZBrain Documentation
  • ZBrain Documentation
    • Components of ZBrain
    • Getting started with ZBrain
    • 📚Knowledge base
      • How to create a knowledge base?
      • Knowledge source
      • Information schema
      • File summary
      • Automated reasoning
      • Retrieval testing
      • Knowledge base settings
    • 📱APP
      • How to create a new app?
      • How to set up and customize your app?
      • How to access the app reporting dashboard?
    • 🤖ZBrain AI agents
      • Get started with agents on ZBrain
      • Deploying pre-built agents
      • Creating custom AI agents
        • Agent setup
        • Define input sources
        • Define Flow
          • Key elements of a Flow
          • Flow Components
            • Activepieces Platform
            • AITable
            • Airtable
            • Amazon S3
            • Amazon SNS
            • Amazon SQS
            • Amazon Textract
            • Apify
            • Apollo
            • Approval
            • Asana
            • Azure Communication Services
            • Azure Cosmos DB
            • Azure Document Intelligence
            • Azure OpenAI
            • Azure Translation
            • Bannerbear
            • Baserow
            • Beamer
            • Bedrock Claude
            • Bettermode
            • Binance
            • Bing Search
            • Blackbaud
            • Bonjoro
            • Box
            • Brevo
            • Brilliant Directories
            • Bubble
            • CSV
            • Calendly
            • Certopus
            • Clearout
            • Clockodo
            • Code
            • Confluence
            • Connections
            • Constant Contact
            • Contiguity
            • Contentful
            • Customer.io
            • Crypto
            • Databricks
            • Data Mapper
            • Date Helper
            • DeepL
            • Delay
            • Discord
            • Discourse
            • Drip
            • Dropbox
            • Dust
            • Facebook Pages
            • Figma
            • Files Helper
            • Flowise
            • Flowlu
            • Formbricks
            • Frame
            • Freshdesk
            • Freshsales
            • GCloud Pub/Sub
            • GenerateBanners
            • GhostCMS
            • GitHub
            • GitLab
            • Gmail
            • Google Calendar
            • Google Contacts
            • Google Docs
            • Google Drive
            • Google Forms
            • Google Gemini
            • Google My Business
            • Google Search
            • Google Search Console
            • Google Sheets
            • Google Tasks
            • Groq
            • Hacker News
            • Heartbeat
            • HubSpot
            • HTTP
            • Image Helper
            • Inputs
            • Instagram for Business
            • Intercom
            • Invoice Ninja
            • Jira Cloud
            • Jotform
            • Kimai
            • Kizeo Forms
            • LeadConnector
            • Line Bot
            • Linear
            • LinkedIn
            • LinkedIn Actions
            • LLMRails
            • Lusha
            • MailerLite
            • Mailchimp
            • Mautic
            • Microsoft Dynamics 365 Business Central
            • Microsoft Dynamics CRM
            • Microsoft Excel 365
            • Microsoft OneDrive
            • Microsoft Outlook Calendar
            • Microsoft Teams
            • Mixpanel
            • MongoDB
            • Notion
            • Odoo
            • OpenAI
            • OpenRouter
            • Pastebin
            • PDF
            • Postgres
            • PostHog
            • Pushover
            • Qdrant
            • Queue
            • Razorpay
            • Router
            • Salesforce
            • SendGrid
            • ServiceNow
            • SFTP
            • SharePoint
            • Slack
            • SMTP
            • Snowflake
            • SOAP
            • Spotify
            • Stability AI
            • Stable Diffusion Web UI
            • Storage
            • Stripe
            • SurrealDB
            • SurveyMonkey
            • Taskade
            • Telegram Bot
            • Text Helper
            • Trello
            • Twilio
            • Twitter
            • Utilities
            • WhatsApp Business
            • WordPress
            • XML
            • YouTube
            • ZBrain
            • Zendesk
            • ZeroBounce
            • Zoho Books
            • Zoho CRM
            • Zoho Invoice
            • Zoom
          • How to Define a Flow?
          • How to Test Each Step in the Flow?
        • Configure Additional Settings
        • Test and Deploy Agents
        • How to access, monitor, and manage agent performance and tasks?
    • Settings
    • 📖API Tutorials
      • 📚Knowledge base
        • Automated Reasoning
      • 📱APP
      • 🤖Agents
Powered by GitBook
On this page
  1. ZBrain Documentation
  2. ZBrain AI agents
  3. Creating custom AI agents
  4. Define Flow
  5. Flow Components

Databricks

PreviousCryptoNextData Mapper

Last updated 23 days ago

The Databricks piece in ZBrain Flow provides integration with Databricks' unified data analytics platform, allowing you to execute SQL queries and manage data processing jobs directly from your workflows. This powerful connector enables you to interact with Databricks workspaces without manual intervention. With Databricks integration, you can build automations that run data transformations, query data lakes, monitor job executions, and incorporate big data processing into your business processes. This piece is ideal for data teams looking to automate their analytics workflows, integrate data processing with other business systems, and create consistent, repeatable data pipelines.

How to Use Databricks in ZBrain Flow

Select Databricks as Your Connection

  • Click on the '+' button in the Flow and search for Databricks.

  • Select Databricks.

  • Decide on the action you need, then select it. ZBrain Flow provides several options:

    • Run SQL Command – Execute SQL queries against Databricks warehouses.

    • Create Databricks Job – Define and configure new data processing jobs.

    • Get Job Status – Monitor the execution status of Databricks jobs.

    • Run Job – Trigger the execution of Databricks jobs.

How to Connect to Your Databricks Workspace

Before using any Databricks actions in ZBrain Flow, you'll need to set up a connection to your Databricks environment. This is a one-time setup that will allow you to access your analytics platform securely.

To create your Databricks connection:

  • From any Databricks action, click on the connection dropdown and select 'Create connection'.

  • In the popup window that appears, you'll need to:

    • Enter a descriptive 'Connection Name' to identify this Databricks connection

    • In the 'Instance Name' field, enter your Databricks workspace URL (e.g., )

    • From the 'Grant Type' dropdown, select 'Client Credentials' as the authorization method

    • In the 'Client Id' field, enter your Databricks application client ID

    • In the 'Client Secret' field, enter your Databricks application client secret

    • Click 'Save' to store this connection

How to Execute SQL Queries in Databricks

Configuration Steps:

  • Connect to your Databricks workspace following the steps in the "How to Connect to Your Databricks Workspace" section.

  • In the 'Warehouse Path' field, enter the HTTP path to your SQL warehouse. This typically looks like: "/sql/warehouses/abc12345"

  • In the 'Query' field, enter the SQL statement you want to execute. You can run any SQL command supported by your Databricks SQL endpoint.

How to Create a Databricks Job

Configuration Steps:

  • Connect to your Databricks workspace following the steps in the "How to Connect to Your Databricks Workspace" section.

  • From the 'Task Types' dropdown, select the kind of processing you want to perform:

    • Notebook Task - to run a Databricks notebook

    • Python Wheel Task - to execute Python code packaged as wheel files

  • In the 'Job Name' field, enter a descriptive name for your job.

  • In the 'Cluster ID' field, optionally specify an existing cluster to run the job. Leave empty for Databricks to create a job cluster automatically.

  • In the 'Cron Schedule' field, optionally enter a cron expression to schedule recurring job runs.

  • From the 'Timezone' dropdown, select the time zone for scheduled job execution. This affects when cron-scheduled jobs will run.

  • In the 'Max Concurrent Runs' field, set the maximum number of job instances that can run simultaneously. The default is 1, which prevents multiple instances of the same job from running at once.

How to Get Job Status

To get a job status, first connect to your Databricks workspace following the steps in the "How to Connect to Your Databricks Workspace" section. Then, provide the ID of the job for which you need the job status.

How to Run a Job

To run a job, first connect to your Databricks workspace following the steps in the "How to Connect to Your Databricks Workspace" section. Next, specify the ID of the job you want to run.

🤖
dbc-f12da6c0-8761.cloud.databricks.com