Databricks Install & Auth

Overview

Set up Databricks CLI and SDK with authentication credentials.

Prerequisites

• Python 3.8+ with pip

• Databricks workspace URL (e.g., https://adb-1234567890.1.azuredatabricks.net )

• Personal Access Token or OAuth credentials

• Admin access for Unity Catalog setup

Instructions

Step 1: Install Databricks CLI and SDK

set -euo pipefail

Install CLI v2 (recommended)

pip install databricks-cli

Install Python SDK

pip install databricks-sdk

Install dbx for local development

pip install dbx

Step 2: Configure Authentication

Option A: Personal Access Token (Simplest)

Interactive configuration

databricks configure --token

Enter workspace URL: https://adb-1234567890.1.azuredatabricks.net

Enter token: dapi1234567890abcdef...

Option B: Environment Variables

Set environment variables

export DATABRICKS_HOST="https://adb-1234567890.1.azuredatabricks.net" export DATABRICKS_TOKEN="dapi1234567890abcdef..."

Or create .env file

cat > .env << 'EOF' DATABRICKS_HOST=https://adb-1234567890.1.azuredatabricks.net DATABRICKS_TOKEN=dapi1234567890abcdef... EOF

Option C: OAuth (Recommended for Production)

Using OAuth U2M (user-to-machine)

databricks configure --oauth

Using OAuth M2M (service principal)

export DATABRICKS_HOST="https://adb-1234567890.1.azuredatabricks.net" export DATABRICKS_CLIENT_ID="your-client-id" export DATABRICKS_CLIENT_SECRET="your-client-secret"

Step 3: Configure Profiles (Multi-Workspace)

~/.databrickscfg

[DEFAULT] host = https://adb-dev-1234567890.1.azuredatabricks.net token = dapi_dev_token...

[staging] host = https://adb-staging-1234567890.1.azuredatabricks.net token = dapi_staging_token...

[production] host = https://adb-prod-1234567890.1.azuredatabricks.net token = dapi_prod_token...

Step 4: Verify Connection

Test CLI connection

databricks workspace list /

Test with specific profile

databricks workspace list / --profile staging

Test SDK connection

python -c " from databricks.sdk import WorkspaceClient w = WorkspaceClient() print(f'Connected to: {w.config.host}') print(f'Clusters: {len(list(w.clusters.list()))}') "

Output

• Databricks CLI installed and configured

• Python SDK installed

• Authentication credentials stored

• Connection verified to workspace

Error Handling

Error Cause Solution

Invalid token

Expired or incorrect token Generate new PAT in User Settings

Workspace not found

Wrong host URL Verify workspace URL format

Permission denied

Token lacks permissions Request admin access or correct scopes

Connection refused

Network/firewall issue Check VPN, firewall rules

SSL certificate error

Corporate proxy Set --insecure or configure proxy

Examples

Python SDK Initialization

from databricks.sdk import WorkspaceClient

Auto-detect from environment/config file

w = WorkspaceClient()

Explicit configuration

w = WorkspaceClient( host="https://adb-1234567890.1.azuredatabricks.net", token="dapi1234567890abcdef..." )

With profile

w = WorkspaceClient(profile="production")

Service Principal Authentication

from databricks.sdk import WorkspaceClient from databricks.sdk.config import Config

config = Config( host="https://adb-1234567890.1.azuredatabricks.net", client_id="your-client-id", client_secret="your-client-secret" ) w = WorkspaceClient(config=config)

Azure AD Authentication

from databricks.sdk import WorkspaceClient from azure.identity import DefaultAzureCredential

Uses Azure AD managed identity

w = WorkspaceClient( host="https://adb-1234567890.1.azuredatabricks.net", credentials_provider=DefaultAzureCredential() )

Resources

• Databricks CLI Documentation

• Databricks SDK for Python

• Authentication Methods

• Personal Access Tokens

Next Steps

After successful auth, proceed to databricks-hello-world for your first cluster and notebook.