Databricks | Insights

Prerequisites

Before you configure the Databricks connector, ensure the following requirements are met.

Databricks access

• Account Admin role in your Databricks account.
• Admin role in all Databricks workspaces.
• MANAGE access to all workspaces' secret scopes.
• (Optional) Databricks CLI installed on the machine where you will run the setup script. See Install or update the Databricks CLI.

Identity Security Insights access

• Administrator access to Identity Security Insights.
• Your Insights site is provisioned and active.

🚧

Important

Third-party documentation is subject to change. Updates might not be reflected in BeyondTrust documentation. For the most up-to-date information, visit the websites linked in this topic.

Configuration paths

Before you begin, identify the configuration options that apply to your environment. Your selections determine which steps you complete.

Service principal setup method

Installation method

Select this method during connector creation.

Step 1: Create the connector in Insights

1. From Insights Home, select > Connectors.
   The Connectors page displays.
2. Click Total configured.
3. Click Create Connector and select Databricks from the list.

4. Provide a human-readable name for the Databricks connector in the Connector Name field.

5. Select an installation method:
   • Databricks CLI (recommended if you can install Databricks CLI)
   • REST API
6. Download the PowerShell script and run it on the local machine.

ℹ️

Keep this panel open. You return to this page after running the script to enter credentials.

Step 2: Run the setup script

Run the downloaded PowerShell script on a local machine with access to your Databricks account.

The script:

• Creates the service principal in Databricks (if it does not already exist)
• Creates a client secret for the service principal (optional)
• Assigns the following permissions to the service principal:
  • Account Admin in Databricks account
  • Admin role in all Databricks workspaces
  • MANAGE scope for all workspaces' secret scopes

ℹ️

If you selected Databricks CLI as the installation method, ensure Databricks CLI is available on the machine before running the script. See (Install or update the Databricks CLI | Databricks Documentation).

Once the script completes, record the output values. You need these for the Final step.

• Account Id
• Client Id
• Client Secret

If the script completed successfully, proceed directly to the Final step.

Step 3: Create the Service Principal manually

Skip this step if the setup script completed successfully in Step 2.

Create Service Principal

1. Click User management on the sidebar.
2. On Service Principals tab, click Add service principal.
3. Enter BT-Insights-Databricks-SP service principal’s name.
4. Click Add service principal.

ℹ️

For more information, see Manage service principals | Databricks Documentation.

Create credentials for Service Principal

1. In the sidebar, click User management.
2. On the Service principals tab, find and click the username.
3. Select Credentials & secret tab.
4. Click Generate secret.
5. Enter the secret lifetime, e.g.: 365 and click Generate.
6. Record the Client Id and Secret.

Step 4: Assign permissions manually

Skip this step if the setup script completed successfully in Step 2.

Assign Account Admin role to Service Principal

1. In the sidebar, click User management.
2. On the Service principals tab, find and click the username.
3. Select Roles tab.
4. Select Account admin to activate.

ℹ️

For more information, see Manage service principals | Databricks Documentation.

Assign Admin role to service principals for all workspaces

1. On the menu on the left, navigate to Workspaces.
2. Do the following step for each workspace:
   1. Click on workspace name.
   2. Navigate to Permissions tab.
   3. Click Add permissions.
   4. Search for BT-Insights-Databricks-SP.
   5. Select Admin for Permission.
   6. Click Save.

Assign MANAGE scope to all workspaces' secret scopes

Databricks CLI is required for this section. For more information, see Databricks CLI documentation.

1. Obtain the Databricks Account ID by clicking on the user profile located in the top right corner of the Databricks account portal.
2. Log in to Databricks account:

   databricks auth login --host <https://accounts.cloud.databricks.com> --account-id {AccountId}

3. List all workspaces:

   databricks account workspaces list --output json

4. For each workspace that is not in FAILED state:
   1. Get workspace deployment_name from the output above
   2. Log in to each workspace

      databricks auth login --host https://{workspace_deployment_name}.cloud.databricks.com --profile {workspace_deployment_name}

   3. List all secret scopes

      databricks secrets list-scopes --profile {workspace_deployment_name}

   4. For each secret scope
      1. Assign secret scope MANAGE permission to service principal

         databricks secrets put-acl {scope_name} {sp_client_id} MANAGE --profile {workspace_deployment_name}

Final Step: Complete the connector in Insights

1. Return to the Create Databricks Connector panel in Insights.

   

2. Enter the following:

   • Account ID: Your Databricks account ID (e.g., 123e4567-e89b-12d3-a456-426614174111)
   • Client ID: The Databricks service principal application ID.
   • Client Secret: The Databricks service principal secret value.

3. Click Create Connector.

Navigate to > Connectors > Total Configured to confirm the connector was successfully created and review connector settings.