How to Connect Azure DevOps to Databricks

Connecting your Databricks workspace to Azure DevOps is a valuable step for enabling CI/CD, version control, and collaboration across your data workflows. With the new Databricks free version offering more flexibility than the old one, there’s even more reason to set this up properly from the start.

Here’s a clear and simple walkthrough to help you link Azure DevOps with Databricks and understand why it matters.

Getting Started: Linking Your Account

To connect your Azure DevOps account to Databricks:

1. Go to your Databricks workspace
   • Click your user icon
   • Navigate to Settings
   • Select Linked Account
2. Switch to Azure DevOps
   • Head to your Azure DevOps portal
   • Open User Settings
   • Go to Personal Access Tokens
   • Click New Token
     • Name it something like databricks full access
     • Choose a duration (e.g., 90 days or up to a year)
   • Click Create
   • Copy the token and store it securely
3. Return to Databricks
   • Enter your email address and paste the access token
   • Save your settings
   • Your account should now be linked

Why This Matters

Connecting Azure DevOps to Databricks unlocks a number of powerful capabilities:

• CI/CD Automation
  Automate deployments of notebooks and workflows through pipelines connected to your DevOps repos.
• Version Control
  Track changes, manage code history, and roll back if something breaks. This adds traceability and control to your development process.
• Team Collaboration
  Work on the same branch, leave comments, and collaborate on notebooks simultaneously. This helps teams stay aligned and move faster.
• Rapid Iteration & Feedback Loops
  By using pull requests and reviews, teams can continuously improve and release faster. Feedback becomes part of the development cycle.
• Deployment Flexibility
  Use asset bundles and structured pipelines to deploy across environments — from dev to production.
• Testing Integration
  Run unit tests and integrate quality checks as part of your DevOps setup.

Azure DevOps offers a great foundation for improving how teams collaborate, ship code, and deliver data products. Setting this up with Databricks makes your workspace ready for scalable and maintainable data engineering.