This is simple way to copy everything from a repo to a databricks workspace.
In DevOps there seem to be two types of pipelines: 1 PipeLines 2 Releases
The Pipelines is more sophisticated including compilation, unit testing, etc to have an artifact package for release.
The Releases can be as simple as just copying stuff from a repo to another place. Here we look at only the simple Releases to copy Databricks notebooks and files from a repo onto a databricks workspace.
On dev.azure.com, go to the devops space, make sure a repo is setup (github). On the Dev instance of databricks, go to the Repo tab and point it to the github instance.
Check in the notebooks to the repo.
Come back to Azure Devops website, go to the Pipelines tag and the Releases subtag. Click "Create Release".
The Release pipeline has two parts 1. Artifacts 2. Stages
In the Artifacts, select the repo's branch (e.g. master branch) to be the trigger. Click the 'flash' icon (on both the artifacts and stages)? to select the trigger, e.g. push or pull request. Done
In the Stages, click on the job to create a agent job. Select platform, e.g. windows, linux, etc. Use windows-latest for example.
Then add tasks to the agent job using the + sign and search for proper tasks component. To deploy to databricks
add a Use Python Version 3.x task
add a "Configure Databricks" task, and put in databricks URL and access token.
add "Databricks Deploy Notebooks" task, and put in the Azure region, repo's path, target path in the Workspace and the token.
Done.
Create a pull request to test it. Note, it only needs to approve the pull request to trigger the release. Don't approve and then 'complete' it straight away because the complete would delete the pull request merge where the release pipeline is reading the latest version. When release is finished, then click complete / set an auto complete.
or
Create Release to test it.
Note there are different providers to the databricks tasks, e.g. microsoft and others, some maybe working some maybe not.