Connect Your Development Environment to MLflow
This guide shows you how to connect your development environment to an MLflow Experiment. You can run MLflow on your local machine, self-host the open source MLflow service, or use a managed offering, such as Databricks Managed MLflow.
Prerequisites
- OSS MLflow
- Databricks
- Python Environment: Python 3.9+ with pip installed
- Databricks Workspace: Access to a Databricks workspace
This guide describes using a Databricks Personal Access Token. MLflow also works with the other Databricks-supported authentication methods.
Setup Instructions
- OSS MLflow
- Databricks - Local IDE
- Databricks - Notebook
Step 1: Install MLflow
pip install --upgrade "mlflow>=3.1"
Step 2: Configure Tracking
MLflow supports different backends for tracking your experiment data. Choose one of the following options to get started. Refer to the Self Hosting Guide for detailed setup and configurations.
Option A: Database (Recommended)
Set the tracking URI to a local database URI (e.g., sqlite:///mlflow.db
). This is recommended option for quickstart and local development.
import mlflow
mlflow.set_tracking_uri("sqlite:///mlflow.db")
mlflow.set_experiment("my-genai-experiment")
Option B: File System
MLflow will automatically use local file storage if no tracking URI is specified:
import mlflow
# Creates local mlruns directory for experiments
mlflow.set_experiment("my-genai-experiment")
File system backend is in Keep-the-Light-On (KTLO) mode and will not receive most of the new features in MLflow. We recommend using the database backend instead. Database backend will also be the default option soon.
Option C: Remote Tracking Server
Start a remote MLflow tracking server following the Self Hosting Guide. Then configure your client to use the remote server:
import mlflow
# Connect to remote MLflow server
mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("my-genai-experiment")
Alternatively, you can configure the tracking URI and experiment using environment variables:
export MLFLOW_TRACKING_URI="http://localhost:5000"
export MLFLOW_EXPERIMENT_NAME="my-genai-experiment"
Step 3: Verify Your Connection
Create a test file and run this code:
import mlflow
# Print connection information
print(f"MLflow Tracking URI: {mlflow.get_tracking_uri()}")
print(f"Active Experiment: {mlflow.get_experiment_by_name('my-genai-experiment')}")
# Test logging
with mlflow.start_run():
mlflow.log_param("test_param", "test_value")
print("✓ Successfully connected to MLflow!")
Step 4: Access MLflow UI
If you are using local tracking (option A or B), run the following command and access the MLflow UI at http://localhost:5000
.
# For Option A
mlflow ui --backend-store-uri sqlite:///mlflow.db --port 5000
# For Option B
mlflow ui --port 5000
If you have the remote tracking server running (option C), access the MLflow UI at the same URI.
When using the remove tracking server, you may hit an access denied error when accessing the MLflow UI from a browser.
Invalid Host header - possible DNS rebinding attack detected
The common cause for this error is the tracking server does not allow requests from your origin to prevent clickjacking attacks. You need to set up an allowlist of CORS origins in the tracking server configuration in this case. Refer to the Network Security Guide for more details.
Step 1: Install MLflow
Install MLflow with Databricks connectivity:
pip install --upgrade "mlflow[databricks]>=3.1"
Step 2: Create an MLflow Experiment
- Open your Databricks workspace
- Go to Experiments in the left sidebar under Machine Learning
- At the top of the Experiments page, click on New GenAI Experiment
Step 3: Configure Authentication
Choose one of the following authentication methods:
Option A: Environment Variables
- In your MLflow Experiment, click Generate API Key
- Copy and run the generated code in your terminal:
export DATABRICKS_TOKEN=<databricks-personal-access-token>
export DATABRICKS_HOST=https://<workspace-name>.cloud.databricks.com
export MLFLOW_TRACKING_URI=databricks
export MLFLOW_EXPERIMENT_ID=<experiment-id>
Option B: .env File
- In your MLflow Experiment, click Generate API Key
- Copy the generated code to a
.env
file in your project root:
DATABRICKS_TOKEN=<databricks-personal-access-token>
DATABRICKS_HOST=https://<workspace-name>.cloud.databricks.com
MLFLOW_TRACKING_URI=databricks
MLFLOW_EXPERIMENT_ID=<experiment-id>
- Install the
python-dotenv
package:
pip install python-dotenv
- Load environment variables in your code:
# At the beginning of your Python script
from dotenv import load_dotenv
# Load environment variables from .env file
load_dotenv()
Step 4: Verify Your Connection
Create a test file and run this code to verify your connection:
import mlflow
# Test logging to verify connection
print(f"MLflow Tracking URI: {mlflow.get_tracking_uri()}")
with mlflow.start_run():
print("✓ Successfully connected to MLflow!")
Step 1: Install MLflow
Databricks runtimes include MLflow, but for the best experience with GenAI capabilities, update to the latest version:
%pip install --upgrade "mlflow[databricks]>=3.1"
dbutils.library.restartPython()
Step 2: Create a Notebook
Creating a Databricks Notebook will create an MLflow Experiment that is the container for your GenAI application. Learn more about Experiments in the MLflow documentation.
- Open your Databricks workspace
- Go to New at the top of the left sidebar
- Click Notebook
Step 3: Configure Authentication
No additional authentication configuration is needed when working within a Databricks Notebook. The notebook automatically has access to your workspace and the associated MLflow Experiment.
Step 4: Verify Your Connection
Run this code in a notebook cell to verify your connection:
import mlflow
# Test logging to verify connection
print(f"MLflow Tracking URI: {mlflow.get_tracking_uri()}")
with mlflow.start_run():
print("✓ Successfully connected to MLflow!")
Next Steps
Now that your environment is connected to MLflow, try the other GenAI quickstarts:
- Instrument your app with tracing: Follow the quickstart to instrument your first GenAI app
- Evaluate your app's quality: Use the evaluation quickstart to systematically test and improve your app's quality