MLFlow Tutorial | ML Ops Tutorial

Purpose / Problem Solved

• MLflow centralizes experiment tracking, model management, reproducibility, and deployment so teams stop relying on ad-hoc Excel/Google Sheets and manual file transfers.
• Solves versioning for:
  • Experiments (parameters, metrics)
  • Artifacts (model files, requirements)
  • Model selection, registration, and staging (dev → prod)
  • Integration with CI/CD and cloud platforms

What the tutorial covers (high-level)

1. Installing and running MLflow locally

• Install: pip install mlflow
• Run the UI: mlflow ui (default: http://localhost:5000)
• In code: set the tracking URI to point to the MLflow server (e.g., mlflow.set_tracking_uri(...))

2. Experiment tracking (core workflow)

Typical sequence:

1. mlflow.set_experiment(...) and mlflow.set_tracking_uri(...)
2. with mlflow.start_run():
   • mlflow.log_params(...) — store model parameters
   • mlflow.log_metrics(...) or mlflow.log_metric — store evaluation metrics
   • mlflow.log_model(...) — store trained model artifacts
   • mlflow.log_artifact(...) — store extra files

Practical tips:

• Use sklearn.metrics.classification_report(output_dict=True) to capture accuracy/recall/F1 as a dict and log those metrics.
• Run multiple experiments in a loop (vary models, hyperparameters, preprocessing like SMOTE) to populate MLflow runs.
• Give runs explicit names (or rely on auto-generated names).

MLflow UI features:

• Tabular and chart views of runs
• Metric plots, scatter plots, and filters
• Visual run comparisons (e.g., compare recall for minority class across runs)
• Download artifacts (model pickles/XGBoost files), requirements.txt, and conda.yaml for packaging

3. Handling class-imbalance example

• Demonstrated models: Logistic Regression, Random Forest, XGBoost, and XGBoost + SMOTE oversampling
• Showed trade-offs between recall and precision for different approaches
• MLflow visual comparisons help pick the best model based on business metrics (e.g., recall for minority class in anomaly detection or fraud)

4. Model Registry / Model Management

• Registration options:
  • Register during log_model(...) (immediate)
  • Register later via the register_model API
• Model URI formats:
  • Run-based: runs:/<run-id>/model (run-id from run details)
  • Registry-alias: models:/<model_name>/<version_or_alias>
• Versioning: each registration creates versions (v1, v2, …)
• Aliases / stages: use tags/aliases like Challenger / Champion or stages Dev → Production
• Loading a registered model:
  • mlflow.<flavor>.load_model(...)
  • Or use models:/MyModel/Champion to avoid hardcoding version numbers
• MLflowClient / API: programmatically transition or copy model versions between stages and integrate with pipelines

5. Packaging and deployment notes

• Artifacts typically include model files, requirements.txt, and conda.yaml to simplify containerization and deployment (Docker)
• MLflow supports deployment and integration with Databricks, cloud providers (AWS), and CI/CD tools (e.g., Jenkins)
• The tutorial mentions packaging a model and deploying it to production environments

6. Centralized tracking with DagsHub (cloud-hosted MLflow)

Motivation:

• Multiple team members need a shared MLflow server and dataset versioning

Workflow:

1. Create a GitHub repo and push notebook/code
2. Create a DagsHub account and connect the GitHub repo (DagsHub can version code + data)
3. Install the client: pip install dsub and set the DagsHub tracking URI in code
4. Set environment variables / API key (username + public key/token) for authentication if required
5. Publish experiments/metrics to the DagsHub-hosted MLflow server; runs and registered models appear in the cloud UI

Benefits:

• Central experiment visibility
• Dataset versioning
• Collaborative workflow

Practical troubleshooting / tips

• If you see errors like “experiment ID doesn’t exist”: rerun the cell or recreate the experiment.
• Ensure model URI format is exact when registering/loading models; using aliases avoids hardcoding version numbers.
• For DagsHub API errors: check cached credentials and set required environment variables (username, token, tracking URI).
• MLflow documentation is comprehensive — consult it for deeper features and edge cases.

Resources / Code Availability

• The tutorial uses Jupyter notebooks and code provided by the presenter (noted available in the video description).
• Strong recommendation to read the MLflow documentation and the provided notebooks for hands-on learning.

Main speakers / Sources

• Video instructor / tutorial author (unnamed in the transcript) — provides a hands-on MLflow + DagsHub walkthrough and code examples
• Referenced tools and docs: MLflow documentation, DagsHub platform, GitHub, and cloud/deployment references (Databricks, AWS, Jenkins)