Video summary

Jonathan Frankle, Databricks | Databricks Data+AI Summit 2026

Main summary

Key takeaways

Technology

Summary (tech concepts + product/AI runtime themes)

  • TheCUBE hosts a discussion with Jonathan Frankle (Chief AI Scientist at Databricks) about whether Databricks can act as a “runtime layer” for enterprise AI cognition, emphasizing that intelligence should not just reason, but take action—linking runtime + application layers as a combined system.

Databricks’ “systems” foundation for agents (Lakehouse + Lakebase)

  • Lakebase is positioned as more than a database: it enables agent-safe experimentation via:
    • Separation of storage and compute: agents can create/act on data primitives without paying for compute when not needed—important for agents and reinforcement learning.
    • Branching (GitHub-like for data/workspaces): agents can “fork” database states, test changes, and merge/revert to avoid breaking large enterprise datasets/workspaces (including examples with very large data scale).
  • The framing: this foundation helps agents operate safely over large environments.

Model layer strategy (flexible model choices + custom models)

  • Databricks’ approach prioritizes maximizing choice:
    • Open-source models
    • Access to closed models
    • Databricks/custom models, including products like Agent Bricks powered by custom models
  • Efficiency focus includes making models faster and cheaper, including:
    • Custom RL (reinforcement learning) pipelines
    • Repeated emphasis on moving toward hardware efficiency (“bring models closer to hardware efficiency”)

Evaluation + improvement loop (move beyond hype)

  • Central theme: “once you can measure, you can improve.”
  • Evaluation work is described as enabling:
    • Faster/better/cheaper model behavior
    • Improved customer results via Databricks AI Runtime tooling
  • Example discussed: RL-style results for tasks requiring hundreds of tool/data calls (described as “RAG,” but framed as going beyond standard retrieval due to ~200 database calls), with claims of better cost/speed/quality than closed models for that workflow.

Token efficiency is no longer “token maxing”

  • The conversation challenges token-maxing culture. Frankle agrees the industry has shifted toward:
    • Value
    • Cost
    • Governance
    • Security
  • Recommended approach:
    1. Prove AI can solve the problem (POC) even if cost is higher initially.
    2. Then optimize for efficiency (smaller models, prompt optimization, better harnesses; for high-use cases: RL and fine-tuning open models).

Agent deployment guidance (process, not rigid steps)

  • Frankle rejects a single “1-2-3 step progression.”
  • Core guidance: “Respect AI” and respect humans (agents are powerful but fallible).
  • Practical workflow:
    • Talk to domain users (identify what’s frustrating/boring/tedious)
    • Bootstrap and evaluate
    • Use frameworks integrated into MLflow to structure the improvement cycle
    • Iterate via prompt optimization
    • Keep humans in the loop for quality/oracle feedback so agents improve without harming people’s work

Custom model adoption: when to fine-tune

  • Custom models are presented as a commitment decision:
    • Fine-tune/customize when a use case is stable and will consume significant tokens over time.
    • Don’t start with fine-tuning—prove first using POC + evaluation
  • “Switching costs” framing:
    • If the use case changes or the experiment fails, prefer lower-commitment options (base models + prompt optimization/GEPA + MLflow-managed eval).
    • If committed, expect ongoing maintenance: re-run pipelines as models/hardware improve (“relationships take work”).

Evaluation for agents beyond single-model QA

  • Frankle argues:
    • Even qualitative risks like security can be made measurable.
    • Measurement plus governance/quality improves trust and deployability.
  • Since agents may involve interacting models and model disagreement, evaluation must cover:
    • Robust evaluation across generations/releases
    • Awareness that model upgrades may require prompt/harness changes and sometimes re-training/fine-tuning

Ontology re-framed as agent memory

  • Traditional ontologies (hand-built knowledge graphs) are reframed as:
    • A form of agent memory
    • Derived from data rather than being “hand authored from scratch”
  • Purpose: help agents avoid repeatedly re-discovering semantics, acting as breadcrumbs to speed and improve repeated multi-question interactions.
  • Measurement idea: compare performance with vs. without memory/notes across sequences of queries.

From search/chat to runtime/task execution

  • Frankle contrasts a “chatbot/search mentality” (single answer) with the agent paradigm:
    • Multi-step reasoning
    • Tool use
    • Reinforcement learning
    • Configured execution resembling scheduling/operating system behavior
  • Mentioned products:
    • Knowledge Assistant v2.0: optimized for speed (about 3X faster, slightly higher quality)
    • KARL: optimized for quality (minutes-long, about ~200 tool calls; cheaper than closed models for similar tasks)

Routing + governance of model calls

  • Routing is described as hard but crucial:
    • Selecting the right model(s) for the task
    • Cost control and quality targeting along a Pareto curve
  • Databricks Unity AI Gateway is mentioned as enabling:
    • Control over calls to models
    • Ability to route/swap models
    • Cost control via budgets
  • Routing schemes from major model providers may change or disappear—implying the challenge is persistent.

What Databricks is focused on next

  • Near-term priorities:
    • Cost/value + ROI
    • Governance + security
    • Enabling customers to build and deploy best-in-class agents with:
      • custom models
      • custom RL
    • Reduce trust/deployment barriers through governance and quality

Main speakers/sources

  • Jonathan Frankle — Chief AI Scientist, Databricks
  • John Furrier — Host, theCUBE (Silicon Valley-based tech interview program)

Original video