Jason Choi - Safety Filters for Uncertain Dynamical Systems: Control Theory & Data-driven Approaches

Main ideas / lessons

• Why safety filtering is needed in robotics and autonomy

  • Robots can often perform tasks, but there is a critical gap between what they can do and what they can guarantee safely.
  • Safety is especially important for safety-critical systems (e.g., advanced aerospace vehicles with complex dynamics and evolving configurations).
  • Core research framing: handle 1) complicated uncertain/high-dimensional dynamics, and 2) the growing use of data-driven methods for decision-making, while still guaranteeing safety.

• What a “safety filter” does (conceptually)

  • A safety filter is a final layer in a decision pipeline.
  • It receives a desired/nominal control (from learning, MPC, human command, etc.).
  • If the desired action could be unsafe, the filter overrides it with a safe alternative.
  • Goal structure:
    1. Minimize deviation from the desired control (performance consideration).
    2. Enforce safety constraints so the state remains safe for all time.
    3. Ensure the selected control is physically feasible (belongs to an admissible control set).

• Two fundamental subproblems in designing safety filters

  1. Safe set verification / computation
     • Determine a subset of the designer’s specified safe region that is actually provably safe.
     • Key difficulty: some states inside the safe region can still lead to failure later.
  2. Control design to enforce safety
     • Construct (or enforce via constraints/optimization) a control policy that keeps trajectories inside the verified safe set.

---

Methodologies and design principles

1) Control-theoretic approaches to safety filters (three main families)

A. Reachability-based safety filters (Hamilton–Jacobi viewpoint)

• Purpose

  • Compute a set of states from which safety can be ensured over a time horizon despite control choices.

• Key objects

  • Let the unsafe/failure region be ( \mathcal{T} ).
  • Compute an (inevitable) reachability set capturing states that will inevitably reach failure within the horizon.
  • The complement yields a viable kernel: states from which there exists a control strategy that avoids failure (at least for the horizon).

• Maximal control invariant safety

  • Extend the horizon to infinity to obtain an infinite-horizon maximal control invariant set.

• How the filter is applied

  • Use the value function (from the Hamilton–Jacobi PDE) to evaluate proximity to the boundary.
  • If near the boundary (value near 0 / about to violate), override the desired control with the optimal reachability-based safety control.
  • If safely inside, apply the desired control.

• Practical limitations highlighted

  • Poor scalability to high-dimensional systems.
  • Chattering behavior (abrupt switching between desired and safety control), which can be unsafe/unrealistic with actuator delays.

B. Control Barrier Function (CBF) safety filters (QP-based)

• Core idea

  • Replace “hard stop at boundary” behavior with a smooth safety mechanism.
  • Ensure safety before the boundary is hit by constraining the evolution of a barrier function.

• Invariance concepts

  • Start from Nagumo-type conditions (forward invariance) but adapt to control by requiring existence of control actions on/near the boundary.

• CBF definition sketch

  • Use a continuously differentiable function (B(x)) so that enforcing a specific inequality guarantees forward invariance of the safe set.
  • At the boundary, the CBF constraint reduces to the Nagumo condition.
  • Use a class-(\mathcal{K}) comparison function (often with parameter (\gamma)) to create an “exponential underbound” to keep trajectories away from violation.

• Optimization implementation

  • For control-affine systems (dynamics linear in control), the CBF inequality becomes linear in the control input.
  • Combine with a quadratic objective (stay close to desired control) and constraints → results in a fast online Quadratic Program (QP).

• Role of (\gamma) (tradeoff)

  • Smaller (\gamma): more conservative / restrictive.
  • Larger (\gamma): less conservative but can become more myopic (intervenes mainly close to the boundary).

• Key caveat

  • Guarantees depend on rigorous correctness of the CBF model/derivation.
  • If control bounds exist, validating feasibility of the CBF condition can be complex and may resemble reachability-like difficulty.

• Behavioral advantage

  • Produces smoother control intervention than reachability-based methods (less chattering).

C. Model Predictive Control (MPC) safety filters (terminal set + feasibility)

• Setup

  • Discretize dynamics and solve a predictive optimization over a horizon.

• Problem structure (three constraint themes)

  • Minimize deviation from desired control (performance).
  • Enforce:
    1. Target safety constraints (avoid unsafe states).
    2. Input constraints (actuator limits).
    3. Terminal set constraint (S_T).

• Why terminal sets matter

  • Choose a terminal set (S_T) that is control invariant.
  • If at the end of the horizon the trajectory is inside (S_T), invariance ensures it can remain safe forever afterwards.

• Safety mechanism

  • If the MPC problem is feasible, the controller has a backup plan that preserves safety long-term.

• Key caveat: horizon length

  • Short prediction horizon → terminal constraint becomes more restrictive → safe set shrinks and becomes more “myopic.”

• Safety-set representation

  • Often implicit: whether a state is in the safe region is inferred from whether the MPC is feasible from that state.

---

2) Model uncertainty and data-driven safety filters (GP-CBF / “mod-aware”)

• Problem

  • Safety filters derived from a nominal model ( \tilde{f}, \tilde{g} ) can fail when true dynamics ( f, g ) differ.

• Illustrated failure

  • A CBF-QP designed using a manufacturer-provided model may appear correct in simulation, but rollout can leave the CBF safe set.

• Proposed fix: incorporate data-driven learning of uncertainty

  • For a GP-CBF safety filter:
    • Rewrite the CBF constraint to include an unknown uncertainty term capturing mismatch between true and nominal dynamics.
    • Use a Gaussian Process (GP) to learn this uncertainty term from experimental data.

• Chance-constrained safety

  • Replace hard constraints with probabilistic (chance) constraints.
  • Use GP confidence bounds so that with high probability, the uncertainty term lies within a bound; then enforce safety using a worst-case bound.

• Computational tractability

  • Exploit structure (uncertainty term is control-affine / control-dependent in a particular way) so the resulting constraint can be cast as a Second-Order Cone Program (SOCP), preserving online efficiency.

• Feasibility / visibility analysis

  • GP uncertainty introduces conservatism; constraints can become infeasible if GP uncertainty (variance) is too large.
  • Emphasized principle: an explicit tradeoff between safety and uncertainty.

• Online learning / recursive feasibility

  • The method may choose exploratory yet still safe controls to gather data and reduce uncertainty, aiming for recursive visibility (maintaining feasibility as learning progresses).

• Scalability enhancement

  • Addresses GP inference cost scaling via online data selection so runtime scales more favorably (reducing complexity dependence, as described).

---

3) Discriminating Hyperplane (DH) safety filters for control-affine systems

• Motivation

  • Certificate-function-based methods (CBFs, reachability value functions) are powerful but:
    • Safety constraint design may be underdetermined (many valid certificates lead to different behaviors).
    • It can be awkward/inefficient to craft and tune certificate functions.
  • Goal: generalize/abstract safety constraints without relying on a chosen certificate function for constraint design.

• Key geometric concept

  • Start from the Nagumo condition idea: at boundary points, safety requires selecting control directions that “point into” the safe set.
  • Extend by focusing on a state-dependent discriminating hyperplane in the control input space:
    • Valid controls lie on one side of the hyperplane.

• Core abstraction

  • Learn or compute a hyperplane that partitions controls into:
    • safe vs unsafe actions (as a sufficient condition for safety at the boundary, then extended inward).

• Learning approaches described

  • Supervised learning
    • Requires a pre-verified control invariance set to generate labels.
  • Reinforcement learning (RL)
    • Does not require pre-verified invariance; trains while using the DH filter.

• Empirical messages from examples

  • In training tasks, vanilla RL/PG can violate safety frequently.
  • DH-filter-based training greatly reduces safety violations.
  • For Half Cheetah (more “black box” dynamics):
    • Vanilla learning can become aggressive and fall into unsafe behaviors.
    • Safety-only baselines can become overly conservative and reduce performance.
    • DH safety filter enables aggressive task behavior while preventing catastrophic unsafe events by intervening when unsafe conditions are imminent.

• Takeaway

  • DH safety filtering is presented as a general abstraction of safety constraints for control-affine systems, enabling learning without explicit certificate design.

---

Wrap-up / overarching themes

• All three classical methods (reachability, CBF, MPC) share a structural pattern:
  • Safety relies on certificates (value/barrier functions, or terminal invariance sets).
• Under uncertainty:
  • Safety requires robust/probabilistic formulations or data-driven augmentation to maintain guarantees.
• DH filtering provides an alternative abstraction:
  • It generalizes safety constraints via control-space hyperplane separation, potentially reducing reliance on explicit certificate construction.

---

Speakers / sources featured

• Jason Choi (speaker; PhD student at UC Berkeley; collaborator/author mentioned)
• Siy[a] (host/instructor; welcomes Jason and moderates questions)
• Claire Tomlin (co-advisor mentioned)
• Caltech Sho (spelled in subtitles as “Cal Sho”; co-advisor mentioned)
• Melanie Zinger (advisor/collaborator mentioned)
• Aaron Ames (advisor/collaborator mentioned)
• Kim Weish (collaborator mentioned for reachability/overall control-theoretic part)
• Andrew Taylor (spelled in subtitles as “Andrew tayor”; professor/collaborator mentioned)
• Professor names (appears as “Professor names”; context suggests an additional collaborator/institution member)
• Fernando and on and B (collaborators mentioned for GP-CBF uncertainty project; names partially obscured in subtitles)
• Armen (co-collaborator mentioned; also referenced as having given a prior lecture on related concepts)
• Audience members (asked questions via the chat/interruptions; no names provided)