Video summary

Discover what’s next: The biggest updates from WWDC26 | Meet with Apple

Main summary

Key takeaways

Technology

Tech recap of WWDC26 (Meet with Apple) — main updates and how they fit together

Core AI “intelligence” (multi-model foundation + system integration)

Single “front door” for AI models via Foundation Models

  • One common API so app code can target different model types:
    • on-device models
    • Private Cloud Compute models
    • frontier models
    • custom local models
  • On-device intelligence
    • private and offline-capable
    • fast performance on Apple silicon
    • no inference bill
  • Private Cloud Compute
    • used for heavier reasoning
    • designed to preserve privacy
  • Open-source direction
    • the framework will support open-source availability
  • Swift APIs everywhere
    • the same Swift interface can be used server-side for end-to-end AI workflows
  • MLX support
    • enables custom model workflows
    • integrates into broader application patterns

Apple Intelligence models through Foundation Models

  • Guided generation
    • prompt/response flow where output structure is controlled and type-safe with Swift
  • Tool calling
    • connect to live app/system data (e.g., weather, calendar) and perform actions
  • Custom instructions
    • define role/style/behavior
  • Streaming responses
    • support real-time UI updates
  • Vision-capable prompts
    • text + images for tasks like captions and landmark/visual understanding

Bring your own models with on-device frameworks

  • Core AI
    • deployment/runtime toolkit for custom models optimized for Apple silicon (CPU/GPU/Neural Engine)
    • benefits:
      • privacy (local execution)
      • offline availability
      • no per-inference developer cost
    • full lifecycle support:
      • optimization/conversion/inference/tuning/debugging
      • integration into apps
  • MLX
    • open-source framework for research, training, fine-tuning, and running local inference servers
    • supports distributed workloads across multiple Macs
    • efficient on Apple silicon (including GPU acceleration via Metal and unified memory optimization)
    • ecosystem compatibility mentioned (examples):
      • Ollama
      • LM Studio
      • VLM

Agentic experiences (multi-step + multi-agent orchestration)

  • Shift from single prompt/response to goal-driven agents that can:
    • call tools
    • act on outcomes
  • Example:
    • travel app uses a planner agent (calendar + on-device model)
    • a booking agent (server model + booking tool)
  • Dynamic profiles (new capability)
    • preserves shared context across a continuous session
    • retains transcript (instructions, prompts, responses, tool calls/outputs) while switching models/capabilities
    • balances privacy, performance, and cost per stage via different active profiles

Evaluations framework (quality assurance for unpredictable outputs)

  • Recognizes that LLM outputs can be hard to validate using normal unit tests
  • Provides:
    • defined metrics
    • automated grading (including LLM-based evaluators)
    • trend analysis across large datasets
    • optional synthetic sample generation for broader coverage
  • Designed to work with Foundation Models so quality checks stay consistent regardless of model choice

Integration with the system (Siri, Spotlight, widgets, live activities)

Make apps “discoverable” across system surfaces

  • Uses App Intents / App Intents framework
    • Entities = “nouns” (content items like events, messages, photos)
    • Intents = “verbs” (actions like send, reply, create, open, etc.)
  • App Schema domains
    • Apple-provided blueprints (system/phone/mail/clock/calendar/camera/photos, etc.)
    • allow Siri/Apple Intelligence to reason about app/system concepts consistently
  • View annotations
    • system maps on-screen UI elements to entities
    • enables actions like “open the third event”
  • Cross-app actions
    • Siri can combine entities/actions across multiple apps in one request
    • example: pull attendees from a calendar app and send via Mail
  • Interaction donations
    • learns from how users operate in your UI
    • helps disambiguate app choice and predict next actions

Siri rebuilt with Apple Intelligence

  • Introduced as a new interaction model featuring:
    • personal context (e.g., understands your calendar event list)
    • on-screen awareness (references what’s currently displayed)
    • multi-app execution in a natural conversation flow

Additional system surfaces mentioned

  • Widgets
  • Live Activities
  • Control Center
  • Action Button
  • Spotlight Search
  • (and other integration points in the same ecosystem)

Adaptability & design system updates (Liquid Glass + universal app resizability)

Liquid Glass evolves

  • improved readability via better diffusion
  • clearer separation using:
    • “darken edge”
    • brighter specular highlights
  • User personalization
    • a slider from ultra clear → fully tinted
  • Apps using Liquid Glass get improvements automatically without recompiling

iPhone app resizability (major UX/product behavior change)

  • iPhone apps can run responsively on iPadOS
  • and via iPhone mirroring on Mac
  • apps must handle multiple canvas sizes:
    • side-by-side layouts
    • external display layouts
  • described as “universal resizability” (one app adapts to whatever size it’s given)

Design guidance for resizability

  • Recompile with Xcode 27
    • SwiftUI handles resizability
    • no new API adoption required (per the recap)
  • Use:
    • Size Classes
    • system components for graceful layout adaptation
  • New visibility priority behavior
    • determines what stays visible vs moves to overflow (toolbars/menus)

Developer tools: Xcode 27 “agentic coding”

Coding agents across the full workflow

  • Agents added across:
    • build
    • debug
    • test
    • distribute
  • Agents work with your preferred model(s), with integrations mentioned for:
    • Anthropic (Claude)
    • OpenAI (Codeex)
    • Google (Gemini)

Agent protocol + context integration

  • Agent client protocol
    • supports custom agents/models
    • includes local models on the Mac
  • Model context protocol
    • connects to tools like Figma and GitHub

Agent workflow features

  • Plan mode
    • agent drafts a step-by-step plan before code is written
    • user can revise
  • Inline annotations
    • fine-grained context and feedback
  • Documentation-aware coding
    • local optimized Apple docs archive
    • framework search for latest APIs
  • Previews
    • visually verify and generate multiple UI directions incrementally
  • Localization automation
    • translates full apps while preserving contextual/UI/pluralization details
  • Testing automation
    • runs existing suites automatically
    • can generate/modernize tests
  • Debugging support
    • crash fixing assistance
    • performance assessment
    • debugging with tools like LLDB

Security controls

  • agent file access requires explicit permission
  • protected by macOS security

App Store & monetization updates (discovery, subscriptions, automation)

Discovery/marketing enhancements

  • New creative assets
    • large images/videos may appear on product pages and search results
  • assets can be approved without submitting a new binary
  • App Store Connect tools:
    • Product page previews
      • screenshots, header/icon, descriptions
      • iPhone/iPad variants
      • multiple languages
      • light/dark mode
    • Asset library
      • manage and track approved assets and where they apply
  • Personalized collections
    • tailored topic collections based on user interests

In-app purchase / StoreKit improvements

  • subscription retention messaging (churn prevention)
  • subscription bundles
    • cheaper than purchasing separately
    • can include multi-app bundles and partner bundles
  • subscription suites
    • multiple related apps under the same developer account
  • new subscription purchase group models:
    • Group purchases (invite others)
    • Volume purchasing via Apple School Manager / Apple Business
  • 12-month commitment option
    • auto-renewable subscriptions with a lower monthly rate
  • Batch submission improvements
    • submit in-app purchase items, app updates, or in-app events together in a single review submission

App Store Connect API updates

  • API supports:
    • the new subscription/12-month commitment option
    • submission enhancements
    • asset library
  • changes to asset download management to reduce installation time

Practical workflow guidance

  • Use creative asset approvals for campaigns
  • leverage retention messaging, bundles/suites, and group purchases
  • automate via App Store Connect API

Main speakers / sources

  • Josh Tidsbury (Apple, Technology Evangelism) — overall recap and framing
  • Shashank (Apple, Technology Evangelism) — Intelligence: Foundation Models, Core AI, MLX, agents, evaluations
  • Alan Schaefer (Apple, Technology Evangelism) — Integration: Siri + App Intents, system experiences
  • Maho / Maho (Design Evangelist) — Design updates: Liquid Glass + iPhone resizability, system components
  • Angelica (Apple, Technology Evangelism) — Developer tools: Xcode 27 agentic coding
  • Latika (Product marketing, marketplace technologies/app store) — App Store updates: discovery, subscriptions, StoreKit, App Store Connect API

Original video