Bongkar AI Workflow Eps. 1 - Asep Bagja

High-level theme

A live interview and demonstration showing how a developer (Mas Asep) integrates large language models (LLMs) and agent tooling into everyday software workflows. The session covers concrete tooling choices, architecture patterns, and practical trade-offs (latency, tokens, security, review).

Guest background & projects

• Mas Asep: full‑time web/dev engineer.
• Projects: music‑tech product Nanas (nanas.ee), various app‑store apps.
• Activities: conference attendance across Europe.
• Notable achievement: won an Anthropic/cloud hackathon with a real‑time AI “bandmate” that generates MIDI tracks.

Key technical concepts & architecture

MIDI as a data protocol

• Uses standard MIDI messages (note on/off, velocity, control change).
• MIDI 1.0 values 0–127; MIDI 2.0 exists but is not widely adopted.

Real‑time AI music generation pipeline

• Input: live MIDI stream from an instrument.
• First layer: local WASM rule‑based engine to ensure deterministic, low‑latency behavior and to provide a fallback when the LLM is slow.
• Every ~8 seconds the engine batches ~4 bars and sends them to an LLM (Opus‑46) to generate new MIDI tracks; the LLM returns MIDI‑like data (added tracks, harmonies).
• Latency considerations: first token ~15 ms typical; batching plus the WASM rulebase avoids perceptible lag.

Multi‑agent behavior

• Cloud spawns subagents automatically for tasks (e.g., code exploration).
• A liberal spawn policy allows multiple subagents to edit files concurrently.
• Asep mainly relies on default subagents rather than hand‑authoring many custom ones.

Token & model strategy

• Multiple accounts (personal, work, API/hackathon credits) to separate billing/token pools.
• Model selection by task:
  • Opus‑46: heavy planning / creative work (music generation).
  • Sonnet: cheaper model for routine coding to save tokens.
  • GLM: occasional backup.
• Uses a “high effort” parameter selectively (useful for deep exploration but can “overthink” simple tasks).
• Often uses a “dangerously skip permission” option to bypass repeated confirmations (convenience vs risk trade‑off).

Deployment & development environment

• Development on a VPS (example: 8 GB RAM, 4 cores) to avoid local disk/storage issues for heavy frameworks (Next.js, etc.).
• Git workflow:
  • develop branch used as staging.
  • Cloud code auto‑generates plans and can create branches/PRs.
  • PRs previewed on Vercel.
  • Merges remain manual to preserve review control.
• Cloud workspace can include multiple directories/repos so agents can read and edit across projects.

Tool integrations & ecosystem

• Cloud Code / Anthropic “cloud” features: central tooling (cloud.md, plan mode, agent system).
• Vercel: PR previews and deployments.
• Design integrations: Figma / Sketch + MCP (Model Context Protocol); designers instructed to name layers and structure files for LLM use.
• Xcode MCP integration: cloud code can communicate with a local Xcode MCP server for iOS projects.
• Graptil / Graptile: automated code review bot integrated with PRs (useful for triage, some false positives).
• Cursor / Copilot: historically used for autocomplete; cloud code reduced dependence on some of these features.

Security & QA

• Cloud exposes a security‑review endpoint (e.g., /security) that lists issues and proposes fixes; developer approval typically required.
• Asep prefers functional/manual testing and human review before merging; automated reviews help but do not replace humans.

Practical tips & observations

• Keep cloud.md short and focused (rules and essential instructions). Long cloud.md files bloat context and harm agent effectiveness.
• Follow a plan → execute → PR → preview → merge workflow; prefer manual merge to ensure staging checks pass.
• Use WASM/local rulebase for real‑time systems that cannot wait for full LLM round trips.
• Save tokens by using cheaper models for mundane tasks and reserve expensive models for creative/planning phases.
• Maintain manual coding practice: keep some pet projects coded without AI to retain debugging and deep understanding skills.
• When using design assets (Figma), have designers name layers and structure files properly so LLMs can use them reliably.

Guides / workflow steps (Asep’s process)

1. Write a short request / problem statement in cloud.md (or prompt).
2. Cloud auto‑generates a plan (plan mode) and saves it to the repo directory.
3. Cloud/agents execute the plan (spawn default subagents as needed).
4. Generated changes are put in a feature branch and a PR is created.
5. Deploy the PR to Vercel for preview and manual verification.
6. Human merges the PR into develop/main and optionally cleans up branches.
7. Use Graptil/automated review to surface issues; run the security check endpoint and decide on fixes.

Tools & models mentioned

• Opus‑46 (music generation, high‑effort tasks)
• Sonnet (cheaper model for coding)
• GLM (backup)
• WASM rulebase (local, low‑latency fallback)
• Cloud Code / Anthropic “cloud” features (cloud.md, plan mode, agent system)
• Vercel (preview/deployment)
• Figma, Sketch, MCP (Model Context Protocol)
• Xcode MCP integration
• Graptil / Graptile (automated code review)
• Cursor, Copilot (autocomplete/history)

Trade‑offs called out

• Speed vs safety: skipping permission flows speeds development but increases risk of regressions.
• Automation vs human checks: agents accelerate feature creation (hours vs days) but manual testing and human merging remain important.
• Cost vs capability: high‑quality models cost tokens; multi‑account strategies and cheaper models mitigate cost.
• Context length: keep orchestration docs short; store long architecture docs separately.

Practical examples covered

• MIDI bandmate demo: live MIDI in → WASM rulebase → LLM adds extra MIDI tracks in real time → output to audio engine.
• VPS dev setup and using SCP to copy screenshots from a local Mac into the VPS project folder.
• Using cloud.md to store rules and trigger automatic plan generation.

Main speakers / sources

• Host: Imre (Imbregi)
• Guest: Mas Asep (Asep Bagja)