Guarda questa intervista per capire gli agenti AI [Alessio Pomaro]

Cost vs. Scale in Production

Even if AI tests are cheap in “cents,” costs can quickly balloon in production. Once an agent runs thousands of interactions/iterations per day, daily spend can reach thousands of euros.

• Higher-performing models can help.
• But “more capable” doesn’t mean perfect understanding—models may not infer anything that wasn’t explicitly provided.

Gap Between Demos and Real Production

A major mismatch exists between quick demos and real engineering work. The speaker argues that most effort is not building the agent/model (demos can be created quickly), but building the application layer, including:

• Defining required data inputs
• Designing tool/data flows
• Ensuring scalability and cost control
• Adding observability (being able to see what’s happening)

Observability Requirement for Agent Systems

For agent-based systems (including multi-agent setups), you must track:

• What tools are used
• What reasoning steps occur
• What exchanges happen between agents

Then apply monitoring/evaluation to detect and diagnose issues.

“Solid” Web Interactions via Web MCP (Google Proposal)

Traditional “agentic web” approaches often rely on a built-in browser plus computer vision to interpret UI elements (e.g., buttons, product sheets, cart actions). This can introduce interpretation and brittleness.

Web MCP aims to make interactions more programmatic/robust by exposing explicit front-end APIs, removing ambiguity about the UI.

Result: websites can remain visually unchanged for humans, while exposing an agent-ready layer so agents can call functions like:

• add-to-cart
• configure

…rather than relying on visual clicking.

How to Evaluate “Solidity” of an Agent (Practical Guidance)

To judge how reliable an agent solution is:

1. Run testing on small datasets to assess:
   • Functionality
   • Robustness
   • Cost
   • Edge cases / attacks
2. Move to production gradually to understand real-world behavior and limits.

Emphasis: rely on internal, real-world benchmarking rather than vendor-reported benchmark tables.

Benchmark Skepticism

The interview reinforces a key idea: “trusting benchmarks” from model/tool providers can be marketing-heavy. The most trustworthy benchmark is internal testing on your real tasks and data.

Prompt Engineering Is Still Important (But Evolves)

The claim that “prompt engineering is dead” is rejected. Even with stronger models, better performance mostly means they can follow detailed instructions more reliably.

• You should still invest in structured, detailed prompts to reduce ambiguity.
• Prompting becomes especially more controllable through system prompts and agent instruction frameworks.

Tools for Structured Prompting

The discussion mentions Digital Director and Digital Art Director systems that:

• Take unstructured ideas
• Ask questions to fill missing details
• Output structured prompts compatible with templates

They’re suggested as particularly useful for visual models, where being objective is harder than with text.

Reduce Downstream Interactions via Better Instructions

Better instructions/context can reduce the number of follow-up iterations, improving convergence speed.

The speaker also recommends:

• Interactive “ask the system questions first”
• Metaprompting: using an AI to review a prompt for gaps/deficits and suggest corrections

Visual Model Control & Replicability (State of the Art)

Current image/video models are approaching:

• Usable outputs
• Useful control
• Replicability (repeatable results via hyper-structured prompts)

However, results still depend heavily on input quality, such as:

• structured prompts
• correct terminology
• storyboard-like instructions

Working on Prompts vs. Interactive Trial Loops for Visuals

A key practice is refining the prompt/instructions so the prompt becomes a reusable asset.

• Iterations inside the visual generation model may not be reproducible.
• The prompt itself can be.

Application Advantage: More Experimentation, Not Just “Same Capability”

A competitive edge isn’t only the ability to generate an artifact. It’s the ability to run many more cheap tests, expanding the creative/search space.

Example mindset:

• Show multiple generated drafts/pilots early to stakeholders
• Instead of spending heavily on only a few iterations

Concrete Token/Cost Optimization Techniques for Agents (Reported Experiments)

Savings can be achieved without switching to a cheaper model, by optimizing the agent application layer:

1. Streamline what goes into the agent loop If MCP-extracted data isn’t needed for decision-making, do extraction/cleanup outside the loop and pass only processed summaries/insights into a smaller/cheaper model.

2. Prune MCP toolsets Some MCPs expose up to ~100 tools. Remove tools you don’t use to reduce context overhead (even function signatures consume space).

3. Optimize payload size If a tool call returns 100 rows by default but you only need 10, instruct/structure calls to request fewer rows.

4. Compress tool outputs Instead of using raw MCP outputs, use function-call wrappers that intercept and compress/clean data before it reaches the agent (token reduction).

Underlying message: don’t “hammer everything with MCP.” Sometimes traditional pipelines (e.g., Python pre-processing) are the right tool.

Irregular Frontier / Reliability Gap

AI can excel at advanced tasks but fail at “elementary for humans” activities.

Proposed responses:

• Know model limits and add compensating external tools (e.g., do calculations in Python, then feed results back)

• Use strong observability

• Use graders (small/cheaper AI evaluators) to detect anomalies at multiple workflow stages and optimize away failure modes

Avoid AI Obsession: Choose the Right Tool

Not every workflow needs AI. Sometimes simple tools (e.g., Canva) or small amounts of code (e.g., “four lines of Python”) solve the problem better.

Main Speakers / Sources

• Alessio Pomaro (computer engineer; Head of AI at Search on Media Group; LinkedIn Top Voice Italy; lecturer; author; main interviewee)
• Interview host: unnamed narrator (“What’s Next” podcast/interview host)