01.10 Algorütm: AI tuleb panka. Kes vastutab, kui mudel eksib?

Episode Overview: “01.10 Algorütm: AI tuleb panka. Kes vastutab, kui mudel eksib?”

The episode focuses on how large language models (LLMs) are being integrated into bank software development—and, crucially, how responsibility, risk, security, and governance should work when models make mistakes.

Key Takeaways

1) LLMs can fit into banks’ change culture—if integrated carefully

Banks tend to adopt new technology slowly due to stability and security requirements. However, major strategies are shifting toward integrating LLM capabilities into existing systems rather than treating them as experimental add-ons.

2) Code-development adoption is already happening, but data protection gates it

Initial interest in tools like coding copilots was cautious, but LLM usage is now appearing more commonly in normal development workflows. This is only possible after investments in safeguards to prevent data leakage.

3) The real shift is from “prompt and done” to “context and process engineering”

Even if developers type less code, teams still need to build reliable context—rules, constraints, and business/legacy logic that LLMs must follow.

Experiments converting legacy code showed that:

• much time goes into iterative prompting,
• review and correction are necessary,
• human workflows and psychology matter (some people lose patience and prefer writing code themselves).

4) Humans don’t disappear; responsibilities and workflows change

Panelists argue LLMs can assist with routine tasks like drafting, information gathering, and compliance support. But they cannot fully replace accountable experts, because LLMs are not deterministic and can “hallucinate” plausible but incorrect answers.

5) Regulatory pressure increases the need for stronger review and risk controls

Stricter AI/software regulations intensify questions such as:

• Who approves generated code?
• Who is responsible if it’s wrong?

While LLMs can speed up compliance searches and summarization, a human expert must still validate final outputs. As a result, bottlenecks may shift from coding to:

• security checks,
• vulnerability analysis,
• expert reviews.

6) Governance requires evaluation frameworks and practical “guardrails”

Instead of relying only on generic benchmarks, banks need assessment metrics and frameworks that test LLM performance within their own environment and within defined risk boundaries. Models and methods should be selected based on acceptable risk.

7) Organizational control functions will evolve like DevOps/platform functions

The episode predicts growth of dedicated “LLM teams” (or similar AI/platform engineering competencies) to manage:

• pipelines,
• monitoring,
• controls,
• secure release integration.

This helps prevent bottlenecking on a small specialist group.

8) External vs. internal models depends on confidentiality and architecture

Banks may refuse to send highly classified data to third-party cloud models—even when protections exist. In those cases, panelists highlight:

• local/isolated infrastructure,
• university experiments using smaller “Gemma/Kenma”-type models,
• masking sensitive data where needed, and/or
• running models internally to reduce external exposure.

9) Masking data—and even masking structure—is complex

Beyond hiding sensitive fields, banks may also need to obscure structure patterns (for example, replacing named fields with generic placeholders). This can require a large engineering pipeline, and errors in identifying structure can still occur.

An alternative considered is more isolated internal execution, where some masking burdens can be reduced.

10) Trust involves more than accuracy—also autonomy, privacy, and threat actors

The panel emphasizes that companies and individuals will need to define where trust boundaries should be for agentic use (LLMs that act autonomously via calendars, email, and systems). Convenience increases risk, and attackers can also leverage LLMs effectively.

In banking contexts, data leakage and surveillance concerns make caution essential.

11) “Future view”: LLMs are powerful tools, not general intelligence

Progress is framed as tool-driven improvement rather than proof of general intelligence. Near-term advancement centers on better integration patterns, such as:

• deterministic parts for verifiable operations (e.g., SQL generation + validation),
• limiting LLM uncertainty to bounded steps.

12) Practical workflow philosophy: pair LLM creativity with deterministic verification

A suggested pattern is:

1. let the LLM help with planning or generating a query/report,
2. validate results with deterministic checks,
3. run outputs through traditional systems.

This preserves accountability by making the “responsibility surface” visible and auditable.

Presenters / Contributors

• Martin Kapp (host/presenter)
• Eerik Jõgi (co-host/presenter)
• Kristiina (guest; software architect from SEB Pank, also referenced as working at/with SCB in the intro)