Summary of "The AI Upskill Most Teams Are Overlooking | Gartner CIO Leadership Forum"

High-level summary

Core message: In the AI era, the highest-impact upskill many organizations are overlooking is human-first “soft” skills — critical thinking, judgment, contextual awareness, skepticism, and the ability to prompt and interpret models. These skills are the differentiator between AI value and AI failure and must be trained, measured and operationalized across the enterprise (not left to HR alone).

Mandi Bishop (Gartner) recommends operationalizing a “Soft Skills Academy” with exercises, scoring rubrics and maturity models that integrate with hiring, training, product sprints and governance to reduce AI risk and increase value realization.

Frameworks, playbooks and processes

Soft Skills Academy
- Components: structured exercises, scoring rubrics, maturity models, documentation and repositories.
- Use cases: hiring assessments, continuous training, role-specific enablement.
REFLECT prompt framework (deconstructing prompts)
- Elements to specify in prompts: Role, Format, Language, Example, Context, Task.
- Purpose: create precise, low-iteration prompts.
Hallucination Hunt (automation-bias busting exercise)
- Purpose: adversarial testing of model outputs to surface hallucinations, compliance gaps and contextual misses.
- Output: document fixes and consequences.
So What Studio (framing & interpretation exercise)
- Process: provide a neutral AI summary → assign role personas → have teams interpret “so what” for each role → rewrite prompts and outputs to match role needs (e.g., Gantt chart vs. table).
Prompt maturity model (novice → developing → proficient)
- Novice: minimal context; high iteration/hallucination risk.
- Developing: most REFLECT elements included; still iterative.
- Proficient: full REFLECT usage, action-oriented language, minimal iterations.
Practical team exercises
- Reverse engineering: start with bare task prompts; teams fill missing REFLECT elements.
- Variable swap: change one element at a time in a complete prompt to observe output variance.

Key metrics, KPIs and survey findings

Perceptions and adoption indicators:

50% of CIOs say core human judgment/soft skills are in their top-5 hiring criteria.
Almost half of respondents report problem-solving/critical thinking as top-3 skills needed for IT in an AI-augmented environment.
75% reported that even beginner-level prompt training affects AI value realization.
Only ~5% of CEOs reported highly positive ROI from AI investments (as of the referenced January).
Almost half of CEOs reported they are not yet seeing ROI from their AI investments.

Responsibility split:

5% believe this is primarily HR’s responsibility.
More than a third believe it’s primarily the CIO’s responsibility.
More than half recognize it must be a partnership across functions.

Recommended assessment metrics to capture:

Detection rate of hallucinations/errors in exercises.
Quality of documentation (steps taken to validate and fix).
Ability to articulate impacts (compliance, reputational, operational).
Prompt maturity score (novice → proficient).
Time/iterations to reach acceptable model output (proxy for efficiency / cost of AI use).

Concrete examples and case studies

Healthcare hallucination hunt
- Scenario: a developer prompts an LLM to generate code to de-identify patient IDs. The generated code assumed “internal” meant only redact last four digits and missed edge cases where logs could be shared externally.
- Risks: compliance breach, reputational and operational damage.
- Remediation: adversarial input testing, document validation steps, score developer on detection/fix, assess production implications.
So What Studio example
- A neutral AI summary of a race can be interpreted very differently by an engineer versus a sponsor. The exercise trains teams to produce role-specific outputs (e.g., Gantt chart for PM, table/RFP format for vendor) and to re-prompt models accordingly.
Racing metaphor
- AI is likened to a Formula 1 car: a high-performance tool where human judgment is the drivetrain/steering/navigation. Training should focus on safe competence before production use.

Actionable recommendations (operational steps)

Build and operationalize a Soft Skills Academy
- Start small: run a hallucination hunt as an icebreaker at the next all-team meeting.
- Create measurable rubrics: detection, documentation, explanation of impact, correction.
- Partner with HR to use exercises for hiring and continuous assessment.
Make REFLECT prompting mandatory in projects and sprints
- Add REFLECT-based prompt templates into sprint checklists and PRDs.
- Log prompt templates and best examples in a searchable repository.
Integrate exercises into everyday workflows
- Use reverse engineering and variable-swap exercises in team workshops.
- Include “so what” role-interpretation in product reviews and status updates.
Governance and risk controls
- Require adversarial testing before productionizing model-generated code or agent behaviors.
- Document decisions, fixes and potential impacts for audit trails.
Communities and scaling
- Create Slack/Teams channels for peer learning and to share hallucination-hunt and so-what exercises.
- Encourage employees to author and share exercises via enterprise copilots.
Measurement and ROI focus
- Track prompt maturity, error-detection rates and iterations-to-acceptable-output and tie improvements to business outcomes (cycle time, compliance incidents avoided, time saved).
- Use soft-skill assessment data in hiring and promotion decisions.

Organizational implications and leadership guidance

Ownership: this is cross-functional. CIOs should lead partnerships with HR, legal and business units to operationalize soft-skill learning and assessments.
Prioritize human competencies as a strategic capability — treat them like engineering tooling: train, test, score and iterate.
Emphasize safe practice environments (low-risk “cones”) to let teams learn without catastrophic consequences.
Frame training as fun, team-based and measurable to drive adoption and reduce career-risk anxiety.

High-level investing / ROI note

CEOs are not broadly seeing AI ROI yet (almost half report no ROI; only ~5% report highly positive ROI). Bishop connects this gap to insufficient human capability to extract value — improving human skills is a lever to improve AI ROI.