Where do LLMs fall short in CBT-guided reasoning?

LLMs can know CBT theory yet still default to Validation & Reflection when a conversation calls for a different strategy. CBT Lens combines clinical grounding with multiple reasoning paths to choose a response, then measures whether that guidance changes the model’s usual behavior.

To be published in the proceedings of the 14th International Conference on Affective Computing and Intelligent Interaction (ACII 2026) ↗

Read the paper ↗Try CBT Lens ↗
01 / The cognitive model

Start by breaking down the user’s query.

Automatic
thought Emotion
(shame) Behavior Behavior Emotion
(sadness)
“I knowI’m a burden on other peopleso I keep things to myselfand put on a smile,even when I’m not okay.”
Beck’s cognitive model ↗ An automatic thought is the meaning a person gives a situation (“I’m a burden”); emotions are what they feel (shame and sadness); behaviors are what they do (stay quiet and put on a smile).
02 / The gap

An empathetic reply doesn't necessarily reflect clinical structure.

Current models often validate and reflect, even when guided discovery or another CBT strategy may be more appropriate. The responses below show the current state of GPT-5.3 and a mental health application.

Same opening query · LLM vs. mental health appNo prior user history

GPT-5.3

Explains and suggests

That sounds really heavy to carry—feeling like you have to hide what you’re going through just to avoid being a burden. When someone cares about others, they often start minimizing their own needs. That doesn’t make your feelings less valid—it means you’ve learned to protect others at your own expense.

A few things worth gently questioning:

  • “I’m a burden” → is that a fact, or a fear?
  • If someone you cared about felt this way, would you see them as a burden?
  • Are you giving others the chance to choose?

You don’t have to suddenly open up to everyone. Maybe there’s a small step that feels safe:

  • sharing one honest sentence with someone you trust
  • or even just saying “I’ve been a bit off lately”

You deserve space too—not just to function, but to be real. If you want, we can break this down together:

  • where that “burden” belief started
  • or how to test it in a low-risk way without feeling exposed
Assumes why the person feels this way, overwhelms them with too much information, then immediately guides them toward action.

Ash — AI for Mental Health

Asks before interpreting

That sounds really heavy – not just worrying you might be a burden, but feeling sure that you are. I’m not going to argue with you or try to talk you out of it. I just want to understand. What makes you feel certain you’re a burden to other people?

Invites the person to explain the belief before suggesting what to do.
03 / The approach

Clinical grounding.

01 — CONCEPTUALIZE

Read the cognitive context

Break a user narrative into elements of Beck’s cognitive conceptualization framework.

02 — GROUND

Retrieve relevant concepts

Connect the user’s context to a SNOMED CT knowledge graph and validate the concepts’ relevance with natural language inference.

03 — SELECT

Choose a CBT strategy

Use multiple chains of thought to select from Validation & Reflection, Socratic Questioning, or an Alternative Perspective.

04 / Protocol Leverage Force

Did our framework change the model’s baseline behavior?

Protocol Leverage Force (F) measures how strongly CBT guidance reorients a response away from the model’s baseline and toward the user’s utterance.

baselinecentroid r m v F selected protocol Baseline response the model’s usual answer shift toward the user’s utterance resulting leverage of CBT guidance Not to scale
Adapted from Huygens’ centrifugal-force analogy. The center is the model’s baseline response; the circle shows its distance to the CBT-guided response. From the selected protocol (m), one arrow points toward the user’s words, while the outward arrow represents the framework’s leverage (F).

Across three open-weight models and RealCBT-derived user case studies, models often favored Validation & Reflection. F helps quantify how much CBT guidance changed a model’s baseline response. Human reviewers brought expertise in clinical psychology, psychology research, mental health AI, and industry practice.

Read the methods ↗
05 / The affective story

What do valence and arousal show us?

Valence describes how positive or negative an emotion feels. Arousal describes its level of physiological activation or energy. These are the two dimensions of Russell’s Circumplex Model of Affect ↗. The paper compares estimates from user language in CBT-MCoT synthetic sessions with RealCBT human transcripts, scoring each turn with the NRC VAD Lexicon.

CBT-MCoT synthetic sessions RealCBT human transcripts Conversation start Conversation end
1 10 user turns1 60+ RealCBT client turns

Mistral vs RealCBT

Gemma3 vs RealCBT

GPT-OSS vs RealCBT

Recreated in code from the paper’s Figure 6. The axes and turn ranges follow the figure; individual coordinates are visually approximated from it. Each path represents a cumulative mean over turns, averaged across 14 transcripts.
The team
Vaishnavi SinhaUniversity of Southern CaliforniaComputer SciencePooja GuttalUniversity of Maryland, Baltimore CountyComputer Science & Electrical EngineeringPranay Deep Reddy KatikeUniversity of Maryland, Baltimore CountyComputer Science & Electrical EngineeringVishal SinhaUniversity of California, IrvineElectrical Engineering & Computer ScienceGerald NdawulaUniversity of Maryland, Baltimore CountyComputer Science & Electrical EngineeringLira YoonUniversity of Maryland, Baltimore CountyPsychologyAndrea KleinsmithUniversity of Maryland, Baltimore CountyInformation SystemsManas GaurUniversity of Maryland, Baltimore CountyComputer Science & Electrical Engineering
Reference
@article{sinha2026llms,
  title={Where do LLMs Fall Short in CBT-Guided Affective Reasoning?},
  author={Sinha, Vaishnavi and Guttal, Pooja and Katike, Pranay Deep Reddy and Sinha, Vishal and Ndawula, Gerald and Yoon, Lira and Kleinsmith, Andrea and Gaur, Manas},
  journal={arXiv preprint arXiv:2607.02885},
  year={2026}
}