UC San Diego Researchers Train AI Agents to Model Partners’ Beliefs

Researchers at the University of California, San Diego developed an AI framework that enables agents to model a partner’s beliefs, intentions and likely behavior during an interaction, improving performance across several language and multimodal benchmarks.

The work was developed by Hanan Gani, Lulu Shao and Manmohan Chandraker. It uses Qwen2.5-7B-Instruct as its primary language model, with additional tests involving LLaMA2-7B-Chat, Mistral-7B-Instruct-v0.3 and the multimodal Qwen2.5-VL-Chat.

The system represents both first-order mental states, such as what another participant believes or intends, and second-order states, including what that participant believes about the acting agent.

Training proceeds in three stages. The framework first develops an amortized recursive model of a partner’s mental state. It then uses that representation with a belief-conditioned reward model to assess possible actions before converting the resulting behavior into a standalone policy. During deployment, the policy operates without running the mental-state and reward models alongside it.

On the Sotopia benchmark, the Qwen2.5-7B-Instruct system delivered an average relative improvement of 11.5% over the base model across two partner settings. On BigToM, it raised paired true-belief and false-belief accuracy by an average of 26.7 percentage points across six reported settings.

The framework also improved results in multimodal testing. Its normalized MMRole score was 1.023, compared with 0.994 for the MMRole-Agent baseline. On the out-of-domain split, it scored 1.019 versus 0.981 for the baseline.

The MMRole training data contained 15,607 annotated inter-role dialogue turns. Of 15,919 total training turns, 15,607 passed automated validation, representing 98.04%. A blinded review of 100 annotations produced an average score of 4.45 out of 5, with 90 receiving scores of at least 4.

Additional experiments covered ToMi, Craigslist-Bargain and FANToM, spanning text-only and multimodal interactions. The researchers also released the project’s code.