Images Shifted Three AI Models’ Coding Choices in a Study

Researchers affiliated with Google DeepMind and Stanford University found that simple images shifted three AI models toward recursive code on a binary-tree task, showing that visual cues can affect answers to an unrelated programming prompt.

The team tested Anthropic’s Claude Opus 4.7, OpenAI’s GPT-5.4 and Google’s Gemini 3.1 Pro on inorder traversal of a binary tree. A recursive solution uses a function that calls itself; an iterative solution uses loops. In the main setup, each model produced 20 solutions per configuration after a prompt directed it to inspect an image.

Without an image, Claude and GPT-5.4 used recursion in 0% of their samples. A red heart raised both models’ rates to 100%, or all 20 solutions. Gemini 3.1 Pro used recursion in 40% of its samples without an image and 90% with a yellow checkmark.

The researchers tested 288 combinations of 24 shapes and 12 colors. Results varied by model, and branching arrows did not reliably steer models more than less intuitive symbols. Random noise and naturalistic photos also did not produce systematic shifts toward recursion in the tested models. Text captions matching the images failed to reproduce the same patterns; reported correlations between caption-driven and image-driven steering were −0.05 for Claude, 0.31 for Gemini and 0.17 for GPT-5.4.

In separate trials, some images still shifted model behavior when inserted without instructions to examine them. The symbols that affected responses in those trials often differed from the ones that worked when the prompt told models to look at the image.

A rule-based classifier also found mismatches between some responses and the models’ returned traces. In Gemini cases where a symbol changed behavior, 71.5% of traces were classified as dismissing it as irrelevant. For GPT-5.4, 74.7% of traces in cases where a symbol had no effect were classified as treating it as relevant. These percentages reflect the classifier’s labels, not a direct measure of what a model understood. Claude Opus 4.7 did not return traces for this analysis.

The paper was submitted Oct. 26 at 8:51:17 p.m. ET (Oct. 27 at 00:51:17 UTC). The work is preliminary: its tasks were synthetic, and it does not establish how visual cues affect models in real-world settings.