MME-CC: A Challenging Multi-Modal Evaluation Benchmark of Cognitive Capacity

Notably, GPT-5 (high) achieves the highest performance in Spatial Reasoning with a score of 30.3%, a result that appears attributable to its capabilities in sub-tasks requiring complex spatial orientation and object counting (e.g., Indoor Directional Reasoning and Indoor Deduplication Counting). Gemini-2.5-Pro, in contrast, demonstrates a clear advantage in Visual Knowledge Reasoning by attaining a leading score of 70.7%. In the domain of Geometric Reasoning, the Doubao model's performance advantage appears localized to the Jigsaw Puzzle task, whereas Gemini-2.5-Pro shows more robust results across other geometric reasoning challenges.

^* 5 students who did not participate in question setting; all held advanced degrees.

Analysis of chain-of-thought behavior reveals that MLLMs follow a three-stage reasoning process: problem understanding, core analysis, and conclusion formation. While models repeatedly revisit visual information throughout reasoning, excessive verification often reduces efficiency. On spatial and geometric tasks, most models achieve only 20%-30% accuracy, with maze tasks showing < 2% success rates. Adding textual guidance ("describe the image first") yields consistent improvements, suggesting that initial visual descriptions stabilize subsequent reasoning by anchoring perception.

Analysis of failure cases reveals three recurring error patterns: (1) Orientation judgment errors where models fail to preserve object orientation across views and struggle with reference-frame alignment, particularly in indoor layout tasks; (2) Entity identity inconsistency in multi-view settings, leading to double counting or omission of the same entities; and (3) Over-reliance on literal descriptions where models prioritize visual content over task-specific counterfactual constraints in instructions, producing answers that conflict with required reasoning.