Vision-Language Model-Based Emotional Understanding and Interpretable Feedback for Art Therapy Support

Abstract

Artworks can evoke complex, subjective, and heterogeneous emotional responses among viewers, whereas conventional approaches often reduce such responses to a single emotion label and therefore fail to jointly preserve affective diversity, visually grounded evidence, and linguistic interpretation. This paper proposes a large vision-language framework for artwork-evoked emotion understanding and interpretable feedback generation, with potential relevance to professionally guided reflection and discussion in art-therapy-oriented settings. Given an artwork image and an affect-oriented instruction, visual and textual representations are jointly encoded, and bidirectional cross-modal attention is introduced to strengthen the correspondence between regional visual content and affective semantics. During emotion inference, an affective prototype memory is employed to estimate multi-category emotion distributions, while uncertainty estimation is incorporated to avoid overly confident interpretations of ambiguous artistic expressions. To improve interpretability, an evidence-sensitive grounding mechanism identifies and aggregates color, subject, texture, and compositional regions associated with the predicted affective responses, producing a visually grounded emotion representation. An explanation decoder subsequently generates natural-language feedback conditioned on the predicted emotion distribution, regional visual evidence, and confidence information, while semantic consistency constraints maintain agreement among affective predictions, grounded regions, and explanatory text. Comparative experiments on the ArtEmis dataset demonstrate strong overall performance in terms of BLEU-1, BLEU-4, METEOR, and ROUGE-L. The results indicate that the proposed framework improves lexical correspondence, content coverage, and linguistic consistency in artwork emotion explanations, providing a reliable technical foundation for organizing artwork-evoked emotional responses and visual evidence in art-therapy-oriented auxiliary analysis.

Cover Image
Downloads
Published
2025-07-01
Section
Articles