意象知识增强的社交媒体头像情感分析

doi:10.3778/j.issn.1002-8331.2408-0216

摘要/Abstract

摘要： 图像具有丰富的情感信息，这些信息可以被快速、直观地获取。头像作为一类特殊的图像，与用户的自我认知具有很强的关联性，用户通过头像内容所具有的意象来反映这种自我认知。然而现有的图像情感分析工作缺乏对于意象的考虑，因此在VAD情感模型基础上，扩展了意象情感维度，用于表示意象所反映的用户自我认知。为了衡量VAD情感和意象情感综合反映出的用户情感，引入心理能量度量。通过构建一个情感协同融合的心理能量预测模型学习图像特征和意象知识，利用注意力机制学习二者之间的相关性，分析头像心理能量。在真实数据集上进行实验，该模型心理能量维度的NDCG（normalized discounted cumulative gain）指标为0.499，优于其他表现最好的基线模型5.50%，验证了所提出方法的有效性。

关键词: 社交媒体头像, 意象, 心理能量, 情感分析

Abstract: Images contain rich affective information, which can be obtained quickly and intuitively. As a special type of image, avatars have a strong correlation with user’s self-cognition. Users reflect their self-cognition through the imagery contained in their avatars. However, the existing image affection analysis work lacks consideration of imagery. Therefore, on the basis of VAD affection model, this paper expands the imagery affection dimension to represent the user’s self-cognition reflected by the imagery. To measure the user’s affection reflected by VAD affection and imagery affection, psychological energy measurement is introduced. This paper proposes an affections collaborative fusion for psychological energy prediction model to analyze the psychological energy of avatars. The model learns image features and imagery knowledge, and uses attention mechanism to learn the correlation between them, thereby analyzing the psychological energy of avatars. The model is tested on the real dataset, and the NDCG (normalized discounted cumulative gain) of the psychological energy dimension is 0.499, which is 5.50% better than other best performing baseline models, verifying the effectiveness of the proposed method.

Key words: social media avatars, imagery, psychological energy, affection analysis

刘俊岭, 安宁, 孙焕良, 许景科. 意象知识增强的社交媒体头像情感分析[J]. 计算机工程与应用, 2025, 61(22): 267-277.

LIU Junling, AN Ning, SUN Huanliang, XU Jingke. Affection Analysis of Social Media Avatars Enhanced by Imagery Knowledge[J]. Computer Engineering and Applications, 2025, 61(22): 267-277.

参考文献

[1] ZHAO S C, YAO X X, YANG J F, et al. Affective image content analysis: two decades review and new perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(10): 6729-6751.
[2] 曹高辉, 巩洪村, 梅潇. 社交媒体用户自我呈现变更行为的影响因素研究: 以头像更换为例[J]. 情报科学, 2020, 38(6): 44-52.
CAO G H, GONG H C, MEI X. The influencing factors of social media users’ self-presentation change behavior: taking the change of avatar as an example[J]. Information Science, 2020, 38(6): 44-52.
[3] 朱建军, 曹昱. 情绪词典: 你的感受试图告诉你什么[M]. 北京: 中国人民大学出版社, 2023: 91-95.
ZHU J J, CAO Y. Emotional dictionary: what are your feelings trying to tell you? [M]. Beijing: China Renmin University Press, 2023: 91-95.
[4] 朱建军. 释梦: 升级版[M]. 北京: 中国人民大学出版社, 2016: 227-361.
ZHU J J. Interpretation of dreams: upgraded version[M]. Beijing: China Renmin University Press, 2016: 227-361.
[5] EKMAN P. An argument for basic emotions[J]. Cognition and Emotion, 1992, 6(3/4): 169-200.
[6] PLUTCHIK R. Emotion: a psychoevolutionary synthesis[M]. New York: Harper & Row, 1980.
[7] SCHLOSBERG H. Three dimensions of emotion[J]. Psychological Review, 1954, 61(2): 81-88.
[8] WANG W N, YU Y L, JIANG S M. Image retrieval by emotional semantics: a study of emotional space and feature extraction[C]//Proceedings of the 2006 IEEE International Conference on Systems, Man and Cybernetics. Piscataway: IEEE, 2007: 3534-3539.
[9] YANULEVSKAYA V, VAN GEMERT J C, ROTH K, et al. Emotional valence categorization using holistic image features[C]//Proceedings of the 2008 15th IEEE International Conference on Image Processing. Piscataway: IEEE, 2008: 101-104.
[10] MACHAJDIK J, HANBURY A. Affective image classification using features inspired by psychology and art theory[C]//Proceedings of the 18th ACM International Conference on Multimedia. New York: ACM, 2010: 83-92.
[11] LI B, REN H, JIANG X K, et al. SCEP: a new image dimensional emotion recognition model based on spatial and channel-wise attention mechanisms[J]. IEEE Access, 2021, 9: 25278-25290.
[12] WANG G, TAN L, SHANG Z L, et al. Multimodal dual emotion with fusion of visual sentiment for rumor detection[J]. Multimedia Tools and Applications, 2024, 83(10): 29805-29826.
[13] ZHAO J, YANG F P. Fusion with GCN and SE-ResNeXt network for aspect based multimodal sentiment analysis[C]//Proceedings of the 2023 IEEE 6th Information Technology, Networking, Electronic and Automation Control Conference. Piscataway: IEEE, 2023: 336-340.
[14] ANASOSALU VASU P K, GABRIEL J, ZHU J, et al. FastViT: a fast hybrid vision transformer using structural reparameterization[C]//Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2024: 5762-5772.
[15] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[16] BIO国际组织教材编写组. 心理咨询与治疗基础[M]. 北京: 人民日报出版社, 2007.
BIO International Organization Textbook Editorial Group. Basic counseling and psychotherapy[M]. Beijing: People’s Daily Press, 2007.
[17] CARL G J. Psychological types[M]. Zurich: Rascher Verlag, 1921.
[18] LAKOFF G, JOHNSON M. Metaphors we live by[M]. Chicago: University of Chicago Press, 2003.
[19] CARL G J. The symbolism of transformation: anticipatory analysis of schizophrenia[M]. Beijing: International Culture Publishing Company, 2011: 115.
[20] 朱建军. 我是谁: 心理咨询与意象对话技术[M]. 北京: 中国城市出版社, 2001.
ZHU J J. Who am I: psychological counseling and image dialogue technology[M]. Beijing: China City Press, 2001.
[21] LANG P J, BRADLEY M M, CUTHBERT B N. International affective picture system (IAPS): technical manual and affective ratings[M]. NIMH Center for the Study of Emotion and Attention, 1997: 39-58.
[22] KIM H R, KIM Y S, KIM S J, et al. Building emotional machines: recognizing image emotions through deep neural networks[J]. IEEE Transactions on Multimedia, 2018, 20(11): 2980-2992.
[23] LI J N, LI D X, XIONG C M, et al. BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation[C]//Proceedings of the 39th International Conference on Machine Learning, 2022.
[24] BAI J Z, BAI S, CHU Y F, et al. Qwen technical report[J]. arXiv:2309.16609, 2023.
[25] NIELSEN F ?. A new ANEW: evaluation of a word list for sentiment analysis in microblogs[J]. arXiv:1103.2903, 2011.
[26] ZHAO S C, JIA Z Z, CHEN H, et al. PDANet: polarity-consistent deep attention network for fine-grained visual emotion regression[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 192-201.
[27] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[28] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409. 1556, 2014.
[29] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: ACL, 2019: 4171-4186.
[30] KIM Y. Convolutional neural networks for sentence classification[J]. arXiv:1408.5882, 2014.
[31] YADAV A, VISHWAKARMA D K. A deep multi-level attentive network for multimodal sentiment analysis[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2023, 19(1): 1-19.
[32] ZHANG H, LIU Y N, XIONG Z Y, et al. Visual sentiment analysis with semantic correlation enhancement[J]. Complex & Intelligent Systems, 2024, 10(2): 2869-2881.
[33] 赵恒, 程志华. 田径运动员心理能量与运动性疲劳的相关性研究[J]. 体育科技文献通报, 2008, 16(9): 18.
ZHAO H, CHENG Z H. Correlative study of sports mental energy related with exercise fatigue[J]. Bulletin of Sport Science & Technology, 2008, 16(9): 18.