Topic Trend Prediction of Scientific Papers Based on Spatiotemporal Information Enhancement

doi:10.3778/j.issn.1002-8331.2203-0287

Abstract

Abstract: In recent years, with the increase of social investment in scientific research, the number of research results in various fields has increased significantly. Accurately and effectively predicting the trends of future research topics can help researchers discover future research hotspots. However, due to the increasingly close correlation between various research themes, there is a certain dependency relationship between a large number of research themes. Viewing a single research theme in isolation and using traditional sequence problem processing methods cannot effectively explore the spatial dependencies between these research themes. To simultaneously capture the spatial dependencies and temporal changes between research topics, a spatiotemporal convolutional network is proposed. The network combines a graph convolutional neural network（GCN） and temporal convolutional network（TCN）, specifically, GCNs are used to learn the spatial dependencies of research topics and use space dependence to strengthen spatial characteristics. TCN is used to learn the dynamics of research topics’ trends. Optimization is based on the calculation of weighted losses based on time distance. Compared with the current mainstream sequence prediction models and similar spatiotemporal models on the paper datasets, experimental results show that, in research topic prediction tasks, the model can effectively capture spatiotemporal relationships and the predictions outperform state-of-art baselines.

Key words: science data forecasting, graph neural network, dynamic graph learning, dilated convolution, time series forecasting

摘要： 近年来，随着社会对科研投入的增大，各种领域的研究成果数量显著增加，准确有效地预测未来研究主题的趋势可以帮助科研人员发现未来研究热点，各种领域的研究成果数量显著增加，准确有效地预测未来研究主题的趋势可以帮助科研人员发现未来研究热点。然而由于各个研究主题之间关联逐渐紧密，大量研究主题之间存在一定的依赖关系，孤立地看待单个研究主题并采用传统的序列问题处理方法，无法有效地挖掘这些研究主题之间的空间依赖。为了同时捕获研究主题之间的空间依赖以及时间变化，提出了一种基于时空信息增强的科技论文主题趋势预测模型，该模型结合了图卷积神经网络（GCN）和时间卷积网络（TCN），具体来说，GCN用于学习研究主题的空间表示，并利用空间依赖加强空间特征，TCN用于学习研究主题趋势的动态变化，并根据时间距离计算加权损失进行优化。在论文数据集以及公开数据集上与当前主流的序列预测模型以及类似的时空模型进行了对比，实验结果表明，在研究主题预测任务以及其他动态图任务中，该模型可以有效捕获时空关系并且优于当前最新的基准算法。

关键词: 科技数据预测, 图神经网络, 动态图学习, 膨胀卷积, 时间序列预测

ZHENG Changwei, XUE Zhe, LIANG Meiyu, DU Junping, KOU Feifei. Topic Trend Prediction of Scientific Papers Based on Spatiotemporal Information Enhancement[J]. Computer Engineering and Applications, 2023, 59(14): 86-93.

郑长伟, 薛哲, 梁美玉, 杜军平, 寇菲菲. 基于时空信息增强的科技论文主题趋势预测[J]. 计算机工程与应用, 2023, 59(14): 86-93.

References

[1] LI Wenling，JIA Yingmin，DU Junping.State estimation for stochastic complex networks with switching topology[J].IEEE Transactions on Automatic Control，2017，62（12）：6377-6384.
[2] LI Wenling，JIA Yingmin，DU Junping.Recursive state estimation for complex networks with random coupling strength[J].Neurocomputing，2017，219：1-8.
[3] SHI Chuan，HAN Xiaotian，SONG Li，et al.Deep collaborative filtering with multi-aspect information in heterogeneous networks[J].IEEE Transactions on Knowledge and Data Engineering，2019，33（4）：1413-1425.
[4] LI Wenling，JIA Yingmin，DU Junping.Variance-constrained state estimation for nonlinearly coupled complex networks[J].IEEE Transactions on Cybernetics，2017，48（2）：818-824.
[5] KIPF T N，WELLING M.Semi-supervised classification with graph convolutional networks[J].arXiv：1609.02907，2016.
[6] XUE Zhe，DU Junping，ZHENG Changwei，et al.Clustering-induced adaptive structure enhancing network for incomplete multi-view data[C]//Proceedings of IJCAI，2021：3235-3241.
[7] VELI?KOVI? P，CUCURULL G，CASANOVA A，et al.Graph attention networks[J].arXiv：1710.10903，2017.
[8] PAN L，SHI C，DOKMANI? I.Neural link prediction with walk pooling[J].arXiv：2110.04375，2021.
[9] XUE Zhe，DU Junping，DU Dawei，et al.Deep correlated predictive subspace learning for incomplete multi-view semi-supervised classification[C]//Proceedings of IJCAI，2019：4026-4032.
[10] PANAGOPOULOS G，NIKOLENTZOS G，VAZIRGIANNIS M.Transfer graph neural networks for pandemic forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2021：4838-4845.
[11] 李有红，王学军，谌裕勇，等.一种融合邻边属性的个人社交网络社区发现算法[J].计算机工程，2021，47（7）：81-87.
LI Youhong，WANG Xuejun，CHEN Yuyong，et al.A community discovery algorithm fused with adjacent edge attribute for personal social network[J].Computer Engineering，2021，47（7）：81-87.
[12] KOU Feifei，DU Junping，HE Yijiang，et al.Social network search based on semantic analysis and learning[J].CAAI Transactions on Intelligence Technology，2016，1（4）：293-302.
[13] ZHOU X，LIANG W，WANG K I K，et al.Academic influence aware and multidimensional network analysis for research collaboration navigation based on scholarly big data[J].IEEE Transactions on Emerging Topics in Computing，2021，9（1）：246-257.
[14] QIN Y，SONG D，CHEN H，et al.A dual-stage attention-based recurrent neural network for time series prediction[C]//Twenty-Sixth International Joint Conference on Artificial Intelligence，2017.
[15] LAI G K，CHANG W C，YANG Y M，et al.Modeling long-and short-term temporal patterns with deep neural networks[C]//The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval（SIGIR’18），New York，NY，USA，2018：95-104.
[16] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017.
[17] WANG Z，MA Y，LIU Z，et al.R-transformer：recurrent neural network enhanced transformer[J].arXiv：1907.05572，2019.
[18] XUE Zhe，DU Junping，DU Dawei，et al.Deep low-rank subspace ensemble for multi-view clustering[J].Information Sciences，2019，482：210-227.
[19] FANG Yuke，DENG Weihong，DU Junping，et al.Identity-aware CycleGAN for face photo-sketch synthesis and recognition[J].Pattern Recognition，2020，102：107249.
[20] HU Weiming，GAO Jun，LI Bing，et al.Anomaly detection using local kernel density estimation and context-based regression[J].IEEE Transactions on Knowledge and Data Engineering，2018，32（2）：218-233.
[21] BAI S，KOLTER J Z，KOLTUN V.An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[J].arXiv：1803.01271，2018.
[22] 贺小伟，徐靖杰，王宾，等.基于GRU-LSTM组合模型的云计算资源负载预测研究[J].计算机工程，2022，48（5）：11-17.
HE Xiaowei，XU Jingjie，WANG Bin，et al.Research on cloud computing resource load forecasting based on GRU-LSTM combination model[J].Computer Engineering，2022，48（5）：11-17.
[23] ZHAO L，SONG Y，ZHANG C，et al.T-GCN：a temporal graph convolutional network for traffic prediction[J].IEEE Transactions on Intelligent Transportation Systems，2019，21（9）：3848-3858.
[24] GUO S，LIN Y，FENG N，et al.Attention based spatial-temporal graph convolutional networks for traffic flow forecasting[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2019：922-929.
[25] LI J，HAN Z，CHENG H，et al.Predicting path failure in time-evolving graphs[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining，2019：1279-1289.
[26] BAI L，YAO L，LI C，et al.Adaptive graph convolutional recurrent network for traffic forecasting[C]//Advances in Neural Information Processing Systems，2020：17804-17815.
[27] CHEN J，WANG X，XU X.GC-LSTM：graph convolution embedded LSTM for dynamic network link prediction[J].Applied Intelligence，2021：1-16.
[28] 于瑞云，林福郁，高宁蔚，等.基于可变形卷积时空网络的乘车需求预测模型[J].软件学报，2021，32（12）：3839-3851.
YU Ruiyun，LIN Fuyu，GAO Ningwei，et al.Passenger demand forecast model based on deformable convolution spatial-temporal network[J].Journal of Software，2021，32（12）：3839-3851.
[29] 李慧博，赵云霄，白亮.基于深度神经网络和门控循环单元的动态图表示学习方法[J].计算机应用，2021，41（12）：3432-3437.
LI Huibo，ZHAO Yunxiao，BAI Liang.Dynamic graph representation learning method based on deep neural network and gated recurrent unit[J].Computer Applications，2021，41（12）：3432-3437.
[30] 孙一珺，胡辉.基于动态图卷积的加权点云分类网络[J].计算机工程与应用，2022，58（20）：240-246.
SUN Yijun，HU Hui.Weighted point cloud classification network based on dynamic graph convolution[J].Computer Engineering and Applications，2022，58（20）：240-246.