Neural Network for Moving Small Target Pedestrian Detection Based on Episodic Memory

doi:10.3778/j.issn.1002-8331.2203-0032

Abstract

Abstract: Reliable detection of small target pedestrians from visual scenes is a critical basis for building future intelligent video surveillance systems. However, the tiny sizes and blurred textures of moving small targets make it difficult for these existing traditional methods of pedestrian target detection to cope. To address this problem, the paper proposes an artificial visual neural network（STPDNN） model for moving small target pedestrian detection, by means of the neural structure properties of locust vision systems and the episodic memory cognition mechanism of the medial temporal lobe（MTL） inhuman brains. The proposed neural network consists of two parts: presynaptic and postsynaptic sub-networks. Therein, the presynaptic network obtains visual motion cues characterizing the low-order features of targets by simulating the neural processing mechanisms of locust visual systems; the postsynaptic network extracts the episodic memory higher-order information of pedestrian targets from low-order visual signals to response selectively motion targets. The results of systematic experiments show that the proposed STPDNN can effectively detect moving small target pedestrians in different visual scenes. This research work involves dynamic visual information processing of pedestrian targets inspired by biological optic nerve mechanisms, which can provide novel ideas andcreative methods for pedestrian detection and moving behavior analysis in future intelligent video surveillance.

Key words: moving target detection, small target-pedestrian, episodic memory, locust vision nerve, intelligent video surveillance

摘要： 从视觉场景中可靠地检测小目标行人对象是构建未来人工智能视觉系统的重要基础。由于运动小目标的视感尺寸小且纹理特征模糊，导致现有的传统行人目标检测方法难以应对。针对该问题，基于蝗虫视觉系统的神经结构特性，借助人类大脑内侧颞叶（MTL）情景记忆认知机理，提出一种适用于运动小目标行人检测的人工视觉神经网络（STPDNN）模型。所提出的神经网络包括两部分：突触前和突触后子网络。其中，突触前网络模拟蝗虫视觉系统加工处理视觉信号的神经机理，获得表征目标对象低阶特征的视觉运动线索；突触后网络从低阶视觉信号中提取出行人目标的情景记忆高阶信息，以实现对运动目标的偏好性响应。系统性的实验结果表明，提出的STPDNN可有效检测视觉场景中的运动小目标行人对象。该研究工作涉及生物视神经机理启发的行人目标动态视觉信息加工处理，可为智能视频监控中的行人检测识别与运动行为分析提供新思想、新方法。

关键词: 运动目标检测, 小目标行人, 情景记忆, 蝗虫视觉神经, 智能视频监控

ZHANG Benkang, HU Bin. Neural Network for Moving Small Target Pedestrian Detection Based on Episodic Memory[J]. Computer Engineering and Applications, 2022, 58(15): 169-183.

张本康, 胡滨. 基于情景记忆的运动小目标行人检测神经网络[J]. 计算机工程与应用, 2022, 58(15): 169-183.

References

[1] GAWANDE U，HAJARI K，GOLHAR Y.Pedestrian detection and tracking in video surveillance system：issues，comprehensive review，and challenges[J].Recent Trends in Computational Intelligence，2020：1-24.
[2] CAO J，PANG Y，XIE J，et al.From handcrafted to deep features for pedestrian detection：a survey[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021.
[3] WANG X，ZHENG S，YANG R，et al.Pedestrian attribute recognition：a survey[J].Pattern Recognition，2022，121：108220.
[4] KELE? M F，FRYE M A.Object-detecting neurons in drosophila[J].Current Biology，2017，27（5）：680-687.
[5] KELE? M F，HARDCASTLE B J，ST?DELE C，et al.Inhibitory interactions and columnar inputs to an object motion detector in drosophila[J].Cell Reports，2020，30（7）：2115-2124.
[6] TANAKA R，CLARK D A.Object-displacement-sensitive visual neurons drive freezing in drosophila[J].Current Biology，2020，30（13）：2532-2550.
[7] YUE S，RIND F C.Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement[J].IEEE Transactions on Neural Networks，2006，17（3）：705-716.
[8] YUE S，RIND F C.Visual motion pattern extraction and fusion for collision detection in complex dynamic scenes[J].Computer Vision and Image Understanding，2006，104（1）：48-60.
[9] WANG H X，PENG J T，YUE S N.A directionally selective small target motion detecting visual neural network in cluttered backgrounds[J].IEEE Transactions on Cybernetics，2018，50（4）：1541-1555.
[10] WANG H X，WANG H T，ZHAO J N，et al.A time-delay feedback neural network for discriminating small，fast-moving targets in complex dynamic environments[J].IEEE Transactions on Neural Networks and Learning Systems，2021：1-15.
[11] WANG H，ZHAO J，WANG H，et al.Attention and prediction guided motion detection for low-contrast small moving targets[J].arXiv：2104.13018，2021.
[12] HU B，YUE S，ZHANG Z.A rotational motion perception neural network based on asymmetric spatio temporal visual information processing[J].IEEE Transactions on Neural Networks and Learning Systems，2017，28（11）：2803-2821.
[13] HU B，ZHANG Z.Bio-plausible visual neural network for spatio-temporally spiral motion perception[J].Neurocomputing，2018，310：96-114.
[14] HU B，ZHANG Z，LI L.Lgmd-based visual neural network for detecting crowd escape behavior[C]//2018 5th IEEE International Conference on Cloud Computing and Intelligence Systems（CCIS），2018：772-778.
[15] HU B，ZHANG Z.Bio-inspired visual neural network on spatio-temporal depth rotation perception[J].Neural Computing and Applications，2021，33（16）：10351-10370.
[16] RIND F C，BRAMWELL D I.Neural network based on the input organization of an identified neuron signaling impending collision[J].Journal of Neurophysiology，1996，75（3）：967-985.
[17] RIND F C.Motion detectors in the locust visual system：from biology to robot sensors[J].Microscopy Research and Technique，2002，56（4）：256-269.
[18] HADELN J，HENSGEN R，BOCKHORST T，et al.Neuroarchitecture of the central complex of the desert locust：tangential neurons[J].Journal of Comparative Neurology，2020，528（6）：906-934.
[19] SIMMONS P J，RIND F C，SANTER R D.Escapes with and without preparation：the neuroethology of visual startle in locusts[J].Journal of Insect Physiology，2010，56（8）：876-883.
[20] VAZ A P，WITTIG J H，INATI S K，et al.Replay of cortical spiking sequences during human memory retrieval[J].Science，2020，367（6482）：1131-1134.
[21] RUTISHAUSER U，REDDY L，MORMANN F，et al.The architecture of human memory：insights from human single-neuron recordings[J].Journal of Neuroscience，2021，41（5）：883-890.
[22] ULLMAN S.Using neuroscience to develop artificial intelligence[J].Science，2019，363（6428）：692-693.
[23] PANG Y，CAO J，WANG J，et al.JCS-Net：joint classification and super-resolution network for small-scale pedestrian detection in surveillance images[J].IEEE Transactions on Information Forensics and Security，2019，14（12）：3322-3331.
[24] LI G，YANG J，KANG Z.Pedestrian detection algorithm based on improved YOLOv3_tiny[C]//Proceedings of 2021 Chinese Intelligent Automation Conference.Singapore：Springer，2022：98-106.
[25] ADARSH P，RATHI P，KUMAR M.YOLO v3-tiny：object detection and recognition using one stage improved model[C]//2020 6th International Conference on Advanced Computing and Communication Systems（ICACCS），2020：687-694.
[26] YOU Y，CHEN H，LI Y，et al.Small-scale pedestrian detection method based on attention mechanism and adaptive feature fusion[J].Journal of Physics：Conference Series.IOP Publishing，2021，2035（1）：012023.
[27] BOCHKOVSKIY A，WANG C Y，LIAO H Y M.Yolov4：optimal speed and accuracy of object detection[J].arXiv：2004.10934，2020.
[28] HAN B，WANG Y，YANG Z，et al.Small-scale pedestrian detection based on deep neural network[J].IEEE Transactions on Intelligent Transportation Systems，2019，21（7）：3046-3055.
[29] HUANG S，RAMANAN D.Expecting the unexpected：training detectors for unusual pedestrians with adversarial imposters[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2017：2243-2252.
[30] JOCHER G，ALEX S，AYUSH C，et al.Yolov5[EB/OL].
（2020）.https：//github.com/ultralytics/yolov5.
[31] ZAJAC F E，NEPTUNE R R，KAUTZ S A.Biomechanics and muscle coordination of human walking：Part I：introduction to concepts，power transfer，dynamics and simulations[J].Gait & Posture，2002，16（3）：215-232.
[32] ZAJAC F E，NEPTUNE R R，KAUTZ S A.Biomechanics and muscle coordination of human walking：part II：lessons from dynamical simulations and clinical implications[J].Gait & Posture，2003，17（1）：1-17.
[33] ROUPA I，DA SILVA M R，MARQUES F，et al.On the modeling of biomechanical systems for human movement analysis：a narrative review[J].Archives of Computational Methods in Engineering，2022：1-44.
[34] SUTOU S.Hairless mutation：a driving force of humanization from a human-ape common ancestor by enforcing upright walking while holding a baby with both hands[J].Genes to Cells，2012，17（4）：264-272.
[35] TAN U.A new syndrome with quadrupedal gait，primitive speech，and severe mental retardation as a live model for human evolution[J].International Journal of Neuroscience，2006，116（3）：361-369.
[36] BOYER P.Evolutionary economics of mental time travel?[J].Trends in Cognitive Sciences，2008，12（6）：219-224.
[37] SUDDENDORF T，ADDIS D R，CORBALLIS M C.Mental time travel and the shaping of the human mind[J].Philosophical Transactions of the Royal Society B：Biological Sciences，2009，364（1521）：1317-1324.
[38] QUIROGA R Q.Concept cells：the building blocks of declarative memory functions[J].Nature Reviews Neuroscience，2012，13（8）：587-597.
[39] REDDY L，PONCET M，SELF M W，et al.Learning of anticipatory responses in single neurons of the human medial temporal lobe[J].Nature Communications，2015，6（1）：1-8.
[40] DE FALCO E，ISON M J，FRIED I，et al.Long-term coding of personal and universal associations underlying the memory web in the human brain[J].Nature Communications，2016，7（1）：1-11.
[41] REBER T P，FABER J，NIEDIEK J，et al.Single-neuron correlates of conscious perception in the human medial temporal lobe[J].Current Biology，2017，27（19）：2991-2998.
[42] STARESINA B P，REBER T P，NIEDIEK J，et al.Recollection in the human hippocampal-entorhinal cell circuitry[J].Nature Communications，2019，10（1）：1-11.
[43] REY H G，GORI B，CHAURE F J，et al.Single neuron coding of identity in the human hippocampal hormation[J].Current Biology，2020，30（6）：1152-1159.
[44] SCHMELING F，TEGTMEIER J，KINOSHITA M，et al.Photoreceptor projections and receptive fields in the dorsal rim area and main retina of the locust eye[J].Journal of Comparative Physiology A，2015，201（5）：427-440.
[45] YUE S，RIND F C.A collision detection system for a mobile robot inspired by the locust visual system[C]//Proceedings of the 2005 IEEE International Conference on Robotics and Automation，2005：3832-3837.
[46] CLARK D A，DEMB J B.Parallel computations in insect and mammalian visual motion processing[J].Current Biology，2016，26（20）：1062-1072.
[47] FU Q，HU C，PENG J，et al.A robust collision perception visual neural network with specific selectivity to darker objects[J].IEEE Transactions on Cybernetics，2019，50（12）：5074-5088.
[48] OH S，HOOGS A，PERERA A，et al.AVSS 2011 demo session：a large-scale benchmark dataset for event recognition in surveillance video[C]//2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance（AVSS），2011：527-528.
[49] VIDEEZ Y.Shot of a seagull flying with blue sky on background in 4K[EB/OL].（2022）.https：//www.videezy.com/animals-and-wildlife/11580-shot-of-a-seagull-flying-with-blue-sky-on-background-in-4k.
[50] REDMON J，FARHADI A.Yolov3：an incremental improvement[J].arXiv：1804.02767，2018.
[51] WANG C Y，BOCHKOVSKIY A，LIAO H Y M.Scaled-yolov4：scaling cross stage partial network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：13029-13038.