Abstract: Segmenting and tracking are a challenging problem in video processing.To solve the problem of identification and tracking of the players in drastic basketball game，this paper presents a real-time method to implement segmentation and tracking of players from a basketball video.Different from other common segment method，players are segmented by the use of the background subtraction based on two level（pixel level and frame level）；Mahalanobis distances between a player and a model are used to identify the team that the player belongs to；This kernel-based object tracking method has been used to track multiple players in a stinging basketball game.Experiment results show that this real-time method deals well with identification of the players，and can track multiple players in partial occlusion，and can track robustly when the players’ shape changes greatly.