Abstract: Reinforcement learning is an effective way for accomplishing task in multi-robot system.While much of the work has focused on optimizing discounted cumulative reward,optimizing average reward is sometimes a more suitable criterion for multi-robot coordination.Learning algorithms based on discounted rewards,such as Q learning,can attain a well result at the action-level,but it cannot perform well at the task-level.However,learning methods based on average reward,such as the Monte Carlo algorithm,are capable of achieving the optimal result through cooperation at the task-level.Real robot experiment shows that the algorithm adopting the average reward is superior to the one adopting the discounted cumulative reward.