Abstract: Reinforcement Learning（RL） is a kind of unsupervised learning method for agent to acquire optimal behavior sequence to adapt to unknown environments with a clue of reward.Now RL is widely used in agent systems.One of difficult problems for RL is action selecting，which means how to balance the relation exploitation and exploration.A counter mechanism on the basis of Q learning combined with ε-greedy strategy is presented so that the parameters of ε-greedy can be adjusted in steps when choosing actions.The simulation results of Grid World verify the effectiveness of the method.