关键词: 桌面虚拟环境, 笔式交互, 三维交互, 交互框架

Abstract: Integrating pen-based interaction into 3D interaction is a new way for researching 3D interaction paradigm.Including two main components for generating interaction primitives and interaction tasks，an architecture for pen-based 3D interaction in desktop virtual environment is proposed.After encapsulating high-level events and interaction primitives from lexical and grammar level，the interaction task integration mechanism is presented based on interaction context，user mediation and gesture recognition.Three built-in interaction techniques using gesture，constrain perception and hybrid interaction can efficiently reduce user’s recognition burden caused by task decomposition and mode switch，which therefore result in natural interaction.

Key words: desktop virtual environment, pen-based interaction, three dimensional interaction, interaction architecture