Abstract: Integrating pen-based interaction into 3D interaction is a new way for researching 3D interaction paradigm.Including two main components for generating interaction primitives and interaction tasks，an architecture for pen-based 3D interaction in desktop virtual environment is proposed.After encapsulating high-level events and interaction primitives from lexical and grammar level，the interaction task integration mechanism is presented based on interaction context，user mediation and gesture recognition.Three built-in interaction techniques using gesture，constrain perception and hybrid interaction can efficiently reduce user’s recognition burden caused by task decomposition and mode switch，which therefore result in natural interaction.

Key words: desktop virtual environment, pen-based interaction, three dimensional interaction, interaction architecture

摘要： 笔输入和三维交互结合是研究三维交互范型的一种新方法。提出一个桌面虚拟环境下的笔式三维交互框架，包含交互原语和交互任务构造两个核心组件。首先采用词法到语法的二级封装机制来生成高级事件和交互原语，然后综合交互上下文、用户修正和手势识别实现基本交互任务到复杂交互任务的整合机制。框架中内嵌的手势交互、约束感知和混合交互技术能有效降低任务分解和模式切换带来的认知负担，提高交互自然性。

关键词: 桌面虚拟环境, 笔式交互, 三维交互, 交互框架