Abstract
There’s a general need of remote collaboration on physical tasks in ev- eryday life and work, such as obtaining assistance and instruction from a remote expert for assembling furniture or repairing a machine. While mul- timodal interfaces, such as those integrating video and audio, were found to be useful for communication, users may still suffer from the ambiguity of referential expressions in remote collaboration (e.g., indicating which object in the workspace to handle). We propose a novel annotation-based collabora- tive interface that allows users to add narratable annotations of objects, such as alphabetic labels, in a dynamic video-mediated workspace. By explicitly supporting object narratability, users may directly refer to objects by read- ing out the labels, which may relieve the inconvenience of making references to objects using sophisticated descriptions of object attributes or deictic pro- nouns (e.g., “here”, “there” etc.). This improves working efficiency greatly between user communication. We divide our system into two parts: “live- action annotation” and ”virtual annotation”. In live-action annotation, there is a remote user that can guide the client user by giving annotations imme- diately. While in virtual annotation, we have a server that stored defined instruction annotations to help client user on physical tasks step by step.