Abstract
Information search is a common yet important task in everyday work and life. It remains a challenging design issue how to help users search for information or things they don’t necessarily know how to express with words. Also, even when people know how to express, the cognitive cost required to retrieve the concepts and formulate the queries can be excessive. In this paper, we present Click-Search, a search user interface that allows people to indicate their search intents by merely selecting and cropping segments of image contents. The system automatically converts selected image segments to keywords based on known associations between image pixels and semantic labels created by prior crowdsourced image tagging. Through a user study, we found that Click-Search can support a range of searching activities effectively. We discuss the implications of the new approach of searching through interactions with images.