Call For Papers
International ACM Workshop on Crowdsourcing for Multimedia held in conjunction with ACM Multimedia 2013, Oct 21 - Oct 25 2013, Barcelona, Spain
The power of crowds-leveraging a large number of human contributors and the capabilities of human computation-has enormous potential to address key challenges in the area of multimedia research. Applications range from the exploitation of unsolicited user contributions, such as using tags to aid understanding of the visual content of yet-unseen images, to utilizing crowdsourcing platforms and marketplaces like Amazon's Mechanical Turk and CrowdFlower, which micro-outsource tasks such as semantic video annotation to a large population of workers. Further, crowdsourcing offers a time- and resource-efficient method for collecting large volumes of input for system design and evaluation, making it possible to optimize multimedia systems more rapidly and to address human factors more effectively.
CrowdMM 2013 will solicit novel contributions to multimedia research that make use of human intelligence, but also take advantage of human plurality. We will especially encourage contributions that propose solutions for the key challenges that face widespread adoption of crowdsourcing paradigms in the multimedia research community. These include: identification of optimal crowd members (e.g., user expertise, worker reliability), providing effective explanations (i.e., good task design), controlling noise and quality in the results, designing incentive structures that do not breed cheating, adversarial environments, gathering necessary background information about crowd members without violating privacy, controlling descriptions of task. Particular emphasis will be put on contributions that successfully combine human and automatic methods in order to address multimedia research challenges.
This workshop encourages theoretical, experimental, and methodological developments advancing state-of-the-art knowledge of crowdsourcing techniques for multimedia research and novel applications using crowdsourcing to solve traditional challenges in multimedia research. Topics include, but are not limited to the use of crowds, wisdom of crowds, or human computation in multimedia, in the following areas of research:
- Creation: content synthesis, authoring, editing, and collaboration, summarization and storytelling
- Evaluation: evaluation of multimedia signal processing algorithms, multimedia analysis and retrieval algorithms, or multimedia systems and applications
- Retrieval: analysis of user multimedia queries, evaluating multimedia search algorithms and interactive multimedia retrieval
- Annotation: generating semantic annotations for multimedia content, collecting large-scale input on user affective reactions
- Human factors: designing or evaluating user interfaces for multimedia systems, usability study, multi-modal environment, human recognition and perceptions
- Novel applications (e.g., human as an element in the loop of computation)
- Effective Learning from crowd-annotated or crowd-augmented datasets
- Quality assurance and cheat detection
- Economics and incentive structures
- Programming languages, tools and platforms providing enhanced support
- Inherent biases, limitations and trade-offs of crowd-centered approaches
CrowdMM 2013 welcomes submissions of full papers, as well as short papers reporting work-in-progress. Full papers must be no longer than 6 pages (inclusive of all figures, references and appendices). Short papers are 2 pages, and will be presented as Posters in an interactive setting.
All submissions must be written in English and must be formatted according to the ACM Proceedings style. They must contain no information identifying the author(s) or their organization(s). Reviews will be double-blind. Papers will be judged on their relevance, technical content and correctness, and the clarity of presentation of the research.
Detailed of the submission procedures will soon be available.
Accepted full and short papers will appear in the ACM Multimedia 2013 Workshop Proceedings and in the ACM Digital Library. Outstanding workshop papers will qualify for submission in extended form for a fast-track review at IEEE Transactions on Multimedia.
Submission due: June 28, 2013
Author notification: July 15, 2013
Camera ready due: July 25, 2013
Workshop date: October 22, 2013
Wei-Ta Chu, National Chung Cheng University, Taiwan
Martha Larson, Delft University of Technology, Netherlands
Kuan-Ta Chen, Academia Sinica, Taiwan
Publicity and Web:
Wei Tsang Ooi, National University of Singapore, Singapore