Projects

						Visual Search + New Approaches in Content-Based Image Retrieval Researchers Thomas Vetter, Bernhard Egger Project Partners Michael Renner & Theodore Davis, HGK Basel (lead); SodaTech AG; KEYSTONE AG Funding SNF Project Duration 1.10.2011 - 30.9.2013 With an exponential growth in both the generation and sharing of digital photographs, the topic of Content-Based Image Retrieval (CBIR) has become increasingly important across all disciplines. This is particularly true as the model of manually tagging content and context within every media file becomes unsustainable. While automation in keyword tagging can be found in products such as facial recognition software, by maintaining textual descriptions of visual media, both the stored labels and attempts to match them are grossly limited by the semantic gap. Nevertheless, with the inevitable advancement of image processing to transcode qualitative attributes into quantifiable data, design will play an everincreasing role in providing the bridge for users to access this information through visual means. Visual Search + explores the ways in which this bridge can be built, through the formation of an interdisciplinary team consisting of junior researchers from the departments of computer vision and interface design.   This research strives to make progress beyond the well-explored low-level CBIR attributes (color, contour, texture) and broaden the contextual interpretation of images by computers through reducing the sampled imagery to one type, portraits. A focus is placed on images involving the face, due to its crucial role in everyday communication. Furthermore, while the innate human ability to recognize and process features within the face exists across all mediums, there remains a great limitation to articulate and describe them in detail. Using the 3D Morphable Face Model, a technology capable of analyzing 2D photographs in regard to face attributes such as eye gaze, head tilt, age, sex, mouth activity, facial expression, beard, among others, this collaboration pushes the technology to meet time restrictions imposed by the real world environments of content management and retrieval. It is with this challenge that the 3D-MFM will also be utilized to generate an unlimited set of sample training imagery, used to teach Fast Image Detectors how to describe and quantify visual attributes with concise speed and accuracy.   For each facial attribute extracted, a visual module will be designed for the query of numerically stored data. Exploring the possibilities and usability of both 2D and 3D widgets, visual qualities within the image will be retrieved through the abstracted visual representation of those very same qualities. Realized through a web application environment (Linux, Apache, MySQL, PHP), this will allow for the seamless connection of an online graphical user interface, with a database used to store all extracted information, after having been populated through offline image processing.   Through a collaborative venture of the Graphics and Vision Research Group at the University of Basel and the Visual Communication Institute of FHNW HGK Basel, Visual Search + yields the real world application and user consideration of cutting-edge computer vision techniques, while providing the field of design an opportunity to realize investigations of interface design through the underpinning of developing technology.     Publications Pose Normalization for Eye Gaze Estimation and Facial Attribute Description from Still Images [pdf] Bernhard Egger, Sandro Schönborn, Andreas Forster and Thomas Vetter IN: Proceedings of the 36th German Conference of Pattern Recognition, GCPR 2014, Münster, Germany, September 2-5.   Interfacing CBIR: Designing Interactive Widgets to Query Attribute Data in Face Image Retrieval. Ted Davis, in HCI International 2014 Proceedings (HCII 2014 – DUXU), Crete, Greece (June 2014)   A Monte Carlo Strategy to Integrate Detection and Model-Based Face Analysis [pdf] Sandro Schönborn, Andreas Forster, Bernhard Egger and Thomas Vetter IN: Proceedings of the 35th German Conference of Pattern Recognition, GCPR 2013, Saarbrücken, Germany, September 3-6, LNCS Volume 8142, 2013, pp 101-110 . DOI: http://dx.doi.org/10.1007/978-3-642-40602-7_11