Learning visual similarity for product design with convolutional neural networks
Sean Bell,
and Kavita Bala
ACM Transactions on Graphics (SIGGRAPH 2015), 34(4), August 2015
Download: [ Paper PDF (13.2 MB)]
 |
Abstract: Popular sites like Houzz, Pinterest, and LikeThatDecor, have communities of users helping each other answer questions about products in images. In this paper we learn an embedding for visual search in interior design. Our embedding contains two different domains of product images: products cropped from internet scenes, and prod- ucts in their iconic form. With such a multi-domain embedding, we demonstrate several applications of visual search including identify- ing products in scenes and finding stylistically similar products. To obtain the embedding, we train a convolutional neural network on pairs of images. We explore several training architectures including re-purposing object classifiers, using siamese networks, and using multitask learning. We evaluate our search quantitatively and qualita- tively and demonstrate high quality results for search across multiple visual domains, enabling new applications in interior design.
Links:
- Supplemental material (zip, 164.5 MB)
- Talk Slides (keynote, 212.8 MB)
- Embedding viewer