Two Electrical Engineering Profs Land Awards from Yahoo! Labs

Two electrical engineering professors from the Jacobs School of Engineering, Gert Lanckriet and Nuno Vasconcelos, have won spots in the 2013 Yahoo! Faculty Research and Engagement Program (FREP). 

Check out the full list of awardees at the Yahoo! Labs Faculty Research and Engagement Program website.

This program funds help academics across the globe collaborate with Yahoo! research scientists on new, exciting internet research studies and experiments.

Lanckriet and Vasconcelos, from the Department of Electrical and Computer Engineering at UC San Diego, are two of 27 recipients from 24 universities in 7 countries were selected based on their areas of science and interest to Yahoo!.

A summary of Lanckriet’s project is below.

Machine Learning with Highly Unstructured Multimedia DataAn increasingly important aspect of information in today’s world is that it is highly unstructured and heterogeneous. For example, a search for “miss world 2013” yields fundamentally different modes of information about the topic via news articles, blogs, videos, pictures etc.  While there are some commonalities in the information provided in these modalities (i.e., they are about the beauty pageant), the nature of information differs widely amongst the modalities.

Videos and pictures are innately visual in nature, and differ from news articles and blogs which are textual in nature. Even within the textual sources, there exist differences: news articles are more factual in nature, while blogs are opinionated. In multimedia applications, such information is often available in a variety of unstructured formats: blogs may contain one or more pictures, or none, embedded videos, audio tracks, and/or tweets, or not, etc.

Ideally, an algorithm to make inferences from the data (e.g., whether or not the judges were fair in deciding the winner) would be given access to all available information sources, since each source may bring some unique information to the table. Machine learning with such unstructured, heterogeneous data is challenging and an active topic of research. Existing approaches assume a rigidly structured input format. They cannot accommodate missing modalities (incomplete information) or modalities with multiple instances (over-complete information).  In this project, we intend to design and develop principled approaches to learning from heterogeneous, unstructured multimedia data.

One of the problems that we are interested in is artist recommendation. Current approaches rely on either collaborative filtering methods (relying on the wisdom of the crowd to determine if two artists are similar), or content based methods (two artists are considered similar if the audio content of their songs are similar) to determine if two artists are similar.  We are interested in developing a multimodal artist similarity, based on heterogeneous unstructured social and multimedia data that relates to artists.  This can include band videos and pictures (which implicitly provide information about the “look-and-feel” of an artist), tweets, reviews, and blogs about the artists’ music (providing opinions and information about artists), lyrics, their social network of listeners, etc. We believe that combining all this information through a more accurate multimodal similarity measure will improve recommendation compared to using content or collaborative filtering only.