Zhenguo Yang: Multimodal Data Representation Learning for Event Detection from Photo-Sharing Social Media Data

Time

11:00-12:00, Mar. 23rd, 2017

 

Location

212, Jingyuan
 

Abstract

Social media platforms (e.g., Flickr, Facebook) provide new ways for users to share their photos and experiences, generating huge amounts of multimedia resources that are available on the Internet. As reported by Flickr, the number of uploaded images reached 7.28 billion in 2015. The massive data resources have attracted a great deal of research interest in exploring real-world concepts using user-shared data, such as dense crowd, 3D objects, ecological phenomena, places of interest, storyline summarization, visual concepts, and events. The speaker focuses on event detection from Flickr-like social media by addressing the problems, including heterogeneity of the multimodal data, the low discriminative power of raw data, and the processing of streaming data. In this talk, the speaker will introduce the proposed a three-stage framework to deal with the three problems. Specifically, to address the heterogeneity problem, we propose to construct bipartite graphs based on data dictionaries. To address the low discriminative power problem, we propose a data representation learning model by incorporating four constraints: dense reconstruction error, low-rank, dictionary density inhomogeneity and local invariance. To address the streaming data, we devise a class-wise data recovery residual model by taking advantage of the rationale of data recovery. The proposed social event detection approach achieves the highest performance in terms of multiple metrics for the MediaEval Social Event Detection 2014 dataset. Finally, the speaker will give a conclusion and indicate some possible directions in this talk.

 

Lecturer

Zhenguo Yang