As intelligent systems increasingly blend into our everyday life, artificial social intelligence becomes a prominent area of research. Human language offers a unique unconstrained approach to probe through questions and reason through answers about social situations. This unconstrained approach extends previous attempts to model social intelligence through numeric supervision (e.g. sentiment and emotions labels). This is the cornerstone of the Social-IQ dataset. The dataset contains rigorously annotated and validated videos, questions and answers, as well as annotations for the complexity level of each question and answer. Social-IQ brings novel challenges to the field of artificial intelligence which sparks future research in social intelligence modeling, visual reasoning, and multimodal question answering.

Get Started

You can download the Social-IQ from our github page. We will periodically update the data and add new descriptors under the CMU-Multimodal SDK format. You can download the evaluation scripts as well.

C5IjwQdVMAYm6YS.jpg
Leader Board

You can participate in the Social-IQ challenge(s). The private test set will be used to evaluate the results. 

Leaderboard will keep track of the top performing models.

Join Our Mailing List
Submit to Leaderboard

Submit to the leaderboard of the Social-IQ. The submission process is easy and currently open. Your results will be added to the leaderboard. To acquire the private test data, please contact Amir Zadeh (abagherz@cs.cmu.edu).

Watch the CVPR Talk

Watch the Social-IQ presentation at CVPR 2019. We discuss the overall statistics of the datasets as well as baseline performances. You can also have a look at our poster

 

1250

Videos

7500

Questions

30k

Correct Answers

22.5k

Incorrect Answers

News

[06/15/2019] We are releasing a public test set for Social-IQ 1.1 on July 15th with 100 publicly available test data. 

[06/15/2019] Watch the CVPR 2019 talk. 

[06/15/2019] Github is now live!

[06/15/2019] Computational sequences are available for download through CMU Multimodal SDK.

[06/15/2019] Social-IQ Version 1.0 released. 

 
 

Team

citations.jpeg

Amir Zadeh is receiving his Ph.D. in artificial intelligence from Carnegie Mellon University, School of Computer Sciences.

Amir Zadeh

photo.png

Paul Liang is receiving his Ph.D. in Machine Learning from Machine Learning Department, Carnegie Mellon University School of Computer Sciences. 

Paul Pu Liang

Siverling.jpg

Nicole Siverling is the lab manager at Multicomp Lab and a manager of resources in Social-IQ project.

Nicole Siverling

image1_edited.jpg

Michael Chan is an undergraduate student at School of Computer Sciences, Carnegie Mellon University. He is the webmaster of Social-IQ and developer of CMU-Crowd.

Michael Chan

IMG_0644_edited.jpg

Amy Lee is an undergraduate student at Tepper School of Business, Carnegie Mellon University. She is the annotation manager of Social-IQ project.

Amy Lee

image001.jpg

Louis-Philippe Morency is an associate professor at Language Technologies Institute, Carnegie Mellon University School of Computer Sciences and the Director of Multicomp Lab.

Louis-Philippe Morency