Andrew Owens

Please turn on JavaScript to view email address.
Office: EECS 4231
Assistant Professor of Electrical Engineering and Computer Science
University of Michigan

Google Scholar  ·  GitHub  ·  CV

I'm an assistant professor at The University of Michigan in the EECS department. If you'd like to learn more about the computer vision group at Michigan, please look here!

I did my PhD at MIT CSAIL, where I was advised by William Freeman and Antonio Torralba, and was a postdoc at UC Berkeley with Alyosha Efros and Jitendra Malik. Before that, I was an undergrad at Cornell.

Research highlights
      You can learn more about my research directions here.

Research group
PhD students:
   Daniel Geng
MS students:
   Zhangxing Bian ·  Ziyang Chen ·  Oscar de Lima ·  Xixi Hu

Resources
Teaching

Publications
Planar Surface Reconstruction from Sparse Views
Linyi Jin, Shengyi Qian, Andrew Owens, David F. Fouhey
arXiv 2021
project page · paper · video · code · bibtex
@article{jin2021planar, title={Planar Surface Reconstruction from Sparse Views}, author={Jin, Linyi and Qian Shengyi and Owens, Andrew and Fouhey, David F}, journal={arXiv}, year={2021} }
We create a planar reconstruction of a scene from two very distant camera viewpoints.
Space-Time Correspondence as a Contrastive Random Walk
Allan Jabri, Andrew Owens, Alexei A. Efros
NeurIPS 2020 (Oral)
project page · paper · code · bibtex
@article{jabri2020spacetime, title={Space-Time Correspondence as a Contrastive Random Walk}, author={Jabri, Allan and Owens, Andrew and Efros, Alexei A}, journal={Neural Information Processing Systems (NeurIPS)}, year={2020} }
A simple, self-supervised method for video representation learning. Train a random walker to traverse a graph derived from a video. Learn an affinity function that makes it return to the place it started.
Self-Supervised Learning Of Audio-Visual Objects From Video
Triantafyllos Afouras, Andrew Owens, Joon Son Chung, Andrew Zisserman
ECCV 2020
project page · paper · code · bibtex
@article{afouras2020selfsupervised, title={Self-supervised learning of audio-visual objects from video}, author={Afouras, Triantafyllos and Owens, Andrew and Chung, Joon Son and Zisserman, Andrew}, journal={European Conference on Computer Vision (ECCV)}, year={2020} }
We learn from unlabeled video to represent a video as a set of discrete audio-visual objects. These can be used as drop-in replacements for face detectors in speech tasks, including 1) multi-speaker source separation, 2) active speaker detection, 3) correcting misaligned audio and visual streams, and 4) speaker localization.
CNN-generated images are surprisingly easy to spot... for now
Sheng-Yu Wang, Oliver Wang, Richard Zhang, Andrew Owens, Alexei A. Efros
CVPR 2020 (Oral)
project page · paper · code · bibtex
@article{wang2019cnn, title={CNN-generated images are surprisingly easy to spot... for now}, author={Wang, Sheng-Yu and Wang, Oliver and Zhang, Richard and Owens, Andrew and Efros, Alexei A}, journal={Computer Vision and Pattern Recognition (CVPR)}, year={2020} }
Forensics classifiers trained to spot one type of CNN-generated image generalize surprisingly well to images made by other networks, too.
Detecting Photoshopped Faces by Scripting Photoshop
Sheng-Yu Wang, Oliver Wang, Andrew Owens, Richard Zhang, Alexei A. Efros
ICCV 2019
project page · paper · video · code · bibtex
@article{wang2019detecting, title={Detecting Photoshopped Faces by Scripting Photoshop}, author={Wang, Sheng-Yu and Wang, Oliver and Owens, Andrew and Zhang, Richard and Efros, Alexei A}, journal={International Conference on Computer Vision (ICCV)}}, year={2019} }
We detect manipulated face photos, using only training data that was automatically generated by scripting Photoshop.
Learning Individual Styles of Conversational Gesture
Shiry Ginosar*, Amir Bar*, Gefen Kohavi, Caroline Chan, Andrew Owens, Jitendra Malik
CVPR 2019
project page · paper · video · bibtex
@article{ginosar2019learning, title={Learning Individual Styles of Conversational Gesture}, author={Ginosar, Shiry and Bar, Amir and Kohavi, Gefen and Chan, Caroline and Owens, Andrew and Malik, Jitendra}, journal={Computer Vision and Pattern Recognition (CVPR)}, year={2019} }
We predict a speaker's arm/hand gestures from audio.
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Andrew Owens, Alexei A. Efros
ECCV 2018 (Oral)
paper · project page · video · talk · slides (key, ppt) · code · bibtex
@article{owens2018audio, title={Audio-visual Scene Analysis with Self-Supervised Multisensory Features}, author={Owens, Andrew and Efros, Alexei A}, journal={European Conference on Computer Vision (ECCV)}, year={2018} }
We use self-supervision to learn a multisensory representation that fuses the audio and visual streams of a video. We apply it to: 1) sound-source localization, 2) action recognition, 3) on/off-screen audio source separation.
Fighting Fake News: Image Splice Detection via Learned Self-Consistency
Minyoung Huh*, Andrew Liu*, Andrew Owens, Alexei A. Efros
ECCV 2018
paper · project page · video · code · bibtex
@article{huh2018fighting, title={Fighting Fake News: Image Splice Detection via Learned Self-Consistency}, author={Huh, Minyoung and Liu, Andrew and Owens, Andrew and Efros, Alexei A}, journal={European Conference on Computer Vision (ECCV)}, year={2018} }
We detect images that are not "self-consistent", using an anomaly detection model that was trained only on real images.
More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch
Roberto Calandra, Andrew Owens, Dinesh Jayaraman, Justin Lin, Wenzhen Yuan, Jitendra Malik, Edward H. Adelson, Sergey Levine
RA-L 2018
RA-L 2018 Best Paper Award Finalist
paper · video · project page · bibtex
@article{calandra2018more, title={More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touch}, author={Calandra, Roberto and Owens, Andrew and Jayaraman, Dinesh and Lin, Justin and Yuan, Wenzhen and Malik, Jitendra and Adelson, Edward H and Levine, Sergey}, journal={Robotics and Automation Letters (RA-L)}, year={2018} }
We train a robot to adjust its grasp, using both vision and touch sensing.
MoSculp: Interactive Visualization of Shape and Time
Xiuming Zhang, Tali Dekel, Tianfan Xue, Andrew Owens, Qiurui He, Jiajun Wu, Stefanie Mueller, William T. Freeman
UIST 2018
paper · project page · bibtex
@article{zhang2018mosculp, title={MoSculp: Interactive Visualization of Shape and Time}, author={Zhang, Xiuming and Dekel, Tali and Xue, Tianfan and Owens, Andrew and Wu, Jiajun and Mueller Stefanie and Freeman, William T.}, journal={User Interface Software and Technology (UIST)}, year={2018} }
We summarize complex motions using a representation called a motion sculpture.
The Feeling of Success: Does Touch Sensing Help Predict Grasp Outcomes?
Roberto Calandra, Andrew Owens, Manu Upadhyaya, Wenzhen Yuan, Justin Lin, Edward H. Adelson, Sergey Levine
CoRL 2017
paper · project page · bibtex
@article{calandra2017feeling, title={The feeling of success: Does touch sensing help predict grasp outcomes?}, author={Calandra, Roberto and Owens, Andrew and Upadhyaya, Manu and Yuan, Wenzhen and Lin, Justin and Adelson, Edward H and Levine, Sergey}, journal={Conference on Robot Learning (CoRL)}, year={2017} }
Touch sensing makes it easier to tell whether a grasp will succeed.
Shape-independent Hardness Estimation Using Deep Learning and a GelSight Tactile Sensor
Wenzhen Yuan, Chenzhuo Zhu, Andrew Owens, Mandayam Srinivasan, Edward H. Adelson
ICRA 2017
paper · video · bibtex
@inproceedings{yuan2017shape, title={Shape-independent Hardness Estimation Using Deep Learning and a GelSight Tactile Sensor}, author={Yuan, Wenzhen and Zhu, Chenzhuo and Owens, Andrew and Srinivasan, Mandayam A and Adelson, Edward H}, booktitle={International Conference on Robotics and Automation (ICRA)}, year={2017}, }
We can estimate the hardness of an object by analyzing the way that it deforms a touch sensor.
Ambient Sound Provides Supervision for Visual Learning
Andrew Owens, Jiajun Wu, Josh McDermott, William T. Freeman, Antonio Torralba
ECCV 2016 (Oral)
paper · journal paper (2018) · project page · models · bibtex
@inproceedings{owens2018ambient, title={Learning Sight From Sound: Ambient Sound Provides Supervision for Visual Learning}, author={Owens, Andrew and Wu, Jiajun and McDermott, Josh H and Freeman, William T and Torralba, Antonio}, booktitle={International Journal of Computer Vision (IJCV)}, year={2018}, } @inproceedings{owens2016ambient, title={Ambient Sound Provides Supervision for Visual Learning}, author={Owens, Andrew and Wu, Jiajun and McDermott, Josh H and Freeman, William T and Torralba, Antonio}, booktitle={European Conference on Computer Vision (ECCV)}, year={2016}, }
When we train a neural network to predict sound from sight, it learns to recognize objects and scenes — without using any labeled training data.
Visually Indicated Sounds
Andrew Owens, Phillip Isola, Josh McDermott, Antonio Torralba, Edward H. Adelson, William T. Freeman
CVPR 2016 (Oral)
paper · project page · video · bibtex
@inproceedings{owens2016visually, title={Visually indicated sounds}, author={Owens, Andrew and Isola, Phillip and McDermott, Josh and Torralba, Antonio and Adelson, Edward H and Freeman, William T}, booktitle={Computer Vision and Pattern Recognition (CVPR)}, year={2016} }
What sound does an object make when you hit it with a drumstick? We use sound as a supervisory signal for learning about materials and actions.
Camouflaging an Object from Many Viewpoints
Andrew Owens, Connelly Barnes, Alex Flint, Hanumant Singh, William T. Freeman
CVPR 2014 (Oral)
paper · project page · video · code · bibtex
@inproceedings{owens2014camouflaging, title={Camouflaging an object from many viewpoints}, author={Owens, Andrew and Barnes, Connelly and Flint, Alex and Singh, Hanumant and Freeman, William}, booktitle={Computer Vision and Pattern Recognition (CVPR)}, year={2014} }
We texture a 3D object so that it is hard to see, no matter where it is viewed from.
Shape Anchors for Data-Driven Multi-view Reconstruction
Andrew Owens, Jianxiong Xiao, Antonio Torralba, William T. Freeman
ICCV 2013
paper · project page · bibtex
@inproceedings{owens2013shape, title={Shape anchors for data-driven multi-view reconstruction}, author={Owens, Andrew and Xiao, Jianxiong and Torralba, Antonio and Freeman, William}, booktitle={International Conference on Computer Vision (ICCV)}, year={2013} }
Some image regions are highly informative about 3D shape. We use this idea to make a multi-view reconstruction system that exploits single-image depth cues.
SUN3D: A Database of Big Spaces Reconstructed using SfM and Object Labels
Jianxiong Xiao, Andrew Owens, Antonio Torralba
ICCV 2013
paper · project page · video · bibtex
@inproceedings{xiao2013sun3d, title={SUN3D: A Database of Big Spaces Reconstructed using SfM and Object Labels}, author={Xiao, Jianxiong and Owens, Andrew and Torralba, Antonio}, booktitle={International Conference on Computer Vision (ICCV)}, year={2013} }
A large dataset of 3D-reconstructed indoor scenes.
Discrete-Continuous Optimization for Large-Scale Structure from Motion
David Crandall, Andrew Owens, Noah Snavely, Dan Huttenlocher
CVPR 2011 (Oral)
CVPR Best Paper Award Honorable Mention
paper · journal paper (2013) · project page · video · bibtex
@article{crandall2013pami, author = {David Crandall and Andrew Owens and Noah Snavely and Daniel Huttenlocher}, title = {{SfM with MRFs}: Discrete-Continuous Optimization for Large-Scale Structure from Motion}, journal = {Transactions on Pattern Analysis and Machine Intelligence (PAMI)}, year = {2013}, } @inproceedings{crandall2011cvpr, author = {David Crandall and Andrew Owens and Noah Snavely and Daniel Huttenlocher}, title = {Discrete-Continuous Optimization for Large-scale Structure from Motion}, booktitle = {Computer Vision and Pattern Recognition (CVPR)}, year = {2011} }
Discrete Markov random fields can solve structure-from-motion problems, while incorporating extra information such as GPS and vanishing lines.