Objects as Audio-Visual Modal Sound Fields
Reconstructing how objects sound from multi-view images and a few impact recordings.
RESEARCH COLLECTION
Publications
Exploring visual, audio, and language representations.
* denotes equal contribution. Click a figure to explore the method.
Masked Diffusion Captioning for Visual Feature Learning
Learning visual representations by reconstructing masked captions with an image-conditioned diffusion model.
Masked Autoencoders are Secretly Efficient Learners
A lighter pre-training recipe unlocks the time and sample efficiency of masked autoencoders.
Efficient Vision-Language Pre-training by Cluster Masking
Masking clusters of visually similar patches for efficient vision-language representation learning.
MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections
Combining implicit representations and diffusion to recover 3D structure from limited microscopy projections.
A-ESRGAN: Training Real-World Blind Super-Resolution with Attention U-Net Discriminators
Attention U-Net discriminators for sharper real-world blind super-resolution.
SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation
Multi-view masked learning with Swin Transformers for 3D medical image segmentation.
Technical report
Bridging Audio-Visual Semantics with Language-Guided Synthesis
Language-guided cross-modal synthesis for audio-visual representation learning.