ZME Science
No Result
View All Result
ZME Science
No Result
View All Result
ZME Science

Home → Science → News

Making walls talk – new technique extracts audio from video

Tibi PuiubyTibi Puiu
December 15, 2014 - Updated on November 16, 2020
in News, Technology
A A
Share on FacebookShare on TwitterSubmit to Reddit

RelatedPosts

Rattlesnakes modulate their tail wagging to make you think they’re closer than they are
The world’s most annoying sound: whining
What, really, is the speed of sound?
Researchers produce the loudest sound in the world inside tiny jets of water

A very simple, yet effective optical technique was demonstrated that can transform video inputs, such as the motion of a piece of paper, into audio. To achieve this, the researchers involved exploited a simple principle that describes how sound waves causes objects in their path to vibrate. If you reverse engineer the vibrations, you can effectively decode the sound source and play it back. In effect, the technique could be used to extract audio information from a silent video, like a remote surveillance, granting the walls ears. Of course, the video needs to be shot at high speed since the technique can’t work without many frames per second at its disposal. Also, the demonstrations are far from conclusive, but considering it’s a first version I found it rather impressive.

Turning video into sound

Graffiti artists can make walls turn to life, and speak to the hearts of people through art. Researchers have now given new meaning to the phrase "walls can speak". Image: Theater Fever
Graffiti artists can make walls turn to life, and speak to the hearts of people through art. Researchers have now given new meaning to the phrase “walls can speak”. Image: Theater Fever

Sound is nothing but vibration. When we hear, what we’re actually sensing is air displaced in a signature manner by a mechanical pressure wave which eventually hits special receptor cells in the inner ear. This information is then transformed by nerves and relayed to the brain where it’s decoded. Because it’s a pressure wave, these mechanical vibrations we call sound can cause objects in its path to vibrate as well. This vibration is so tiny that we hardly notice it, unless you have really powerful speakers. But even with you home stereo, you might be able to notice the effect sound waves have on other objects if the latter are small enough. The vibrations, although usually with small amplitudes, can be detected and analyzed algorithmically, and audio reconstructed based on those calculations.

The researchers from the Catholic University of America used a thin sheet of paper for their tests. Matrix points were spread on the image of the paper so that the sound vibration imaged by a high-speed camera could be mapped. The Gauss-Newton algorithm and a few other measures were applied to process the image, then a simple model enacted the original audio information of the sound waves.

Credit: Optical Engineering
Credit: Optical Engineering

If you have a video of two people taking, you can tell what their conversation is about by lipreading. If you can’t see the people’s faces, though, this can’t work. This technique might thus be useful to discern conversations by studying the vibrations of objects around people.

“One of the intriguing aspects of the paper is the ability to recover spoken words from a video of objects in the room,” said journal Associate Editor Reiner Eschbach, a Research Fellow at Xerox Corp. “The paper shows that the sound creates minute vibrations in objects and that these vibrations ― given the right equipment ― can be picked up from a video signal. This is an interesting foray into a new application space and will, in my view, trigger interesting research in the field.”

Of course, the technique needs to be further refined before anyone can read anything from vibrating bricks. The paper was published in the journal Optical Engineering.

Tags: soundvibration

ShareTweetShare
Tibi Puiu

Tibi Puiu

Tibi is a science journalist and co-founder of ZME Science. He writes mainly about emerging tech, physics, climate, and space. In his spare time, Tibi likes to make weird music on his computer and groom felines. He has a B.Sc in mechanical engineering and an M.Sc in renewable energy systems.

Related Posts

News

A Nearby Star Sings a Stellar Tune, and Scientists can Hear Its Age

byTudor Tarita
2 days ago
Science

Aztec Death Whistle Was Designed to Haunt the Mind, Brain Scans Confirm

byTibi Puiu
6 months ago
News

Scientists found a way to make sound travel in only one direction

byTibi Puiu
7 months ago
Mind & Brain

You can literally hear the sound of silence, surprising study says

byTibi Puiu
11 months ago

Recent news

Scientists Blasted Human Cells With 5G Radiation and the Results Are In

May 15, 2025

Orange Cats Are Genetically Unlike Any Other Mammal and Now We Know Why

May 15, 2025

Scientists Found ‘Anti Spicy’ Compounds That Make Hot Peppers Taste Milder

May 15, 2025
  • About
  • Advertise
  • Editorial Policy
  • Privacy Policy and Terms of Use
  • How we review products
  • Contact

© 2007-2025 ZME Science - Not exactly rocket science. All Rights Reserved.

No Result
View All Result
  • Science News
  • Environment
  • Health
  • Space
  • Future
  • Features
    • Natural Sciences
    • Physics
      • Matter and Energy
      • Quantum Mechanics
      • Thermodynamics
    • Chemistry
      • Periodic Table
      • Applied Chemistry
      • Materials
      • Physical Chemistry
    • Biology
      • Anatomy
      • Biochemistry
      • Ecology
      • Genetics
      • Microbiology
      • Plants and Fungi
    • Geology and Paleontology
      • Planet Earth
      • Earth Dynamics
      • Rocks and Minerals
      • Volcanoes
      • Dinosaurs
      • Fossils
    • Animals
      • Mammals
      • Birds
      • Fish
      • Amphibians
      • Reptiles
      • Invertebrates
      • Pets
      • Conservation
      • Animal facts
    • Climate and Weather
      • Climate change
      • Weather and atmosphere
    • Health
      • Drugs
      • Diseases and Conditions
      • Human Body
      • Mind and Brain
      • Food and Nutrition
      • Wellness
    • History and Humanities
      • Anthropology
      • Archaeology
      • History
      • Economics
      • People
      • Sociology
    • Space & Astronomy
      • The Solar System
      • Sun
      • The Moon
      • Planets
      • Asteroids, meteors & comets
      • Astronomy
      • Astrophysics
      • Cosmology
      • Exoplanets & Alien Life
      • Spaceflight and Exploration
    • Technology
      • Computer Science & IT
      • Engineering
      • Inventions
      • Sustainability
      • Renewable Energy
      • Green Living
    • Culture
    • Resources
  • Videos
  • Reviews
  • About Us
    • About
    • The Team
    • Advertise
    • Contribute
    • Editorial policy
    • Privacy Policy
    • Contact

© 2007-2025 ZME Science - Not exactly rocket science. All Rights Reserved.