Research

With Ola Rønning and Andrzej Wąsowski I work on a simple question: how can an underwater vehicle know where it is when nothing is installed on the seabed? Our setup is a floating buoy with a hydrophone array and an inertial sensor (IMU) that listens to beacons. The trouble is the waves. They rock the buoy, so the array keeps changing its orientation, and the usual direction-finding methods stop working well.

In FP-MUSIC we use the IMU to correct for that motion before estimating the direction, and then use a matched filter to get the range. I presented this at RSE’26 in Barcelona and in a SQuaRe group talk. A second paper, You Should Be Properly Scoring Your Odometry, asks how we should judge these estimators in the first place.

What I learned

• The motion of the sensor has to be part of the model. Correcting for the waves raised the accuracy of telling two beacons 2 m apart from about 45% to 75% in simulation.
• A single error number can fool you. When we scored odometry filters on how well they knew their own uncertainty, all four were overconfident.
• In the worst case a filter claimed centimeter accuracy while its real error was kilometers.

Papers: FP-MUSIC · Scoring odometry

Mapping with sound has been overtaken by cameras and lidar, mostly because sound is noisy. But echoes still work in the dark, in bad weather, and against walls that do not reflect light. In this work, with Letizia Marchegiani and Jesper Rindom Jensen, I used machine learning to clean up the map that a normal signal processing method produces, by removing outliers and artefacts.

What I learned

• Machine learning works well here as a helper for the signal processing, not as a replacement for it.
• In simulation the cleaned-up maps still worked at a signal-to-noise ratio of −10 dB and in different reverberant rooms. So far this has only been tested in simulation.

Papers: A machine learning framework for acoustic reflector mapping

When a robot sends out a sound, every wall sends back an echo. If I can tell when each echo arrives and from which direction, I can work out where the walls are. That is harder than it sounds, because the robot’s own motors are loud and it is not obvious which echo belongs to which wall.

I used expectation-maximization (EM) to estimate the arrival times and the directions together, with a ring of microphones around a loudspeaker. I also made a version for a single microphone, and a later one that takes the loudspeaker’s own response into account.

What I learned

• A model-based method holds up much better in noise than simply picking peaks from the room impulse response. It still gave good estimates when the noise was about 10 dB stronger.
• Estimating time and direction together means I do not have to match echoes to walls first.
• The hardware matters. A loudspeaker never plays a perfect pulse, and estimating that imperfection inside the algorithm improved the arrival time accuracy by about 10% at low signal-to-noise ratio.

Papers: Modified EM algorithm · EM methods · Multichannel TOA and DOA · Distance estimation with ego-noise

Once I know where one reflector is, I can start building a map. The first method needs only one microphone and one loudspeaker, which many robots such as NAO already have, and it can find glass walls that cameras and lasers miss. The next paper uses the robot’s own motor noise to estimate how far away a wall is. The journal paper turns all of this into a full mapping framework with two algorithms, one for a single microphone and one for several.

What I learned

• Sound can sense things that light cannot, using hardware a robot already has.
• Ego-noise can be used as a signal instead of only being a problem.
• The hardest lesson was the gap between simulation and a real room. In simulation the framework found a reflector up to 1.5 m away at a signal-to-diffuse-noise ratio of 0 dB, but in the real environment it needed 10 dB to reach the same 80% accuracy.

Papers: Spatial map generation · Detecting reflectors with ego-noise · Model-based reflector localization

Perceiving the World With Sound, written with Robin Kerstens, is an introduction to how robots use sound. It starts with passive listening, finding where a sound such as a person speaking comes from. It then builds up to active sensing, where the robot sends out a sound and listens for the echoes, including neural networks and map building.

What I learned

• Robots do not have to rely on vision.
• The step I find most interesting is the one from listening to probing, sending a sound and reading its echo. That idea comes from animals that use echolocation, and it is what my other papers build on.

Papers: Perceiving the World With Sound