Jörg Rychen, Julie Semoroz, Alexander Eckerle, Richard HR Hahnloser, Rebecca Kleinberger
University of Zurich and ETH Zurich, Association Motonomy, University of Munich, MIT Media Lab
[Paper] [Github]
Abstract: Cetaceans show high cognitive abilities and strong social bonds. Acoustics is their primary modality for interacting with conspecifics and sensing the environment. Research of their echolocation behaviors and of their abilities to communicate with conspecifics and with humans has typically made use of visual signaling and interactive tactile systems adapted from research on primates or birds, even though cetaceans primarily use the acoustic modality to interact with conspecifics and to sense the environment. Research on cetacean vocal behaviors would therefore benefit from a purely acoustic communication system in which acoustic signals flow in both directions simultaneously (full-duplex). We present the design and implementation of such as a system for acoustically interacting with cetaceans in the wild, featuring digital echo-suppression. We tested the system in a pilot experiment in Arctic Norway, where killer whales and humpback whales cruise the waters. We achieved an echo suppression of about 18 dB leaving room for technical improvements that we address in the discussion. Nevertheless, the full-duplex system allowed us to vocally interact with the underwater acoustic scene by being able to listen while producing sounds. We first describe the potential of such a system for potential future interactions with cetaceans. We then present our pilot deployment and highlight the ability the system offers to actively experience the underwater sonic environment. Finally, we give examples of our initial explorative attempts to vocally interact with wild killer whales and humpback whales.
Video Caption. Some video caption and acknowledgement
Figure 1. Full-duplex communication system with echo suppression. The signal that is broadcast over the speaker (red) is transferred acoustically to the hydrophone (yellow), where it is superimposed over the external signal of interest (green). Echo suppression is a digital filter that models the transfer of the signal from speaker to microphone with an impulse response function (IRF). This modeled signal is subtracted from the raw microphone signal to yield a pure external signal (green)
Figure 2. Four classes of acoustic interactive experiments with cetaceans that can benefit from a full-duplex system: A: Computerized interactive playback experiments. Prerecorded acoustic stimuli are played back depending on the behavior of the animal. B: Phantom echo (artificial sonar target). A low latency digital signal processing unit detects incoming echolocation clicks, and then emits a computed echo to simulate a virtual target. C: Telecommunication between two animals to study their communication. This approach provides separated recordings of individuals, the signals can be modified or filtered in real-time, and additional signals can be injected. D: Direct human–animal vocal interaction. This setup allows humans to interactively experience the underwater acoustic space.
Figure 3. A: We installed the full-duplex interaction system on a small motorboat. The experimenter wore a headset and interacted with the underwater sonic environment. B: We mounted an underwater loudspeaker (LL916, Lubell Labs, USA) and four hydrophones (HTI-96, High Tech Inc., USA, red arrows) 70 cm below the speaker to a plastic frame made of polyethylene (PE-HD), that minimizes acoustic disturbances). The speaker was lowered overboard to about 6 m into the water. C: To protect the electronics from saltwater splashes, they were placed inside the cabin of the motorboat.
Figure 4. Overview of the full-duplex system architecture. We used a controller (CompactRio, NI-9057, National Instruments, USA) with a real-time Linux operating system. The integrated FPGA interfaces with input and output (IO) modules and runs the signal processing including the adaptive LMS filters for echo-suppression. Digital-to-analog converters (DAC) and analog-to-digital converters (ADC) with 24-bit sigma-delta converters were run at a sample rate of 51.2 kHz. All IO modules were driven by a common clock for synchronized sampling of all channels. The two output channels of the first NI-9260 module were amplified and connected to an analog stereo headphone. A directional condenser microphone was connected to a microphone preamplifier and then to the analog input module NI-9250. The underwater loudspeaker (Lubell Labs, USA) was driven by a mono 100-W class-D amplifier (TPA3116D2, Texas Instruments), followed by an impedance matching circuit (Transformer Model AC202B, Lubell Labs, USA). The signal was generated by one channel of the second NI-9260 module. We used four hydrophones (High Tech Inc., USA) with integrated ICP preamplifiers. These were connected directly to the four input channels of the NI-9234 module, which delivered an excitation current of 2 mA to each of the preamplifiers. We used the factory calibration measurement for hydrophones with sensitivities of -170 dB re 1 V/µPa (+/- 0.3 dB). The system used about 10 W and the 12 V, 100 Wh battery allowed for a run-time of at least 6 h. All signals on the FPGA were saved to a file on an SD-card. Live inspection of the signals and device control was made possible thanks to a user interface running on a portable computer. All programming was done in LabVIEW (National Instruments, USA).
Figure 6. Four example spectrograms of interactions with humpback and killer whales. The horizontal axis is time, with extent indicated in seconds. The vertical axis is linear in frequency starting from zero with extent indicated in kHz. Green-yellow is the signal from the hydrophones and indicate all background sounds including animal vocalization. Overlaid in purple is the human signal broadcast through the speaker. A: Humpback growls alternating with human quacks. B: Down sweep chorus with orcas. C: Rhythmic humpback whups alternating with human whup imitations. D: Orca click trains presumably aimed at our acoustic system right after a human high-frequency sound. Some of the clicks overloaded the signal (vertical lines down to the lower edge).