Listen to the animal.
Short excerpts of hydrophone recordings: humpback song, dolphin whistles, sperm-whale codas, and killer-whale calls.
This HTML file needs the app server to load its scripts and recordings. Open the running orchestra. If it is not running, start it with uv run start.
Marine biodiversity has a sound of its own. This orchestra transforms real recordings into music for human ears, guided by measured rhythms and pitch contours. The prototype plays a small selection of short excerpts from open research collections. Choose your musicians on the map, or hear a finished piece.
Choose recordings to check saved versions.
Choose Base for a full composition or Stable Audio for a faster alternative. Each model keeps its own version.
This server has no CUDA GPU. Composing runs on the CPU and may take a while. Finished pieces are ready to play immediately.
Listen to the recorded voices and the music they helped shape.
Individual voices become musical guides, combined into one score for the composition. Coloured ticks mark source phrases. Select a point on any track to listen from there.
Their musical guides, combined on the same timeline
Real recordings enter the system. Signal analysis measures their phrases. Those measurements become a playable guide, which a generative model uses to compose music.
Short excerpts of hydrophone recordings: humpback song, dolphin whistles, sperm-whale codas, and killer-whale calls.
Onsets, durations, click groups, and pitch contours are extracted from audio. Tonal calls and clicks have different representations.
Contours become tones and clicks become mallet strikes. Each player's instrument and register are combined with your trims, timing, and arrangement.
Your selected model transforms the combined guide into music. A deterministic response provides a comparison that follows the measured events by construction.
The diagrams above illustrate the process. The spectrograms and phrase marks below come from the selected recording.
We selected excerpts from the DCLDE killer-whale detection and classification collection for its regional and acoustic diversity. Its authors describe it as the largest curated DCLDE audio-and-annotation dataset at the time of publication in 2025.
Dataset paperThese figures describe the research collection. This prototype uses a small selection of its excerpts, together with excerpts from earlier DCLDE workshops (2011–2022), the Alfred Wegener Institute's polar recordings on PANGAEA, and Orcasound, OpenWhistle and Dominica Sperm Whale Project recordings, to bring different waters and species into the same composition. The source collections are listed below.
Human and marine-mammal hearing differ. Registers can be shifted into a musical range, while measured event times and contour shapes remain the guide. The timing scores below test how closely the generated music follows the measured event times.
NOAA: the variety of marine mammal soundsTiming scores compare musical onsets with the animal's measured onsets, against the same output shifted in time. They help distinguish a response to this phrase from music that simply sounds appropriate.
The recordings are animal sounds. The measurements are signal analysis. The guide is a human-designed mapping and the composition is generated music. This process does not decode animal meaning.
We used Claude Code and OpenAI Codex to help develop and debug the software. Livia Zaharia and Anton Kulaga define the project concept, artistic direction and methodological choices, make the final decisions, and take responsibility for the work.
Measuring the recordings. Signal analysis extracts call timing and pitch contours. Our code maps those measurements to instruments and registers, then combines them into a playable audio guide.
Generating the music. We run ACE-Step 1.5's Base and Turbo checkpoints locally. Base uses 50 steps and guidance controls; Turbo provides an eight-step preview whose instrumentation and detail can differ. We also offer Stable Audio 3 Small Music, a smaller eight-step model. Each model receives the same guide, caption, duration and seed, with its own transformation controls, and keeps a separate generated version. The diffusion transformer generates music from the combined audio guide and a caption. Our current pipeline leaves the model's optional language-model planning stage disabled.
Checking the response. A deterministic musical response provides a comparison, and timing scores test how closely the generated music follows the measured events. Original recordings, constructed guides and generated music remain separately playable.
Jewellery methods and digital tests are on the Sound to silver page.
The prototype plays a few dozen excerpts, each under half a minute. They are a small subset, chosen by hand from open collections that hold years of recordings and many thousands of annotated calls. Each card says what we took.
Recorders on AWI moorings listen to the polar oceans year-round, and the Ocean Acoustics group publishes the recordings and their detection tables on PANGAEA. We take Antarctic blue whales from two moorings on the Greenwich meridian in the Weddell Sea, choosing each window from the published detections. Ilse Van Opzeeland, a co-author of both Weddell Sea recordings, works at AWI's Ocean Acoustics Lab.
Audio: Thomisch, Spiesecke, Lefering, Menze, Preis, Van Opzeeland & Boebel (2024), AWI232-11; Thomisch, Spiesecke, Van Opzeeland & Boebel (2024), AWI227-12. Detections: Thomisch, El-Gabbas, Spiesecke & Boebel (2023), AWI232-11 and AWI227-12.
OPUS recording archiveThe DCLDE workshops publish recordings with analysts' labels for detection and classification research. 2027: killer whales and humpbacks from the Salish Sea, the outer coast and Alaska. 2013: right and fin whales at Stellwagen Bank. 2015: blue whales off the Channel Islands. 2022: minke whales, striped and rough-toothed dolphins on a Hawaiian survey. 2011: dolphin whistles traced at Palmyra Atoll and in the mid-Atlantic.
NOAA NCEI passive acoustic archive; data from NOAA NEFSC, PIFSC and SWFSC, Scripps Institution of Oceanography, Fisheries and Oceans Canada, JASCO, ONC, Orcasound, SIMRES, SMRU Consulting and the University of Alaska.
NOAA passive acoustic data DCLDE 2027 dataset paperHumpback song at the Orcasound Lab hydrophone, with Emily Vierling's call annotations.
orcasound.netBottlenose dolphin whistle sequences recorded at Dolphin Reef, Eilat.
OpenWhistle on Hugging FaceCoda clips from the Dominica Sperm Whale Project, placed in sequence with short silences we added.
DSWP on Hugging FaceChair and legend silhouettes by Chris huh (CC BY-SA 3.0) and Margot Michaud (CC0).
phylopic.orgOnly the animal recordings come from these collections. The guides, the constructed responses and the generated music are made by this project.
Add players in the concert hall to adjust their phrases here.
In the first study, cover follows the guide's timing more closely. Lego tracks mostly ignore it.
Nothing composed in this session yet.
Inline uses your supplied base with an open gap between the ends and a smooth middle spine. Recording amplitude sets the outward height; frequency sets the spacing of the peaks. Smooth transverse connections appear at the wave peaks while the outer rails stay fixed. Compare Neutral and Sound, or press Play sound to listen.
Preparing the ring…
Sound shapes these forms through artistic mappings chosen by Livia. The same approach can be applied to other forms she designs.
Planned editions. Website visitors will be able to choose a recording and download its sound-shaped STL. At exhibitions, a small batch of 3D printed pieces made from the default sound combinations will be available to take or buy to support the research.
A limited cast-silver edition is planned for visitors who donate to marine-science and conservation causes. Each exhibition will name the beneficiary and state the donation amount.
Earlier whole-ring bending tests checked finger openings, wall thickness and gaps on Livia's Inline and Roots models. The current additive previews still need their parts joined and checked before fabrication.
The pilot asks which features of a dolphin whistle can survive a compact inscription. Its input is the annotated pitch contour from an OpenWhistle recording. We sample the contour at 32 points, store each pitch in 8 bits, and add 16 bits for start and end times: 272 bits per whistle. A constant-amplitude tone reconstructed from those measurements is the output. A separate version maps time around a ring band and pitch across its width, forming a groove.
Using the dolphin-trained OpenWhistle encoder, we trained a classifier on original recordings and tested it on the same 420 held-out whistles in each condition. Recording sessions are separate across the training, validation and test sets. The labels are six whistle types: five signature types associated with named dolphins and one shared non-signature type. They identify types, rather than verified callers.
| Audio tested | Macro-F1 |
|---|---|
| Original recordings | 0.822 |
| Tones from 272-bit pitch contours | 0.617 |
| Reversed contours | 0.229 |
| Flat tones at each whistle's mean pitch | 0.256 |
| Amplitude-only reconstruction, following Hardata (2021) | 0.096 |
Some distinctions between whistle types remain in the contour tones. Reversing or flattening the contour reduces the classifier's score, suggesting that contour shape matters to this model. This tests a model's response, not what dolphins hear or what their calls mean.
The groove version also retained type distinctions after simulated blur and surface noise. That digital simulation does not establish casting tolerance: printing, casting and polishing tests remain to be done.
Whale & Dolphin Orchestra grows from Livia Zaharia's practice as a Romanian architect, established parametric jewellery designer and science-art artist. It brings her work with computational form into marine sound and silver. Anton Kulaga contributes bioinformatics, longevity research and open-source software.
A Romanian architect, Livia works across computational design, contemporary jewellery and science art. Under Paral Design, she uses Grasshopper, COMPAS and Python to generate forms, then 3D-prints and casts them mainly in silver, sometimes with amber, walnut husks or quartz. Her jewellery has been shown at Romanian Jewelry Week each year since 2021.
Anton works in longevity research and bioinformatics at the Systems Biology of Aging Group. He builds open-source computational tools. For this project, he built the map, the recording-analysis pipeline, the music generation software and the website.
The Biology collection follows different stages and parts of a cell through jewellery. The Nanot of Power is one sterling-silver pendant among several works.
An interactive science-art experience made with GlucoseDAO and Longevity Genie. Visitors explore biological traits; Livia shapes their selections into printable crystals, while Anton contributes biology and software.
A walk-through 3D town where Livia's jewellery forms grow into architecture.
The repository holds this website, the Python engine that measures the recordings and composes the pieces, and the research behind both: datasets, models, art precedents and the silver studies.
Recordings come from the Alfred Wegener Institute's polar recordings on PANGAEA, the NOAA DCLDE workshop datasets, Orcasound, OpenWhistle and the Dominica Sperm Whale Project. Photographs of Livia's jewellery and her portrait: Livia Zaharia. To talk about the project, write to Livia on Instagram or LinkedIn, or open an issue on GitHub.