Seadragon Whistles Exchange

Leafy Seadragon demoed at International Conference in Boston, July 2007

2007-10-08T15:21:00.000-05:00

Leafy Seadragon, the first open source application to research cetacean communication by using two-way underwater acoustic interactions, was successfully demonstrated at the 3rd International Workshop on Detection and Classification of Marine Mammals using Passive Acoustics, on July 24, 2007, in Boston.

Ken Ekstrom from MIT did a great job with his audio equipment and personal whales and dolphins recordings.

Leafy was also successfully tested in a sea harbor, using a laptop and two hydrophones from Aquarian Audio, and without any marine mammals nearby.

Example 3: Whistle s1at40

2006-12-02T17:36:00.000-05:00

s1at100 : The target shape at 100 fsps for whistles s1at10 (see post Example 2) and s1at40 (in this post). This kind of definition (i.e., 100 frequency samples per second, fsps) cannot be achieved with Seadragon 2.1 and the best that it can do today is 40 fsps, second diagram, because of the capacity limits of current x86 computers. Whistle s1 at 100 fsps:

s1at40: The frequency values that we can use at 40 fsps are graphed below, each column represents a frequency value that we can write in the xml file signals_to_read.xml for a specific time:

Vertical Scale: In the two diagrams above, the vertical scale is the same, at approximately one horizontal line for each distinct frequency value that we can acquire at 48,000 voltage samples per second (vsps) using FFT on a series of 1024 voltage samples (one series per frequency value). With a normal FFT like we are using in Seadragon and this voltage sampling rate, we get approximately 20 lines per kHz, or 50 Hz per line. By contrast, in the diagram in post Example 2 for s1at10, the vertical scale is 1000 Hz per line. For s1at10, the fsps of 10 and the shape of the signal are such that we don't need to write more frequency values in the xml for that whistle. But at 40 fsps, we can write more frequency samples to shape the whistle better, thus we get s1at40.

s1at40 vs. s1at10: Whistle s1at10 has approximately the same frequency values at the same time positions as whistle s1at10 therefore these two whistles should be similar acoustically. Both s1at10 and s1at40 may be representing the same acquired whistle, i.e., they could be representing the same actual dolphin whistle.

Seadragon should be able to recognize the actual whistle by matching it with either s1at10 or s1at40. You should not have both whistles in the same xml file because one of these may be ignored during recognition (i.e., the pattern matching process). You should use s1at10 in the xml file with the Seadragon application which is running at 10 fsps and use s1at40 with the Seadragon which is running at 40 fsps.

This example is meant to make you aware that the difference in the sampling rate for the frequency values, the fsps, is causing significant differences between the representation at 10 and 40 fsps for this kind of whistles. These differences are not caused by the voltage sampling rate because they both use the same rate of 48,000 voltage samples per second (vsps). The key frequency samples are the same for both s1at10 and s1at40. Only the frequency sampling rate differs. This shows why it is preferable to use a computer that allows us to run at 40 fsps instead of 10 fsps.

Recall that s1at10 is drawn in another post, Example 2, and that both s1at10 and s1at40 attempt to represent the whistle drawn in the first diagram in this post, which would require 100 fsps.

About the two diagrams in this post, please note that the main difference between the first diagram (the target or ideal shape) and the second one (the best possible shape today) is also the frequency sampling rate. The first diagram is based on a sampling rate of 100 frequency values per second (fsps), while with the current, normal, top of the line, off-the-shelf, laptops we can only reach 40 fsps. Changing the fsps requires minor code changes in Seadragon and a future version could run on a computer that could support 100 fsps. These days, only a top-of-the-line server computer (i.e., very expensive) has the resources to support 100 fsps.

Higher Voltage Sampling Rate: One of the main next improvements for Seadragon is probably not a higher fsps but a higher vsps (voltage samples per second) so that we may acquire and emit at 100 kHz or above. Currently we can acquire up to 24 kHz (effectively up to 18 kHz) and we can emit effectively up to 10 kHz. This limitation is caused by the built-in audio analog-to-digital and digital-to-analog hardware in off-the-shelf computers; hardware which is designed for human acoustics and not for dolphins. To be able to emit good quality whistles at 100 kHz we would need a vsps of 400,000 or more, instead of the current 48,000, which would require additional hardware. Fortunately, based on what we know so far, it seems that many dolphin species do not commonly emit whistles at 100 kHz and that a frequency range of 1 to 10 kHz for whistles may be sufficient for many species, such as Stenella, Tursiops, and Beluga, for example.

The xml text to insert in file signals_to_read.xml prior to starting the application:

<object class="org.leafyseadragon.jse.signal.StoredSignal">
<void property="hz40ps">
<array class="java.lang.Double" length="40">
<void index="0"><double>3000.0</double></void>
<void index="1"><double>3000.0</double></void>
<void index="2"><double>3000.0</double></void>
<void index="3"><double>3050.0</double></void>
<void index="4"><double>3100.0</double></void>
<void index="5"><double>3250.0</double></void>
<void index="6"><double>3500.0</double></void>
<void index="7"><double>3750.0</double></void>
<void index="8"><double>4000.0</double></void>
<void index="9"><double>4250.0</double></void>
<void index="10"><double>4500.0</double></void>
<void index="11"><double>4750.0</double></void>
<void index="12"><double>5000.0</double></void>
<void index="13"><double>5250.0</double></void>
<void index="14"><double>5500.0</double></void>
<void index="15"><double>5750.0</double></void>
<void index="16"><double>5900.0</double></void>
<void index="17"><double>5950.0</double></void>
<void index="18"><double>6000.0</double></void>
<void index="19"><double>5950.0</double></void>
<void index="20"><double>5900.0</double></void>
<void index="21"><double>5750.0</double></void>
<void index="22"><double>5500.0</double></void>
<void index="23"><double>5250.0</double></void>
<void index="24"><double>5100.0</double></void>
<void index="25"><double>5050.0</double></void>
<void index="26"><double>5000.0</double></void>
<void index="27"><double>5050.0</double></void>
<void index="28"><double>5100.0</double></void>
<void index="29"><double>5250.0</double></void>
<void index="30"><double>5500.0</double></void>
<void index="31"><double>5750.0</double></void>
<void index="32"><double>5900.0</double></void>
<void index="33"><double>5950.0</double></void>
<void index="34"><double>5950.0</double></void>
<void index="35"><double>6000.0</double></void>
<void index="36"><double>5950.0</double></void>
<void index="37"><double>6000.0</double></void>
<void index="38"><double>6050.0</double></void>
<void index="39"><double>6000.0</double></void>
</array>
</void>
<void property="signalType">
<string>LEX_SIGNAL</string>
</void>
<void property="text">
<string>s1at40</string>
</void>
<void property="uid">
<string>s1at40</string>
</void>
</object>

Copyright (c) 2006-2007 Serge Masse.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation. A copy of the license is included in http://www.gnu.org/licenses/fdl.txt

Example 2: Whistle s1at10

2006-12-02T12:45:00.000-05:00

s1at10 is equivalent to the hardcoded whistle s1 which is included in the software. This is a good example of limitations of the slower sampling rate of 10 frequency samples per second (fsps), because when there are large frequency variations between samples, e.g., 1000 Hz in some cases in this whistle, then the resulting whistles may be more linear than desired, i.e., without sufficient graduation or curves in the shape. A similar whistle at 40 fsps, s1at40, in a future post, will show the advantage of the faster sampling rate. Other whistles in future posts will show smaller variations between samples.

The approximate shape on a spectrogram:__ +------------+ 10 kHz | | __ | | __ | | __ | | _6 kHz | oo oo | _5 kHz | o oo | __ | o | _3 kHz | oo | __ | | __ | | _0 kHz +------------+ ___ 0 9 = 1 secondIndexes 0 to 9 at 10 fsps covers 1 second.

The effective maximum frequency that Seadragon 2.1 can emit is 10 kHz. Although Seadragon 2.1 can acquire frequencies has high as 24 kHz.

The xml code to insert in file signals_to_read.xml:

<object class="org.leafyseadragon.jse.signal.StoredSignal">
<void property="hz10ps">
<array class="java.lang.Double" length="10">
<void index="0">
<double>3000.0</double>
</void>
<void index="1">
<double>3000.0</double>
</void>
<void index="2">
<double>4000.0</double>
</void>
<void index="3">
<double>5000.0</double>
</void>
<void index="4">
<double>6000.0</double>
</void>
<void index="5">
<double>6000.0</double>
</void>
<void index="6">
<double>5000.0</double>
</void>
<void index="7">
<double>5000.0</double>
</void>
<void index="8">
<double>6000.0</double>
</void>
<void index="9">
<double>6000.0</double>
</void>
</array>
</void>
<void property="signalType">
<string>LEX_SIGNAL</string>
</void>
<void property="text">
<string>s1at10</string>
</void>
<void property="uid">
<string>s1at10</string>
</void>
</object>

Copyright (c) 2006 Serge Masse.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation. A copy of the license is included in http://www.gnu.org/licenses/fdl.txt

Example 1: whistle s9_1s code explained

2006-11-21T19:48:00.000-05:00

s9_1s is the name (aka. "text" near the bottom of the code in the post below) for the whistle and is what is typed by the human user to emit that whistle and what is displayed in the Message window when a dolphin emits a similar whistle (a whistle that matches this one).

hz10ps means that the frequency (hz) sampling rate is 10 per second (ps). If the program uses a frequency sampling rate of 40 samples per second instead of 10 samples per second, then the program will internally convert the whistle to that rate (40 per second) by calculating the intermediate frequency values automatically.

length="10" means that there are 10 values in the array or list (each value is a frequency value in Hz or cycle per second) and because the frequency sampling rate is 10 per second (hz10ps) then this means that the whistle will last 1 second when emitted, and that it will match an acquired signal of that duration at the given frequencies (9 kHz).

index="0" means that the frequency value is at position 0 in the list. The first position in this list is always zero.

<double>9000.0</double> means that the frequency value is 9000.0 cycles per second, or 9000.0 Hz or 9.0 kHz.

Copyright (c) 2006 Serge Masse.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation. A copy of the license is included in http://www.gnu.org/licenses/fdl.txt

Example 1: xml code for whistle s9_1s = 9 kHz tone lasting 1 second

2006-11-21T19:11:00.000-05:00

Like all whistles, you insert this xml code in file signals_to_read.xml before the last line </java>:

<object class="org.leafyseadragon.jse.signal.StoredSignal">
<void property="hz10ps">
<array class="java.lang.Double" length="10">
<void index="0">
<double>9000.0</double>
</void>
<void index="1">
<double>9000.0</double>
</void>
<void index="2">
<double>9000.0</double>
</void>
<void index="3">
<double>9000.0</double>
</void>
<void index="4">
<double>9000.0</double>
</void>
<void index="5">
<double>9000.0</double>
</void>
<void index="6">
<double>9000.0</double>
</void>
<void index="7">
<double>9000.0</double>
</void>
<void index="8">
<double>9000.0</double>
</void>
<void index="9">
<double>9000.0</double>
</void>
</array>
</void>
<void property="signalType">
<string>LEX_SIGNAL</string>
</void>
<void property="text">
<string>s9_1s</string>
</void>
<void property="uid">
<string>s9_1s</string>
</void>
</object>

Copyright (c) 2006 Serge Masse.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation. A copy of the license is included in http://www.gnu.org/licenses/fdl.txt

First Post

2006-11-21T18:49:00.000-05:00

This blog will contain whistles usable by the free Seadragon program to research dolphin communication.

The current version of the Seadragon program is 2.1. Future versions will probably include additional properties in the xml code for a whistle, such as human_source, cetacean_source, program_version, creation_date, etc.