15,610,865 members
Articles / Multimedia / GDI+
Article
Posted 21 Dec 2008

106.5K views
83 bookmarked

Draw into sound

Rate me:
The article illustrates how to draw into a spectral representation of an audio file.

Introduction

The idea is to draw pictures into sounds. In particular, draw pictures into the spectral representation of the sound. A sound can be represented in many ways. In particular, it can be drawn with Amplitude-Time representation (Waveform) and Frequency-Time representation (Spectrum). The first shows the amplitude of the sound in time:

The X-axis is the time while the Y-axis is the amplitude of the sound. The next kind of view shows the audio signal as components of different frequencies:

Here, the X-axis is time while the Y-axis is the frequency. In particular, lighter colors (white) means higher value of the component of the specific frequency, while darker colors (black) means a lower component. Here, it’s possible to see how the sound changes in time and frequency.

The Fast Fourier Transform

The Fourier Transform is an operation that transforms a signal (in our case, the audio signal) into its frequency domain representation. So, it is possible to see which frequencies are present in the input signal. The FFT is an algorithm that computes the Fourier Transform efficiently.

The inverse operation (Inverse Fast Fourier Transform) takes data from the frequency domain and gives the values in the time domain. We can use this algorithm to draw a sound. Just imagine the draw as an input like if it was the spectral representation of the signal. Then, we apply the IFFT algorithm, and we retrieve the audio waveform which is used to create the output wave file.

The Program

The blackboard is the frequency-time representation of the audio signal. It is possible to paint into it using the color of the bar on the left side of the start button. Clicking the bar, it’s possible to change the color. The best results can be obtained using darker colors (like dark gray). When you click the Start button, the IFFT algorithm begins to compute data and the output audio file is generated.

Just draw something:

We can view the results with programs like Cooledit.

The Code

The program is written in C#. We use two fundamental libraries here:

• Wave File Library written by Garrett Hoofman. The library has been used to produce the Wav output file.
• A library that implements the IFFT algorithm.
C#
```using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WaveLibrary;

namespace Img2Wav
{
class Core_Img2Wav
{

private const double MAX_DATA = +50;
private const double MIN_DATA = -50;

private const String NoInputBitmap = "No input bitmap";

public Bitmap InputBitmap { get; set; }
public WaveFile OutputWav { get; set; }

public void Start()
{
int NumSamples = InputBitmap.Width * InputBitmap.Height;
byte[] Samples = new byte[NumSamples];
OutputWav = new WaveFile(1, 16, 44000);
if (InputBitmap == null) throw new Exception(NoInputBitmap);
double[] data = new double[InputBitmap.Height];

int w = 0;
for (int i = 0; i < InputBitmap.Width; i++)
{
for (int j = 0; j < InputBitmap.Height; j++)
{
Color C = InputBitmap.GetPixel(i,j);
data[j] = (C.R + C.G + C.B ) / 3;
}

FFT_Img2Wav.inverse(data);

for (int x = 0; x < data.Length; x++)
{
Samples[w] = (byte)(MAX_DATA * data[x]);
w++;
}
}

OutputWav.SetData(Samples,NumSamples);
}
}
}```

Written By
Software Developer (Senior)
Italy
Angelo Gattuso graduated in November 2003 from "Politecnico di Torino" in Software Engineering. He is a Software Engineer with experience in projecting and developing Businness Applications. Areas of expertise include ASP .Net design and develompemnt of three-tier applications.

He has interests in music, reading and cooking even if he prefers eating.

 First Prev Next
 How do I even find the .exe (app)? YellaFella27-Jul-21 4:01 YellaFella 27-Jul-21 4:01
 My vote of 5 Bill SerGio, The Infomercial King23-Apr-15 4:28 Bill SerGio, The Infomercial King 23-Apr-15 4:28
 C# Wave Library Member 1108725116-Sep-14 8:34 Member 11087251 16-Sep-14 8:34
 I've moved the C# Wave Library mentioned in the article to Github: https://github.com/gambitsunob/WaveLibrary[^] ~ Garrett Hoofman http://ghoofman.com
 Re: C# Wave Library Franc Morales11-Oct-22 17:54 Franc Morales 11-Oct-22 17:54
 How to see the frequency Member 859082926-Jun-12 16:08 Member 8590829 26-Jun-12 16:08
 My vote of 5 winxpdll11-Sep-11 10:10 winxpdll 11-Sep-11 10:10
 My vote of 5 abdillahi26-Apr-11 23:08 abdillahi 26-Apr-11 23:08
 My vote of 4 LucianPopescu22-Feb-11 22:29 LucianPopescu 22-Feb-11 22:29
 Great, but has some bugs M\$VBC++27-Dec-10 4:33 M\$VBC++ 27-Dec-10 4:33
 My vote of 5 victorbos31-Jul-10 7:33 victorbos 31-Jul-10 7:33
 Simply fascinating victorbos31-Jul-10 7:03 victorbos 31-Jul-10 7:03
 Loss "WaveLibrary" in package kenj_huang010317-Jun-10 0:12 kenj_huang0103 17-Jun-10 0:12
 Re: Loss "WaveLibrary" in package victorbos31-Jul-10 6:54 victorbos 31-Jul-10 6:54
 Re: Loss "WaveLibrary" in package filmee2426-Apr-14 1:22 filmee24 26-Apr-14 1:22
 Terrible article dude! Bill SerGio, The Infomercial King16-Jan-10 13:10 Bill SerGio, The Infomercial King 16-Jan-10 13:10
 Re: Terrible article dude! evol7612-Mar-10 10:31 evol76 12-Mar-10 10:31
 Re: Terrible article dude! victorbos31-Jul-10 7:11 victorbos 31-Jul-10 7:11
 Re: Terrible article dude! Bill SerGio, The Infomercial King31-Jul-10 12:37 Bill SerGio, The Infomercial King 31-Jul-10 12:37
 Spectrogram Patt Rojazz5-Nov-09 7:35 Patt Rojazz 5-Nov-09 7:35
 Re: Spectrogram evol7612-Mar-10 10:33 evol76 12-Mar-10 10:33
 Better Image Tools mkane726-Sep-09 12:59 mkane7 26-Sep-09 12:59
 Good Idea! kmslove26-Jun-09 5:15 kmslove 26-Jun-09 5:15
 Very Funny Idea! johannesnestler21-Dec-08 21:58 johannesnestler 21-Dec-08 21:58
 Last Visit: 31-Dec-99 18:00     Last Update: 25-Mar-23 4:50 Refresh 1