FFT of Large Data


(Burhan Eyimaya) #1

Hi,

How can we compute FFT of a large array for ex 10 million samples? I tried to take FFT in chunks with size 1024 then combine the results. Is this correct way to do this?
I have doubts about this because if I change the size for ex 2048 the result changes. Is this normal?

My sample code is :

int numData = m_Buffer.Length;
m_FftBuffer = new Complex[numData];

int fftSize = 1024;
Complex[] fftBuffer = new Complex[fftSize];

int srcIndex = 0;
int destIndex = 0;
while (srcIndex < numData)
{
		for (int x = 0; x < fftSize; x++)
		{
			if (srcIndex < numData)
				fftBuffer[x] = new Complex(m_Buffer[srcIndex], 0.0);
			else
				fftBuffer[x] = new Complex(0.0, 0.0);

			srcIndex++;
		}

		Fourier.Forward(fftBuffer);

		for (int x = 0; x < fftSize; x++)
		{
			 if (destIndex >= numData)
                           break;
			
                    m_FftBuffer[destIndex] = fftBuffer[x];
			destIndex++;
		}
} 

Thanks,
Burhan Eyimaya


(Peter Vanderwaart) #2

About sample size and the FFT, figure out the largest sample size that is convenient and practical. There isn’t too much to gain by going to a huge number because there is a limit on what you can display on a monitor which is, say, 2000 pixels across. Similar for printing on paper.

I doubt the approach you describe is what you want to do. The way to reduce your sample size to a practical number is to take every nth observation, for some number n.

You want to plan your analysis by deciding what frequency range you are interested seeing. Here is an example: Suppose you have 1,0000,000 observations taken over a span of one year, and you are interested in cycles that are about 1 day long. If you want 20 observations per day and your FFT wants a sample of 2048, then data range will be about 2048/20 = approx 102 days.

You have 1,000,000 * 102/365 = approx 280,000 data points in that range. So you pick one point, skip 280,000/2048 = approx 135, pick one, skip 135, etc. (calculations are approximate).


(Burhan Eyimaya) #3

Hi Peter,

Thank you for your response.
I undersand you suggestion but I am trying to draw spectrogram of a wav file.
Does this method work for sound processing?
For example if a wav file contains 20 million samples, and If I take 1024 samples for FFT I will skip aprroximately 19530 samples and take one sample etc. Will this reflect the general frequency graph of the wav file?


(Peter Vanderwaart) #4

HI. I don’t know how spectrogams of .wave files are usually done, so I can’t help you much. I think you need to think about what the x-axis represents, i.e. what a given point on the x-axis means.


(Burhan Eyimaya) #5

The usage concept is the detail, spectrogram or another thing it doesn’t matter. I just want to compute the FFT of a large array in chunks. I want to know how to do this. Is there anyone who can help me?


(Peter Vanderwaart) #6

The idea of skipping 19530 is the best way I know to get a spectrogram of the whole file.

If it bothers you to throw away most of the data, think about it like this. If you had temperature data for whole year measured every 1/100 th of a second, you would not need all that detail. It would be enough to only use one observation per hour.