Multichannel FFT with MonitorSpectralNode

1

Hey.

This is probably one for @rich.e but is there a particular reason MonitorSpectralNode only supports FFT on a single channel? (and averages all channels above this?)

I imagine there was some difficultly in implementing this with the current setup but i’m now doing a project where this would be very useful and thinking of implementing (a probably fairly naive) one on my own cinder fork:

mMonitorSpectralNode->getMagSpectrum( channelIndex )

Any advice or forewarning before I get stuck in? I think this feature could probably benefit a number of users so would be great if it could make its way back into master at some point.

2

I did similar thing a while ago, had to subclass the MonitorNode class and doing fft separately for each channel inside the new node class.

some code:
in new node’s initialize function:

{
       MonitorNode::initialize();

       if( mFftSize < mWindowSize )
           mFftSize = mWindowSize;
       if( ! isPowerOf2( mFftSize ) )
          mFftSize = nextPowerOf2( static_cast<uint32_t>( mFftSize ) );

       mFft = unique_ptr<dsp::Fft>( new dsp::Fft( mFftSize ) );
       mFftBuffer = audio::Buffer( mFftSize );
       mBufferSpectral = audio::BufferSpectral( mFftSize );

       mMagSpectrum.resize(getNumChannels());
       for(int i = 0; i < getNumChannels(); i++){
           mMagSpectrum[i].resize( mFftSize / 2 );
       }

       mWindowingTable = makeAlignedArray<float>( mWindowSize );
       generateWindow( mWindowType, mWindowingTable.get(), mWindowSize );
}

prepareprocessing function:

void MultiChannelSpectralNode::prepareProcessing(){
    uint64_t numFramesProcessed = getContext()->getNumProcessedFrames();
    if( mLastFrameMagSpectrumComputed == numFramesProcessed )
        return;
    mLastFrameMagSpectrumComputed = numFramesProcessed;
    fillCopiedBuffer();
}

and finally the getMagSpectrum function:

const std::vector<float>& MultiChannelSpectralNode::getMagSpectrum(size_t channel){
    if(channel >= getNumChannels()) return mMagSpectrum[0];
        
    // window the copied buffer and compute forward FFT transform
    dsp::mul( mCopiedBuffer.getChannel(channel), mWindowingTable.get(), mFftBuffer.getData(), mWindowSize );
    mFft->forward( &mFftBuffer, &mBufferSpectral );
        
    float *real = mBufferSpectral.getReal();
    float *imag = mBufferSpectral.getImag();
        
    // remove Nyquist component
    imag[0] = 0.0f;
        
    // compute normalized magnitude spectrum
    const float  magScale = 10.f / mFft->getSize();
    const size_t specSize = mMagSpectrum[channel].size();
        
    for( size_t i = 0; i < specSize; i++ ) {
        float re = real[i];
        float im = imag[i];
        mMagSpectrum[channel][i] = mMagSpectrum[channel][i] * mSmoothingFactor + std::sqrt( re * re + im * im ) * magScale * ( 1 - mSmoothingFactor );
    }
        
    return mMagSpectrum[channel];
}

hope this helps.
-seph

3

just put the class onto gist:
header
src

4

Haha thanks Seph!

That is almost char for char what I had started writing this morning (though integrating with the original class rather than subclassing).

Very useful to compare against.

Cheers.

5

One thing that I was thinking about was separating the processing from the getMagSpectrum().

At the moment every call to getMagSpectrum( channel ) will recompute that channels FFT. I see how the prepareProcessing()' function reduces the need to have a vector ofmLastFrameMagSpectrumComputed(one for each channel) but by moving thenumFramesProcessed` check to a different function it doesnt have the benefit of skipping an unrequired FFT::forward.

I guess there a two options for this:

  1. use a vector of mLastFrameMagSpectrumComputed and add the numFramesProcessed check back into getMagSpectrum( channel ) so it can skip the FFT when it needs to.

  2. have a processFFT()' function that does all the FFTing for every channel andgetMagSpectrum( channel )` just becomes a simple getter… This would mean all the channels would have to get processed at the same time but this is probably the most common use case.

Just a thought.

6

yeah correct. I’d say the second option sounds neat. And i think the data will be passed in altogether too.
In my use case I pulled all channel’s data once in the update and so didn’t think about the recompute.

1 Like
7

Just wanted to say that I think it’d be great to get multi-channel FFT functionality into MonitorSpectralNode. Currently it does downmix, so the only way to get separate mag spectrums is to first split your signal with a ChannelRouterNode. Though it is clear that MonitorSpectralNode should support this internally, and you could still force the downmixing with Node::ChannelMode and Node::Format::channels(). I think I just ran out of time when I first wrote it to properly think it through, so if you’all want to take a stab and PR it in, would love to check it out.

Only thing to be careful about is the thread safety with getMagSpectrum(), since that is usually called from a non-audio thread, while the audio thread is doing the processing (or at least filling the buffer).

cheers,
Rich

8

FWIW I’m working on an app that also requires performing an FFT on two different input channels and have the following code working with router nodes for FFT analysis for each channel (left/right):

mInputDeviceNode = ctx->createInputDeviceNode();
auto monitorFormat = audio::MonitorSpectralNode::Format().fftSize(1024).windowSize(512);

// create monitor spectral nodes:
mMonitorSpectralNode[0] = ctx->makeNode(new audio::MonitorSpectralNode(monitorFormat));
mMonitorSpectralNode[1] = ctx->makeNode(new audio::MonitorSpectralNode(monitorFormat));

// create channel router nodes and routes:
// Set the ChannelRouterNode to use only 1 channel
auto format = audio::Node::Format().channels(1);

auto channelRouter1 = ctx->makeNode(new audio::ChannelRouterNode(format));
auto channelRouter2 = ctx->makeNode(new audio::ChannelRouterNode(format));

mInputDeviceNode >> channelRouter1->route(0, 0) >> mMonitorSpectralNode[0];
mInputDeviceNode >> channelRouter2->route(1, 0) >> mMonitorSpectralNode[1];
1 Like