Audio Effects DSP Board

[markseel]

I suppose many folks would prefer to put this board in a Hammond-like case and add pots and a footswitch.  But I'm putting mine in small 2.5"x2.5"x0.95" cases milled from solid stainless steel (no pots or switches) 

Here's an early prototype with an old board that smaller and had the USB jack in a different location:

[markseel]

I can have my machinist make these out of solid aluminum (stainless steel us *really* expensive - harder to machine) for a reasonable cost.  Thinking of offering those as incentives for the kickstarter campaign.  Anyone interested?

[MetalGuy]

It definitely looks solid and maybe is quite heavy  Machinists hate stainless steel stuff for sure.

[markseel]

Here's a FlexFX application example.  It doesn't implement any algorithms but just shows what configuration, events, and functions are used to make a complete application.  I put them in one file for illustration.  This is the only code that needs to be written for the board - all of the USB, I2S, MIDI, I2C and UART code is precompiled and part of the downloadable framework.  An example of how to build (compile this file and link with the framework code to make the firmware executable image) is earlier in this thread.

Code Select Expand


// ================================================================================================
// FlexFX sample application
// ================================================================================================

typedef unsigned char byte;

// ================================================================================================
// System Configuration.  Set these constants to configure USB and I2C.
// ================================================================================================

// ------------------------------------------------------------------------------------------------
// USB Audio configuration.
//
// usb_fs_bit_mask: Each bit indicates what sampling frequency is to be supported for USB audio.
//                  Any combination of bits can be set as long as the ADC/DAC can be configured
//                  accordingly via the I2C functions.
//                  Bit 0 for 44k1, bit 1 for 48K0, bit 2 for 88k2, bit 3 8 for 96k,
//                  bit 4 for 176k4, bit 5 for 192k, bit 6 for 352k8, bit 7 for 384k0
//
// usb_audio_product_id:  16-bit value representing the USB audio product ID
// usb_audio_vendor_id:   16-bit value representing the USB audio vendor ID
// usb_audio_product_str: 10 character string representing the USB audio product name
// usb_audio_vendor_str:  10 character string representing the USB audio vendor name
// ------------------------------------------------------------------------------------------------

const byte  usb_fs_bit_mask       = 0x02;
const int   usb_audio_product_id  = 0x1234;
const int   usb_audio_vendor_id   = 0x1234;
const char* usb_audio_product_str = "FlexFX";
const char* usb_audio_vendor_str  = "Example";

// ------------------------------------------------------------------------------------------------
// I2S Bus configuration.  The I2S bus consists of MCLK (master audio clock), BCLK (sample bit
// clock), WCLK (word select clock), four SDOUT wires for up to four DAC's, and four SDIN wires for
// up to four ADC's.  All sample words are 32-bits supporting 16/24/32 bit samples with either left
// or right justification within the 32-bit word slot.  Time division multiplexing is supported -
// 2 (stereo), 4, 6, 8, 10, 12, 14, and 16 samples per audio cycle, per SDIN/SDOUT wire is allowed
// for up to 64x64 channel I2S/TDM.  The word select format can be I2S or PCM.
//
// i2s_sync_format: 0 for I2S format (low-high WCLK transition one BCLK period before audio cycle.
//                  1 for PCM format (high-low WCLK transition aligned with each audio cycle.
//
// i2s_tdm_slot_count: Defines how many samples are transfered on each SDIN/SDOUT wire for each
//                     audio cycle.  Minimum is 2 (stereo), max is 16.  Must be even number.
// ------------------------------------------------------------------------------------------------

const int i2s_sync_format    = 1;   // 0 for I2S, 1 for PCM
const int i2s_tdm_slot_count = 2;   // 2 (stereo), 4, 6, 8, 10, 12, 14, 16

// ================================================================================================
// System Control.  Use these functions to control system peripherals (using I2C) and to
// distribute effects parameters to audio processing threads (using MIDI data).
// ================================================================================================

// ------------------------------------------------------------------------------------------------
// I2C Functions.  Use these functions to configure the audio ADC/DAC/CODEC upon USB audio config
// change events or to sense pot/switch values via low-speed ADC's and/or I2C port expanders.
//
// i2c_write: Write a sequence of bytes to the addressed device
// i2c_read:  Read a sequence of bytes from the addressed device
// i2c_write_reg:  Write an 8-bit value to the specified 8-bit register of the addressed device
// i2c_read_reg:  Read an 8-bit value from the specified 8-bit register of the addressed device
// ------------------------------------------------------------------------------------------------

extern void i2c_write    ( byte dev_addr, const byte* data, int count );
extern void i2c_read     ( byte dev_addr, byte* data, int count );
extern void i2c_write_reg( byte dev_addr, byte reg_num, byte reg_value );
extern byte i2c_read_reg ( byte dev_addr, byte reg_num );

// ------------------------------------------------------------------------------------------------
// MIDI write function.  Use this function to send MIDI-based property data to each audio thread,
// typically as the result of reading new pot/switch values in the function 'handle_timer' below.
// ------------------------------------------------------------------------------------------------

extern void write_midi_property( const int property[6] );

// ================================================================================================
// Non-audio event handlers.  Code in these event handlers will not effect audio flow.
// ================================================================================================

// ------------------------------------------------------------------------------------------------
// USB and Timer event handlers.  Implement ADC/DAC/CODEC changes via I2C here.
//
// handle_usb_mute: Occurs whenever a USB audio channel is muted or un-muted.
//
// handle_usb_volume_change: Occurs whenever a USB audio channel volume is changed.  0 is minimum
//                           volume and 255 is maximum volume.
//
// handle_usb_rate_change: Occurs whenever the sample rate is changed by the USB host.  The
//                         application should use the I2C functions to set the audio master clock
//                         and configure the ADC/DAC accordingly.
//
// handle_timer: Called at a rate of 1000 Hz. Implement low-speed ADC reading (for pots and/or
// switches) in this function.
// ------------------------------------------------------------------------------------------------

void handle_usb_mute( int audio_channel, int mute_flag )
{
    //i2c_write_reg( 0x00, 0x00, 0x00 );
    //i2c_write_reg( 0x00, 0x00, 0x00 );
}
void handle_usb_volume_change( int audio_channel, byte volume )
{
    //i2c_write_reg( 0x00, 0x00, 0x00 );
    //i2c_write_reg( 0x00, 0x00, 0x00 );
}
void handle_usb_rate_change( int frequency )
{
    //i2c_write_reg( 0x00, 0x00, 0 );
    //i2c_write_reg( 0x00, 0x00, 0 );
}

void handle_timer( void )
{
    //int property[6];
    //int x = i2c_read_reg( 0x00, 0x00 );
    //i2c_write_reg( 0x00, 0x00, 0 );
    //write_midi_property( property );
}

// ================================================================================================
// Audio event handlers.
//
// The initialization functions are all called before audio flow starts and should be used to
// initialize data used by audio processing threads.  The audio processing functions are all called
// once for every audio cycle meaning that they will be expected finish processing before the next
// audio cycle starts. Each processing thread runs in its own 32-bit core and has approximately 100
// MIPs or processing allocation.  The processing functions A1-A5, B1-B5, and C1-C5 form a 15-stage
// processing pipeline for 64 audio samples at each pipeline stage.
//
// Data arriving to thread A1 is from the USB and I2S interfaces and is arranged as 32 USB samples
// followed by 32 I2C samples.  The 64-sample array departing from thread C5 is sent to the USB and
// I2S interfaces and is therefore assumed to be arranged as 32 USB samples followed by 32 I2C
// samples. The 64-sample array passed between all other processes (A2-A5,B1-B5,C1-C4) is user
// defined.
//
// thread_nn_initialize: Initialize data for audio thread nn (A1-A5, B1-B5, or C1-C5)
//
// thread_nn_process: Process audio samples for one audio cycle. 'samples' points to an array of
//                    64 fixed-point (Q1.31 format) audio samples. 'property' points to an array
//                    if 6 integers forming the property used to configure audio processing
//                    functions in real time.  Property[0] is the 32-bit property ID. Property[1:5]
//                    contains the five 32-bit values for the property.
// ================================================================================================

void thread_a1_initialize( void )
{
}
void thread_a1_process( int* samples, const int* property )
{
}

void thread_a2_initialize( void )
{
}
void thread_a2_process( int* samples, const int* property )
{
    // Process audio samples from thread A1, results are sent to thread A3.
    if( property[0] != 0 ) {
        // Check property ID (property[0]) to see of this property applies to this processing
        // thread.
    }
}

void thread_a3_initialize( void )
{
}
void thread_a3_process( int* samples, const int* property )
{
}

void thread_a4_initialize( void )
{
}
void thread_a4_process( int* samples, const int* property )
{
}

void thread_a5_initialize( void )
{
}
void thread_a5_process( int* samples, const int* property )
{
}

void thread_b1_initialize( void )
{
}
void thread_b1_process( int* samples, const int* property )
{
}

void thread_b2_initialize( void )
{
}
void thread_b2_process( int* samples, const int* property )
{
}

void thread_b3_initialize( void )
{
}
void thread_b3_process( int* samples, const int* property )
{
}

void thread_b4_initialize( void )
{
}
void thread_b4_process( int* samples, const int* property )
{
}

void thread_b5_initialize( void )
{
}
void thread_b5_process( int* samples, const int* property )
{
}

void thread_c1_initialize( void )
{
}
void thread_c1_process( int* samples, const int* property )
{
}

void thread_c2_initialize( void )
{
}
void thread_c2_process( int* samples, const int* property )
{
}

void thread_c3_initialize( void )
{
}
void thread_c3_process( int* samples, const int* property )
{
}

void thread_c4_initialize( void )
{
}
void thread_c4_process( int* samples, const int* property )
{
}

void thread_c5_initialize( void )
{
}
void thread_c5_process( int* samples, const int* property )
{
}

// ================================================================================================
// ================================================================================================


[markseel]

An example:

1) Pass USB audio output to I2S out (the DAC) and pass I2S input (the ADC) to USB audio input.  USB samples occupy the first 32 samples of the 64-byte sample array and I2S samples occupy the second set of 32 samples.  If no threads are modifying the 64-byte array then audio samples simply loopback to each interface (USB out --> USB in, I2S out --> I2S in).  Copying samples[0:1] to samples[32:33] and also samples[32:33] to samples[0:1] would result in (for stereo) USB out --> I2S out and I2S in --> USB in.  See thread A1.   

Code Select Expand


void thread_a1_initialize( void )
{
}
void thread_a1_process( int* samples, const int* property )
{
    int tmp1 = samples[0]; // Store USB sample[0] (left channel)
    int tmp2 = samples[1]; // Store USB sample[1] (right channel)
    samples[0] = samples[32]; // USB left = I2S left
    samples[1] = samples[33]; // USB right = I2S right
    samples[32] = tmp1; // I2S left = USB left
    samples[33] = tmp2; // I2S right = USB right
}


2) A simple volume control for I2S outputs [0:1] (left and right for stereo).  The initial volume is initialized to 0.  In the thread the samples are multiple by the fractional volume level which ranges from 0 to 0.9999999 (samples and volume are both in Q1.31 format).  The thread also handles a MIDI property that updates the volume level.  See thread A2.

Note: The MIDI property could have arrived to this thread from three sources; (1) USB MIDI from a MIDI host (e.g. a PC), (2) from the UART RX line if connected to an external MIDI source, or (3) from the function 'write_midi_property' that may have been called in the 'handle_timer' function after reading a low-speed ADC using the I2C functions.

Code Select Expand


#include "core_dsp.h"
#define PROPERTY_VOLUME_ID 0x00000001
static a2_volume_q31;
void thread_a2_initialize( void )
{
    a2_volume_q31 = Q31( 0.0 );
}
void thread_a2_process( int* samples, const int* property )
{
    samples[32] = multiply( 31, samples[32], a2_volume_q31 ); // Apply volume to left channel
    samples[33] = multiply( 31, samples[33], a2_volume_q31 ); // Apply volume to right channel
    if( property[0] == PROPERTY_VOLUME_ID ) { // See if it's our property
        a2_volume_q31 = property[1]; // Only one of the five 32-bit property values are used here
    }
}


[stringsthings]

This looks really interesting.  Following the thread for news/updates.

[pruttelherrie]

Note: The MIDI property could have arrived to this thread from three sources; (1) USB MIDI from a MIDI host (e.g. a PC), (2) from the UART RX line if connected to an external MIDI source, or (3) from the function 'write_midi_property' that may have been called in the 'handle_timer' function after reading a low-speed ADC using the I2C functions.

Wow, it gets cooler and cooler!

[markseel]

Note: The MIDI property could have arrived to this thread from three sources; (1) USB MIDI from a MIDI host (e.g. a PC), (2) from the UART RX line if connected to an external MIDI source, or (3) from the function 'write_midi_property' that may have been called in the 'handle_timer' function after reading a low-speed ADC using the I2C functions.

There's some important reasons for this; I only want one area in the FW that handles incoming MIDI data to keep the code simple and to allow for distribution of data.  Keeping to one protocol for USB, UART, and internal (via the timer callback function) also keeps the user code simple - you only parse one data format.  Handling all MIDI in a single part of the FW makes it easy to distribute MIDI data to all interfaces regardless of where the data originates:
1) MIDI over USB is distributed to USB (looped back), to the UART and to the user's algorithms
2) MIDI via the UART RX is sent to UART TX (pass through), to USB and to the user's algorithms
3) MIDI data from the user's timer callback is send to USB, UART TX and to the user's algorithms.
So if you have multiple interferes they can stay in sync as far as settings go and allow both USB and UART interfaces to be updated regardless of what interface made the change.  Also you could use this then for a USB MIDI and UART MIDI bridge.

[pruttelherrie]

Smart. The only thing I can see going "wrong" is when you use pots connected to the low speed ADC to control certain values: these will be mirrored to the USB-MIDI and the UART-TX, but the other way around is not possible. I say quote-wrong-quote because nothing goes really wrong in the breaking-stuff-sense-of-the-word, but the setting of the pot might not represent the actual value in the DSP if it's modified over USB-MIDI or UART-RX.

... if I understand you correctly ...

Other than that, this enables on-device controls in parallel with host-based controls, that's wicked!

Of course, one could always read pots by a microcontroller which communicates over the UART with the XMOS.

[markseel]

Good point.  I agree with you - XMOS itself or a micro-controller sensing pots and updating properties/settings works in only one direction as far as keeping everything sync'd and that things would get out of sync easily.

Hmmm, maybe if the MCU was listening to UART RX for MIDI data from XMOS (in case parameters were updated elsewhere e.g. USB) then I suppose the MCU could light up an LED that indicated that the pots were out of sync -- or something along those lines.  Or you could use rotary encoders and I2C digi-pots (and some kind of display?) but that adds work and complexity.  Not an easy one to solve I guess.

A while back I thought of putting two tiny led's below each knob with either one lit up indicating that the pot value was out of sync with the actual property value.  The left led indicating that the pot's value was too large and that knob needed to be turned counter-clockwise until it reached the correct value, and the other led doing the same except for the value being too small and requiring the knob to be turned clockwise.  LED's would turn of once the pot value matched. 

[markseel]

Here's a data-flow diagram of the framework.

[pruttelherrie]

Hmmm, maybe if the MCU was listening to UART RX for MIDI data from XMOS (in case parameters were updated elsewhere e.g. USB) then I suppose the MCU could light up an LED that indicated that the pots were out of sync -- or something along those lines.  Or you could use rotary encoders and I2C digi-pots (and some kind of display?) but that adds work and complexity.  Not an easy one to solve I guess.

I imagine something like the UI of a digital scope, with 'soft buttons' (context-sensitive function buttons) and rotaries. I would then use the UART and completely bypass the analogue domain (the digi-pots)

A while back I thought of putting two tiny led's below each knob with either one lit up indicating that the pot value was out of sync with the actual property value.  The left led indicating that the pot's value was too large and that knob needed to be turned counter-clockwise until it reached the correct value, and the other led doing the same except for the value being too small and requiring the knob to be turned clockwise.  LED's would turn of once the pot value matched.

One could implement this with those transparant shaft pots with a duo-led behind it!

Here's a data-flow diagram of the framework.

What's the total delay/latency from Tile 0 back to Tile 0? < 1ms like you explained earlier in this thread?

[markseel]

What's the total delay/latency from Tile 0 back to Tile 0? < 1ms like you explained earlier in this thread?

Each stage of the audio chain operates on one audio cycle at a time so latency is extremely low.  The latency through the audio processing pipeline is 24 samples.  I2S input and I2S output adds 3 samples for each (6 total) due a buffer and the port's hardware double-buffering.  So for I2S in to I2S out total latency is 30 audio cycles (0.625 msec for 48 kHz Fs and ~0.157 msec for 192 kHz Fs).

Select ADC/DAC/CODEC's with low latency digital filters.  The AK4588's filters have latencies of 5 and 6.8 samples for ADC and DAC respectively.  Analog in to analog out latency would then be around 42 samples (0.875 msec for 48 kHz Fs and 0.219 msec for 192 kHz) -- well under 1 msec and probably as good or better than any other audio effects system available?

Even a not-so-low latency CODEC like the AK4556 (18/Fs for ADC and 21/Fs for DAC) would still provide sub-1msec latency at 192 kHz Fs --> (30+18+21)/192K ~= 0.36 msec. 

USB audio packets (isochronous USB audio 2.0) are transferred at 8000Hz (0.125msec period) and have to be buffered.  The firmware packet FIFO's are 8 deep for incoming packets and 2 deep for outgoing packets adding 1 msec of USB input (host to XMOS) latency and 0.25 msec of USB output (XMOS to host) latency to firmware's audio processing latency of 24/Fs.

Latency through the host OS (data moving through the USB driver and through OS services/API's) varies - I'm not sure how it compares for OS X, Windows, and Linux.

[pruttelherrie]

Ok, thanks for the explanation.

USB latency and host latency is not a big concern when you do all monitoring by the DSP and not through the DAW, thats the whole point of mixing/monitoring in the interface.

[markseel]

Code sample time!

I added the ability to measure how long your DSP code takes to run for each of the 15 processing threads.  Here's an example added to thread A1:

Code Select Expand


void thread_a1_process( int* samples, const int* property, int ticks )
{
    static int pass = 1;

    // Check the tick count for the previous pass through this function.
    // If pass == 2 then we're in our second pass and displaying tick count for pass #1.
    // If pass == 3 then we're in our second pass and displaying tick count for pass #2.
    // Note that pass #1 tick count will always be zero - so have to check for passes after #1.

    if( pass == 2 ) printf( "Pass %u: %04u ticks (%u Hz)\n", pass-1, ticks, 100000000/ticks );
    if( pass == 3 ) printf( "Pass %u: %04u ticks (%u Hz)\n", pass-1, ticks, 100000000/ticks );

    // Make sure processing time does not exceed one audio cycle period.
    // Skip this check for passes before 4 since they will have taken a long time due printing.
    // One tick is 1e-8 second, so one audio cycle at 48 kHz Fs is about 2083 ticks.
    // Subtract 20 ticks for the overhead required to enter/exit this function.
    // The 'tick' value is the tick count for the previous pass.

    if( pass > 3 && ticks > 2083 - 20 )
        printf( "Thread A1 took too long! (%u ticks)\n", ticks );

    pass++;
}


Printed results are:

Code Select Expand


bash-3.2$ xsim test.xe
Hello!
Pass 1: 0018 ticks (5555555 Hz)
Pass 2: 3206 ticks (31191 Hz)
Thread A1 took too long! (2781 ticks)
Goodbye.
bash-3.2$


Pass #1 is measured and printed out in pass #2.
Pass #2 is measured and printed out in pass #3.
Pass #2 takes a lot of ticks due to printing.
Passes #3 and above are checked for a timing violation.  Pass #3 was a violation due to printing.

The function, not doing any DSP, can run without exceeding a cycle of 1/5.5 MHz.  That will go down of course as you add DSP functions.  You can see that calling printf takes a lot of processing time ... so be sure to disable that sort of code when doing real-time audio.  Just use it during simulation and for testing.

Note: You can simulate your code without a board although it's painfully slow.  I'll add a post on that later.

[markseel]

The firmware dev kit is available on GitHub:
https://github.com/markseel/flexfx_kit.git

Below I cloned the repo, built the example, and simulated it.

Code Select Expand


bash-3.2$ mkdir github_flexfx_test
bash-3.2$ cd github_flexfx_test
bash-3.2$ git clone https://github.com/markseel/flexfx_kit.git

Cloning into 'flexfx_kit'...
remote: Counting objects: 75, done.
remote: Compressing objects: 100% (69/69), done.
remote: Total 75 (delta 8), reused 71 (delta 6), pack-reused 0
Unpacking objects: 100% (75/75), done.
Checking connectivity... done.

bash-3.2$ cd flexfx_kit
bash-3.2$ /Applications/XMOS_xTIMEcomposer_Community_14.1.1/SetEnv.command
bash-3.2$ ./build.sh example

example.c:157:14: warning: unused variable 'sample' [-Wunused-variable]
    unsigned sample = Q31( magnitude );
             ^
1 warning generated.
xmap: Warning: port "XS1_PORT_1J" on tile[0] is not connected to any pins in this package.
xmap: Warning: port "XS1_PORT_1K" on tile[0] is not connected to any pins in this package.
xmap: Warning: port "XS1_PORT_1I" on tile[0] is not connected to any pins in this package.
Constraint check for tile[0]:
  Cores available:            8,   used:          6 .  OKAY
  Timers available:          10,   used:          8 .  OKAY
  Chanends available:        32,   used:         26 .  OKAY
  Memory available:       262144,   used:      60712 .  OKAY
    (Stack: 4940, Code: 33156, Data: 22616)
Constraints checks PASSED.
xmap: Warning: More than 6 cores used on a tile. Ensure this is not the case on tile running XUD.
Constraint check for tile[1]:
  Cores available:            8,   used:          7 .  OKAY
  Timers available:          10,   used:          7 .  OKAY
  Chanends available:        32,   used:         15 .  OKAY
  Memory available:       262144,   used:      15836 .  OKAY
    (Stack: 2124, Code: 10624, Data: 3088)
Constraints checks PASSED.
xmap: Warning: More than 6 cores used on a tile. Ensure this is not the case on tile running XUD.
Constraint check for tile[2]:
  Cores available:            8,   used:          7 .  OKAY
  Timers available:          10,   used:          7 .  OKAY
  Chanends available:        32,   used:         15 .  OKAY
  Memory available:       262144,   used:       8876 .  OKAY
    (Stack: 628, Code: 5372, Data: 2876)
Constraints checks PASSED.
xmap: Warning: More than 6 cores used on a tile. Ensure this is not the case on tile running XUD.
Constraint check for tile[3]:
  Cores available:            8,   used:          7 .  OKAY
  Timers available:          10,   used:          7 .  OKAY
  Chanends available:        32,   used:         15 .  OKAY
  Memory available:       262144,   used:       8860 .  OKAY
    (Stack: 652, Code: 5336, Data: 2872)
Constraints checks PASSED.

bash-3.2$ xsim example.xe

Hello!
Pass 1: 0018 ticks (5555555 Hz)
Pass 2: 3206 ticks (31191 Hz)
Thread A1 took too long! (2781 ticks)

^Cbash-3.2$


[markseel]

Here's a dimensional drawing of the board showing the USB connector (left), mounting holes, input/output pins along the top (for power, ground, UART, I2C and I2S/TDM), and the JTAG/XTAG interface.

[pruttelherrie]

I'm getting

Code Select Expand


bash-3.2$ xsim example.xe
ERROR: No port 'XS1_PORT_1A' found on node 32788 core 0


Am I doing something wrong or missing something? Installed latest (14.2.4) XMOS tools; running on Mac OS 10.10.5.

[markseel]

Not sure what that means yet - I'll upgrade my version and rebuild ...

[pruttelherrie]

I didn't actually git clone, I downloaded the -master.zip and unzipped and cd'ed into that.