Efficient Digital Implementation of Extreme Learning Machines for Classification

Sergio Decherchi, Paolo Gastaldo, Alessio Leoncini, Rodolfo Zunino.

The availability of compact, fast circuitry for the support of artificial neural systems is a long-standing and critical requirement for many important applications. This work addresses the implementation of the powerful Extreme Learning Machine (ELM) model on reconfigurable digital hardware. The design strategy first provides a training procedure for ELMs, which effectively trades off prediction accuracy and network complexity. This in turn facilitates the optimization of hardware resources.

The research analyzes two implementation approaches: one involving FPGA devices, and one embedding low-cost, low-performance devices such as CPLDs. Experimental results show that, in both cases, the design approach yields efficient digital architectures with satisfactory performances and limited costs.

**Available material**

This page make available the VHDL code that implements the ELM classifier. All the material is packed into a password protected zip file. Please contact Paolo Gastaldo (paolo.gastaldo@unige.it) to get the password.

The implementation refers to a classifier tested on the MNIST dataset; the neural network has 81 inputs and 18 neurons in hidden layer. The training phase was completed offline; hence, the number of neurons and the hidden weights are hardcoded into the digital design.

A signed 2-complement fixed-point representation is adopted; 16 bits encode all numerical quantities, with 4 bits for the integer part and 12 bits for the fractional part.

**The following files are provided:**

Implementation on FPGA devices:

*BinaryELM.vhd*(and dependencies) the ELM classification circuit*InputMemory.vhd*the memory block that stores all the test patterns*Test.vhd*the testbench that runs the experiment (simulation time at least 90 ms)

Implementation on CPLD devices:

*BinaryELM.vhd*(and dependencies) the ELM classification circuit*InputMemory.vhd*the memory block that stores all the test patterns*Test.vhd*the testbench that runs the experiment (simulation time at least 900 ms)

Common files

*TestSet.txt*contains the 799 81-dimensional patterns used as test set*TestSetLabels.txt*contains the labels of the test patterns*PowersMatrix.txt*contains the weights assigned to the hidden layer (*InputShifter*part of*BinaryELM*)*Alphas.txt*contains the weights assigned to the ouptut layer after training*Predictions.txt*outputs predicted for the test patterns by a Matlab simulation of the ELM network (64-bit floating point numbers)

**Guidelines:**

- Implement the VHDL code with
*BinaryELM*as top level entity - Start the simulation by using testbench
*Test.vhd*; this step generates the Test.out file. The file contains the outputs of the ELM network represented as binary values - Convert binary values into real values (16 bits, 4 bits for the integer part, 12 bits for the fractional part) by using this converter ( win32 source | linux source )
- Evaluate the classification performance of hardware implementation by using this spreadsheet