Mathematics Colloquia and Seminars

Return to Colloquia & Seminar listing

Reservoir Computing (RC) is a recent addition to the field of recurrent neural networks, with the added advantage of a simple and fast training procedure. This talk presents the use of reservoirs when applied to several time-­‐series datasets including real-­‐world data collected from an engineering application. Through this analysis an antagonistic trade-­‐off between a reservoir's non-­‐linear mapping and ability to recall inputs from the past was observed when data appeared to require high amounts of non-­‐linearity and input recall, which hindered performance. To overcome this trade-­‐off, a reservoir was combined with two feedforward layers of neurons to give Reservoirs with Random Static Projections (R2SP). These two layers were borrowed from the field of extreme learning machines (ELMs), where it was conjectured that these would allow the reservoir to be tuned towards maximising its memory capacity while the non-­‐linear transformation of the input was taken care of by these new layers. The R2SP architecture, along with a standard reservoir and traditional recurrent neural networks were applied to several datasets, where the R2SP outperformed the standard reservoir approach. The properties of the standard reservoir and R2SP were analysed where it was found that the reservoir of the R2SP was tuned towards input recall, as less non-­‐linear transformation of the input was required from it. The advantages of using RC approaches was apparent not only from the improvement in performance they offered, but also the drastic reduction in the complexity of their training procedures.