I tried similar approaches long ago (~2 years now?) with something related to RNN-RBM and it showed some slight glimmer of promise, and still think there might be some clever ways to combine concatenative methods and deep learning to avoid a lot of the noise issues present in parametric models. Then again, maybe it just needs to train longer - it's always hard to tell. I liked jungle.horse, awesome stuff!