Category: Deep Learning

ML Algorithms Addendum: Hopfield Networks

09/20/2017 Artificial Intelligence Computational Neuroscience Deep Learning Generic Machine Learning Machine Learning Algorithms Addenda Neural networks Python 2 Comments
Hopfield networks (named after the scientist John Hopfield) are a family of recurrent neural networks with bipolar thresholded neurons. Even if they are have replaced by more efficient models, they represent an excellent example of associative memory, based on the shaping of an energy surface. In the following picture, there’s…

Quickprop: an almost forgotten neural training algorithm

09/15/2017 Artificial Intelligence Computational Neuroscience Deep Learning Generic Machine Learning Machine Learning Algorithms Addenda Neural networks Python 3 Comments
Standard Back-propagation is probably the best neural training algorithm for shallow and deep networks, however, it is based on the chain rule of derivatives and an update in the first layers requires a knowledge back-propagated from the last layer. This non-locality, especially in deep neural networks, reduces the biological plausibility…

Artificial Intelligence is a matter of Language

09/11/2017 Artificial Intelligence Complex Systems Computational Neuroscience Deep Learning Generic Machine Learning Neural networks NLP Philosophy of Mind No Comments
“The limits of my language means the limits of my world.” (L. Wittgenstein)   When Jacques Lacan proposed his psychoanalytical theory based on the influence of language on human beings, many auditors remained initially astonished. Is language an actual limitation? In the popular culture, it isn’t. It cannot be! But,…

An annotated path to start with Machine Learning

09/09/2017 Artificial Intelligence Deep Learning Generic Keras Machine Learning Neural networks Python Scikit-Learn Tensorflow Theano 2 Comments
“Do not worry about your difficulties in Mathematics. I can assure you mine are still greater.” (A. Einstein)   Machine Learning is becoming more and more widespread and, day after day, new computer scientists and engineers begin their long jump into this wonderful world. Unfortunately, the number of theories, algorithms, applications,…

Twitter Sentiment Analysis with Gensim Word2Vec and Keras Convolutional Networks

08/07/2017 Convnet Deep Learning Generic Keras Neural networks NLP Python Tensorflow 64 Comments
Fork Word2Vec (https://code.google.com/archive/p/word2vec/) offers a very interesting alternative to classical NLP based on term-frequency matrices. In particular, as each word is embedded into a high-dimensional vector, it’s possible to consider a sentence like a sequence of points that determine an implicit geometry. For this reason, the idea of considering 1D…

SVD Recommendations using Tensorflow

08/02/2017 Machine Learning Python Tensorflow 6 Comments
Recommendation system based on the user-item matrix factorization have become more and more important thanks to powerful and distributable algorithms like ALS, but sometimes the number of users and/or items is not so huge and the computation can be done using directly a SVD (Singular Value Decomposition) algorithm. In this…

Lossy image autoencoders with convolution and deconvolution networks in Tensorflow

07/29/2017 Convnet Deep Learning Neural networks Python Tensorflow 6 Comments
Fork Autoencoders are a very interesting deep learning application because they allow a consistent dimensionality reduction of an entire dataset with a controllable loss level. The Jupyter notebook for this small project is available on the Github repository: https://github.com/giuseppebonaccorso/lossy_image_autoencoder. The structure of a generic autoencoder is represented in the following figure:…

Keras-based Deepdream experiment based on VGG19

07/09/2017 Convnet Deep Learning Keras Machine Learning Python Tensorflow Tensorflow Theano No Comments
Fork I’ve just published a repository (https://github.com/giuseppebonaccorso/keras_deepdream) with a Jupyter notebook containing a Deepdream (https://github.com/google/deepdream) experiment created with Keras and a pre-trained VGG19 convolutional network. The experiment (which is a work in progress) is based on some suggestions provided by the Deepdream team in this blog post but works in a slightly different…