## A Brief (and Comprehensive) Guide to Stochastic Gradient Descent Algorithms

Stochastic Gradient Descent (SGD) is a very powerful technique, currently employed to optimize all deep learning models. However, the vanilla algorithm has many limitations, in particular when the system is ill-conditioned and could never find the global minimum. In this post, we’re going to analyze how it works and the…

## A virtual Jacques Lacan discusses about Artificial Intelligence

“In other words, the man who is born into existence deals first with language; this is a given. He is even caught in it before his birth.” (J. Lacan)   A virtual discussion with Jacques Lacan is a very hard task, above all when the main topic is Artificial Intelligence,…

## Linearly Separable? No? For me it is! A Brief introduction to Kernel Methods

This is a crash-introduction to kernel methods and the best thing to do is starting with a very simple question? Is this bidimensional set linearly separable? Of course, the answer is yes, it is. Why? A dataset defined in a subspace Ω ⊆ ℜn is linearly separable if there exists a (n-1)-dimensional hypersurface…

## Hopfield Networks addendum: Brain-State-in-a-Box model

The Brain-State-in-a-Box is neural model proposed by Anderson, Silverstein, Ritz and Jones in 1977, that presents very strong analogies with Hopfield networks (read the previous post about them). The structure of the network is similar: recurrent, fully-connected with symmetric weights and non-null auto-recurrent connections. All neurons are bipolar (-1 and…

## Elon Musk’s AI “Child Warning” marketing approach

“I disapprove what you say, but I will defend to the death the right to say it.” (E. B. Hall – misattributed to Voltaire) Elon Musk is surely a great entrepreneur. His companies SpaceX, Tesla, and OpenAI are pioneering the world of Artificial Intelligence with a clear and pragmatic approach.…

## ML Algorithms Addendum: Hopfield Networks

Hopfield networks (named after the scientist John Hopfield) are a family of recurrent neural networks with bipolar thresholded neurons. Even if they are have replaced by more efficient models, they represent an excellent example of associative memory, based on the shaping of an energy surface. In the following picture, there’s…

## Quickprop: an almost forgotten neural training algorithm

Standard Back-propagation is probably the best neural training algorithm for shallow and deep networks, however, it is based on the chain rule of derivatives and an update in the first layers requires a knowledge back-propagated from the last layer. This non-locality, especially in deep neural networks, reduces the biological plausibility…

## A model-free collaborative recommendation system in 20 lines of Python code

Model-free collaborative filtering is a “lightweight” approach to recommendation systems. It’s always based on the implicit “collaboration” (in terms of ratings) among users, but it is computed in-memory without the usage of complex algorithms like ALS (Alternating Least Squares) that can be executed in parallel environment (like Spark). If we assume…

## Artificial Intelligence is a matter of Language

“The limits of my language means the limits of my world.” (L. Wittgenstein)   When Jacques Lacan proposed his psychoanalytical theory based on the influence of language on human beings, many auditors remained initially astonished. Is language an actual limitation? In the popular culture, it isn’t. It cannot be! But,…