• Blog
  • Podcasts
  • Books
  • Resume / CV
  • Bonaccorso’s Law
  • Essays
  • Contact
  • Testimonials
  • Disclaimer

Giuseppe Bonaccorso

Artificial Intelligence – Machine Learning – Data Science

  • Blog
  • Podcasts
  • Books
  • Resume / CV
  • Bonaccorso’s Law
  • Essays
  • Contact
  • Testimonials
  • Disclaimer

SVD Recommendations using Tensorflow

08/02/2017 Machine Learning Python Tensorflow 6 Comments

Recommendation system based on the user-item matrix factorization have become more and more important thanks to powerful and distributable algorithms like ALS, but sometimes the number of users and/or items is not so huge and the computation can be done using directly a SVD (Singular Value Decomposition) algorithm. In this post, we’re going to discuss a very simple implementation based on Tensorflow.

The first element to define is the user-item matrix. Normally this matrix associates each user with each product through a rating (0 means that no rating has been provided) or explicit feedback. This approach is effective whenever it’s possible to ask the user to rate the items, but there are many situations where this isn’t possible. In all those cases, an implicit feedback can be employed. For example:

  • For a movie/video, it’s possible to consider how many times the user watched it and the average duration
  • For a song, it’s useful to consider the number of times it was played
  • For a blog/magazine, a valid measure is provided by the time spent on each single page

Many measures can be used effectively; the only condition is that they must be time-invariant metric functions. If there are n users and m products, the user-item matrix is:

The SVD (limiting the computation to the p greatest singular values) allows to define the matrix as:

 

The columns of U contain the left singular vector, while the rows of transposed V contain the right singular vectors. Sigma is a diagonal matrix containing the singular values.

In this model, we are assuming that a rating is determined by a set k user latent factors and k item ones. A latent factor is a variable that we cannot directly observe but it’s possible to estimate. This concept is very important because the theoretical foundation of the whole model is built upon it. Just to understand, we can say that a user rates a generic product according to the gender, age range, education, interests and so forth. At the same time, a product can be virtually split into different components that contribute the the overall rating: brand, features, price, etc. Even if we expect those latent factors, we cannot easily determine them, therefore we use a factorization method. The original user-item matrix will be split into two factors: a user-factor matrix and a factor-product matrix. The multiplication removes the latent factors, rebuilding the original user-item matrix.

If we assume to have k factors, we can truncate the SVD:

In this way, we are selecting the top k singular values and the corresponding singular vectors. Predictions can be obtained after defining two auxiliary matrices:

The predicted rating for user i and product j is:

where the first term is the average rating per user. However, in this example we are interested in finding the top k suggestions for all users, therefore we are not going to consider it as it’s a constant after defining the user.

In the example, we assume to have 5000 users and 2000 products. The number of factors (k) is rather high (500) and can be dramatically reduced. The ratings are bounded between 1 and 5 (0 meaning no rating). Every user has rated 500 random products and we want to suggest the top 10 items. In this case, I’m not controlling if the item has already been rated, in fact there are situations (like Youtube) where the recommendations contain video frequently seen and others (like Netflix) where it’s more useful to suggest new (never-seen) items.

The computational time is about 5 seconds on a i7 CPU.

See also:

A model-free collaborative recommendation system in 20 lines of Python code – Giuseppe Bonaccorso

Model-free collaborative filtering is a “lightweight” approach to recommendation systems. It’s always based on the implicit “collaboration” (in terms of ratings) among users, but it is computed in-memory without the usage of complex algorithms like ALS (Alternating Least Squares) that can be executed in parallel environment (like Spark).

Share:

  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)
  • Click to share on Pocket (Opens in new window)
  • Click to share on Tumblr (Opens in new window)
  • Click to share on Reddit (Opens in new window)
  • Click to share on Pinterest (Opens in new window)
  • Click to share on Skype (Opens in new window)
  • Click to share on WhatsApp (Opens in new window)
  • Click to share on Telegram (Opens in new window)
  • Click to email this to a friend (Opens in new window)
  • Click to print (Opens in new window)

You can also be interested in these articles:

recommendationssvdtensorflow

Lossy image autoencoders with convolution and deconvolution networks in Tensorflow

Assessing clustering optimality with instability index

6 thoughts on “SVD Recommendations using Tensorflow”

  1. Yin
    11/02/2017 at 8:09

    Hi, I want to ask whether “ratings_t = tf.matmul(Uk, Si)” should be “ratings_t = tf.matmul(Su, Si)”?

    Reply
    • Giuseppe Bonaccorso
      11/02/2017 at 8:45

      Hi,
      thank you very much. It was a typo. In the formula I wrote Su dot Si, but in the code there was a mistake. Corrected.

      Reply
  2. Vincent
    03/15/2018 at 8:56

    Compute reduced matrices

    Sk = tf.diag(St)[0:nb_factors, 0:nb_factors]
    Uk = Ut[:, 0:nb_factors]
    Vk = tf.transpose(Vt)[0:nb_factors, :]

    Hi, I just want to ask , there should be a transpose operation when calculating the Vk matrix. Or else the shape of ratings would be incorrect.

    Reply
    • Giuseppe Bonaccorso
      03/18/2018 at 12:11

      Hi,
      the SVD implementation (like in SciPy) returns the V matrix already transposed.
      Which shape are you obtaining?

      Reply
  3. Yongguang Zhang
    05/16/2018 at 11:06

    That’ what I need! thx!
    BTW, I think it’s will look better if u wrote the code of training process.
    I mean cost and train_op.
    It’s make it easier to understand for the beginer.
    Thanks again.

    Reply
    • Giuseppe Bonaccorso
      05/16/2018 at 19:26

      Hi,
      thanks! In this case, the training process is done through the SVD. An alternative approach which is based on a cost function (mean squared error) and a training operator is the Non-Negative Matrix Factorization. I’m going to add this example in a future post.

      Reply

Leave a Reply Cancel reply

Follow Me

  • linkedin
  • twitter
  • facebook
  • googlescholar
  • youtube
  • github
  • amazon
  • medium

Search articles

Latest blog posts

  • Mastering Machine Learning Algorithms Second Edition 02/03/2020
  • EphMrA 2019 Switzerland one day meeting 08/30/2019
  • Machine Learning Algorithms – Second Edition 08/28/2018
  • Recommendations and User-Profiling from Implicit Feedbacks 07/10/2018
  • Are recommendations really helpful? A brief non-technical discussion 06/29/2018

Subscribe to this blog

Join 2,199 other subscribers

Follow me on Twitter

My Tweets
Copyright © Giuseppe Bonaccorso. All Rights Reserved
Proudly powered by WordPress | Theme: Doo by ThemeVS.
loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.