• Blog
  • Podcasts
  • Books
  • Resume / CV
  • Bonaccorso’s Law
  • Essays
  • Contact
  • Testimonials
  • Disclaimer

Giuseppe Bonaccorso

Artificial Intelligence – Machine Learning – Data Science

  • Blog
  • Podcasts
  • Books
  • Resume / CV
  • Bonaccorso’s Law
  • Essays
  • Contact
  • Testimonials
  • Disclaimer

Theano GPU vs pure Numpy (CPU)

07/11/2016 Deep Learning Generic Machine Learning Python Theano 2 Comments

In this benchmark, I’ve used a Windows 10 Pro 64 Bit computer with Intel Core i7 6700HQ 2.60 GHz with 32 Gb RAM and NVIDIA GeForce GTX 960M. As a programming environment, I’ve used Python 2.7 (Anaconda distribution) and Jupyter.

The task is very simple, integrating this expression (simple but effective):

The code I’ve written is this (without matplotlib functions and float32 numbers, in order to use the GPU):

import math
from datetime import datetime

import numpy as np
import matplotlib.pyplot as plt

import theano
import theano.tensor as T
from theano import function, shared

# Define constants
a = 0
b = math.pi
precision = 10000000.0
delta = ((b-a) / precision)

# Define x linear space
xs = np.linspace(a, b, num=precision).astype(np.float32)

# Define Theano function
xss = shared(xs, 'xss')
deltas = shared(delta, 'delta')
sinvx = T.sum(T.sin(xss) * deltas)
sf = function([], sinvx)

# Number of iterations
num_executions = 500
execution_times = []

# Theano test
for i in range(num_executions):
    t0 = datetime.now()
    res = sf()
    t1 = datetime.now()

    execution_times.append((t1-t0).microseconds)
   
et = np.array(execution_times)

print ('Theano:')
print ('Result: %f' % res)    
print('Average execution time: %d (us)' % np.average(et))

execution_times = []
 
# Numpy test
for i in range(num_executions):
    t0 = datetime.now()
    res = (sin(xs) * delta).sum()
    t1 = datetime.now()
    
    execution_times.append((t1-t0).microseconds)

et = np.array(execution_times)

print('Numpy:')
print('Result: %f' % res)
print('Average execution time: %d (us)' % np.average(et))

Final results are:

Using gpu device 0: GeForce GTX 960M (CNMeM is enabled with initial size: 20.0% of memory, CuDNN 3007)

Theano:
Result: 2.000000
Average execution time: 39690 (us)

Numpy:
Result: 2.000001
Average execution time: 158240 (us)

So, Numpy is on average 300% slower than Theano (with GPU support). Here the diagrams:

theano

numpy

The spikes should be due to CPU overload, multitasking or memory swapping. However, it’s absolutely clear that Theano (I’m going to test also Tensorflow) should be the best choice if you want to implement deep learning algorithms (in particular if you have a good GPU).

See also:

Machine Learning Algorithms – Giuseppe Bonaccorso

My latest machine learning book has been published and will be available during the last week of July. From the back cover: In this book you will learn all the important Machine Learning algorithms that are commonly used in the field of data science.

Share:

  • Click to share on Twitter (Opens in new window)
  • Click to share on Facebook (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)
  • Click to share on Pocket (Opens in new window)
  • Click to share on Tumblr (Opens in new window)
  • Click to share on Reddit (Opens in new window)
  • Click to share on Pinterest (Opens in new window)
  • Click to share on Skype (Opens in new window)
  • Click to share on WhatsApp (Opens in new window)
  • Click to share on Telegram (Opens in new window)
  • Click to email this to a friend (Opens in new window)
  • Click to print (Opens in new window)

You can also be interested in these articles:

benchmarkcpudeep learninggpumachine learningnumpypythontheano

Archimedes: a simple exercise with Keras and Scikit-Fuzzy

Fixed-delay smoothing in HMM with Numpy

2 thoughts on “Theano GPU vs pure Numpy (CPU)”

  1. Jim
    08/12/2017 at 2:02

    Your example code needs a slight correction. You should clear the execution_times array between the Theano and Numpy runs. As it is, the Theano execution times are included in the Numpy average. Just add a, execution_times = [], after the Theano results are printed and before the Numpy calculations are started.

    Best regards,

    Jim

    Reply
    • Giuseppe Bonaccorso
      08/12/2017 at 8:10

      Thanks Jim. Indeed it was a typo when copying from the Jupyter notebook. However, the calculations are correct (confirmed also by the plots).

      Reply

Leave a Reply Cancel reply

Follow Me

  • linkedin
  • twitter
  • facebook
  • googlescholar
  • youtube
  • github
  • amazon
  • medium

Search articles

Latest blog posts

  • Mastering Machine Learning Algorithms Second Edition 02/03/2020
  • EphMrA 2019 Switzerland one day meeting 08/30/2019
  • Machine Learning Algorithms – Second Edition 08/28/2018
  • Recommendations and User-Profiling from Implicit Feedbacks 07/10/2018
  • Are recommendations really helpful? A brief non-technical discussion 06/29/2018

Subscribe to this blog

Join 2,199 other subscribers

Follow me on Twitter

My Tweets
Copyright © Giuseppe Bonaccorso. All Rights Reserved
Proudly powered by WordPress | Theme: Doo by ThemeVS.
loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.