IPython Interactive Computing and Visualization Cookbook

Over 100 hands-on recipes to sharpen your skills in high-performance numerical computing and data science with Python

IPython Interactive Computing and Visualization Cookbook

Cookbook
Cyrille Rossant

Over 100 hands-on recipes to sharpen your skills in high-performance numerical computing and data science with Python
$29.99
$49.99
RRP $29.99
RRP $49.99
eBook
Print + eBook
$12.99 p/month

Want this title & more? Subscribe to PacktLib

Enjoy full and instant access to over 2000 books and videos – you’ll find everything you need to stay ahead of the curve and make sure you can always get the job done.
+ Collection
Free Sample

Book Details

ISBN 139781783284818
Paperback512 pages

About This Book

  • Leverage the new features of the IPython notebook for interactive web-based big data analysis and visualization
  • Become an expert in high-performance computing and visualization for data analysis and scientific modeling
  • A comprehensive coverage of scientific computing through many hands-on, example-driven recipes with detailed, step-by-step explanations

Who This Book Is For

Intended to anyone interested in numerical computing and data science: students, researchers, teachers, engineers, analysts, hobbyists... Basic knowledge of Python/NumPy is recommended. Some skills in mathematics will help you understand the theory behind the computational methods.

Table of Contents

Chapter 1: A Tour of Interactive Computing with IPython
Introduction
Introducing the IPython notebook
Getting started with exploratory data analysis in IPython
Introducing the multidimensional array in NumPy for fast array computations
Creating an IPython extension with custom magic commands
Mastering IPython's configuration system
Creating a simple kernel for IPython
Chapter 2: Best Practices in Interactive Computing
Introduction
Choosing (or not) between Python 2 and Python 3
Efficient interactive computing workflows with IPython
Learning the basics of the distributed version control system Git
A typical workflow with Git branching
Ten tips for conducting reproducible interactive computing experiments
Writing high-quality Python code
Writing unit tests with nose
Debugging your code with IPython
Chapter 3: Mastering the Notebook
Introduction
Teaching programming in the notebook with IPython blocks
Converting an IPython notebook to other formats with nbconvert
Adding custom controls in the notebook toolbar
Customizing the CSS style in the notebook
Using interactive widgets – a piano in the notebook
Creating a custom JavaScript widget in the notebook – a spreadsheet editor for pandas
Processing webcam images in real time from the notebook
Chapter 4: Profiling and Optimization
Introduction
Evaluating the time taken by a statement in IPython
Profiling your code easily with cProfile and IPython
Profiling your code line-by-line with line_profiler
Profiling the memory usage of your code with memory_profiler
Understanding the internals of NumPy to avoid unnecessary array copying
Using stride tricks with NumPy
Implementing an efficient rolling average algorithm with stride tricks
Making efficient array selections in NumPy
Processing huge NumPy arrays with memory mapping
Manipulating large arrays with HDF5 and PyTables
Manipulating large heterogeneous tables with HDF5 and PyTables
Chapter 5: High-performance Computing
Introduction
Accelerating pure Python code with Numba and just-in-time compilation
Accelerating array computations with Numexpr
Wrapping a C library in Python with ctypes
Accelerating Python code with Cython
Optimizing Cython code by writing less Python and more C
Releasing the GIL to take advantage of multicore processors with Cython and OpenMP
Writing massively parallel code for NVIDIA graphics cards (GPUs) with CUDA
Writing massively parallel code for heterogeneous platforms with OpenCL
Distributing Python code across multiple cores with IPython
Interacting with asynchronous parallel tasks in IPython
Parallelizing code with MPI in IPython
Trying the Julia language in the notebook
Chapter 6: Advanced Visualization
Introduction
Making nicer matplotlib figures with prettyplotlib
Creating beautiful statistical plots with seaborn
Creating interactive web visualizations with Bokeh
Visualizing a NetworkX graph in the IPython notebook with D3.js
Converting matplotlib figures to D3.js visualizations with mpld3
Getting started with Vispy for high-performance interactive data visualizations
Chapter 7: Statistical Data Analysis
Introduction
Exploring a dataset with pandas and matplotlib
Getting started with statistical hypothesis testing – a simple z-test
Getting started with Bayesian methods
Estimating the correlation between two variables with a contingency table and a chi-squared test
Fitting a probability distribution to data with the maximum likelihood method
Estimating a probability distribution nonparametrically with a kernel density estimation
Fitting a Bayesian model by sampling from a posterior distribution with a Markov chain Monte Carlo method
Analyzing data with the R programming language in the IPython notebook
Chapter 8: Machine Learning
Introduction
Getting started with scikit-learn
Predicting who will survive on the Titanic with logistic regression
Learning to recognize handwritten digits with a K-nearest neighbors classifier
Learning from text – Naive Bayes for Natural Language Processing
Using support vector machines for classification tasks
Using a random forest to select important features for regression
Reducing the dimensionality of a dataset with a principal component analysis
Detecting hidden structures in a dataset with clustering
Chapter 9: Numerical Optimization
Introduction
Finding the root of a mathematical function
Minimizing a mathematical function
Fitting a function to data with nonlinear least squares
Finding the equilibrium state of a physical system by minimizing its potential energy
Chapter 10: Signal Processing
Introduction
Analyzing the frequency components of a signal with a Fast Fourier Transform
Applying a linear filter to a digital signal
Computing the autocorrelation of a time series
Chapter 11: Image and Audio Processing
Introduction
Manipulating the exposure of an image
Applying filters on an image
Segmenting an image
Finding points of interest in an image
Detecting faces in an image with OpenCV
Applying digital filters to speech sounds
Creating a sound synthesizer in the notebook
Chapter 12: Deterministic Dynamical Systems
Introduction
Plotting the bifurcation diagram of a chaotic dynamical system
Simulating an elementary cellular automaton
Simulating an ordinary differential equation with SciPy
Simulating a partial differential equation – reaction-diffusion systems and Turing patterns
Chapter 13: Stochastic Dynamical Systems
Introduction
Simulating a discrete-time Markov chain
Simulating a Poisson process
Simulating a Brownian motion
Simulating a stochastic differential equation
Chapter 14: Graphs, Geometry, and Geographic Information Systems
Introduction
Manipulating and visualizing graphs with NetworkX
Analyzing a social network with NetworkX
Resolving dependencies in a directed acyclic graph with a topological sort
Computing connected components in an image
Computing the Voronoi diagram of a set of points
Manipulating geospatial data with Shapely and basemap
Creating a route planner for a road network
Chapter 15: Symbolic and Numerical Mathematics
Introduction
Diving into symbolic computing with SymPy
Solving equations and inequalities
Analyzing real-valued functions
Computing exact probabilities and manipulating random variables
A bit of number theory with SymPy
Finding a Boolean propositional formula from a truth table
Analyzing a nonlinear differential system – Lotka-Volterra (predator-prey) equations
Getting started with Sage

What You Will Learn

  • Code better by writing high-quality, readable, and well-tested programs; profiling and optimizing your code, and conducting reproducible interactive computing experiments
  • Master all of the new features of the IPython notebook, including the interactive HTML/JavaScript widgets
  • Analyze data with Bayesian and frequentist statistics (Pandas, PyMC, and R), and learn from data with machine learning (scikit-learn)
  • Gain valuable insights into signals, images, and sounds with SciPy, scikit-image, and OpenCV
  • Learn how to write blazingly fast Python programs with NumPy, PyTables, ctypes, Numba, Cython, OpenMP, GPU programming (CUDA and OpenCL), parallel IPython, MPI, and many more

In Detail

IPython is at the heart of the Python scientific stack. With its widely acclaimed web-based notebook, IPython is today an ideal gateway to data analysis and numerical computing in Python.

IPython Interactive Computing and Visualization Cookbook contains many ready-to-use focused recipes for high-performance scientific computing and data analysis. The first part covers programming techniques, including code quality and reproducibility; code optimization; high-performance computing through dynamic compilation, parallel computing, and graphics card programming. The second part tackles data science, statistics, machine learning, signal and image processing, dynamical systems, and pure and applied mathematics.

Authors

Table of Contents

Chapter 1: A Tour of Interactive Computing with IPython
Introduction
Introducing the IPython notebook
Getting started with exploratory data analysis in IPython
Introducing the multidimensional array in NumPy for fast array computations
Creating an IPython extension with custom magic commands
Mastering IPython's configuration system
Creating a simple kernel for IPython
Chapter 2: Best Practices in Interactive Computing
Introduction
Choosing (or not) between Python 2 and Python 3
Efficient interactive computing workflows with IPython
Learning the basics of the distributed version control system Git
A typical workflow with Git branching
Ten tips for conducting reproducible interactive computing experiments
Writing high-quality Python code
Writing unit tests with nose
Debugging your code with IPython
Chapter 3: Mastering the Notebook
Introduction
Teaching programming in the notebook with IPython blocks
Converting an IPython notebook to other formats with nbconvert
Adding custom controls in the notebook toolbar
Customizing the CSS style in the notebook
Using interactive widgets – a piano in the notebook
Creating a custom JavaScript widget in the notebook – a spreadsheet editor for pandas
Processing webcam images in real time from the notebook
Chapter 4: Profiling and Optimization
Introduction
Evaluating the time taken by a statement in IPython
Profiling your code easily with cProfile and IPython
Profiling your code line-by-line with line_profiler
Profiling the memory usage of your code with memory_profiler
Understanding the internals of NumPy to avoid unnecessary array copying
Using stride tricks with NumPy
Implementing an efficient rolling average algorithm with stride tricks
Making efficient array selections in NumPy
Processing huge NumPy arrays with memory mapping
Manipulating large arrays with HDF5 and PyTables
Manipulating large heterogeneous tables with HDF5 and PyTables
Chapter 5: High-performance Computing
Introduction
Accelerating pure Python code with Numba and just-in-time compilation
Accelerating array computations with Numexpr
Wrapping a C library in Python with ctypes
Accelerating Python code with Cython
Optimizing Cython code by writing less Python and more C
Releasing the GIL to take advantage of multicore processors with Cython and OpenMP
Writing massively parallel code for NVIDIA graphics cards (GPUs) with CUDA
Writing massively parallel code for heterogeneous platforms with OpenCL
Distributing Python code across multiple cores with IPython
Interacting with asynchronous parallel tasks in IPython
Parallelizing code with MPI in IPython
Trying the Julia language in the notebook
Chapter 6: Advanced Visualization
Introduction
Making nicer matplotlib figures with prettyplotlib
Creating beautiful statistical plots with seaborn
Creating interactive web visualizations with Bokeh
Visualizing a NetworkX graph in the IPython notebook with D3.js
Converting matplotlib figures to D3.js visualizations with mpld3
Getting started with Vispy for high-performance interactive data visualizations
Chapter 7: Statistical Data Analysis
Introduction
Exploring a dataset with pandas and matplotlib
Getting started with statistical hypothesis testing – a simple z-test
Getting started with Bayesian methods
Estimating the correlation between two variables with a contingency table and a chi-squared test
Fitting a probability distribution to data with the maximum likelihood method
Estimating a probability distribution nonparametrically with a kernel density estimation
Fitting a Bayesian model by sampling from a posterior distribution with a Markov chain Monte Carlo method
Analyzing data with the R programming language in the IPython notebook
Chapter 8: Machine Learning
Introduction
Getting started with scikit-learn
Predicting who will survive on the Titanic with logistic regression
Learning to recognize handwritten digits with a K-nearest neighbors classifier
Learning from text – Naive Bayes for Natural Language Processing
Using support vector machines for classification tasks
Using a random forest to select important features for regression
Reducing the dimensionality of a dataset with a principal component analysis
Detecting hidden structures in a dataset with clustering
Chapter 9: Numerical Optimization
Introduction
Finding the root of a mathematical function
Minimizing a mathematical function
Fitting a function to data with nonlinear least squares
Finding the equilibrium state of a physical system by minimizing its potential energy
Chapter 10: Signal Processing
Introduction
Analyzing the frequency components of a signal with a Fast Fourier Transform
Applying a linear filter to a digital signal
Computing the autocorrelation of a time series
Chapter 11: Image and Audio Processing
Introduction
Manipulating the exposure of an image
Applying filters on an image
Segmenting an image
Finding points of interest in an image
Detecting faces in an image with OpenCV
Applying digital filters to speech sounds
Creating a sound synthesizer in the notebook
Chapter 12: Deterministic Dynamical Systems
Introduction
Plotting the bifurcation diagram of a chaotic dynamical system
Simulating an elementary cellular automaton
Simulating an ordinary differential equation with SciPy
Simulating a partial differential equation – reaction-diffusion systems and Turing patterns
Chapter 13: Stochastic Dynamical Systems
Introduction
Simulating a discrete-time Markov chain
Simulating a Poisson process
Simulating a Brownian motion
Simulating a stochastic differential equation
Chapter 14: Graphs, Geometry, and Geographic Information Systems
Introduction
Manipulating and visualizing graphs with NetworkX
Analyzing a social network with NetworkX
Resolving dependencies in a directed acyclic graph with a topological sort
Computing connected components in an image
Computing the Voronoi diagram of a set of points
Manipulating geospatial data with Shapely and basemap
Creating a route planner for a road network
Chapter 15: Symbolic and Numerical Mathematics
Introduction
Diving into symbolic computing with SymPy
Solving equations and inequalities
Analyzing real-valued functions
Computing exact probabilities and manipulating random variables
A bit of number theory with SymPy
Finding a Boolean propositional formula from a truth table
Analyzing a nonlinear differential system – Lotka-Volterra (predator-prey) equations
Getting started with Sage

Book Details

ISBN 139781783284818
Paperback512 pages
Read More