Title

class: center, middle

# Predictive modeling with Python
## trends and tools

Olivier Grisel - PyData Berlin 2016

.affiliations[
  ![Inria](images/inria-logo.png)
  ![scikit-learn](images/scikit-learn-logo.png)
]

---
class: singleimg
# Big RAM is eating big data

![Inria](images/cumfq-kdnuggets.png)

.credits[
  Source: http://datascience.la/big-ram-is-eating-big-data-size-of-datasets-used-for-analytics/
  by Szilard Pafka
]

---
class: singleimg
# Big RAM is eating big data

![Inria](images/cumfq-kdnuggets-linear.png)

.credits[
  Source: http://datascience.la/big-ram-is-eating-big-data-size-of-datasets-used-for-analytics/
  by Szilard Pafka
]

---
# Big RAM is eating big data

.middlebelowheader[

### Datasets size increase by **20%** y/y

### Big EC2 instance RAM size increase by **50%** y/y

Year	| Type	       | RAM (GiB)
------|--------------|---------
2007  | m1.xlarge    | 15
2009  | m2.4xlarge   | 68
2012  | hs1.8xlarge  | 117
2014  | r3.8xlarge   | 244
2016  | x1.32xlarge  | 1952

### ⇒ single node in-memory analytics forever!?

]

---
class: middle, center, singleimg

![Consulting service: you bring your big data problems to me, I say
  "your data set fits in RAM", you pay me $10,000 for saving you $500,000.
](images/your-data-set-fits-in-RAM.png)

---
# But...

.middlebelowheader[

### What if we have a sample bias in the KDnuggets survey?

### Maybe favorite tools prevent the KDnuggets readers to work with larger datasets:

- Setting up a "Big Data" cluster requires sysadmin skills
- Hosting or renting a "Big Data" cluster is too expensive
- Cluster-induced latency makes analysis much less fun / productive

]

---
# Agenda

.middlebelowheader[

### Evolution on predictive modeling tools

### Scaling out analysis with dask/distributed

### Scaling out feature engineering with MPP

]

---
class: center, middle, bgheader
background-image: url(images/rocks-sea-14004414605_fe7c3dbe1d_k.jpg)
background-size: cover

# Evolution of predictive modeling tools

---
background-image: url(images/01-predictive-modeling-flow.png)
background-size: contain

---
background-image: url(images/02-predictive-modeling-flow.png)
background-size: contain

---
background-image: url(images/03-small-data-xl.png)
background-size: contain

---
background-image: url(images/04-small-medium-python.png)
background-size: contain

---
background-image: url(images/05-small-medium-python-pandas.png)
background-size: contain

---
background-image: url(images/06-small-medium-pandas-sklearn.png)
background-size: contain

---
background-image: url(images/07-big-hive.png)
background-size: contain

---
background-image: url(images/08-big-hive-python.png)
background-size: contain

---
background-image: url(images/09-big-redshift-python.png)
background-size: contain

---
background-image: url(images/10-big-spark-spark.png)
background-size: contain

---
class: center, middle, bgheader
background-image: url(images/forest-9026372290_ffed331779_k.jpg)
background-size: cover

# Where do predictive models fit?

---
background-image: url(images/11-big-data-archictecture.png)
background-size: contain

---
background-image: url(images/12-big-data-tools-mix-1.png)
background-size: contain

---
background-image: url(images/13-big-data-tools-mix-2.png)
background-size: contain

---
class: center, middle, bgheader
background-image: url(images/racks-8533890844_02fa24474d_o.jpg)
background-size: cover

# Scaling predictive modeling

---
# The need for scaling out in ML

.middlebelowheader[

### I/O intensive feature engineering and model scoring

#### Loading, filtering, joining, aggregating: SQL-land

#### Click log ⇒ Session features ⇒ User activity features

### CPU intensive model fitting

#### Hyper-parameter search and cross-validation

#### Gradient Boosting, Random Forests, Large Neural Networks

]

---
# Limitations of PySpark

.middlebelowheader[

### Python driver -> Scala / JVM -> Python worker

#### Latency induced by the networked architecture

#### Complex traceback / errors for non-scala developers

### No pure Python local mode for PySpark

#### Impossible to use profiler or ipdb for inner calls

]

---
# Dask + Distributed

.middlebelowheader[

### Collections oriented API:

- `dask.bag`
- `dask.array`
- `dask.dataframe`

### `dask.delayed`: procedural definition of the compute graph

### `distributed` provides cluster scheduler (optional)

]

---
# Dask + Distributed

.middlebelowheader[

<p style="width: 750px; text-align: center">
<video height="400px" controls
  src="videos/distributed_demo_with_dask.ogv" />
</p>
[Link to notebook](
  https://github.com/ogrisel/docker-distributed/blob/master/examples/sklearn_parameter_search.ipynb)
]

---
# Dask + Distributed limitations

.middlebelowheader[

### Young project (under very active development)

### `dask.dataframe` ⊂ `pandas.DataFrame` or PySpark

### No distributed shuffle:

#### Cannot do distributed `merge` / SQL join

#### Cannot do distributed `grouby` + `aggregate`

]

---
class: center, middle, bgheader
background-image: url(images/floor-2249694239_c27a3ea043_o.jpg)
background-size: cover

# Scaling feature engineering with MPP

---
class: bunchoflogos
# Massively Parallel Processing

.middlebelowheader[

![AWS Redshift](images/redshift-logo.png)
![Google BigQuery](images/bigquery-logo.png)

![Apache Impala](images/impala-logo.png)
![Citus](images/citusdata-logo.png)
![Presto](images/prestodb-logo.png)

]

---
# Problem with the use of SQL in MPP

.middlebelowheader[

## Great for ad-hoc queries but no plotting

## SQL strings in Python code is sad :(

### Not easy to write test / CI

## Standard ORM not a solution

]

---
# MPP in Python

.middlebelowheader[

### `blaze` & `ibis` provide 'dataframe' like Python API

### Under the hood it can generate SQL for MPP engines

### `blaze` also targets non-SQL backends (pandas, MongoDB, PySpark...)

]

---
class: middle, mmedium

```python
>>> import blaze as bz
>>> iris = bz.Data('postgresql://localhost::iris')
>>> iris
    sepal_length  sepal_width  petal_length  petal_width      species
0            5.1          3.5           1.4          0.2  Iris-setosa
1            4.9          3.0           1.4          0.2  Iris-setosa
2            4.7          3.2           1.3          0.2  Iris-setosa
3            4.6          3.1           1.5          0.2  Iris-setosa

>>> bz.by(iris.species, smallest=iris.petal_length.min(),
...                      largest=iris.petal_length.max())
           species  largest  smallest
0      Iris-setosa      1.9       1.0
1  Iris-versicolor      5.1       3.0
2   Iris-virginica      6.9       4.5
```

---
class: center, middle, bgheader
background-color: rgb(30, 100, 0)

# Conclusion

---
class: middle, center, singleimg

![Changing stuff and see what happens](images/changing-stuff-models.png)

---
# Secrets of the success of Python (& R) in Data Science

.middlebelowheader[

### Iterative exploration with built-in plotting tools

### Low latency of single host in-memory computing

### Easy to install, easy to teach: no-sysadmin required

### Rich ecosystem of libraries

]

---
# Conclusion

.middlebelowheader[

### PyData ecosystem is evolving towards big data / big compute

### We should never sacrifice usability on a single host

### dask, tensorflow and ibis / blaze show that it's possible

]

---
# Thank you

.middlebelowheader[
<div style="display: flex">

<div style="width: 50%; padding: 1em">
<p><img height="35px" src="images/carina-logo.svg" /></p>
<p>
<strong>Rackspace</strong> for free cloud resources to support SciPy
projects and for helping the Python community in general.
</p>
</div>

<div style="width: 50%; padding: 1em">
<p><img height="35px" src="images/gcp-logo.png" /></p>
<p>
<strong>Google</strong> for GCP credits to test distributed Python with
kubernetes.
</p>
</div>

</div>

<div style="margin: 1em auto; width: 400px">
<p><img height="50px" src="images/inria-logo.png" /></p>
<p>
<strong>Inria</strong> for supporting my work on scikit-learn and related
projects.
</p>
</div>
]

---
class: middle

# Thank you for your attention!

- https://github.com/dask

- https://github.com/ogrisel/docker-distributed

- [dask/distributed YouTube channel](
    https://www.youtube.com/playlist?list=PLRtz5iA93T4PQvWuoMnIyEIz1fXiJ5Pri)

---
class: medium

# dask/distributed on GKE

.middlebelowheader[
```
$ gcloud config set compute/zone europe-west1-d
$ gcloud container clusters create cluster-1 \
    --num-nodes 3 \
    --machine-type n1-highcpu-32 \
    --scopes bigquery,storage-rw \
    --wait
Creating cluster cluster-1...done.
[...]
$ gcloud container clusters get-credentials \
    cluster-1
```
]

---
class: medium

# dask/distributed on GKE

.middlebelowheader[
```
$ git clone https://github.com/ogrisel/docker-distributed
[...]
$ cd docker-distributed
$ kubectl create -f kubernetes/
service "dscheduler" created
service "dscheduler-status" created
replicationcontroller "dscheduler" created
replicationcontroller "dworker" created
service "jupyter-notebook" created
replicationcontroller "jupyter-notebook" created
```
]

---
Background image credits

- https://www.flickr.com/photos/jemimus/8533890844/

- https://www.flickr.com/photos/antcaz/2249694239/

- https://www.flickr.com/photos/benjamine-s/14004414605

- https://www.flickr.com/photos/a-herzog/9026372290