Scaling Models for Production

When deploying models in real-world environments, you may encounter large datasets, distributed infrastructure, or high inference demand. In this recipe we’ll explore techniques to scale model training and prediction, including leveraging n_jobs, joblib parallelism, connecting to external backends like Dask (https://www.dask.org/), and designing for batch serving.

Getting ready

You’ll need tools to run parallel inference and synthetic data to benchmark performance.

1. Load libraries:

   import numpy as np
   from sklearn.datasets import make_classification
   from sklearn.ensemble import RandomForestClassifier
   from sklearn.model_selection import cross_val_score
   import time

2. Train a forest model on synthetic data:

   X, y = make_classification(n_samples=2000, n_features=50, random_state=2024)
   clf = RandomForestClassifier(n_estimators=100, n_jobs=-1, random_state=2024)
   clf.fit(X, y)

Next, let’s try predicting on a random batch size of text data.

How...