Plotting outside of Spark
If you wish to use other tools to plot the data, you can first take a sample of the Spark data and plot it using another package such as ggplot
. Note that some versions of Spark may now have ggplot integrated and available for use within Spark. However this example will show another example of extracting data which can be used by other packages.
Collecting a sample of the results
We will take a 2% sample of all of the predictions and then print some of the results. Note that the Spark sample function has a different syntax from the base R sample function we used earlier. You could also specify this as SparkR::sample
to make sure you are invoking the correct function:
local = collect(sample(preds, F,.02)) head(local)
![](https://static.packt-cdn.com/products/9781785886188/graphics/image_10_019.png)
Examining the distributions by outcome
Next, you can run ggplot
to graphically display the errors grouped by outcome. The resulting boxplots show that the three quartiles for diabetes are above the non-diabetic patients. This demonstrates that the model...