TensorFrames – quick start

After all this preamble, let's jump start our use of TensorFrames with this quick start tutorial. You can download and use the full notebook within Databricks Community Edition at http://bit.ly/2hwGyuC.

You can also run this from the PySpark shell (or other Spark environments), like any other Spark package:

# The version we're using in this notebook
$SPARK_HOME/bin/pyspark --packages tjhunter:tensorframes:0.2.2-s_2.10  

# Or use the latest version 
$SPARK_HOME/bin/pyspark --packages databricks:tensorframes:0.2.3-s_2.10

Note, you will only use one of the above commands (that is, not both). For more information, please refer to the databricks/tensorframes GitHub repository (https://github.com/databricks/tensorframes).

Configuration and setup

Please follow the configuration and setup steps in the following order:

Launching a Spark cluster

Launch a Spark cluster using Spark 1.6 (Hadoop 1) and Scala 2.10. This has been tested with Spark 1.6, Spark 1.6.2, and Spark 1.6.3 ...