IntelliJ is a very popular IDE, which a lot of engineers use for developing Spark applications. I also like the Zeppelin IDE, which is very interactive, with good visualization capabilities, and supports Python, Scala, Java, and SQL.
To keep it simple and focused on our goals of working with Spark, we will use the Spark shell most of the time in this book.