This is the first step. This recipe shows how to install Apache Kafka.
Ensure that you have at least 4 GB of RAM in your machine; the installation directory will be /usr/local/kafka/ for Mac users and /opt/kafka/ for Linux users. Create these directories.
Go to the Apache Kafka home page at http://kafka.apache.org/downloads, as in Figure 1-1, Apache Kafka download page:
Figure 1-1. Apache Kafka download page
The current available version of Apache Kafka is 0.10.2.1, as a stable release. A major limitation with Kafka since 0.8.x is that it is not backward-compatible. So, we cannot replace this version for one prior to 0.8. Once you've downloaded the latest available release, let's proceed with the installation.
Remember, for Mac users, replace the directory /opt/ for /usr/local/ in the examples.
We need Java 1.7 or later. Download and install the latest JDK from Oracle's website: http://www.oracle.com/technetwork/java/javase/downloads/index.html
- Change the file mode:
> chmod +x jdk-8u131-linux-x64.rpm- Go to the directory in which you want to perform the installation:
> cd <directory path name>- Run the
rpminstaller with the command:
> rpm -ivh jdk-8u131-linux-x64.rpm- Finally, add the environment variable
JAVA_HOME. This command will write theJAVA_HOMEenvironment variable to the file/etc/profile:
> echo "export JAVA_HOME=/usr/java/jdk1.8.0_131" >> /etc/profileThe following are the steps to install Scala in Linux:
- Download the latest Scala binary from: http://www.scala-lang.org/download
- Extract the downloaded file
scala-2.12.2.tgz:
> tar xzf scala-2.12.2.tgzMost tutorials agree that the best place to set environment variables is in the
/etc/profilefile.- Create the
SCALA_HOMEenvironment variable:
> export SCALA_HOME=/opt/scala- Add the Scala bin directory to the
PATHvariable:
> export PATH=$PATH:$SCALA_HOME/binThe following are the steps to install Kafka in Linux:
- Extract the downloaded file
kafka_2.10-0.10.2.1.tgz:
> tar xzf kafka_2.10-0.10.2.1.tgz- Create the
KAFKA_HOMEenvironment variable:
> export KAFKA_HOME=/opt/kafka_2.10-0.10.2.1- Add the Kafka bin directory to the
PATHvariable:
> export PATH=$PATH:$KAFKA_HOME/binNow Java, Scala, and Kafka are installed.
To do all these steps in command-line mode, there is a powerful tool for Mac users called brew (the equivalent in Linux would be yum).
To install from the command line, we use the following steps:
- With brew,
install sbt(Scala build tool):
> brew install sbtIf you already have it (downloaded in the past), upgrade it:
> brew upgrade sbtThe output is similar to:
> brew upgrade sbt ==> Upgrading 1 outdated package, with result: sbt 0.13.15 ==> Upgrading sbt ==> Using the sandbox ==> Downloading https://github.com/sbt/sbt/releases/download/v0.13.15/sbt-0.13.15.tgz ==> Downloading from https://github-cloud.s3.amazonaws.com/releases/279553/09838df4-23c6-11e7-9276-14 ######################################################################## 100.0% ==> Caveats You can use $SBT_OPTS to pass additional JVM options to SBT: SBT_OPTS="-XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M" This formula is now using the standard lightbend sbt launcher script. Project specific options should be placed in .sbtopts in the root of your project. Global settings should be placed in /usr/local/etc/sbtopts ==> Summary /usr/local/Cellar/sbt/0.13.15: 378 files, 63.3MB, built in 1 minute 5 seconds
- With brew, install Scala:
> brew install scalaIf you already have it (downloaded in the past), upgrade it:
> brew upgrade scalaThe output is similar to:
> brew install scala ==> Using the sandbox ==> Downloading https://downloads.lightbend.com/scala/2.12.2/scala-2.12.2.tgz ######################################################################## 100.0% ==> Downloading https://raw.githubusercontent.com/scala/scala-tool-support/0a217bc/bash-completion/sr ######################################################################## 100.0% ==> Caveats To use with IntelliJ, set the Scala home to: /usr/local/opt/scala/idea Bash completion has been installed to: /usr/local/etc/bash_completion.d ==> Summary /usr/local/Cellar/scala/2.12.2: 44 files, 19.9MB, built in 19 seconds Mist:Downloads admin1$ scala -version Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL
- With brew, install Kafka (it also installs ZooKeeper):
> brew install kafkaIf you already have it (downloaded in the past), upgrade it:
> brew upgrade kafkaThe output is similar to:
> brew install kafka ==> Installing dependencies for kafka: zookeeper ==> Installing kafka dependency: zookeeper ==> Downloading https://homebrew.bintray.com/bottles/zookeeper-3.4.9.sierra.bottle.tar.gz ######################################################################## 100.0% ==> Pouring zookeeper-3.4.9.sierra.bottle.tar.gz ==> Using the sandbox ==> Caveats To have launched start zookeeper now and restart at login: brew services start zookeeper Or, if you don't want/need a background service you can just run: zkServer start ==> Summary /usr/local/Cellar/zookeeper/3.4.9: 242 files, 18.2MB ==> Installing kafka ==> Downloading https://homebrew.bintray.com/bottles/kafka-0.10.2.0.sierra.bottle.tar.gz ######################################################################## 100.0% ==> Pouring kafka-0.10.2.0.sierra.bottle.tar.gz ==> Caveats To have launchd start kafka now and restart at login: brew services start kafka Or, if you don't want/need a background service you can just run: zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties & kafka-server-start /usr/local/etc/kafka/server.properties ==> Summary /usr/local/Cellar/kafka/0.10.2.0: 145 files, 37.3MB
- Take a look at the Apache Kafka download page: http://kafka.apache.org/downloads
- To see more details about brew, visit: https://brew.sh/
