SCALA and APACHE SPARK installation on Ubuntu 16 / 18
Prerequisites Java and Hadoop
Install java and configure Hadoop by referring previous page.
Java Installation
1. sudo apt-get update
2. sudo apt-get install default-jdk
3. java -version
4. update-alternatives --config java
After successful installation of Java and Hadoop:
Install Scala
5. sudo apt-get install scala
After Successful installation type scala in terminal as shown below
6. scala
Just to verify the working of scala, type following println statement
7. println(" Hello welcome to Scala ")
To quit from scala shell
8. :q
Install Spark
first install git before Spark for this
9. sudo apt-get install git
Download latest Spark and Extract it
Example:
Download
10. wget http://mirrors.estointernet.in/apache/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz
Extract
11. sudo tar xvf spark-2.4.3-bin-hadoop2.7.tgz
Create a spark director
12. sudo mkdir -p /usr/local/spark
Move extracted spark 2.7 to /usr/local/spark
13. cd spark-2.4.3-bin-hadoop2.7
14 mv * /usr/local/spark
Update ~/.bashrc file for spark path
15. vim ~/.bashrc
add following path to ~/.bashrc
SPARK_HOME=/usr/local/spark
export PATH=$SPARK_HOME/bin:$PATH
16. source ~/.bashrc
Goto bin directory of spark and start spark shell
17. cd /usr/local/spark/bin
18. ./spark-shell
To verify spark working use println statement
19. println(" Welcome to Apache Spark")