SCALA and APACHE SPARK installation on Ubuntu 16 / 18

Prerequisites Java and Hadoop

Install java and configure Hadoop by referring previous page.

Java Installation

1. sudo apt-get update

2. sudo apt-get install default-jdk

3. java -version

4. update-alternatives --config java

After successful installation of Java and Hadoop:

Install Scala

5. sudo apt-get install scala

After Successful installation type scala in terminal as shown below

6. scala

Just to verify the working of scala, type following println statement

7. println(" Hello welcome to Scala ")

To quit from scala shell

8. :q

Install Spark

first install git before Spark for this

9. sudo apt-get install git

Download latest Spark and Extract it

Example:

Download

10. wget http://mirrors.estointernet.in/apache/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz

Extract

11. sudo tar xvf spark-2.4.3-bin-hadoop2.7.tgz

Create a spark director

12. sudo mkdir -p /usr/local/spark

Move extracted spark 2.7 to /usr/local/spark

13. cd spark-2.4.3-bin-hadoop2.7

14 mv * /usr/local/spark

Update ~/.bashrc file for spark path

15. vim ~/.bashrc

add following path to ~/.bashrc

SPARK_HOME=/usr/local/spark

export PATH=$SPARK_HOME/bin:$PATH

16. source ~/.bashrc

Goto bin directory of spark and start spark shell

17. cd /usr/local/spark/bin

18. ./spark-shell

To verify spark working use println statement

19. println(" Welcome to Apache Spark")