A short primer on Scala

Object-Oriented Meets Functional

Scala is relatively new language based on the JVM. The main difference between other “Object Oriented Languages” and Scala is that everything in Scala is an object. The primitive types that are defined in Java, such as int or boolean, are objects in Scala. Functions are treated as objects, too. As objects, they can be passed as arguments, allowing a functional programming approach to writing applications for Apache Spark.

If you have programmed in Java or C#, you should feel right at home with Scala with very little effort.

You can also run or compile Scala programs from commandline or from IDEs such as Eclipse.

Test on HDP 2.3.2 with Spark 1.4.1

Run Sandbox and connect to it from SSH

Let’s open a shell to our Sandbox through SSH:_

Launch Spark-Shell

To learn and experiment with data, I prefer the interactivity of the Scala shell. Let’s launch the Spark shell as it is a fully capable Scala shell.


Values can be either mutable or immutable. Mutable values are expressed by the var keyword.

Immutable values are expressed by the val keyword.

Type Inference

Scala like Java is a strongly typed language. But, it is very smart in type inference, which alleviates the onus of specifying types from the programmers.



Functions are defined with the def keyword. In Scala, the last expression in the function body is returned without the need of the return keyword.


You can write the function more succinctly by leaving out the braces and the return type as return type can be easily inferred.


Anonymous Functions

Anonymous functions can be assigned as a var or val value. It can also be passed to and returned from other functions.

Or, anonymous functions can be further shortened to

where _ is a the shorthand for whatever the input was.


Scala has very convenient set of collections including Lists, Sets, Tuples, Iterators and Options. When you combine these data structures with anonymous fucntions and closures it is very expressive.


This is far from comprehensive. To learn more visit http://scala-lang.org