Scala Programming Language : Introduction

The word “Scala” is a short form of the term “Scalable Language”. Technically, Scala programming is a combination of the two style of programming approaches and those are “Object Oriented Programming” and “Functional Programming”. The Object Oriented approach makes language very easy to build large applications with big structures and architecture and Functional approach keeps design very modular and pluggable. This is how Scala language is empowered by fusion of both of these approaches. Continue reading “Scala Programming Language : Introduction”

Simple explanation of Hadoop Core Components : HDFS and MapReduce

Before this post we have discussed about what is Hadoop and what kind of issues are solved by Hadoop. Now Let’s deep dive in to various components of Hadoop. Hadoop as a whole distribution provides only two core components and HDFS (which is Hadoop Distributed File System) and MapReduce (which is a distributed batch processing framework). And a complete bunch of machines which are running HDFS and MapReduce are known as Hadoop Cluster.

As you add more nodes in Hadoop Cluster the performance of your cluster will increase which means that Hadoop is Horizontally Scalable.  Continue reading “Simple explanation of Hadoop Core Components : HDFS and MapReduce”

Hadoop Core Concepts : The Bazics

In my earlier post about Brief Introduction of Hadoop, we have understood “What is Hadoop and What kind of problems it solves”. The next step is to understand Hadoop Core Concepts which talks more about,

  • Distributed system design
  • How data is distributed across multiple systems
  • What are the different components involved and how they communicate with each others

Continue reading “Hadoop Core Concepts : The Bazics”

6 Steps to Setup Apache Spark 1.0.1 (Multi Node Cluster) on CentOS

Before we move ahead lets learn a bit on Setup Apache Spark,

So, What is Apache Spark?

Apache Spark is a fast, real time and extremely expressive computing system which executes job in distributed (clustered) environment.

It is quite compatible with Apache Hadoop and more almost 10x faster than Hadoop MapReduce on Disk Computing and 100x faster using in memory computations. It provides rich APIs in Java, Scala and Python along with Functional Programming capabilities. Continue reading “6 Steps to Setup Apache Spark 1.0.1 (Multi Node Cluster) on CentOS”

Why Functional References are called “Lambda Expressions” in Java 8?

Many of us are very curious about the fact that why functional references introduced in Java 8 are know as Lambda Expressions. So let’s reveal the fact of word “Lambda Expressions” and make it very simple to understand.

This fact is very much related to the Evaluation process of any expression. Many of us may have solved the mathematical problems and while solving the problems we may have observed how we are reducing the mathematical expression to some values. Similarly we also may have observed in many programming languages how any expression is evaluated. Continue reading “Why Functional References are called “Lambda Expressions” in Java 8?”

Java 8: Lambda Expressions

Java 8 is completing almost 2 years now and this topic is widely discussed on Internet now a days. Java was originally designed to work OOP as it was the primary paradigm for software development at that time. But today’s programming directed towards concurrent and reactive programming which focuses on functional modularity which is provided by functional languages like Scala.

Concept of functional programming is completely a new era for the hard core Java Programmers and lambda expressions are bridging OOP and FP by adding functional language capabilities to Java. You will find this concept roaming around anonymous classes and functional programming. Continue reading “Java 8: Lambda Expressions”

4 Steps to Setup Scala On CentOS

Scala is a widely used Functional Programming language in today’s era of programming. It is a hybrid language which combines the feature of functional and object oriented programming together. Same as Java it also convert the source code to byte code hence it require JVM in order to execute the program. You can learn more about it from this link. Continue reading “4 Steps to Setup Scala On CentOS”

Setup Multi Node Hadoop 2.6.0 Cluster with YARN

Today is the era of parallel computation and whenever we talk about processing very large chunk of datasets the first word that comes in everyone’s mind is HADOOP. Apache Hadoop sits at the peak of Apache Project lists. In this post I’ll explain you all steps of setting up a Bazic Multi Node Hadoop Cluster (we’ll setup two node cluster).

Here I have used two machines for cluster setup you can repeat the steps of setting up slave nodes on more machines in order to create bigger Hadoop cluster. Continue reading “Setup Multi Node Hadoop 2.6.0 Cluster with YARN”