Looking at spark reduceByKey example, we can say that reduceByKey is one step ahead then reduce function in Spark with the contradiction that it is a transformation operation. Let’s understand this operation by some examples in Scala, Java and Python languages. Continue reading “Apache Spark reduceByKey Example”
Apache Spark reduce Example
Here in spark reduce example, we’ll understand how reduce operation works in Spark with examples in languages like Scala, Java and Python. Spark reduce operation is an action kind of operation and it triggers a full DAG execution for all lined up lazy instructions. Continue reading “Apache Spark reduce Example”
Apache Spark filter Example
In spark filter example, we’ll explore filter method of Spark RDD class in all of three languages Scala, Java and Python. Spark filter operation is a transformation kind of operation so its evaluation is lazy. Let’s dig a bit deeper. Continue reading “Apache Spark filter Example”
Apache Spark flatMap Example
Spark flatMap example is mostly similar operation with RDD map operation. It is also defined in RDD abstract class of spark core library and same as map it also is a transformation kind of operation hence it is lazily evaluated. Continue reading “Apache Spark flatMap Example”
Apache Spark map Example
In Apache Spark map example, we’ll learn about all ins and outs of map function. Basically map is defined in abstract class RDD in spark and it is a transformation kind of operation which means it is a lazy operation. Let’s explore it in detail. Continue reading “Apache Spark map Example”
YARN Architecture and Components
We have discussed a high level view of YARN Architecture in my post on Understanding Hadoop 2.x Architecture but YARN it self is a wider subject to understand. Keeping that in mind, we’ll about discuss YARN Architecture, it’s components and advantages in this post. Continue reading “YARN Architecture and Components”
Understanding Hadoop 2.x Architecture and it’s Daemons
Prior to learn the concepts of Hadoop 2.x Architecture, I strongly recommend you to refer the my post on Hadoop Core Components, internals of Hadoop 1.x Architecture and its limitations. It will give you the idea about Hadoop2 Architecture requirement. And we have already learnt about the basic Hadoop components like Name Node, Secondary Name Node, Data Node, Job Tracker and Task Tracker. Continue reading “Understanding Hadoop 2.x Architecture and it’s Daemons”
Java 8 Default Methods in Interface
Interfaces went through major changes while redesign in Java 8 and as a part of that Java 8 default methods in Interface are the major addition. Java 8 interfaces also supports static methods along with default and abstract methods. Let’s dig into the more details to know the facts about default methods in interfaces. Continue reading “Java 8 Default Methods in Interface”
Java 8 Method Reference With Examples
After learning concepts of Java 8 Lambda Expressions and Streams, let’s understand one more Java 8 concept, and that is Java 8 method reference. It is one of the key concept which has shorten the Java syntax by new Syntactic Sugar with functional capabilities. Continue reading “Java 8 Method Reference With Examples”
Java 8 Streams By Examples
Whenever we hear about Java 8 Streams the first thing which strike in mind would be Java I/O stream and classes like InputStream and OutputStream. But here in Java 8 Stream is quite different concept than Java I/O Streams. Java 8 Stream best implementation of functional programming which can be used easily with Java Collection Framework. Continue reading “Java 8 Streams By Examples”