Today is the era of parallel computation and whenever we talk about processing very large chunk of datasets the first word that comes in everyone’s mind is HADOOP. Apache Hadoop sits at the peak of Apache Project lists. In this post I’ll explain you all steps of setting up a Bazic Multi Node Hadoop Cluster (we’ll setup two node cluster).
Here I have used two machines for cluster setup you can repeat the steps of setting up slave nodes on more machines in order to create bigger Hadoop cluster. Continue reading “Setup Multi Node Hadoop 2.6.0 Cluster with YARN”
Most of the when we start to setup any kind of cluster (for Hadoop, Spark etc…) from scratch, we as a beginner often face certain problems while performing steps (like setup Static IP address or setup FQDN) after installation of Linux flavour OS on our VM or Machine. There are very less forums where you can get all of these bazics steps together along with full understanding. Here I have tried to explain you the process of setting up static IP address and FQDN (Fully Qualified Domain Name) in 5 steps. All of these steps are tested on VM/Machine available straight after minimal installation of CentOS 6.4. Continue reading “5 Steps to Setup Static IP And FQDN to CentOS Machine”
Setting up keyless SSH is quite easy on CentOS but still sometimes there are chances that after following all the steps mentioned in How to setup Keyless SSH with non root users in CentOS post it is still not setup properly. There could be many possibilities as a root cause like improper permissions, invalid configuration etc. I have mentioned mainly four debug points which you should follow if your keyless ssh setup seems to be misbehaving. Continue reading “4 Debug Points to check why your Keyless SSH is not working”
While setting up Hadoop or Spark Cluster, primary objective is always to setup keyless SSH between master and slave machines after you set Static IP and FQDN properly. First of all lets understand why it is required to setup passwordless SSH is required for any distributed cluster. Continue reading “How to setup Keyless SSH with non root users in CentOS”
Installing Java on CentOS is one of the easiest exercise ever. However, I’ll let you know two different ways in which Java can be installed on CentOS. We’ll take latest version of Java which is Java 8. Even you can combine all following steps in one shell script and can simply execute that shell script in order to install Java/JDK. Following steps are performed using root user you can also execute the steps with non root user having sudoer rights. Before moving towards installation process I have assumed that CentOS has
wget command installed. If not than install it using below step. Continue reading “2 Ways of installing Java 8 on CentOS”