The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models.

In this article I have shown how you can install hadoop on MacOS specific on Macbook M1 fixing the possible errors. So let’s get started.

Step 1 : Install HomeBrew

/bin/bash -c “$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Use these commands to add homebrew on your bash_profile

echo ‘eval “$(/opt/homebrew/bin/brew shellenv)”’ >> /Users/arjungautam/.bash_profile

eval “$(/opt/homebrew/bin/brew shellenv)”

Step 2 : Install Hadoop using HomeBrew

brew install hadoop

Open the document containing the environment variable settings :

$ cd /opt/homebrew/Cellar/hadoop/3.3.1/libexec/etc/hadoop

then open hadoop-env.sh with any editor, I will use vscode

$code hadoop-env.sh

Make sure to have java jdk installed and JAVA_HOME setup on your system. If you haven’t follow this link ( Install Java 8 or 11 not newer versions)

Find JAVA_HOME on that file and add your local Java path

Use this command to find the path in terminal

$ /usr/libexec/java_home

and add the location on hadoop-env.sh

Step 3: Make changes to Core files

cd /opt/homebrew/Cellar/hadoop/3.3.1/libexec/etc/hadoop
code core-site.xml

and add the following configuration

<configuration> <property>  <name>fs.defaultFS</name>  <value>hdfs://localhost:9000</value> </property></configuration>

Step 4 : Make changes to hdfs files :

cd /opt/homebrew/Cellar/hadoop/3.3.1/libexec/etc/hadoop
code hdfs-site.xml

<configuration>  <property>    <name>dfs.replication</name>    <value>1</value>  </property></configuration>

Step 5 : Make changes to mapred files

cd /opt/homebrew/Cellar/hadoop/3.3.1/libexec/etc/hadoop
code mapred-site.xml

<configuration>    <property>       <name>mapreduce.framework.name</name>       <value>yarn</value>    </property>    <property>    <name>mapreduce.application.classpath</name>  
 <value>
$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*
 </value>    </property></configuration>

Step 6: Make changes to yarn files :

cd /opt/homebrew/Cellar/hadoop/3.3.1/libexec/etc/hadoop
code yarn-site.xml

‍