Eclipse Apache Hadoop Plugin Build 1.0.2

So, there is no good tutorial on how to build the current apache hadoop eclipse plugin, so here it is, it all its minimal glory:

Get Ant:

sudo emerge ant

Get eclipse, and put it somewhere. The location we shall now refer to as $ECLIPSE_HOME (for me ~/Desktop/eclipse/) , you got the idea

Download the apache hadoop src, same version as me, 1.0.2, if you are lazy, here is a link, verify at will:

http://apache.osuosl.org/hadoop/common/hadoop-1.0.2/hadoop-1.0.2.tar.gz

extract hadoop into a directory, we shall now refer to as $HADOOP_HOME. (i.e. ~/Desktop/eclipse/libs/hadoop-1.0.2/ for me)

cd $HADOOP_HOME

Then edit: vim src/contrib/eclipse-plugin/build.properties

add this line:

eclipse.home = $ECLIPSE_HOME

(if you did not export these please replace eclipse home with your real directory, don’t be dumb)

Now, yes you will get build errors, chill.

First:
Get sun’s jdk SE 5 (To build forrest):
http://www.oracle.com/technetwork/java/archive-139210.html
(annoying reg I know)

cd ~/Downloads/
chmod +x jdk-1_5.......(hit tab)
sudo ./jdk-1_5.......(hit tab)
(agree to license, spacebar to skip pages)
mv jdk1.5.0_....(hit tab) /opt/sun-jdk-1.5

Download Apache forrest 0.8 (0.9 will crash… lol)
http://archive.apache.org/dist/forrest/0.8/

mkdir ~/apache
cd ~/apache
tar -xvzf ~/Downloads/apache-forrest-0.8...(hit tab)
mv apache-forrest...(hit tab) forrest-0.8

Now lets build this baby, go to your hadoop directory:

ant clean package -Djava5.home=/opt/sun-jdk-1.5 -Dforrest.home=$HOME/apache/forrest-0.8

replace home with /home/your_user_name
should build, if not, let me know
now go into the built plugin directory:

cd build/contrib/eclipse-plugin/
cp cp hadoop-eclipse-plugin-1.0.3-SNAPSHOT.jar $ECLIPSE_HOME/plugins/

Close & restart eclipse, and BOOM!
Chose open perspective -> other -> Map/Reduce
File->New->Project->MapReduce Project
Select Configure Hadoop Install Directory
choose $HADOOP_HOME
make a project name, and boom
see my other tutorial on getting started with hadoop

Common Errors and Solutions:

1. Make sure you have ant installed, it is a “make” like build system for java:-)

 

sudo emerge -av ant

 

2. If you get an error like “error reading log4j/log4j/jars/log4j-1.2.15.jar; error in opening zip” most likely it is one of two things:

A. Make sure you have log4j installed:

sudo emerge -av log4j

B. If you have it installed on your system and you are still getting that build error try editing the file below:
./.classpath in the hadoop home directory, and change the location of log4j-1.2....... , it should also be in your forrest-0.8/lib/core/ folder

3. If you get stuck, don’t hesitate to ask on the comments!

~ by tripl3fault on April 22, 2012.