Cygwin makes Hadoop MUCH harder to run.

If you’re having problems setting up your DFS file share under Cygwin, see what happened to me. Notice that you have to use the WINDOWS path (C:\\cygwin\\home\\Rajat\\…..) With escape characters.

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -copyFromLocal /home/Rajat/java/mrh/input inputs
copyFromLocal: File /home/Rajat/java/mrh/input does not exist.

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -copyFromLocal C:\cygwin\home\Rajat\java\mrh\input input
copyFromLocal: File C:cygwinhomeRajatjavamrhinput does not exist.

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -copyFromLocal C:\\cygwin\\home\\Rajat\\java\\mrh\\input input
(((THIS WORKED)))

Test:

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -ls
Found 1 items
drwxr-xr-x – Rajat supergroup 0 2009-12-16 23:30 /user/Rajat/input

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -ls input
Found 1 items
-rw-r–r– 1 Rajat supergroup 683956 2009-12-16 23:30 /user/Rajat/input/162523852.csv

Check out this mashup. It will make you cry of laughter.

Normal link -> Here

Finals are over. Two projects still due, but Exams are over!!

In a week, Melissa and I will be en route to India with Tania, Viraj, Anu, Sonia, and many others.

Whenever I go to India, I have a few strange feelings about it. I love being there, learning about my culture and heritage, and of course eating the world’s best cuisine. However, Indians decidedly make us feel like we don’t belong. Everyone we encounter, from the people in the hotels and on the street to my relatives and cousins, make a point of treating me like an outsider. And by outsiders I mean stupid Americans. Every conversation is forced to English and my Bengali is greeted with “VHATT?” and “KEE BOLTHAE CHOOUW?”. Lord know we’re hopelessly overcharged for everything we want to buy. Taxis take circuitous routes, and you’re always handed the most expensive items in every store.

The only one that really bothers me is being treated as someone with only a passing interest in India. I feel badly about it maybe because it is true. As much as I’m interested in India, the economy, relations with the Pakis, the high tech sector, my interest has never evolved further. I have always wanted it to, but to develop a stronger interest, I need to spend more time there. And in order to spend more time there, I must be away from the US for longer periods. That hasn’t been possible until this year.

On this trip, I am planning to immerse myself completely in India; all of our lively culture and the energy of the place. This “Indian-American” stuff isn’t a strong identity. I want to get my culture from the source.

Hi Buddies,
Sorry for the low quality posts as of late. Got a lot of work to do. I’m building a type of Financial Modeling system for a Hadoop cluster in Java. It is a “short squeeze” detector, written for a Hadoop cluster. Tough stuff!

Hope this video clip about grad students makes you laugh!

PS: I’m growing a ponytail.

http://mail-archives.apache.org/mod_mbox/hadoop-common-user/200907.mbox/%3Cd6d7c4410907201717t672e4caet3f369d7e4327ff7b@mail.gmail.com%3E

Seen this?

Rajat@rtde ~/java
$ javac -classpath /home/Rajat/hadoop-0.20.1/hadoop-0.20.1-core.jar -d
wordcount_classes WordCount.java

WordCount.java:5: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.Path;

WordCount.java:6: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.*;
^
WordCount.java:7: package org.apache.hadoop.io does not exist
import org.apache.hadoop.io.*;
^
WordCount.java:8: package org.apache.hadoop.mapred does not exist
import org.apache.hadoop.mapred.*;
^
WordCount.java:9: package org.apache.hadoop.util does not exist
import org.apache.hadoop.util.*;

And running CygWin on Windows? Tried everything for your classpath
huh? Try this:

$ javac -verbose -classpath C:\\cygwin\\home\\Rajat\\hadoop-0.20.1\\hadoop-0.20.1-core.jar -d wordcount_classes WordCount.java

(Change your cygwin path, obviously)

Use the -verbose flag to show the entire class search path.