Dec 17

Cygwin Vs. Hadoop – DFS Copy

Category: general,popular

Cygwin makes Hadoop MUCH harder to run.

If you’re having problems setting up your DFS file share under Cygwin, see what happened to me. Notice that you have to use the WINDOWS path (C:\\cygwin\\home\\Rajat\\…..) With escape characters.

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -copyFromLocal /home/Rajat/java/mrh/input inputs
copyFromLocal: File /home/Rajat/java/mrh/input does not exist.

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -copyFromLocal C:\cygwin\home\Rajat\java\mrh\input input
copyFromLocal: File C:cygwinhomeRajatjavamrhinput does not exist.

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -copyFromLocal C:\\cygwin\\home\\Rajat\\java\\mrh\\input input
(((THIS WORKED)))

Test:


Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -ls
Found 1 items
drwxr-xr-x – Rajat supergroup 0 2009-12-16 23:30 /user/Rajat/input

Rajat@rtde ~/hadoop-0.20.1
$ bin/hadoop dfs -ls input
Found 1 items
-rw-r–r– 1 Rajat supergroup 683956 2009-12-16 23:30 /user/Rajat/input/162523852.csv


1 comment

1 Comment so far

  1. L February 11th, 2010 4:54 am

    thanks, this was super useful, since I ran into the same issue!

    - L

Leave a comment