The error message:
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hr4757/index/term-doc-vectors-forward-index.dat could only be replicated to 0 nodes, instead of 1
How to check if there's something wrong:
@login1:~/Project$ hadoop dfsadmin -report
Configured Capacity: 0 (0 KB)
Present Capacity: 0 (0 KB)
DFS Remaining: 0 (0 KB)
DFS Used: 0 (0 KB)
DFS Used%: ?%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 0 (0 total, 0 dead)
* If everything is running correctly, the higlighted part would be Datanodes available: 1 (1 total, 0 dead) (more than 1 depending on how many processes you requested)
Download the attached file (fix_HDFS.sh) from the bottom of this page. Upload it to your hadoop/bin using SFTP. If you do not know how to upload the file, see this page.
Do this to solve the problem:
hr4757@login1:~$ qstat
job-ID prior name user state submit/start at queue slots ja-task-ID
-----------------------------------------------------------------------------------------------------------------
39661 1.07500 Hadoop-wil hr4757 r 12/08/2010 16:40:02 hadoop@c202-115.longhorn 128
hr4757@login1:~$ ssh c202-115
Warning: untrusted X11 forwarding setup failed: xauth key data not generated
Warning: No xauth data; using fake authentication data for X11 forwarding.
Last login: Wed Dec 8 19:35:50 2010 from login1.longhorn
Rocks Compute Node
Rocks 5.2 (Chimichanga)
Profile built 17:21 07-Dec-2009
Kickstarted 12:27 07-Dec-2009
hr4757@c202-115:~$ cd hadoop/bin/
hr4757@c202-115:~/hadoop/bin$ ls
fix_HDFS.sh hadoop-daemons.sh start-balancer.sh stop-all.sh stop-mapred.sh
hadoop HostList.sh start-cluster-local.sh stop-balancer.sh
hadoop-config.sh rcc start-cluster.sh stop-cluster-local.sh
hadoopD slaves.sh start-dfs.sh stop-cluster.sh
hadoop-daemon.sh start-all.sh start-mapred.sh stop-dfs.sh
hr4757@c202-115:~/hadoop/bin$ chmod +x fix_HDFS.sh
hr4757@c202-115:~/hadoop/bin$ ./fix_HDFS.sh
..........................................
The script will stop Hadoop, delete all the trash files, and restart Hadoop.
hr4757@c202-115:~/hadoop/bin$ hadoop dfsadmin -report
Configured Capacity: 22151593267200 (20.15 TB)
Present Capacity: 22141355106334 (20.14 TB)
DFS Remaining: 22141354737664 (20.14 TB)
DFS Used: 368670 (360.03 KB)
DFS Used%: 0%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
-------------------------------------------------
Datanodes available: 15 (15 total, 0 dead)
hr4757@c202-115:~/hadoop/bin$ exit