Sunday, January 25, 2015

Issue in put files to hadoop - 0 datanodes running

I setup hadoop as I have described in my previous posts. While I was working in some interesting projects on Map-Reduce, I faced an issue on sending some of my local files to the hdfs node. I have started dfs already, but exception says
14/10/25 17:46:20 WARN hdfs.DFSClient: DataStreamer Exception org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/ashansa/input._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation. at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2791) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:606) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:455) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
I faced this issue several times, so I think it will be helpful if I tell you how to overcome this issue. ( I believe this is a common issue we get while running hdfs, since I got the same issue several times and I found that many others also have faced the same issue )

What worked for me?


1. Stop the hadoop cluster with
             bin/stop-all.sh

2. Clean hadoop tmp directory
    You can find the path to your hadoop tmp directory in hdfs-site.xml.
    Check for the following tag in hdfs-site.xml file.
             <name>hadoop.tmp.dir</name>
             <value>root/data/hdfstmp</value>
    You can find the path to your hadoop temp directory in <value> tag.

3. Format node
             bin/hadoop namenode -format

4. Start hadoop cluster
             bin/start-all.sh

With these few simple steps, I was able to overcome this issue.
Hope this will help you too.

100 comments:

  1. There are lots of information about latest technology and how to get trained in them, like Hadoop Training Chennai have spread around the web, but this is a unique one according to me. The strategy you have updated here will make me to get trained in future technologies. By the way you are running a great blog. Thanks for sharing this.

    ReplyDelete
  2. A table is the basic unit of data storage in an oracle database. The table of a database hold all of the user accesible data. Table data is stored in rows and columns. But what is all about the clusters and how to handle it using oracle database system? Expecting a right answer from you. By the way you are maintaining a great blog. Thanks for sharing this in here.
    Oracle Training in Chennai | Oracle Course in Chennai | Oracle Training Center in Chennai

    ReplyDelete
  3. There are lots of information about latest technology, like Hadoop cluster is a special type of computational cluster designed specifically for storing and analyzing huge amounts of unstructured data in a distributed computing environment. This information seems to be more unique and interesting. Thanks for sharing.
    Big Data Training in Chennai | Hadoop Course in Chennai | Big Data Training in Chennai

    ReplyDelete
  4. Your blog is really awesome and I get a lot of great information from this blog. Thanks for sharing such an informative post on blog. Keep posting. It’s really useful and interesting...
    Healthcare Business Analysis Online Training
    Tableau Online Training
    Qlikview Online Training
    ServiceNow Administrator Online Training
    Peoplesoft HRMS Functional Online Training

    ReplyDelete

  5. Thank you for taking the time to provide us with your valuable information. We strive to provide our candidates with excellent care and we take your comments to heart.As always, we appreciate your confidence.
    HADOOP Training in Chennai

    ReplyDelete
  6. Excellant post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    SAS Training in Chennai | SAS Course in Chennai

    ReplyDelete
  7. Great article. It helps to explain complicated things to people who are not competent and explicate why this is important.
    data room services

    ReplyDelete
  8. The strategy you posted was nice. The people who want to shift their career to the IT sector then it is the right option to go with the ethical hacking course.
    Ethical hacking course in Chennai | Ethical hacking training in chennai

    ReplyDelete
  9. we are offering best servicenow training in hyderabad with job assistance and high quality training facilities and well expert faculty .
    to Register you free demo please visit servicenow training in hyderabad

    ReplyDelete
  10. Hi, I am really happy to found such a helpful and fascinating post that is written in well manner. Thanks for sharing such an informative post.R Programming Online Training | Hadoop Online Training

    ReplyDelete

  11. Thanks for sharing this information and keep updating us. This information is very useful to me.
    Ethical Hacking Course in Chennai | Hacking Course in Chennai | Ethical Hacking Course

    ReplyDelete
  12. hai admin.thanks for sharing the post.Any individual who finishes our
    Hadoop Training in Chennai will turn into an ace in Hadoop with hands-on exercises and activities.More course details........ Selenium Training in Chennai
    Dot Net Training in Chennai
    Android Training in Chennai

    ReplyDelete
  13. Thank you so much for sharing this worth able content with us. The concept taken here will be useful for my future programs and i will surely implement them in my study. Keep blogging article like this.

    Hadoop Online Training
    Data Science Online Training

    ReplyDelete
  14. Well Said, you have provided the right info that will be beneficial to somebody at all time. Thanks for sharing your valuable Ideas to our vision.Big Data Hadoop Training in Bangalore | Data Science Training in Bangalore

    ReplyDelete
  15. Updating with the recent skills and applying it is the only tactic to live in our vocation. You have done really a great job by sharing this blog in here. Keep writing blog like this.



    Hadoop Training in BTM Layout

    Hadoop Training in Marathahalli

    ReplyDelete
  16. Informative post about hadoop, i am looking forward for realtime hadoop online training institute.

    ReplyDelete
  17. I simply wanted to write down a quick word to say thanks to you for those wonderful tips and hints you are showing on this site.
    Best Hadoop Training Institute In chennai

    ReplyDelete
  18. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    Selenium Training in Chennai

    ReplyDelete
  19. Best institute for 3d Animation and Multimedia Course training Classes


    Best institute for 3d Animation and Multimedia

    Best institute for 3d Animation Course training Classes in Noida- webtrackker Is providing the 3d Animation and Multimedia training in noida with 100% placement supports. for more call - 8802820025.

    3D Animation Training in Noida

    Company Address:

    Webtrackker Technology

    C- 67, Sector- 63, Noida

    Phone: 01204330760, 8802820025

    Email: info@webtrackker.com

    Website: http://webtrackker.com/Best-institute-3dAnimation-Multimedia-Course-training-Classes-in-Noida.php

    ReplyDelete
  20. Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information. 

    Data Science Training in Chennai
    Data science training in bangalore
    Data science online training
    Data science training in pune
    Data science training in kalyan nagar

    ReplyDelete
  21. Wow it is really wonderful and awesome thus it is very much useful for me to understand many concepts and helped me a lot. it is really explainable very well and i got more information from your blog.
    Devops training in Chennai
    Devops training in Bangalore
    Devops Online training
    Devops training in Pune

    ReplyDelete

  22. Hello! This is my first visit to your blog! We are a team of volunteers and starting a new initiative in a community in the same niche. Your blog provided us useful information to work on. You have done an outstanding job.

    AWS Online Training | Online AWS Certification Course - Gangboard
    AWS Training in Chennai | AWS Training Institute in Chennai Velachery, Tambaram, OMR
    AWS Training in Bangalore |Best AWS Training Institute in BTM ,Marathahalli

    ReplyDelete
  23. Read all the information that i've given in above article. It'll give u the whole idea about it.
    python online training
    python training in OMR
    python training course in chennai

    ReplyDelete
  24. This is very good content you share on this blog. it's very informative and provide me future related information.
    Online DevOps Certification Course - Gangboard
    Best Devops Training institute in Chennai

    ReplyDelete
  25. A very nice guide. I will definitely follow these tips. Thank you for sharing such detailed article. I am learning a lot from you.

    rpa training in electronic-city | rpa training in btm | rpa training in marathahalli | rpa training in pune

    ReplyDelete
  26. Brilliant article shared with lots of knowledge for Hadoop Training in Bangalore

    ReplyDelete
  27. Sap fico training institute in Noida

    Sap fico training institute in Noida - Webtrackker Technology is IT Company which is providing the web designing, development, mobile application, and sap installation, digital marketing service in Noida, India and out of India. Webtrackker is also providing the sap fico training in Noida with working trainers.


    WEBTRACKKER TECHNOLOGY (P) LTD.
    C - 67, sector- 63, Noida, India.
    F -1 Sector 3 (Near Sector 16 metro station) Noida, India.

    +91 - 8802820025
    0120-433-0760
    0120-4204716
    EMAIL: info@webtrackker.com
    Website: www.webtrackker.com

    ReplyDelete
  28. Your info is really amazing with impressive content..Excellent blog with informative concept. Really I feel happy to see this useful blog, Thanks for sharing such a nice blog..
    If you are looking for any Big data Hadoop Related information please visit our website Big Data Training In Bangalore page!

    ReplyDelete
  29. It’s great to come across a blog every once in a while that isn’t the same out of date rehashed material. Fantastic read.
    cloud computing training in chennai | Cloud computing training class in chennai

    ReplyDelete
  30. Clinical sas training in chennai | SAS Training course chennai
    I have to voice my passion for your kindness giving support to those people that should have guidance on this important matter.

    ReplyDelete
  31. I am glad that I saw this post. It is informative blog for us and we need this type of blog thanks for share this blog, Keep posting such instructional blogs and I am looking forward for your future posts.
    Cyber Security Projects for Final Year

    JavaScript Training in Chennai

    Project Centers in Chennai

    JavaScript Training in Chennai

    ReplyDelete
  32. Nice and useful information.Thanks for the effort and the results,impressed with your post.
    big data training in btm layout

    ReplyDelete
  33. Nice blog,I understood the topic very clearly,And want to study more like this.
    Data Scientist Course

    ReplyDelete
  34. I just got to this amazing site not long ago. I was actually captured with the piece of resources you have got here. Big thumbs up for making such wonderful blog page!

    data science course

    ReplyDelete
  35. Informative post about hadoop. the Post is really amazing. blog is informative and the contents are very clear.

    ReplyDelete
  36. It's very useful blog post with inforamtive and insightful content and i had good experience with this information.I have gone through CRS Info Solutions Home which really nice. Learn more details About Us of CRS info solutions. Here you can see the Courses CRS Info Solutions full list.Find the best Hadoop Training with great faculty. Go to know about crs info solutions Workday Training program.

    ReplyDelete
  37. I have Grasp a lots of information's here...Thanks for posting Such a Useful comments...Informative post about hadoop are really Good...Keep updating
    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  38. You have provided a nice article, Thank you very much for this one. And I hope this will be useful for many people. Salesforce Training India 

    ReplyDelete
  39. I am so happy to found your blog post because it's really very informative. Please keep writing this kind of blogs and I regularly visit this blog. Have a look at my services.  
    This is really the best Top 20 Salesforce CRM Admin Development Interview Questions highly helpful. I have found these Scenario based Salesforce developers interview questions and answers very helpful to attempt job interviews. Wow, i got this scenario based Salesforce interview questions highly helpful.

    ReplyDelete
  40. Attend The Data Science Courses From ExcelR. Practical Data Science Courses Sessions With Assured Placement Support From Experienced Faculty. ExcelR Offers The Data Science Courses.
    Data Science Courses

    ReplyDelete
  41. Myself so glad to establish your blog entry since it's actually quite instructive. If it's not too much trouble continue composing this sort of web journals and I normally visit this blog. Examine my administrations.  
    Read these Salesforce Admin Certification Topics which are really helpful. I read these Salesforce Admin and Developer Certification Dumps and very much useful for me. I recommend this Salesforce Developer Training and Certification Course for you.  

    ReplyDelete
  42. Great post! I am actually getting ready to across this information, It’s very helpful for this blog. Also great with all of the valuable information you have Keep up the good work you are doing well.
    CRS Info Solutions Salesforce training for beginners   

    ReplyDelete
  43. I like it and help me to development very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.
    AWS training in chennai | AWS training in annanagar | AWS training in omr | AWS training in porur | AWS training in tambaram | AWS training in velachery

    ReplyDelete
  44. Myself so glad to establish your blog entry since it's actually quite instructive. If it's not too much trouble continue composing this sort of web journals and I normally visit this blog. Examine my administrations.
    Go through these Salesforce Lightning Features course. Found this Salesforce CRM Using Apex And Visualforce Training worth joining. Enroll for SalesForce CRM Integration Training Program and practice well. 

    ReplyDelete
  45. I am so happy to found your blog post because it's really very informative. Please keep writing this kind of blogs and I regularly visit this blog. Have a look at my services.
    I have found this Salesforce training in India worth joining course. Try this Salesforce training in Hyderabad with job assistance. Join Salesforce training institutes in ameerpet with certification. Enroll for Salesforce online training in hyderabad with hands on course.

    ReplyDelete
  46. Going to graduate school was a positive decision for me. I enjoyed the coursework, the presentations, the fellow students, and the professors. And since my company reimbursed 100% of the tuition, the only cost that I had to pay on my own was for books and supplies. Otherwise, I received a free master’s degree. All that I had to invest was my time. getbestmowers

    ReplyDelete
  47. Great post! I am actually getting ready to across this information, It’s very helpful for this blog. Also great with all of the valuable information you have Keep up the good work you are doing well.
    CRS Info Solutions Salesforce training for beginners

    ReplyDelete
  48. Thanks for Sharing This Article.It is very so much valuable content. I hope these Commenting lists will help to my website
    workday studio online training
    best workday studio online training
    top workday studio online training

    ReplyDelete
  49. Keep up the great work, I read few blog posts on this site and I believe that your website is really interesting and has loads of good info.Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

    Data Science Training In Chennai

    Data Science Online Training In Chennai

    Data Science Training In Bangalore

    Data Science Training In Hyderabad

    Data Science Training In Coimbatore

    Data Science Training

    Data Science Online Training

    ReplyDelete
  50. I am looking for and I love to post a comment that "The content of your post is awesome" Great work!

    Simple Linear Regression

    Correlation vs Covariance

    ReplyDelete
  51. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
    Correlation vs Covariance
    Simple linear regression
    data science interview questions

    ReplyDelete
  52. Very nice blogs!!! i have to learning for lot of information for this sites…Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing, data sciecne course in hyderabad

    ReplyDelete
  53. It help to make good advancements in your business that helps you to feel the ultimate confidence knowing that your employees would give you the best performance. It is beneficial especially for new teams who want to do the job well. This gives them the inspiration to carry out the activities efficiently. Salesforce training in Hyderabad

    ReplyDelete