How do I debug Hadoop code?
-
I want to debug the Map Reduce Java code. I tried Eclipse IDE and wanted to debug through TextInputFormat class, i couldn't find java source for it. I tried using Intellij idea to no help. Is there a way, i can download code, run it and debug the jobs locally?
-
Answer:
To be able to debug conveniently, I implemented my own DebugContext class by inheriting the mapper/reducer’s context class, and redirect all output to stderr. This way I can debug the map or reduce function locally and check their output.
Xin Rong at Quora Visit the source
Other answers
Akash Khandelwal
which version of hadoop you are using, if you are using hadoop 0.20.2, the TextInputFormat class is located in the source folder src/mapred, the package name is org.apache.hadoop.mapreduce.lib.input if you want to debug this version of this hadoop, cause this version of hadoop is using ant to manage the project, so first of all, you should download the source code of this version of hadoop, and then you should load the entire project into Eclipse work space, thirdly, run the build.xml ant script and resolve some build path problems, and you can start to view the hadoop codes
Changqing Yang
Related Q & A:
- How can I debug my php code?Best solution by Stack Overflow
- How can I convert Matlab code to c#?Best solution by Stack Overflow
- How can I learn hadoop through projects?Best solution by globalknowledge.com
- How do I get the code for my radio in my Renault Clio?Best solution by Yahoo! Answers
- How do I get the code for the Jumbo remote?Best solution by jumboremotecontrol.com
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.