Find solution
Is that OK to leave a small endless loop script running on server?

How can I run a Pig script in Oozie if I've installed Oozie-4.0.0 in Hadoop-2.2.0?

  • I tried with examples in oozie only java samplemapper and sample reducer class runs correctly. if i am trying am trying to run a example pig script or hive or oozie or any java mapreduce program i am getting the below error: My problem is while executing any kind of mapreduce jobs like pig,hive,sqoop or java MR program in oozie with hadoop-2.2 i am getting ping error in yarn as i mentioned below. if i am executing pig,hive,sqoop or java MR program without oozie in hadoop-2.2 it works well. This is my properties file : nameNode=http://hdfs://localhost:8020 jobTracker=localhost:8032 queueName=default   oozie.libpath=${nameNode}/user/${http://user.name}/share/lib oozie.use.system.libpath=true http://oozie.wf.rerun.failnodes=true   oozieProjectRoot=${nameNode}/user/${http://user.name}/ooziepig appPath=${oozieProjectRoot}/workflow.xml http://oozie.wf.application.path=${appPath} This is my workflow.xml file : <workflow-app name="WorkflowWithPigAction" xmlns="uri:oozie:workflow:0.1"> <start to="pigAction"/> <action name="pigAction"> <pig> <job-tracker>${jobTracker}</job-tracker> <name-node>${nameNode}</name-node> <script>script.pig</script> </pig> <ok to="end"/> <error to="killJobAction"/> </action> <kill name="killJobAction"> <message>"Killed job due to error: ${wf:errorMessage(wf:lastErrorNode())}"</message> </kill> <end name="end" /> </workflow-app> This is my error in yarn: 2014-05-30 10:38:38,079 INFO [IPC Server handler 6 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1401425739447_0003_m_000000_0 I am getting this error continuously.....upto i kill the job. i tried with different versions of oozie in hadoop-2.2.0 i stucked in the same place with the same ping error. This is error in jobhistoryserver: Error: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class javamapreduce.wordcount$Reduce not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1752) at org.apache.hadoop.mapred.JobConf.getCombinerClass(JobConf.java:1139) at org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1517) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.init(MapTask.java:1010) at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:390) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:418) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class javamapreduce.wordcount$Reduce not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1744) ... 11 more Caused by: java.lang.ClassNotFoundException: Class javamapreduce.wordcount$Reduce not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718) ... 12 more This is my log in oozie : 2014-05-29 15:12:49,834  INFO KillXCommand:539 - USER[labuser] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000003-140529150014726-oozie-labu-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000003-140529150014726-oozie-labu-W 2014-05-29 15:12:51,518  INFO CallbackServlet:539 - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000003-140529150014726-oozie-labu-W] ACTION[0000003-140529150014726-oozie-labu-W@pig-node] callback for action [0000003-140529150014726-oozie-labu-W@pig-node] 2014-05-29 15:12:51,551 ERROR CompletedActionXCommand:536 - USER[-] GROUP[-] TOKEN[] APP[-] JOB[0000003-140529150014726-oozie-labu-W] ACTION[0000003-140529150014726-oozie-labu-W@pig-node] XException, org.apache.oozie.command.CommandException: E0800: Action it is not running its in [KILLED] state, action [0000003-140529150014726-oozie-labu-W@pig-node] at http://org.apache.oozie.command.wf.CompletedActionXCommand.eagerVerifyPrecondition(CompletedActionXCommand.java:85) at org.apache.oozie.command.XCommand.call(XCommand.java:251) at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744)

  • Answer:

    AS Abhishek said ,try to give more error details. 1. you can retrieve them  (logs) from Oozie user interface or jobtracker UI. You also share/check your xml file and properties file .first you try to run examples. If you have Hue ,you try to run  oozie from hue .It will simplify the process.

Balaswamy Vaddeman at Quora Visit the source

Was this solution helpful to you?

Other answers

See into the oozie logs. Also in hadoop logs as oozie logs map reducer related detailed logs in there respective log directories instead of oozie logs. Also if it is only getting problem in ping then you should also check the dns name with respect to the one you set in oozie and hadoop configuration files. Can you provide more details of the exception..?

Abhishek Soni

Have you put the jar containing the Reduce class onto hdfs? Do the paths resolve correctly?

Arunraj Nair

NodeManager and Resource Manager running in same system. So i am gettting this error. I started Node manager in second system and my problem is solved.

Suresh Ram

The YARN piece in your log is just an INFO, you can ignore that. The piece of interest is in your job history log: Error: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class javamapreduce.wordcount$Reduce not found at org.apache.hadoop.conf.Configuration.getClass   I believe there is some jar files which are not registered causing the issue. Check and make sure all the environment variables are set correctly for Oozie (since the problem seems specific to Oozie). Also, you may have to search more in the world wide web.

Debarchan Sarkar

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.