Find solution
How to Implement Gateway Service something similar to Oracle API gateway Using Java and Java based Open Source frameworks only?

Distributed Open source job scheduling software?

  • I am looking for a open source job scheduling tool having most of the following characteristics 1) Schedule & run jobs based on time / events / Dependency 2) Retry & restart capability on failure 3) Distributed - (Client/ Server mode) - Where in the jobs can be scheduled and monitored from 1 server and jobs executes on different servers            a) Pass parameters to remote jobs            b) Collect and store log files generated by remote jobs for debugging and analysis purposes 4) Store previous job execution details for analysis 5) Built-in webserver for monitoring & scheduling 6) Provide API's/ Webservices for automation 7) Diverse notifications/ Alerts (email, SMS, Twitter, IM etc) 8) Extend-able, support plugin's 9) Recoverable from server failures 10) Standalone I have found a few tools 1) Quatz - Popular java based ( but not standalone) 2) Acelet Super Scheduler/Super WatchDog 3) Task Forest 4) Job Scheduler There are some recent tools which are specific to Hadoop 5) Azkaban 6) Oozie 7) Cascading 8) Zookeeper It should be a tool which i can install on my server and customized it based on the project requirements. Otherwise i would also be happy if there a web based job scheduling service provider (paid service ofcourse)?

  • Answer:

    It's hard to find an unanswered question in you topic. Actually you made a good job. Now you just need to find out which one of the products listed fits your needs. As for my opinion Job Scheduler could be the winner. Nevertheless you'd better try them yourself.

Mark Napartovich at Quora Visit the source

Was this solution helpful to you?

Other answers

I designed my own. I also added windowing for delivery statistics and a Deadman's switch to stop dependent  jobs in my pipeline from failing. It was distributed and agent based. Web Service agents, Job Running Agents, Job Checking Resources and Resource Agents. It handled Rest/Soap encryption of data, compression, Inbound/Outbound feeds and the ability to delivery files as binary base 64 streams. However, I have heard Camel is capable as well for the delivery end?

Jann van Hamersveld

I wrote https://github.com/earonesty/grun.   Working on a 0mq version, but the current one is fine.  It's pretty simple to set up.

Erik Aronesty

Hi, whilst it may perhaps be more than you are looking for, you might like to take look at the Automation Center from Stonebranch-Opswise http://opswise.com/

Timothy Patrick Collins

The HPC community has been doing this for a while - check out slurm.

Greg Lindahl

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.