How would you chain Unix commands to run a few at a time?
-
Given say some number (say 100) 'whoami' commands, how would you chain commands together to run only a few at a time (say 10)? The goal being to not overwhelm with too many commands at the same time but being able to execute more than one. Consider the numbers variable e.g. use of 'seq' is not allowed. Solution has to be bash compatible. Something in a for-do-done loop....
-
Answer:
I once wrote a tool that does this. It was very simple: it took a list of commands to run and a process pool size, and then ran the commands with that maximum number of processes, starting the next only when one of the currently-running commands finished. For example, if my command list was... expensive-binary foo expensive-binary bar expensive-binary baz expensive-binary qux expensive-binary quux expensive-binary corge expensive-binary grault expensive-binary garply ...and you used a pool size of 3, the program would first start 3 instances of expensive-binary with arguments foo, bar, and baz respectively. When one of them finished, it would start expensive-binary with the argument qux. When another of those would finish, it would start expensive-binary quux. And so on. Then I realized I had essentially written a tiny, unstable subset of http://www.gnu.org/software/parallel/: parallel -a commands -j 3 Note that this is just the tip of the iceberg that is the power of GNU Parallel: it can distribute very complex jobs across multiple machines, with redundancy and dynamic tuning based on completion rate. It is essentially a primitive http://hadoop.apache.org/.
Costya Perepelitsa at Quora Visit the source
Other answers
I'm not familiar with Toby's comment but you could do something like #!/bin/bash for i in {1..10} do for j in {1..10} do echo `whoami` done sleep 1s done Which will print whoami ten times, then wait one second, print whoami ten more times etc. for a total of 100 prints.
Marko Poutiainen
for i in `seq 100`; do <your command> if [ $((i % 10)) -eq 0 ]; then sleep 5 fi done Every 10 iterations will run that sleep. You can do something analogous in other for/while/do constructions keeping track of the amount of iterations to do the same
Gustavo Muslera
GNU make has this built-in. -jN
Toby Thain
How about: COMMAND=$1 N_TIMES=$2 STEP=$3 i=0 j=0 k=1 for i in `seq $STEP $STEP $N_TIMES` do for j in `seq $k $i` do $COMMAND& done wait `jobs -p` k=$((i+1)) done The numbers are variable, like you asked for. wait ensures that the previous process ended before proceeding to the next.
Murukesh Mohanan
Related Q & A:
- How To Run A Command In Ubuntu?Best solution by Yahoo! Answers
- How To Connect Two Different Network Segments Using A Switch And A Router?Best solution by Super User
- How to run a external program in PHP?Best solution by Stack Overflow
- How to paste a line at a time?Best solution by Ask Ubuntu
- How much does it cost to trade in a DS for a DS Lite?Best solution by Yahoo! Answers
Just Added Q & A:
- How many active mobile subscribers are there in China?Best solution by Quora
- How to find the right vacation?Best solution by bookit.com
- How To Make Your Own Primer?Best solution by thekrazycouponlady.com
- How do you get the domain & range?Best solution by ChaCha
- How do you open pop up blockers?Best solution by Yahoo! Answers
For every problem there is a solution! Proved by Solucija.
-
Got an issue and looking for advice?
-
Ask Solucija to search every corner of the Web for help.
-
Get workable solutions and helpful tips in a moment.
Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.