What's the output from this program using fork() and why?

Why does a program's output file go from 0 bytes to 32 MB without any intermediate size (writes all at once?)?

• I wrote a program to split up a database into 30 pieces and run a program called RNAfold to analyze the data from each piece separately. For each piece there is one output file. Then I stitch up the output files in one file again. The problem is I can't stitch them up until ALL the 30 pieces are done running. So I sat and watched the output file sizes out of curiosity to see how long it took and noticed they didn't gradually increase in size, but instead went directly from 0 to 32MB. Why is this? I noticed that a python script I wrote to write to a file did the same thing. 0 bytes when it was writing, and X bytes after I killed the filewriting. Any clue why? I figure any comp sci person should know? Thanks

• Answer:

  Because the program that is writing to the file has it locked. Windows, or your OS, cannot do anything to the file until the program editing it is finished. And secondly, basically what happens is that most programming languages will 'write' to RAM first, and then dump them all into the file when you close it. The actual writing stage happens really fast, compared to the stage while you are calculating the data. I don't nkow which language you are using to do this, but in Java you can have multithreaded processes that wait until other threads have finished to resume working. So if your language has something similar, perhaps you can structure the program to do the stitching together after each analysis thread has reported that it is finished, or something like that. Alternately, you can write a small program that will stitch them together and just run it separately when you see that all 30 files are done.