I have a reducer implemented in perl which takes massively longer to run on the cluster than it does locally. Why might this be?

Sometimes this is a symptom of faulty infrastructure: cpu, disk, etc.  Do you have other reduce tasks that finished quickly on this same machine? In general, this problem is exactly the type that speculative execution (enabled by default, check mapred.{map,reduce}.tasks.speculative.execution is supposed to solve.  If there exists a task that is taking much longer than the rest, then Hadoop should automatically spawn another instance of the task, and occasionally this second instance will finish before the first.  Do you have speculative execution turned off? What do the logs for this particular task show?  In particular, there are R/W loglines which show read/write progress through the reducer.  Do these logs indicate: (a) a stuck reducer, or (b) slow-but-steady progress?

There may be multiple reasons for stragglers in a MapReduce cluster. The following paper in OSDI'10 covers some as seen in production clusters. http://research.microsoft.com/en-us/um/people/srikanth/data/mantri_osdi10.pdf Would be helpful to take a cue to see if any one fits yours.

I realized much later that the problem was due to memory issues on the cluster. I didn't realize that the memory allocated to map and reduce tasks applies to the parent Java process that spawns the streaming script, and that my perl script had to make do with what was left over. I had scaled up the task memory allocation thinking that my perl script would share the budget, so perl was furiously writing to swap because it had only a few megabytes of memory to work with. No error was generated, so I didn't notice until larger runs started causing perl to return out of memory errors.