How to remotly debug python on my apache?

Why don't companies use Python (programming language) for Apache Hadoop programming?

  • I have gone through series of jobs where companies prefer Java with Hadoop and big-data, Hive anything related to big data. Does Python have less support for Hadoop ?

  • Answer:

    I'd argue that, independent of the vertical, for certain roles such as Data Scientists in fact Python is a pivotal language. Just have a look at http://blog.matthewrathbone.com/2013/01/05/a-quick-guide-to-hadoop-map-reduce-frameworks.html that lists four frameworks for Py. Also, there are more and more Hadoop-as-a-Service providers such as http://mortardata.com/ that offer nicely integrated solutions (in their case Pig+Py+Git) and I expect that this segment will grow in the near future.

Michael Hausenblas at Quora Visit the source

Was this solution helpful to you?

Other answers

Most of Hadoop written in Java.  Most extensions and native APIs are simply extended using Java which is why you see so much code for Hadoop in Java.  However, it is simply not true that Python is scarcely used with Hadoop.  When I was doing consulting work for Facebook, we used a couple of Python frameworks almost exclusively for our ETL jobs within the Hadoop clusters.  I wrote some rather complex custom reducers using python through Hive.  There's also plenty of map-reduce libraries available for Python.

Chris Schrader

I think it's  because companies who have their own Apache stack already have competent Java data scientists/engineers. If you're already good with Java, learning python might not be the most productive thing for the company. On top of that, once a majority of your work is in Java, having someone come in and write streaming python map-reduce just makes the whole less consistent and maintainable for future developers.

Jason Liu

Related Q & A:

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.