When do you need large heap for Elasticsearch?

What is the best client library for elasticsearch?

• For our elastic search implementation, currently we are evaluating following client libraries as it simplifies our development effort rather using elasticsearch APIs directly.   1. Jest - REST based client library - https://github.com/searchbox-io/Jest Currently maintained by single person that gives light weight REST based client wrapper. But I found the for builder API we will end up using most of the elastic search library & in most of the cases I don’t see the need of using non-native library. But 2. spring data library - https://github.com/SpringSource/spring-data-elasticsearch Follows the spring model of dealing with data.(repository interface and template pattern) but not officially listed in spring source. Since this deals with native elasticsearch API’s (transport client, Node client), is faster compared to work with json/rest APis route taken by the jest library.   I do see client libraries in scala/groovy/jruby & clojure, but those options are ruled out because we want a pure java library.   Being these wrapper libraries relatively new, if any experience with these and insights into long term implications of using this will be greatly appreciated.

• Answer:

  The best client library for ElasticSearch is ElasticSearch. No, really. The only reason NOT to use the native Java client is that it currently enforces you to use the same version as your ElasticSearch server (an issue that will hopefully be addressed in the upcoming 1.0 release according to some promises). So if you have several ES clusters running different versions, then using the native (or transport) client will be a problem, and you will need to go HTTP (and Jest is the main option I think). If versioning is not an issue, the native client will be your best option as it is cluster aware (thus knows how to route your queries and does not need another hop), and also moves some computation away from your ES cluster (like merging search results that will be done locally instead of on the data node). And using it directly and not through a wrapper means that you always have the latest features and changes, and don't need another layer to implement it.