What are plug-ins and what are they used for?

What Elasticsearch river plugins have you used, and what has been your experience so far?

• Answer:

  I have used the Twitter, RSS, JDBC, MongoDB, CSV and Kafka rivers. And have written a few custom ones for internal use at my job. Some of them don't scale well. Bombard them with a lot of events and things slow down to a crawl at times. Setting some of them up can be a pain. Mainly due to version incompatibilities, etc. given that Elasticsearch versions are released at a firing pace, and you can't ignore the newer versions as they tend to be so much more stable. And then, you end up with a river plugin that takes 6 months to get updated to the latest version. Writing custom rivers can be a pain sometimes, because there's a bunch of concurrency aspects that need to be understood. Although this is true about any Java program, but in case of rivers, it does help to understand the lifecycle when a river is created, when events occur for a river, and when a river is closed. Exception handling also requires quite a lot of attention, else resource leaks are common. P.S. If possible, I'd rather change my feeds into something similar to logs, and use Logstash to push them to Elasticsearch.