How many rake tasks can be run at once in rails?

How should we scale/optimize a slow ruby on rails application?

• Hi everyone, My name is Michael Solovyov and I'm the co-founder & CTO of a rails-based start-up.  We have built a product for other businesses to help their users adopt their products. We've built the application, delivered to a few customers and now we've hit scaling and performance issues. I'd love your feedback on how we proceed as we try to resolve them: 1) These are the problems we've encountered: The main user dashboard takes between 7-15 seconds to load. The maximum requests per second that we can support are 100-150. 2) Why we think they're happening: We're using couchdb with an ORM that has our data modeled relationally. This causes the ORM to launch an extraordinary amount of http requests to couchdb to load all our data. The more data and users in the project, the longer the delay. There's also potentially inefficient code, extra loops etc. We're not using caching to it's fullest extent. 3) What we're using/what we've done: Our architecture: rails '3.2.12', ruby 1.9.3p374, nginx + unicorn (main server), elasticsearch (separate server), couchdb (separate server). All on AWS. We've added some view level caching where we can get away with stale data. We've tried using higher tiered/ssd aws servers but it doesn't look to be our main bottleneck. 4) How we plan on completing it: We're mostly done a rewrite to migrate our data and data model to activerecord. We've hit a snag in migrating s3 attachment code built into the couchdb ORM that we're using.  Following the completion of the migration to sql/activerecord we are going to do the following in an attempt to increase performance: A. Use percono mysql server. B. Run tools such as bullet to analyze performance and where we might have bad queries/lookups. C. Update to latest Rails and Ruby versions. D. Look into Puma and paralelization of DB access  calls. E. Break apart the application into separate services. 5) What we don't know right now: Whether switching to Activerecord or any of the above ideas will inherently give us any performance benefits. Whether we will have any other unexpected surprises or loss of functionality due to switching to ActiveRecord. How long this all will take and whether there's other areas that we need to focus to get sub 1 second performance that we're missing.   Do you all have any feedback/ideas/insights completing projects like these? In particular, I would be curious about: a) How would you approach it? b) Anything that's worked for you in the past on suitable projects? c) Anything to avoid doing? d) Where to find good additional team members & advisors to help us complete the project? e) How long to expect it to take? Here are our application rake stats:http://pastie.org/private/qrxkytk4uur9odndydv5dq

• Answer:

  The answer from Vladimir Legkunets is basically what I would tell you to do in order to begin diagnosing the problems in your application.  Since you talk about what you think is happening rather than what you know to be the case, I'm guessing that you haven't gotten as far as installing New Relic, and that should be your first step.  I'll see if I can add something to what's already been said. First of all, while upgrading to the latest versions of Ruby and Rails is almost never a bad idea, that alone is probably not going to solve your performance problems.  Plenty of apps ran on that stack and were able to support more than 150 req/sec. This might sound overly basic, but make sure you're running your production app in production mode.  Under Rails 3.2, this should give you reduced logging, asset packaging, and other performance improvements. In my experience, a majority of slow applications are caused at least in part by poor performance in the browser.  An app that has to load dozens of CSS and JS assets, potentially from many sources, might give the impression of loading very slowly when in fact it's just getting hung up on one request.  Use the developer tools in Chrome (or your browser's equivalent) to see whether this is the case. Even before you install New Relic (and you should) seeing a browser timeline will give you an indication of where your problem lies.  If you see that there's a long wait on the server for the page to load, then there's a chance that you're on to something concerning DB access, though there could be other reasons for it.  Such a problem could be caused by the ORM, or it might be the way you're using it.  It could also be the way you're using CouchDB - e.g. poor design, improper or lacking use of views, etc.  Only someone familiar with your application and your data will be able to say for sure. As far as getting it fixed, you asked a bunch of questions, but the only one I can really address is the first one (How would you approach it?) since all others need some firm diagnosis first. Browse the application in Chrome with developer tools open.  Use that to find out what's driving the load time. If it is your Rails app, install New Relic and push to production.  You can get something out of their Lite version, but if you think it's a DB problem, you're probably going to have to spring for a premium membership. New Relic will show you where the application is spending the most time. You want to look at long-running actions and view the graphs from those to see what's eating up CPU time.  Also look at memory consumption.  If you're loading as much data as you think you are for each request, you should see some indication of that somewhere, and if you're swapping out a lot, that can slow things down dramatically. Good luck!