What is a database engine?

What Python web framework, template engine and database should I choose for a real-time social network?

  • I just started learning Python and I want to start BIG ( even if a make mistakes in the process :) ) with a project in order to improve my skills. It will be a real-time social network with lots of JOINS (similarity between objects) in the database. Let's say that the project will have millions of users visiting the website everyday just like quora. 1) What is the best framework to achieve this? (rapid development, secure, scalable, fast) 2) What is the best template engine? (very fast) (ex. pytenjin, mako, cheetah or python based template language?) 3) What will be the best database for this? (agile, scalable, availability) (ex. SQL or NOSQL) Other tools that you know can help it's also welcome. Many thanks for your help with this.

  • Answer:

    If you are looking for a realtime web framework to handle large volume async requests, I would suggest twisted or tornado. Personally, I find tornado a lot more accessible and intuitive. The syntax is very similar to webpy, a personal favourite framework of mine. The database you choose should take into consideration the access constraints of the social network. If data is going to be read frequently but not written frequently, you may wish to explore cacheing options. If you need highly relational data (to preserve and query over relationships), you may consider a sql solution. If you need a highly scalable key-val store, you may wish to mess with Amazon's Dynamo which offers built in sharding and extremely low latency.

Mek Karpeles at Quora Visit the source

Was this solution helpful to you?

Other answers

From personal experience, I can attest to Tornado's simplicity and power. The key phrase here is "real-time social network" – to achieve this, you will undoubtedly want a framework that can handle many simultaneous connections. Consider this situation: You are trying to build the site using an Apache server running PHP. You need to have real-time capability, so you are faced with two  options: 1. attempt to poll the DB every x seconds to get the latest updates using AJAX, 2. Implement Comet or something of the sort which is PHP's inferior method of long-polling. In the 1st option, your Apache server will run out of worker threads and proceed to crash if you have even a few hundred connections (maximum.) In the 2nd option, you will have improved performance, but it is still equivalent to sticking an octagonal peg in a round hole (you can do it with a hammer, and it will work...but it is far from ideal.) Now, ideally, as you have already stated, you should be working in a language like Python that is more geared towards this type of situation. (Good, you are.) Then, you will want a server that can handle your requirement (this is where I will shamelessly praise Tornado.) Tornado is a non-blocking HTTP request server. It can support upwards of 10,000 simultaneous connections at once. It is also the same technology that Facebook uses/used for most (if not all) of its real-time capabilities. It is highly scalable and perfect for real-time activity. It also has great documentation and provides new users with demo applications upon its free download. Overall, it's very simple. It also has its own template engine, which I also think is very good. I've heard others say they really enjoyed Mako; however, I cannot say which is "better" because I have not used both. Tornado also includes a lightweight wrapper for MySQL. To implement this, you will have to download MySQLdb. For almost all of your DB needs, on a reasonable level, this will more than suffice. On a side note: For real-time messaging (for example,) it is probably in your best interest to learn how to implement some Python multiprocessing. The multiprocessing module has very good documentation (as does Python in general,) and should you use it, along with Tornado's non-blocking capabilities, any real-time application you decide to implement will be very fast and highly scalable. Please note, if you are not familiar with non-blocking capabilities, you should read about them to make sure you understand their implications (pros and cons.) I hope this helps anyone who stumbled across this question!

John Zimmerman

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.