How do you go about developing an application?

I want to make a web application that displays scores after extracting their information from feeds? How should I go about developing it?

• I am developing for a portal that already exists in Django. I have two options- either scrape the source every time a user opens it (using BeautifulSoup), or save it in the database every minute or so. I could also do some scraping through JavaScript. Please suggest what should be the proper way to do this.

• Answer:

  I think you'll have to save the scores in the database after scraping them from the feeds. You'll have to scrape the scores after some time and update them in the database if they change. If this is the only thing you want to achieve you can use http://www.sqlite.org/ as database and some app that lets you scrape the feeds periodically. A well know app that handles this is http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html.Celery is a bit overkill checkout the links below. This question on stackoverflow gives you some hints on how to do something periodically. http://stackoverflow.com/questions/573618/django-set-up-a-scheduled-job From that list i think https://github.com/tivix/django-cron looks promising but i haven't used this.