What are some good "beginner level" data modeling/analytics approaches to kick start a data science/analytics team?

Charting Seas of Data

• What examples are out there on the web of interactive charts with thousands of data points that can be scrolled through smoothly with little interruption or skipping? So I have this project which is almost done, with a handful of charts that collectively render thousands of data points (5K-10K+). One problem which has been handled in an OK way is that the vast amount of data slow down the app when a user wants to scroll to a certain point. The current work around is for the user to manipulate a slider first, and when they're done, the app reconfigures the charts to align with the slider state. Ideally we would have users manipulate the slider and the chart would change in real time with the changes. But this gets painfully slow due to the circumstances. My team has looked at http://www.weatherspark.com/ for inspiration, but it is incredibly difficult to figure out what they have done, even after a little bit of email and phone correspondence with the creators. Thus I ask if there are other examples of web applications out there that do something similar to Weatherspark: display thousands of data points and allow users to smoothly navigate through them. This is to see if I can get some new ideas of different approaches to the problem. Web app packages that achieve this would be a plus (if they exist). Additionally, detailed explanations as to why this continues to be a challenge, if it is still a challenge, for regular web development (i.e. not Weatherspark) would be awesome. Thank you! PS: I am aware that cutting down on the data to be displayed, or figuring out a way to compactly summarize the data, would solve this problem to some degree.

• Answer:

  Consider http://oicweave.org/? Fairly new and still beta, but I've had some very good luck. Open source. You'll need a server with Apache Tomcat and mySQL. Also try Google FusionTables for speed.