Best way to social network?

What is the best way to crawl Quora or use a Quora API to graph its social network?

• How best to do the following: get and then visualize [1][2] social network behavior on Quora (e.g. one simple way is edges are Follow links (directed, unidirectional or bidirectional), edge strength is the number of shared Followed users; and nodes are possibly clickable to the user's homepage, not sure what do with node color). (Could also do similar for Topics, and show the clouds) Currently Quora Open API does not enable this (). Also it seems user pages have meta-tags robots noindex. So is this forbidden [3], or Anyway then you would have to user homepages and scrape the tags <a href="http://view-source:http://www.quora.com/Stephen-McInerney-1/followers"> and "/username/following" to get those lists, per user. What's the best way? Are there any limits on crawl rate or hits? [3] I'm not talking about exposing anything publicly to Google, and the nodenames could be anonymous. Apart from robots noindex, there seems to be no prohibition to someone manually doing this, if you have enough patience. ( http://the%20usual%20toolscommon visualization tools) PS then we can answer many interesting questions like 's or 's: at least within your subnetwork or clique.

• Answer:

  There are many good ways to graphically illustrate the connections and interactions on Quora. The best way I've seen to do this is to use scripts to cull data from user profile pages that show Everyone that follows a given user The number of times a given user has upvoted by another given user. This is done by looking at the page with all their answers, and then culling all the upvote data. Once you have a dataset that shows each user and all the followers and the number of times users have received upvotes from other users you can start to analyze it and produce graphics. did some really cool posts on this a while ago. Here's one example: It more or less shows who gets upvoted a lot and how strong the upvote interconnection links are between users.[2] The is another graph that helps show the relationships between who a given user is folllowing, who follows that user, and when there is a mutual follow. Color Legend: Green - Mutual Follow Purple - Following, but not followed back by voter Red - Followed by, but not following back voter Black - No following/follower relationship with voter Width of line indicates number of votes received from user[1] Unfortunately you can't really see time stamps on votes, but you could look analyze who voted on someone's early answers as opposed to their later answers to show how voting patterns changed over time. You could animate that. It could also be interesting to do these graphs in specific topics, showing what the interconnections are there as I'm guessing some people would emerge as dominant in different topics. You could do the same type of analysis for following questions. Now that the site is a lot bigger, it's tougher to do this type of stuff, but still possible if you had a script to get the necessary info. It would more or less involve opening up a lot of web pages and just extracting information. The company seems to be OK with a number of efforts to graphically illustrate user interactions on Quora. However you may want to email them to confirm that and get the permission you need. See [1] [2]