What are psychological tests about?

What is a formula to rank people by test score with varying amounts of tests taken?

• I can't seem to find a good formula for what I want to do. It's been a few years since statistics so maybe I am just not searching for the right lingo. What I have: Entities that have multiple "tests" and a percentage score for each test (based on weighted "questions" within each test). So I have the number of tests taken, the average score for all tests, the total possible 'points' based on the weighted points for each question within a test, and the total actual points from "correct" answers. (Excuse my scare quotes (http://en.wikipedia.org/wiki/Scare_quotes) but what I'm actually working with is more like evaluations than school tests) What I want: I want to rank these entities based on "Who is the best performer?" taking into account average grade and the number of "tests" completed. I need a solid way to dampen edge cases. I imagine this is similar to how ratings work in Netflix and Amazon. A product with one 5 star review is not performing better than a product with 10 reviews and a 3.5 star average. Example 1: Person A has taken 100 tests but only has an average of 10%, so they may have total points of 1000. Person B has taken 2 tests and has an average of 90% but may only have 19 points. In this case, Person B needs to be classified as a higher rank than Person A. Example 2: Person A has taken 100 tests and has an average score of 79%, so they may have total points of 790. Person B has taken 10 tests and has an average score of 80%, so they may have total points of 80. In this case, Person A needs to be classified as a higher rank.

• Answer:

  Consider that the number of tests taken by a given candidate is M[i], i being the index of the candidate. Assuming your 'rating' of candidates is going to be relative as your ultimate purpose is to resolve conflicts between candidates with varied amount of training examples (i.e. varied M[i] values) you can adopt the Elo ratings (used in Chess, if you are unfamiliar with it. Also I believe that's what Mark Zuckerberg and Eduardo Saverin used to compare people). This includes giving a base rating to each candidate and then varying it when you compare them with other candidates in your sample space. Read more about it here: http://en.wikipedia.org/wiki/Elo_rating_system