What are good ways to test if product ideas should be implemented, such as testing prototypes or doing link tests to see how many users click?  Examples?

I am currently going through this exercise again, having recently started a new role. To 's point above, I agree that the starting point is to first think about the problem that you are trying to solve and the reasons why. Sometimes people can argue endlessly about possible solutions or ideas. If, however, you want to get buy-in from the decision maker(s) early on, assessing the opportunity (see 's "Opportunity Assessment" template) or outlining the underlying hypotheses (check 's "Lean UX" book) should be half the battle. As for testing possible product ideas, this largely depends on how 'robust' the initial validation of the product idea needs to be. I have got good experiences with any of the following approaches: - Build an MVP: what is the smallest product increment that you can build in the shortest amount of possible time that will provide you with some initial validation of the problem statement/hypothesis which you are trying to address? Your 'minimum viable product' can be a low fidelity prototype (I have tested product ideas just using simple paper sketches) or a digital version to get a sense of people's actual behaviours. For example, I am currently involved in building a tool that let's editors manually publish news stories. Rather than building a full-blown, automated content management system from scratch, we opted to start with a purely manual tool first and take things from there. - A/B test: one of the reasons why I like running A/B tests is that it can give you an insight fairly quickly into the viability of a product idea (and its execution). Even if it takes a while to reach a point of 'statistical significance' with your A/B test; in a lot of cases one can already get a sense early on where an idea is heading. Also, you can then utilise user testing sessions to explore the 'why' behind some of the insights that you derived for the A/B tests.

If your product solves a problem users know they have, spend $100 on an ad network and see how many users you can convince to create accounts, enter credit cards, or whatever.  The in-person variant of this is to socialize it among some existing gathering of people where you have the attention of the people, and see how many people bite. Hacker Dojo started with the in-person variant of this. If your product solves a problem users don't know they have find your smartest UX friend and buy them a nice meal.  Your goal is to figure out how to explain your solution in terms of problems your users do know that they have, and how to present information to help your users buy into the wholeness of your solution.  This "wholeness" issue refers to the fact that often solutions to problems users don't know they have  is applying a known solution to a broader set of problems than they anticipated. After this, your product solves a problem your users know they have because you presented it right, and you can use the first tactic. This was used for http://amacron.com

You could use some usability testing tools to test your prototypes http://verifyapp.com/ or http://www.usabilitytools.com. I know more about the latter one (disclaimer: I work for UsabilityTools). The platform comprises 8 different tools - the one you should look are are in UX Suite. With Click Testing you can test screenshots, mock-ups and other design ideas. The respondents were asked “Where on the website would you look for affordable trips to Asia?” Survey tool on the other hand, allows you to get more detailed insights.

Your approach should vary depending on what type of product you're envisioning. The primary fork in the road has to do with wether (A) your product can be conveyed using a set of visual designs or whether (B) the product will only come to life given dynamic interactions with data. For scenario A, I would suggest working with a designer to create (1) a design that shows how the product will be framed for the user (the entry point), and (2) a set of 3-6 visuals that show what the product will look like. Spend more time choosing the right language and visual hierarchy than obsessing over the visual polish. Design a task on http://usertesting.com around these designs and conduct 6 user tests (costs a little over $200). Within several hours you'll get back videos of users interacting with your designs. This will give you an immediate visceral sense of how your product resonates. You'll either see a path to iterating on the design to make the value come across, or you'll see that users are utterly confused or don't care about your product. If your product does resonate, you'll have the confidence to move forward and user testing materials to socialize the product internally. Nothing is harder to argue with than conclusive and thorough user testing. For scenario B, the path to validation is longer and more expensive. Worry less about the design at first. You'll have to work with an engineer to build a functional prototype that gets to the essence of the type of insight your data-rich product will yield. Focus on creating clear interactions with the data. Since the UI might be very primitive, you probably won't be able to conduct user testing with outside users. Your best bet will be to first socialize the demo internally to people who understand the context. If it's received well, enlist a designer to develop the UI to the point where it is ready for outside user feedback, e.g., through http://usertesting.com. Go through the same validation process described above. Be ready to bail on the product at any point down this road.

This question seems to read, "I have a solution, how do I see if it fixes a problem?" I think this is a bit backwards. A product idea should be based on a known customer problem and attempt to solve it. If there is no customer problem, then testing a prototype or doing link tests could be a massive waste of time. Regardless of the idea, there is an assumption or hypothesis at the root of it that the customer will find this service/feature/product useful in solving some problem. First confirm that the hypothesis is accurate by getting out of the building and talking to customers. After getting that qualitative feedback, building a testing mockups, building a PoC, doing A/B testing, etc will be more productive. For more information along those lines of thinking, I think Steve Blank's Four Steps to the Epiphany is still the best book on customer development.

One that my friends use is Zurb's Verify. Here I interview the team to find out how it works: http://scobleizer.com/2010/11/02/test-your-website-before-you-build-it-first-look-at-zurbs-verify/ -- great for testing ideas, even back-of-napkin drawings to see if your friends will click through the idea.

Some of the product guys behind Amazon.com's interface gave a presentation at SXSW 2010. I don't recall the details (so don't quote me), but the general ideas is that they'd test ideas by releasing them into the wild for minutes at a time. With such a highly-trafficked site, exposing a new feature for just a few minutes can already attract enough usage to determine whether the idea is a hit or a miss. The product team then takes that usage data to help inform decisions. My coworker told me that at Facebook, the product team tests ideas out by deploying them to tiny portions of their servers. By isolating the deployment, they're able to control the experiment and study ripple effects the idea may have across their entire product. Once they gather enough actionable data, they study it to help inform decisions. http://Amazon.com and Facebook seem to have similar methodologies, and both seem influenced by the scientific method. To summarize what I think these companies are doing: Seek out unexpected/undesirable behavior from your users. Ask a "What if?" question (e.g. what if we made the button bigger?) Construct your hypothesis (e.g. if we make the button bigger, more people will click it) Design your experiment to test the hypothesis (e.g. we'll make the button bigger for 30,000 users [be mindful to determine a representative sample size] and see what happens) Do the experiment and collect data Compile results and use it to inform decision-making.

First, look at the current resources that you have - the man hours and money available in your start up. Second, list down the top three priorities that you think your company should be focusing on right now. Third, list down all the product ideas that you have. Make a general rough draft of how many man hours and money it's going to take to implement those ideas. From this exercise, you will see the opportunity cost of implementing those ideas and how are they affecting the top three priorities of your company. You can also think of the possible returns of those ideas if being implemented well. The general guideline would be - you wouldn't know until you try. If the opportunity cost is reasonable in your terms, just try it. You will be surprised at what you will find eventually. User clicks etc. can be very useful to identify patterns and user behavior. For me, working with raw data like that would always be part of my highest priorities.

Regardless of the greatness of a product idea, it has to be "sold" to one's colleagues before any positive action is taken.  Thus, projects presented by those who can "sell" their ideas usually live; the projects which are not "sold" by their presenter usually die.  Thus, any analytical consideration of a project typically takes a back seat to enthusiasm.