What are the pros and cons of usertesting.com versus in-person user testing?

Okay, here are some more broad pros vs cons for a remote testing service vs in person: Pros for in-person: - Oldest approach in the book, and a lot of resources for this (I highly recommend Rocket Surgery Made Easy by Steve Krug if you go down this path) - Can recruit pretty much anywhere (Craigslist, coffee shop, mall, etc) - Easy to follow up with individual participant Cons for in-person: - Often requires more setup time - Scheduling issues (no shows, having problems getting people, etc) - In aggregate can cost more (in terms of your time, prep, paying participants, etc.) - Keeping a cool hand when they tell you things they don't like - Takes you away from other tasks Pros of remote testing service: - Logistics are handled for you - Cost is known up front - Can get this process started at any time - Lower time commitment Cons of remote testing service: - Still a good amount of configuration - Paradox of choice can overwhelm you (especially when there's an empty "what do you want to ask them/what do you want them to do" text box) - Quality control of responses can vary (depending on the service) If you're just starting out AND cost is an issue, I would use a remote testing service to start. Later on you can always add in an in-person testing setup. And for all of these feedback options, you want to do multiple rounds of feedback, as you evolve your mockups, website, or product. BTW, I also happened to build a tool to help you more easily get remote feedback, called UsersThink - http://usersthink.com/ It takes care of the con list for remote feedback, requires no setup (just enter the landing page URL that you'd like feedback on), and is more cost effective than the other testing services out there. I say check it out, should help you with your mockups :)

If you're interested in a remote usability testing for iPhone & iPad apps, I suggest you take a look at Appsee: http://www.appsee.com/?v=3 Since the cost is an issue, you can start with the free plan. Though I work at Appsee, I can say for sure that we offer unique key features: Session playback - provides recordings of real user sessions, which enable understanding of all the actions users perform in the app and what problems they encounter. Touch heatmaps - enable to see exactly how users interact with the app by visualizing every touch they make. This feature helps you find out which elements users focus on the most within your app. In-app analytics - provides key metrics to help you understand how engaged your app users are, their activity overtime, which device and OS they use and their geographic breakdown. Actionable insights - act on pinpoint insights from the Appsee solution to maximize customer engagement, conversions and in-app monetization. Auto-detect UI problems - detect unresponsive actions made by your users, such as taps, swipes or pinches, and helps you to optimize the UI. Easy setup &integration - just drop Appsee SDK in your app and add a single line of code. Takes less than 1 minute! You can contact me with any question or our support team if you need help with integrating the SDK.

http://Usertesting.comhttp://serTesting.com is one of my favorite tools, but it's not good for validating concepts or ideas. It's good for validating design. They aren't going to give you as much useful information about if they'd use your product or how much they'd spend--because you can't see any of the physical cues, followup or choose people that are your target users very specifically (i.e they might not be your target demographic of a customer).

I would add one other point: when setting up your test on any of the unmoderated user testing sites, be it http://usertesting.com or http://trymyui.com, make sure you are specific in your instructions that this is not a website and what you are looking for.  The testers are generally pretty good at following instructions, but they are mostly used to testing higher fidelity websites.