Thanks for participating. Some reactions to your feedback:

1) first and foremost the test has to be compelling (i.e. fun) to the end-user. If not, they won’t complete it – no matter how validity/consistent the results are. Attrition is the first challenge in test construction.

A: Simply not so. The market bears this out. eHarmony neither has a fun nor compelling questionnaire, yet they dominate the market. Actually, users will complete a questionnaire or legitimate assessment if they have intrinsic motivation to tolerate the process. Making it “engaging” is nice, but if the process doesn’t work then all those cosmetics were useless in the end — both for that user and for driving PR for the site.

2) questions should customizable so that the test works on a variety of sites. We’ve found that the same test doesn’t play the same way in the UK as it does in the US or on a Catholic site the way it does on a Jewish site.

A: Not sure what you mean by “cuztomizable.” It’s true that some tests/assessments need to have different content for different audiences, although proper test construction at the onset would’ve revealed this in the first place and designers can make revisions so that one application does work with diverse groups. Test makers in personality and educational testing do this all the time. But, if you mean change formats, questions, etc on a whim, then I’d say that’s a dangerous approach. Changing the wording even slightly on some or all of the questions without a follow up validation is a recipe for a poor test and hence poor user experience.

3) the test should be accessible in terms of price. $350K is laughable. I’m not sure how many takers Dr. Houran is getting but we have to work to get clients accept a $5K set-up fee.

A: Not so. Companies that can afford an original and valid application should be free to enjoy the competitive advantage that application provides. Any company can build their own questionnaire and matching method for free (5K is incredibly expensive for a fun “questionnaire”), but tests and measurements companies routinely get millions of dollars for their work. Building a legitimate assessment that ‘s original, has strong psychometrics (it works) based on modern test theory and is user friendly is costly in terms of time and money. Of course is scalable according to the exact scope and business requirements.

4) finally, the test must be profitable (i.e. make money for the website). It’s no longer enough to simply promote your test on the basis of member conversion. It needs to contribute to subscriber conversion. Why else would you add it? Benevolence is not a trait we’ve encountered much in the dating site owner’s in the last 7 years.

A: Yes, no argument here. Although I do know of some sites that have “tests” simply to compete in the minds of users. It seems that almost every site has a “test” nowadays — so some companies may want one for a value-add to their customers. I guess benevolence is not entirely dead, which is encouraging given this is an industry purportedly devoted to love and happiness.

Thanks,

James Houran, Ph.D.
http://www.OnlineDatingMagazine.com

I think Evan Mark Katz has the best advice. Little wonder his comment has the highest rating so far – a whopping 3.4.

Here’s our two cents…

1) first and foremost the test has to be compelling (i.e. fun) to the end-user. If not, they won’t complete it – no matter how validity/consistent the results are. Attrition is the first challenge in test construction.

2) questions should customizable so that the test works on a variety of sites. We’ve found that the same test doesn’t play the same way in the UK as it does in the US or on a Catholic site the way it does on a Jewish site.

3) the test should be accessible in terms of price. $350K is laughable. I’m not sure how many takers Dr. Houran is getting but we have to work to get clients accept a $5K set-up fee.

4) finally, the test must be profitable (i.e. make money for the website). It’s no longer enough to simply promote your test on the basis of member conversion. It needs to contribute to subscriber conversion. Why else would you add it? Benevolence is not a trait we’ve encountered much in the dating site owner’s in the last 7 years.

Have a good one.

Patrick

Tests and assessments are both gauges or measures of underlying traits or characteristics. Both can use a myriad of different question types to establish trait levels. The only real difference is that with assessments there is evaluative feedback or clinical/practical application. Tests simply give descriptive “grades” or scores, not neccesarily evaluations of those grades per se. That is what an assessment does.

Thanks,

James Houran, Ph.D.

I’ve mentioned astrology before, off to check out your site.

And that’s all he wrote.

JB