The more I think about it, the more I’m excited about the idea of a personality matching shootout. It’s about time we had some true scientifically proven, apples-to-apples comparisons between the various systems in use on today’s dating sites.
Any shootout of this magnitude runs the risk of failure at any time. If we’re going to do this, we need vendors and participating dating sites to be “all in.” I say “we” because its going to take several people to help set it up, run the tests and verify the results. I’m more than happy to participate and help make it happen in whatever role makes sense. Obviously there are privacy concerns, non-disclosure agreements and other hurdles to overcome, but if people are really serious about getting involved, I think we can find common ground for the sake of moving forward.
It’s important to make the shootout meaningful, not just for dating sites, but more importantly, the singles that use them.
Open it up to as many vendors as possible. Here’s who I would like to see participate in the shootout: PerfectMatch, Chemistry, eHarmony, sites running ThomasKnowsPeople’s test and Yahoo! Personals (powered by WeAttract). Let us not forget Fernando Ardenghi, our esteemed colleague in Argentina. I hope he brings his best matching system to the party, as we’re all tired of hearing about him go on about it for years ;-) This is not a complete list by any means. Who else should be invited, who wants to participate?
All companies involved are going to have to set aside some resources to make this happen. We’re not evaluating word processors, but a highly complex systems with lots of moving parts that no-one has ever reliably compared out in the wild successfully, at least to my knowledge. Databases need to be connected to, audit trails tracked and software needs to be written. We need to be able to define and ultimately track progress, bot successful and unsuccessful matches. Then there is the issue of identifying participating websites and a small group of overseers to keep things fair, balanced and on track.
I can see the potential for problems such as creating a list of evaluation criteria and how the results are measured. I’m not a Dr. But I hope to facilitate the discussion between participants about what gets measured and how. Then there is the issue of ranking the results. These are complicated issues that will require all involved to give a little bit in order to make the evaluation as fair as possible.
Perfection is not the goal, but I think we can all get along enough to have a solid set of results that can be used for business deals, marketing materials and a baseline for improvement. And more happy people that can change their Facebook status to “In a relationship”, because thats the end goal, right?
Dating sites will want a list of integration criteria. What resources are required to integrate with each matching platform? What are the expected overall integration costs? How many man-hours from start to finish? Are we talking complicated database queries or “Userplane simple” plug-and play? Where does the matching platform reside? On the dating site’s servers or offsite? How much work is it to pull data from the dating site into the matching system? What does it take to get the results back into member’s profiles? Can you search based on the test questions or are the results baked into search only? I’m just listing a few things off the top of my head here. Any CTO or IT department at a dating site is going to have a lot more questions that what I’ve rattled off here.
Are systems based on monthly licenses or per-query fees? How will a dating site’s costs be affected as they grow? What happens if something blows up in the middle of the night? Who’s responsible for fixing problems?
If your matching platform is wildly accurate, but costs $100k a year and takes 6 months to set up, how does that compare to a $1,000/month service thats less accurate but takes a week to get working? I’m speaking in generalities here, but you get the idea.
Better matches for singles are based on completely different criteria than two business dancing around the table trying to figure out how to enter into a lucrative long-term partnership and all of the associated business rules.
Are the matching platforms a drop-in replacement for a site that already has a system in place, or better suited for sites that don’t have personality testing?
Part of me thinks the tests should go on brand new sites, in part to see how tests work with small numbers of members and scales as the sites grow.
Maybe I’m thinking too much, maybe we make the shootout as simple as possible. For every 1,000 singles, how many meet someone and go on more than 10 dates within three months? Rank the results and we’re done.
As a consultant, hand-waving about tests performed in a controlled environment are not realistic compared to the reality of being part of a live Internet service. That’s why I want to know about the details. And this is a dating industry blog after all. But still, it’s all about more effective matching. That is the end game here.
Added bonus: these general criteria will also be useful to background check, profile and content management/compliance companies.
There will always be a certain amount of inexactitude, but we have to start with some core concepts and ideas and build from there. There is always the chance that the major dating sites won’t want to participate, but I see that as their loss.
There are several top-ten dating sites that have no matching system in place. they spend 10’s of millions of dollars on advertising and couldn’t care less about what people do on their sites, only that they click ads and their credit cards re-bill every month. Perhaps the results of the shootout will put them in a position to consider integrating a matching platform.
What do you think?
