This is the latest in a series of posts that have been languishing in my draft folder. Forgive the somewhat unpolished nature.
Many moons ago someone said in the comments that PlentyOfFish has failed at being a free eHarmony. Sounds crazy, right? But you would be surprised at what goes on behind the scenes at POF and other dating sites in terms of matching voodoo that members are directly affected by.
Today I want to compare how Google continuously improves it’s search index with how dating site compatibility systems evolve (or more importantly, don’t).
Google rolls out updates to it search engine index regularly, even giving them code names.
Dating sites never talk about upgrades to matching systems, in part because people would probably freak out of they met their match with the “old” version of eHarmony.
The moment Google updates its index, Matt Cutts hears from a lot of people who are upset that the website they spent $50,000 on SEO/SEM fell off the first page of Google, or the “Position 6” situation, which at the time was wraught with conspiricy theories. Suffice to say, sometimes Google doesn’t even know what it’s own search engine is doing.
Google knows it doesn’t have to share the guts of what makes it so successful, but it’s more than happy to take on a certain amount of transparency and talk about the outward effects of it’s internal processes. White papers, executive interviews, Charlie Rose appearances, etc.
There appears to be little interest in dating site compatibility matching systems, even with VisualDNA and IntroAnalytics hot on the scene and running big trials that look promising. Dating bloggers don’t say much about matching and the sites themselves say even less. I find this frustrating to no end.
The site that does this and is transparent about it is going to make a killing. OKCupid, maybe Herway and a few others are leaning in this direction. But it’s not nearly enough.
People are messy organisms. We don’t always do the right thing or make the best choices. That’s part of what makes life interesting and online dating so difficult. To that end, I met with Michael Norton a few weeks ago and he had some great things to say about matching systems and virtual dating. Michael has done considerable research into matching systems (and a lot more) back in the day at the MIT Media Lab and is now teaching at Harvard Business School. Michael has done work with Dan Ariely and Jenna Frost, who I’ve mentioned on this blog several times.
I found myself nodding my head at just about everything Michael said. Expect a guest post from him soon to go into more detail. Here’s more on their paper, Online Match-Making with Virtual Dates.
Ok, back to the messy business of matching people. Its my belief that there is an X factor that has yet to be deciphered, which will make the Davinci Code look like Where’s Waldo.
In terms of the amount of questions it takes to get a decent idea of who to match people with, what’s the sweet spot? For eHarmony it’s now around 200, but at Chemistry it’s much less.
What was behind the 200 or so questions eHarmony removed from the signup process?
Redundancy – many tests ask the same question several different ways to be certain you are answering truthfully.
Not useful – What they thought was important to measure, wasn’t.
Or my personal favorite – Marketing came in and said “you guys are killing us, we keep loosing thousands of people every week in the middle of the signup process, shorten the damn test!
Does that mean Chemistry is more efficient at getting a definition of a person with less data?
Lets say for example that even if the original eHarmony algorithm was flawed and based on heuristics (I’m not making any sort of judgement here), one would think they are able to learn as they grow. At least I hope so.
PerfectMatch is somewhere in the middle if I recall correctly. I’ve always wondered what eHarmony did to trim the initial test from 436 questions to 200 or so. IntroAnalytics says they can do it by watching you click on four people’s profiles. How do these systems compare in terms of effectiveness? No, I’m not doing a shootout. When I tried that last time initial discussions devolved into a series unsurmountable hurdles so I gave up.
Lately I’ve been thinking about how companies attempt to match people when the membership grows to millions of people and how matching effectiveness could be improved by looking at the search engine market.
As far as I am concerned dating sites are not even remotely close to being called a success when it comes to matching people. This is not to disparage the decades of research, published papers and hard work that dating sites have based their compatibility system. The reality is that they don’t work anywhere near as well as they should. This is simply where the industry is in it’s lifecycle.
I dislike taking personality tests because it makes me think about myself and my issues. I don’t think I’m alone here. For example, I don’t want to admit on a dating site that I may duck behind a pregnant woman if I hear a car backfire.
OKCupid is a bit easier on the self-esteem. Once you cop to a few bad behaviors the next 200 questions fly by quickly. Although if I dismiss someone from my Quiver, does that mean that women who are like the one I said no to won’t show up again?
I get a bit neurotic when considering that just maybe, Match’s success is based on your preferences regarding weird pets, thunderstorms and brainiacs, the very questions that people tend to complain about the most in my experience. Or that one dumb click on a dating site could hide a plethora of suitable women. Best not to think about it too much.
Online compatibility systems are only a few years old. There are tests based on math, science, pop psychology, DISC, Meyers-Briggs and alot of other psychology that I don’t begin to understand. There is also a lot of hot -ir and snake oil that needs to be evaluated. Unfortunately, dating sites are not willing to share much about they know (and don’t know) about their members.
Saying that an outgoing person should date and introvert, or a builder should date an explorer is theoretically useful, but if you look at the cold hard stats and reality, things start to fall apart pretty quickly.
So that’s where we’re at. Let’s look to Google for some inspiration.
What if dating sites implemented a matching system enabling them to have a 360 degree view of the matching process and the all-important outcome?
Algorithm 1: Tanked, nobody gets to step two of the process. Customer service get slammed with a particularly difficult type of member (money back, complain to press and friends, lawsuit.)
Algorithm 2: Works ok, they went on three dates, then all we know is the guy is back for more.
Algorithm 2: Works ok, they went on three dates, then all we know is the guy is back for more.
Algorithm 3 (Sometime in 2050): Perfection. Everyone matched on these criteria get married and buried together and they send us an email from their honeymoon.
At some point, what happens to the original algorithm? How much of the original eHarmony system is still in place? Has it evolved over time into something that barely resembles the original system?
Chemistry.com says: Our system is designed to continually learn from your feedback: who you like and who you don’t like. Your input is automatically integrated into our matching system so that over time we get a more complete picture of who you are and the kind of person you’re looking for.
In recent years, the online dating industry as a whole has had to settle for modest year-over-year gains. 2010 is the year that the chasm between the major dating sites and the tier two players is going to separate by a wide margin.
At this point in time, marketing spend is is primary differentiator between top-10 sites and the rest of the industry. You really think eHarmony would be as popular as it is if they didn’t borrow $100 million and plow it back into marketing eight years ago? And how ironic that Match, which besides it’s 2-way matching and a few new bells and whistles, still doesn’t really have that much going for it in terms if predicting if two people should go out on a date. Or maybe it does and it just doesn’t work for me (always a possibility.)
Google is famously creative at encouraging these breakthroughs; every year, it holds an internal demo fair called CSI — Crazy Search Ideas — in an attempt to spark offbeat but productive approaches. But for the most part, the improvement process is a relentless slog, grinding through bad results to determine what isn’t working. Dating sites need to do more weekend Hack-a-thons like CSI.
If there is anyone out there listening who actually slings code at a dating site, read WIRED’s How Google’s Algorithm Rules the Web.
Quick bit about VisualDNA. VisualDNA is a way of profiling people through visual personality and lifestyle quizzes. Having signed up eHarmony and Meetic in 2009 to a simple, fun and engaging quiz application designed to boost registrations and subscriptions, they launched the automatic profile writer ProfileWiz.com at the start of 2010. ProfileWiz takes people through a 20 question quiz and writes a detailed (and alarmingly accurate) profile of the member.
The next project for them is to prove that people matched according to their VisualDNA are more likely to want to date than those matched through traditional text profiles. If you would be interested in trialling VisualDNA matching for free, get in contact. Profiling and matching can be done with very low integration as the whole process can be run on a VisualDNA microsite: you just need to deliver the matches to the members.
IntroAnalytics has some nice customer wins happening as well. I hope both companies are able to get their price points and integrations to the point where dating sites become more willing to partner.
I hope that dating sites will hire more women, massively increase their churn rates (more successful and effective) and evolve their matching systems over time like Google does. There are many other things (advertising and branding) the industry needs to address, but that’s a pretty good start for the next five years, don’t you agree?
Thats a wrap, have a great Easter.
