Saturday, September 8, 2007

The new search weapon: humans

Something for the weekend, although after this posting it will be kids, wake boarding and sports on TV until Monday.

Seems Thailand is not the only country censoring the Net -- of course, everyone's doing it ;-)
In Korea there's a big stink about the likes of Naver and Daum censoring posts. You can read all about it at the AsiaMedia site (they copied from Korea Times).

And speaking of Naver, here's a column I wrote recently that covers Korea's most popular search site as well as the newer Mahalo being created by Jason Calacanis. It originally appeared in BroadBand Communities.

The new search weapon: humans

For some reason, there are reports all over the place of late that South Korean search portal Naver has around 70 percent of its local market, while the mighty Google is largely a failure there with its 2 percent share. Perhaps it’s because South Korea organised a large press delegation to its major trade show last month, and that was one of the messages the media got? Or perhaps it’s because The New York Times and Business Week both reported on it (but why? It’s not really that new or newsy) and it went viral from there.

While I’m not interested in how Google is doing in South Korea specifically, I am interested in the model that Naver uses, which I was pointed to by the blog Web 2.0 Asia. Naver is essentially a human-powered search engine, meaning that there is a lot of manual work behind the scenes that goes into making a particular search term relevant.

Apparently a lot of this manual work these days is outsourced to Korean speakers in China, who analyse, index and even produce content for Naver. As a result, if it’s something Korean, such as about one of their soap opera stars, singers or sporting heroes, then you’ll get a much richer collection of links than Google can give you.

It also leads to a downside, as brought up by the Web 2.0 Asia blogger, in that because of the focussed effort needed, and presumable labour costs, Naver has to focus on the hottest topics at the expense of other issues. “The nation's zeitgeist becomes more and more unified – essentially, Naver's top search keyword IS the national zeitgeist,” according to the post, which says this is turning Korea into a “Naverized” nation.

What interests me (and which I haven’t seen brought up in any of the posts on Naver so far) is that there is a major effort going on right now to do a similar human-powered search engine out of Silicon Valley. It's called Mahalo and was co-founded by well-known tech entrepreneur Jason Calacanis, who was also one of the people behind ventures such as Silicon Valley Reporter magazine and the Weblogs Inc site.

Given all the recent attention that has been heaped on Naver, which has been around since 1999, it's odd that Mahalo (means "thank you" in Hawaiian, I've since learned) bills itself as the "world's first human-powered search engine". Makes you question the accuracy of the rest of its information.

Despite this, it's an interesting project. Like Naver, Mahalo uses human guides, both full-time and part-time volunteers, who will create search result pages (known as SeRPs) on the most popular or requested topics. If they haven't created a SeRP on a particular topic, you can request that they do so. End users can also suggest links for topics that have already been covered.

Mahalo is even paying the part timers (US$10-15 per page) when they accept their SeRPs, which are created in the Mahalo Greenhouse. The SeRPs are then reviewed by the full time guides and finally accepted into the Mahalo search engine. While the Greenhouse has only been in operation for five weeks, it reported that it had already accepted more than 525 search result pages from 570 part time guides and counting.

These will be continually added to the 4000 search terms that had been created when Mahalo made its alpha launch at the end of May. The company says it hopes to reach 10,000 search terms by the end of the year, afterwhich it will go into a beta phase.

The site is focused on the top English-language search terms, including verticals such as travel, products, news, entertainment, sports, food, and health. “Google’s mission is to index the world’s information; our mission is to curate that wonderful index,” Calacanis said in a statement released when it was announced. “It’s my belief that humans can play a significant role in the development of search results and we’re going to try to figure out exactly what that role is over the next couple of years." The statement also noted that Mahalo was a five-year project.

If anything, Mahalo reminds me a lot of Wikipedia, given its use of volunteers (although paid ones) from around the world to create content. If it can build up a similar database, it will certainly be a useful addition when you consider the amount of links you need to wade through in Google to get to the information you’re searching for. And even when they don't have a page created for your search term, it defaults to Google in any case.

Of course Google and some of the other search giants could also add humans to the search mix themselves. In fact, that's in part what Yahoo has been doing with its Yahoo Answers service, where you can ask a question and real people behind the scenes will attempt to come up with an answer. (You could even argue that the original Yahoo directory was similar to Mahalo, in that it provided human-generated links on a wide range of subjects.)

Naver, too, is looking to expand. According to the Business Week report, it will export its search model to see if it will work in Japan later this year. And I'd be surprised if others in Asia, particularly places like India with its wealth of human resources, don't try something similar.

Whether it works or not remains to be seen, but given the power of Google these days it's encouraging to see some alternatives springing up. – Geoff Long

No comments: