Wikia Search May Have Trouble Achieving Critical Mass
from the notability dept
Mathew Ingram notes that Jimmy Wales’s company, Wikia, has unveiled a new version of its search engine. The basic premise of the search engine, allowing users to edit search results the way they can edit Wikipedia pages, is clever. But I think Wales is going to have difficult making the project successful. The fundamental problem, I think, is a matter of raw mathematics: there are far, far more potential web searches than there are pages in Wikipedia. Last month I critiqued the business model of Biographicon, a site that’s attempting to create a Wikipedia-style page for everyone. I argued that they’re likely to have trouble making it work because any given page is unlikely to have the critical mass of contributors necessary to make the wiki model work. I think Wikia’s search engine is likely to suffer from an even more serious case of the same problem. Wikipedia achieves this critical mass by limiting itself to subjects that are “notable.” But a search engine can’t have those kinds of limits. People want a search engine to have good responses even for (maybe especially for) obscure searches. And by definition, it won’t be possible to get a bunch of people to contribute to the page for an obscure search term.
Closely related is the problem of bias. Wikipedia strives to take a neutral point of view, presenting all viewpoints fairly and accurately without passing judgment on which one is correct. This often leads to pages being longer than they would otherwise be, but they tend to be reasonable representations of what various people think on the subject at hand. This approach won’t really work with a search engine because people expect the most important search results to be at the top, and deciding which results are the most important is an intrinsically subjective decision. If Wikia’s search engine ever became popular, it could be beset by edit wars that would make the infamous Danzig/Gdansk edit war look tame. Companies pay search engine optimization firms thousands of dollars to improve their Google ranks, a successful Wikia search would likely succumb to the same kinds of pressure, and the site appears to lack Wikipedia’s well-defined procedures for resolving disputes.
Filed Under: search, user generated, wikia, wikis
Comments on “Wikia Search May Have Trouble Achieving Critical Mass”
Not so obscure
Though I agree with the SEO problem (payment for “improved”/skewed search rankings), I don’t buy the “obscure” point.
I ran a large search engine for a few years and the number of “unique” search queries was …er… unique. We had some test queries to test our indexes and result speeds, and even those arbitrarily weird queries eventually were not unique (20 or so pseudo-randomly chosen characters).
At one point in my life (read: when I was younger, kid-free, etc.) I could see me spending time tweaking search results on “linux queries” and “developer queries”, all in the hopes of improving the results for those communities. Done “right”, this type of search engine could replace the need for manually created FAQs.
Hybrid
A common problem with search engines is ambiguous terms. For example, if I search for ‘Python’ I might be looking for a snake or I might be looking for a programming language. A user-edited search could help solve this.
I envision a hybrid system that has a Google-like engine under the hood. But if your search includes certain *notable* keywords or keyword combinations it could leverage it’s users to first create a place where you tell the engine exactly what you mean by that term (if there’s a conflict) and then rate pages that match user-selected results for your variety of that keyword as much more relevant.
Re: Hybrid
I was going to say about the same thing I would love something like google but allow users to edit results to some degree. This way users can mark a page as spam or a repeat of the same article giving more speciffic information and removing most of the repeated content. Also having the ability to link results so if searching for “side effects of coffee” you could have to the side a suppliment with “bennifits of coffee”.
For people that add irrelevant pages and others can vote it down or report it and for results with conflicts someone can review it and decide.
Coding the concept, not words, you will be there.
This is about the point that I have brought along before, even with James’s programmer (Jer). A word or words are not carrying specific meaning when they are standing alone. To get the full meaning of unknown texts, so called “unstructured contents”, the most fundamental unit to measure one or more concepts must be created. I call it “Codon”, “-LCP-” in short form. Then, any unknown contents will be searchable, at least in such theory named “Codonology”.
Anyway, I am trying to use current Wikia as the platform to start the Codonology project. Hopefully, the dream may come true in terms of true “Concept Search Tool”, and reasoning tool as well.
Hua Fang
SEO
I’ve been thinking about this, and SEO is big enough business, SEO scamming enough of a problem, that I think they’ll have too difficult of a time keeping it from being gamed.