Wikipedia founder Jimmy Wales is targeting the fourth quarter of this year for the unveiling of an open-source search engine that he hopes could challenge the dominance of market-leaders Google and Yahoo.
The project is being run through Wikia Inc, a for-profit company founded by Wales that seeks to use a similar model to the Wikipedia community-written and edited encyclopedia. He hopes to provide the tools and technology to allow programmers across the internet to collaborate on the development and testing of a search engine and make the results freely available.
“The essential core principles are that I think search is now a fundamental part of the infrastructure of the internet and it’s really fundamental to society as a whole and therefore as citizens of the world we should be concerned about it being a secretive black box,” he said.
Efforts to create open search engines aren’t new but one of the stumbling blocks they face is a difficulty in running large-scale tests of the search algorithm, said Wales. The algorithm is the code that sits at the heart of the search engine and is responsible for its accuracy or lack thereof.
“To create a full-scale crawling spider of the Web actually requires a great deal of investment in hardware,” he said. Wikia is planning to provide resources to enable full-scale crawling of the World Wide Web so the software can be fully tested and tuned.
The project is still in the planning stages and Wales expects that the first test version due this year will help programmers spot bugs that occur with real-world usage and speed up the development process.
“Probably what we’ll do is launch something in the fourth quarter of this year with a really big warning ‘It sucks, we know it sucks, it’s experimental, don’t panic. This is just an experiment to show what could be and now we’re going to start working to see how we could make it better’,” he said.
Already the project is attracting attention, not just from engineers who want to lend a hand but from companies that are already offering search engines.
