Misconception About Widespead RSS Use By Traditional Online Publishers

Earleir this month, Rich Skrenta of Topix.net wrote about the misconception about how widespead RSS syndication is among traditional online publishers. “Only 7% of the sources Topix.net crawls have XML feeds. I’d estimate that only a few hundreds of the top 3,000 newspapers we crawl have RSS support. The rest we obtain with a news crawler which is good about finding articles on news sites, leaving behind the ads and navigation sidebars. It’s low maintenance so we don’t have to change anything everytime a site redesigns its html. “Even for sites which offer feeds, we’ll generally continue to crawl the […]

Crawling for Local News

Last week, I wrote about Topix.net, which spiders more than 3,000 other local news sites, then lets users enter their local ZIP codes and see a page showing all local news from all local media. The San Jose Mercury News quoted me: Vin Crosbie, a Connecticut media consultant, said his tests of Topix often produced more interesting lists of stories than local newspaper Web sites, in part because it gathers news from so many sources. “I think there would be an interest in this, ”Crosbie said, “and I think if you can provide somebody with a quick one-screen view of […]

Topix Launches

We’re watching the launch of Topix.net, news site whose robots scrape thousands of American news media sites and aggregate and categorize the content for easy browsing by locality. We were initially skeptical of the venture, but have been impressed by what we’ve seen. In fact, a reporter for the San Jose Mercury News, the local newspaper where Topix is based, called us for comment about it, and we told him that I think Topix probably produces a better news Web site for many localities than those localities own newspapers do. Earlier in the day, we had taken a half dozen […]