#[KevinMarks]a lot of technorati secret sauce was correlating the feed version with the html version to decide which was richer. You can also have the case where the h-feed or atom/rss feed is summary not content, where you may need to crawl the post page for full.