GetWorkDone - started and suspended

Posted: 7 months ago (2009-12-27 05:30:37 UTC )

I started on a little project to download HTML documentation automatically for installed gems. That'd be useful because some gems, like Rails, have no (or no useful) RDoc documentation, and just link you to a web page somewhere.

That means no useful offline use of documentation, for instance.

So I wrote a quick spider with Anemone to mirror with a few simple limits -- only a couple of bounces through pages, only on the same site, etc. I figured I could then hook up my little spidering mini-format to something that traversed your gem list and found HTML doc links, and that'd pretty much work.

Unfortunately, Anemone doesn't seem to follow "frame" links in HTML, which means it won't spider the Rails documentation successfully. I'm suspending work on it for now until I'm ready to dig into Anemone a bit more (or until they fix that -- I filed a bug report).