Difference between revisions of "Rewrite PageCreationBot"

Revision as of 19:03, 11 September 2007

OurWork

What (summary)

• New page-building bot
• Still relies on Java/Tomcat to do crawling (for now)
• Carefully tested

Why this is important

• We need to have control over the pages that our created on our site.
• The old bot was known to pollute the database; we need control over all the access points that could screw up our data.
• Gaining mastery over the code so that we can add new features easily.

DoneDone

• Creates news pages based on a template
• Monitoring and logging have been added (tests whether or not the bot succeeds)
  • Output to a log file. Either on each squal box (with aggregation) or an NFS volume. Have emailed Ethan and Michael about this.
• Hooked in to all the old points Bot was
  • Not exactly the same points, but the same end-user functionality.
• Projects:BotTest problems fixed

Bot insertion points into Mediawiki

• /wiki/skins/common/generatePage.js (and some other javascript that we should remove)
• /wiki/extensions/AboutUsDomainRedirect/SpecialRedirectToDomain.php (deprecate and point to CaseSpace)
• /wiki/extensions/CaseSpace/CaseSpace.php (Ultimately, here is where the magic will happen.)
• /wiki/extensions/AboutUsBuildDomain/AboutUsBuildDomain.php should be the best place to keep it.

Schema

• New schema location http://images.aboutus.org/images/b/be/Aboutusbot_new.zip. Its an sql file and not a compressed one.