Difference between revisions of "Rewrite PageCreationBot"

Line 4: Line 4:
 
== What (summary) ==
 
== What (summary) ==
  
* Remove the bot from Thunderclap and shutdown apache
+
* New page-building bot
* Tie in to partner apis
+
* Still relies on Java/Tomcat to do crawling (for now)
 
* Carefully tested
 
* Carefully tested
  
Line 11: Line 11:
 
== Why this is important ==
 
== Why this is important ==
  
Moving the bot off of our Database server will improve site performance. Also, moving it to the same technology we are using for other parts of the system gives us better control over the bot and hence more opportunities to create good features for the bot.
+
* We need to have control over the pages that our created on our site.
 +
* The old bot was known to pollute the database; we need control over all the access points that could screw up our data.
 +
 
  
 
== [[DoneDone]] ==
 
== [[DoneDone]] ==
* Bot on Thunderclap has been deactivated
+
* Creates news pages based on a template
* Apache on Thunderclap has been shutdown
 
* All functionality from original bot has been duplicated
 
 
* Monitoring and logging have been added (tests whether or not the bot succeeds)
 
* Monitoring and logging have been added (tests whether or not the bot succeeds)
 
+
* Hooked in to all the old points Bot was
  
 
[[Category:DevelopmentTeamProject]]
 
[[Category:DevelopmentTeamProject]]
 
</noinclude>
 
</noinclude>

Revision as of 01:56, 14 August 2007

DevelopmentTeam


What (summary)

  • New page-building bot
  • Still relies on Java/Tomcat to do crawling (for now)
  • Carefully tested


Why this is important

  • We need to have control over the pages that our created on our site.
  • The old bot was known to pollute the database; we need control over all the access points that could screw up our data.


DoneDone

  • Creates news pages based on a template
  • Monitoring and logging have been added (tests whether or not the bot succeeds)
  • Hooked in to all the old points Bot was

Retrieved from "http://aboutus.com/index.php?title=Rewrite_PageCreationBot&oldid=8851737"