Scraping Project

I have a scraping project which I need to execute on a scheduled basis. There are a number of webpages I need: some HTML, and some that require a form to be filled out before the data is displayed. I prefer to use an off the shelf scraping package (i.e. VelocityScrape) so that we don't "recreate the wheel" and have to custom program the scraping. The results of the scrape are placed by VelocityScape in an internal DB, which is an "instance" of SQL2005. (i.e. SQL2005 is not installed on the server, but the db is still created and used exclusively by this program) We have about 10 different pages that we need scraped. Some of the fields of each page refer to the same variable, so preferably we can use a similar naming convention for those fields which are identical. The project (group of tasks: login, scrape, store) would be scheduled to run every 30 seconds or so. Preference will be given to coders who have experience in charting (i.e. Dundas charts), although not necessary for this project...as this will be the followup project for us: (1) creating realtime charts that reflect the changes in previous data (2) creating an "alert" system which tells us when data changes by a wide margin. See "deliverables" for a list of pages to scrape. ## Deliverables These are the pages we are currently looking at scraping: Short Term Outage Report [login to view URL] Long Term Outage Report [login to view URL] Long Term Critical Outages [login to view URL] Approved Outages: 1 week [login to view URL] Intertie Availability [login to view URL] BC Current Transmission Outages [login to view URL] Planned BC Transmission 2 week outages (approved) [login to view URL] Participant News [login to view URL] AESO News [login to view URL] Event Log [login to view URL] Design Initiatives: [login to view URL] Unit Reports [login to view URL] Thanks for taking a look! 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows? (depending on the nature? of the deliverables): a)? For web sites or? other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software? installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform Windows 2003 SQL2005 VelocityScape Software