Website monitoring script needs updated

$30-250 USD

Closed

Posted

over 6 years ago

$30-250 USD

Paid on delivery

We need to monitor (scrape) 1000 URL's every 5 minutes for specific changes in information. The problem is we are currently only checking 50 pages of the 1000 every 5 minutes which isn't fast enough with our current code. I need a solution (even a 3rd party paid solution) to be able to check 1000 webpages every 5 minutes. 1. review our current Cron 2. provide a solution to meet our needs If you have previous experience with this type of problem, please mention it.

JavaScript

PHP

Python

Software Architecture

Web Scraping

Project ID: 15780253

About the project

26 proposals

Remote project

Active 6 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

26 freelancers are bidding on average $178 USD for this job

@e3d

Hi, I've dealt with similar problems. it usually can be fixed with multithreading. Please show me your current cron so I can evaluate it and provide you with solution quote. Relevant Skills and Experience web scraping and development Proposed Milestones $99 USD - subject to change upon review of current cron

$99 USD in 3 days

5.0

(348 reviews)

9.0

@schoudhary1553

Greetings sir, i can help you and your 100% satisfaction is assured if you allow me to serve. I can do this task as per your requirement Relevant Skills and Experience I have more than 5 years of experience with excellent knowledge in Javascript, PHP, Python, Software Architecture, Web Scraping . please give me a chance to work on your project Proposed Milestones $200 USD - Website monitoring script needs updated

$200 USD in 3 days

4.9

(263 reviews)

7.9

@technoweb7

A proposal has not yet been provided

$208 USD in 3 days

4.9

(27 reviews)

7.2

@jayshreeganeshha

hi,php+javascript developer here and i can check your current code and do required modifications. thanks Relevant Skills and Experience here are the technologies i am well experienced with 1) javascript,jquery,angular,d3 2) php,mysql 3) html,css Proposed Milestones $100 USD - total

$100 USD in 5 days

5.0

(132 reviews)

7.2

@techwelfare

Hi, We have few questions to clarify before we could start. Please let us know your details requirements so we could analyze it and start working. Thanks & Regards Techwelfare Relevant Skills and Experience we are working for this industry for last 10 years, Please reply us back so we could share our relevant skill. Proposed Milestones $225 USD - Will negotiate the milestones

$225 USD in 15 days

4.7

(215 reviews)

7.5

@topithub

Website monitoring script Relevant Skills and Experience Javascript, PHP, Python, Software Architecture, Web Scraping 7 years experience Proposed Milestones $155 USD - task

$155 USD in 3 days

5.0

(46 reviews)

7.2

@i333

hi there, I have experience with Python scraping and multi-threading and would love to update or create a new script for scraping these 1000 pi pages! I cannot confirm scraping in 5 minutes until I have tested, plus it also is based on pagesize, server speed and the configuration of your machine running the script, so a lot of variables but it should be possible and we can discuss further and test to find precisely. Shall we? :)

$230 USD in 3 days

5.0

(119 reviews)

7.0

@hawkscodeau

I have 7+ years in website design & development with very good experience. Relevant Skills and Experience I am good in Javascript, PHP, Python, Software Architecture, Web Scraping Proposed Milestones $200 USD - Cost

$200 USD in 3 days

4.9

(14 reviews)

6.7

@elbruninh

is PHP code? can you send me the source code? Relevant Skills and Experience web scraping Proposed Milestones $150 USD - full project

$150 USD in 3 days

4.9

(114 reviews)

6.6

@saranaone

I can complete your web scraping requirement on time with good quality results using Python scripting. Relevant Skills and Experience I did similar but not as same kind. I think i can fulfil your needs. Proposed Milestones $100 USD - Fees Thanks

$100 USD in 3 days

5.0

(108 reviews)

5.7

@vicboyvera

Using node.js, it is possible to scrape thousands of webpages every minute. I have done this before with elegance and efficiency in my work at Hewlett Packard (thousands of data sources). I'd love to Relevant Skills and Experience node.js, Web Scraping, cron Proposed Milestones $200 USD - Mass Web Scraping $100 USD - Job Manager If you can use node.js, we're all set. I am confident. Feel free to message me, thanks!

$300 USD in 2 days

5.0

(23 reviews)

5.9

@nvbishr

Where you want run the code? server or local? you can use multi thread process or use multiple VPS. Please send me details. ......................................... Stay tuned, I'm still working on this proposal.

$200 USD in 3 days

4.9

(61 reviews)

5.9

@mrherkar

hello, I could develop a .NET app for you which can check these websites. Relevant Skills and Experience hundreds of web projects Proposed Milestones $99 USD - final app do you need any user interface or all parameter is fixed?

$99 USD in 5 days

5.0

(22 reviews)

5.4

@kartikeyagupta

hello, i can provide you complete soltion to your web scrapping project and cron. It will be a custom python scrapper. Ping me to discuss more Relevant Skills and Experience python/scrapy Proposed Milestones $155 USD - work

$155 USD in 3 days

5.0

(36 reviews)

5.0

@sushilpathariya

I have gone through your Job post and I can understand your job requirement thoroughly. Relevant Skills and Experience I have a total of 7 years of experience in Javascript, PHP, Python, Software Architecture, Web Scraping and had completed a number of projects with some great graphics . Proposed Milestones $155 USD - project cost I’m looking forward to hearing from you soon. Thank you for considering my cover letter. firasat

$155 USD in 3 days

4.9

(35 reviews)

5.2

@AresMTA

Write/modify an existing script so it can check 1000 websites every 5 minutes. Relevant Skills and Experience I'm a very skilled Python developer, with a lot of experience in scraping/multithreading. Proposed Milestones $50 USD - Write/modify an existing script so it can check 1000 websites every 5 minutes.

$50 USD in 3 days

5.0

(49 reviews)

4.9

@lytrungtin

Hello. I have experienced many years of website development. I see your requirement. Các Kỹ Năng và Kinh Nghiệm Liên Quan Hope your response and solve your issues. Nice to meet you. Thanks for reading. Các Mốc Thanh Toán Đề Xuất $222 USD - Total Cost

$222 USD in 10 days

4.9

(15 reviews)

4.7

@freelancerkpis

We are happy to bid on this project. ***** We provides dedicate developers and Development and design services as well. ***** We are having a great team of Web developers with 7-8 years of experie Relevant Skills and Experience Javascript,PHP,Python,Software Architecture,Web Scraping, Proposed Milestones $277 USD - Complete project

$277 USD in 3 days

4.8

(23 reviews)

4.9

@ssi599a5dc458c8b

Let’s chat and discuss about price and work. 12 years experience in same filed you will get 100% correct work. No need to pay upfront money first check work quality and if you satisfied then pay. Relevant Skills and Experience 12 years experience Proposed Milestones $64 USD - I will complete your work as per the guidelines and will deliver you according to time deadline.

$64 USD in 3 days

0.0

(0 reviews)

0.0

@sanjayeu99

I have done something very similar in Javascript and Google cloud. My architecture does not require any cron jobs or any desktop software. It works purely out of Google infrastructure. Relevant Skills and Experience About 20 years experience in programming, design and architecture Proposed Milestones $555 USD - Delivery of full working application

$555 USD in 7 days