Distributed crawler / crawling system / Web mining project
$750-1500 USD
Paid on delivery
You will (or already have)
1) write a distributed crawler system running multiple crawlers with a command an control (in PHP or Java or Node.JS)
2) eventually crawl over 160M pages of a social network ([url removed, login to view]) and defeat potential crawling roadblocks
3) deliver results in a text comma delineated file(s) based on our data format
You will get:
-Massive cloud computing accounts. There will be almost no limits.
-VK Account Directory
-help from our Engineering team.
You will get paid along the way... so don't worry about that. And this will be fun.
Project ID: #1468508
About the project
9 freelancers are bidding on average $1017 for this job
We can help in your project, please check PMB and our ratings/reviews to get idea of our experience.
I Can write the distributed crawler by using scrapy framework, and you can have the output files in .csv format. If you are interested in my offer, please ping us to negotiate. Thanks and regards Srinath
GETTING PROJECTS DONE UNDER BUDGET! New World Solutions brings together some of the brightest minds in the information technology field. From our pool of multi talented, savvy, goal oriented professionals, we s More