Find Jobs
Hire Freelancers

Python service that executes HTTP queries from a list in parallel.

€30-250 EUR

Completed
Posted over 3 years ago

€30-250 EUR

Paid on delivery
The system is composed of 3 milestones. • System sends HTTP GET to remote URL endpoints in parallel. • System records the HTTP response and response code and retries on failures • The functionality of the system is available as a static web page and REST API Milestone 1 - System sends HTTP GET to remote URL endpoints in parallel. Description: Write a python class called Executor that given a list of urls queries the URLs in parallel. The maximum number of concurrent requests is capped. The system uses aiohttp library: [login to view URL] Every time the number of active requests falls below max_concurrent_requests, the system will immediately start the next request from url_list. The system handles exceptions and failures gracefuly. In the even of error or failure only the single affected Postback URL is skipped and other postback URLs are still executed. A warning message is printed to the starnard python log ([login to view URL]()) in such case. An example of how Executor API could look is: url_list=[{"Postback URL":"[login to view URL]"+str(i)} for i in range(0,100)] max_concurent_requests=10 executor=Executor(url_list,max_concurent_requests) [login to view URL]() An integration test must be provided written in python unittest framework that when run starts a local flask server and tests the Executor class against this local server. The approach could be simillar to: [login to view URL] Milestone 2 - System records the HTTP response and response code and retries on failures Extend the executor class to record the HTTP response and the HTTP response code. For every url in the list if the response is HTTP success (2xx) Executor records the response in the result as 'Response 1' key, response code under 'HTTP Code 1' key and sets 'Success' to 'YES'. Any other keys found in the url_list are copied to the result (e.g. aux_data). url_list=[{"Postback URL":"http://www.google.com/q=xxx", "aux_data":"xxx"}] max_concurent_requests=10 number_of_retries=3 miliseconds_between_retries=1000 executor=Executor(url_list,max_concurent_requests, max_retries=3, miliseconds_between_retries=1000) result=[login to view URL]() # The result should be something like this: assertEqual(result['Postback URL'], "http://www.google.com/q=xxx") assertEqual(result['aux_data'], "xxx") assertEqual(result['HTTP Code 1'], 400) assertEqual(result['Response 1'], "<!doctype html><html itemscope="" itemtype="[login to view URL]" lang="de"><head><meta...") assertEqual(result['Success'], "YES") If failure happens (HTTP Code other then 2xx is returned or other problem occurs), the system records the failure in 'HTTP Code 1' and 'Response 1', then waits miliseconds_between_retries and tries again. The result of the second try is recorded as 'HTTP Code 2' and 'Response 2'. And so on until the retry count is equal to number_of_retries. The behaviour of the system is integration tested using the test infrastructure developed in Milstone 1. Given number_of_retries=3 the test checks at least the following scenarios: • first request is a success (HTTP CODE 200) • first request is a failure (HTTP CODE 404) • first request is a failure (500), second request is also a failure (HTTP CODE 405), third request is success (HTTP CODE 200) • first request is a failure (500), second request is also a failure (HTTP CODE 405), third request is a failure (HTTP CODE 405) Milestone 3 - The functionality of the Executor is available as a staic web page and REST API The system accepts an input CSV file from a static web page that has an html form. The system first verifies that auth_token is "1234". Each row of the [login to view URL] is converted into a dictionary in the url_list. The url_list with additional parameters (miliseconds_between_retries,max_concurent_requests,max_retries) is then feed to the Executor. The result of Executor::run() is converted to the CSV and send as HTTP response.
Project ID: 28436595

About the project

11 proposals
Remote project
Active 3 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hello, I've read through your job description carefully and I'm pretty sure that I can finish 3 milestones in 3 days. The key aspect of this project is asyncio(async/await) in python and building RESTful api. I'm pretty good at asyncio and building RESTful API with Flask is easy to do it. As your job description is very clear, I can start working immediately with your current code base. Looking forward to work on this job. Best Regards, Bruno
€250 EUR in 3 days
5.0 (3 reviews)
3.1
3.1
11 freelancers are bidding on average €231 EUR for this job
User Avatar
Hi https://www.freelancer.com/projects/html/Flask-Python-Website-Needs-Work done similar tasks in python Flask, django lets discuss
€250 EUR in 7 days
4.9 (30 reviews)
6.3
6.3
User Avatar
Hi, I've read your project requirements throughout and happy to say I'm able to work on this project will full functionality. I can create Class "Executor" as per requirements and can write unit test for that as well. Please ping me for more details Looking forward to your response! Many thanks
€250 EUR in 10 days
5.0 (16 reviews)
4.3
4.3
User Avatar
Hi, I am a senior sofware developer working on python and other technologies since last few years. Highly experienced with multithreaded applications. I believe with my skill set, I can easily help you with your problem. Please message me so we can discuss further. Thanks
€240 EUR in 7 days
5.0 (6 reviews)
4.0
4.0
User Avatar
Hi I have read the description and details and I can make it with Flask. I have some questions regarding thr project let's discuss it over chat
€222 EUR in 3 days
3.8 (16 reviews)
5.0
5.0
User Avatar
Hello sir. Your project attracted my attention at first glance, because I've extensive experience in Python, Machine Learning Programming, NLP, AI. I'm really confident about your project, and very eager to join your project. If we have a chance to cooperate, I'll do my best to provide wonderful result. Looking forward to your response. Best Regards.
€230 EUR in 15 days
5.0 (20 reviews)
3.9
3.9
User Avatar
Hello . Python is my major and I am sure to finish your project in time perfectly Please contact me and lets discuss in detail Thanks
€100 EUR in 3 days
5.0 (2 reviews)
2.2
2.2
User Avatar
Just send me details what can i help you for this projects then a try working it for 1 or 2 days Thanks
€50 EUR in 1 day
0.0 (0 reviews)
0.0
0.0

About the client

Flag of POLAND
Szczecin, Poland
0.0
0
Payment method verified
Member since May 15, 2015

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.