Read Array of Json objects from file and send to REST
$30-250 USD
Completed
Posted about 9 years ago
$30-250 USD
Paid on delivery
this is GoLang development project:
The to perform the following:
1. Read INI style configuration file with multiple sections
2. Each section will have X number of field value pairs
[personal]
LogginType=DEBUG
Server=[login to view URL]
UploadURL=
ExtractsDirectory=/export/home/xxxx
WebServiceThreads=15
LoggingDirectory=/var/log/
LoggingFileName=[login to view URL]
DefaultKeySpace=XXXXX
UserID=
Password=
[development]
LogginType=DEBUG
Server=[login to view URL]
UploadURL=
ExtractsDirectory=/export/home/xxxx
WebServiceThreads=15
LoggingDirectory=/var/log/
LoggingFileName=[login to view URL]
DefaultKeySpace=XXXXX
UserID=
Password=
3. config file name should be passed in or default
4. log should be created in directory of log with file name
5. the extract directory should be read for 0-Many files and each file should be procced.
6. The extract files large with 15,000 + objects
here is a sample:
[{
"NAME1": "Y",
"NAME2": 1729.0,
"NAME3": "Y",
"NAME4": [
{
"Contact Zip": "33619",
"Date of Contact": "01/21/2015"
}
],
"NAME6": "123456789",
"NAME7": "Walmart",
"NAME8": [
{
"State": "xx",
"Zip Code": "12345",
"Address": "12345 main street",
"Type": "MAILING",
"City": "MyTown"
},
{
"State": "xx",
"Zip Code": "12345",
"Address": "12345 main street",
"Type": "MAILING",
"City": "MyTown"
},
{
"State": "xx",
"Zip Code": "12345",
"Address": "12345 main street",
"Type": "MAILING",
"City": "MyTown"
}
],
"NAME11": "XXXXXXX",
"NAME12": "11/21/2014 14:01:47",
"NAME13": "11/15/2014",
"NAME14": "11/16/1992"
},{
"NAME1": "Y",
"NAME2": 1729.0,
"NAME3": "Y",
"NAME4": [
{
"Contact Zip": "33619",
"Date of Contact": "01/21/2015"
}
],
"NAME6": "123456789",
"NAME7": "Walmart",
"NAME8": [
{
"State": "xx",
"Zip Code": "12345",
"Address": "12345 main street",
"Type": "MAILING",
"City": "MyTown"
},
{
"State": "xx",
"Zip Code": "12345",
"Address": "12345 main street",
"Type": "MAILING",
"City": "MyTown"
},
{
"State": "xx",
"Zip Code": "12345",
"Address": "12345 main street",
"Type": "MAILING",
"City": "MyTown"
}
],
"NAME11": "XXXXXXX",
"NAME12": "11/21/2014 14:01:47",
"NAME13": "11/15/2014",
"NAME14": "11/16/1992"
}]
7. The file should be read either in whole or one object at time to preserve memory
8. each json object should then be passed to the endpoint for processing
9 the sending of end point should be concurrent and the number of concurrency should be configurable by the INI file
10. The endpoint can be either a Channel or REST
Count should maintained as to how many files were read
Count should be maintained as to how many objects where read and sent
All code should be documented. Unit Test should be written for the code
There should be a self standing program to use the classes and features
Dear Sir,
I'm very much delighted to let you know that i did data scraping
with PHP-cURL, Node.js, Selenium from many sites. I just scraped the data from web site
and then wrote the data in mysql database or excel or csv or xml file.
I worked on many similar projects, I have big experience in data mining projects.
I have written hundreds of web scrapers which scrape millions of pages each day.
I'm ready to fulfill your requirement.
I can finish this task in short time, with the best quality. I can assure 100% accuracy.
Please give me the opportunity to do the work.
With Kind Regards,
Debdulal Roy Proshanta