Using request and cheerio, create a node.js script to login to portal a keep session alive

Completed Posted 5 years ago Paid on delivery
Completed Paid on delivery

We have a target B2B portal secured by username/password. We want a node.js/express script to login to the portal and keep the session alive as long as possible and then log back in if the remote server unavoidably destroys the session.

The expected budget for this project is $50. There are follow-up projects available which will ultimately lead to scraping a lot of data out of the remote site and loading the data into MySql.

---------------------------------------------------------------------------

THE FOLLOWING INFORMATION IS FOR CONTEXT ONLY AND IS NOT REQUIRED IN THIS STEP/BID

The express application will *eventually* provide a route through which we can HTTP query our server which will in turn request a secured page from the remote server (i.e. scrape the remote host) and return data back to the browser:

browser <---> *our node.js/expressApp <---> remote server

However, THIS IS NOT REQUIRED IN THIS STEP/BID AND IS ONLY EXPLAINED HERE FOR CONTEXT.

---------------------------------------------------------------------------

THE SCRIPT SHOULD TAKE THE FOLLOWING OUTLINE:

const request = require('request-promise')

const express = require('express')

var isLoggedIn = async (r) => {

var html = await [login to view URL]('[login to view URL]')

return [login to view URL]('Your username is:') // only appears if we have an active session

}

var login = async (r) => {

var html = await [login to view URL]('[login to view URL]', {username: 'superuser', password: 'coolpassword'})

if (!await isLoggedIn(r)) throw "Could not login!"

// save the session/cookies to the database

await [login to view URL](`update cache set cookies = '${[login to view URL]('[login to view URL]')}'`)

}

var keepalive = async (r) => {

await [login to view URL]('[login to view URL]') // resets the idle timeout on remote server

setTimeout(keepalive, randomTimeBetween4And6Minutes)

}

var connect = async () => {

// create a session

var jar = [login to view URL]()

var r = [login to view URL]({jar: jar})

// restore last session/cookies

var cookies = await [login to view URL]('select cookies from cache')

for (cookie of cookies) { [login to view URL](cookie), '[login to view URL]') }

// login if not already logged in

if (!isLoggedIn(r)) login(r)

keepalive(r) // keeps the session alive

}

connect()

// Express/HTTP application and routes

[login to view URL]('/accountNumber', async (req, res, next) => {

var html = await [login to view URL]('[login to view URL]')

var accountNumber = [login to view URL](/'Your Account Number: (\d+)/)[1]

[login to view URL]({status: 'success', accountNumber: accountNumber})

})

[login to view URL](3000)

---------------------------------------------------------------------------

REQUIREMENTS: THE SCRIPT SHOULD DO THE FOLLOWING

Reference the above sample script pseudocode and see that the following steps are required:

* When the express application starts, load cookies stored in MySql from a previous execution, i.e.

var jar = [login to view URL]()

var cookies = await [login to view URL]('select cookies from cache')

for (cookie of cookies) { [login to view URL](cookie), '[login to view URL]') }

* The script should check to see if the session it restored from MySql cookies is still a valid/logged in session. Do this by requesting a page that is generally secured and requires a user to be logged in. Look for indicators in the HTTP response that indicate that the session is logged in or not. If the session is not valid, then try to log in again

* If we login, then save the new session/cookies to MySql

* Keepalive: Request a page periodically to reset the session timeout counter on the server.

* A function to look for an indicator in the HTML of whether our session/cookies are still logged in or not. i.e. [login to view URL]('Your username is:') // only appears if we have an active session

---------------------------------------------------------------------------

A WORKING COPY OF ONE OF THESE SCRIPTS IS AVAILABLE HERE:

[login to view URL]

Express JS JavaScript MySQL Node.js Web Scraping

Project ID: #17268052

About the project

6 proposals Remote project Active 5 years ago

Awarded to:

shapelessds

Highly dedicated, experienced. Have experience creating scraping services, processing thousands of press releases per day.

$50 USD in 3 days
(0 Reviews)
0.0

6 freelancers are bidding on average $104 for this job

CakephpJobs1

Hi, I have used this technique to scrap data from website When i have scrapped data from website where we need login I have good exp there are several factors with each domain with respect to there security a More

$166 USD in 3 days
(38 Reviews)
6.8
vranjan

Hi, I read through the job details extremely carefully and I am absolutely sure that I can do the project very well. Expert in BOTH Backend and Front-end Programming, PHP - OOP, MVC, REST API, HTML/CSS/JAVASCRIPT More

$50 USD in 2 days
(8 Reviews)
4.8
mohammadullah177

I have 6 year experience Freelancer,up work,Fiverr & 99design market place I have seen your project that i can to do easily because I have many experience to Graphic Design,Webdesign,Web Develop & programming .So I cou More

$155 USD in 3 days
(1 Review)
2.7