Crawl the iTunes store app, get the price, app info and store it in a MySQL DB.
We would like to keep track of price and version change, so we can inform our users on our website.
==========================
Here is what we need to save:
Crawl and update fast (once a day):
- App Name
- App Price history.
- App ID
- User rating
- Number of user rating (current and overall)
Crawl slower (update once a week)
- Compatibility (which iphone/ipad, so we can filter by compatibility by a sql request)
- Categorie
- Language
- Developer info, such as name, email, website, etc...
- App version number.
- App screenshots URL.
- App description.
==========================
Functional:
We need to update the list once a day, so it should be really fast.
so that we can inform our user about price changes and version update.
==========================
Work should be done in clean PHP code (easy to read and modify).
You need to crawl information on the around 700 000 app of the applestore.
The code should be able to run on a linux server.
The script should not do something which could looks like a DOS (to prevent our server of being banned).