I'm looking for an expert of scraping
3
Not solved Solved
over 1 year ago

Hello. Do you know an expert of scraping. Il need to scrap the website of meilleursagents.com but my IP address is blacklisted very quickly. I don't know the manipulation to change my IP during the scrap. Thanks for your help...

over 1 year ago

I'm no expert but wouldn't setting up a list of proxies and using those in turn work? I don't think you can reset your IP address just using code, since it depends on your router.

I found a StackOverflow answer with a little more details: here.

over 1 year ago

Have a look to http://scrapinghub.com/crawlera/. The doc is there: http://doc.scrapinghub.com/crawlera.html#ruby (I suggest you to use typhoeus).

Envoy !

(and ping me I you need)

Cancel
Submit your answer