400 Bad Request
« on: March 10, 2015, 04:12:31 PM »
Hi,

I have about 2000 links but now it only gets 270 links in my site map. I check the log and realise that there is a lot of bad request. How can I resolve this?

Regards
Re: 400 Bad Request
« Reply #1 on: March 10, 2015, 04:29:43 PM »
Hello,

it's possible that the server couldn't handle the number of requests generator crawler is sending. You can try to use "Make delay" setting to slow down crawling and avoid overloading server.