I keep getting the error
*************************
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, webmaster@webefforts.co.uk and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.
****************************
I have found if i set crawling to 100 pages it works. But it struggles to do any more.I have my old xml sitemap up, but they are over 6mb still!
I have tried to do more unlimited crawling and now i constantly get a 500 internal server error page??
Any ideas?