I have restarted the crawl several times already. I tried both checking the box "running in the background" and unchecked. It still does not finish. It just stopped at 2002 pages with "Pages left: 292 (+ 8702 queued for the next depth level)".
I have also tried adding the .htaccess file as suggested on another post. When I add this file though, the page comes up as "500 internal server error".
Your immediate assistance is highly appreciated.