Won't crawl the whole site
« on: June 01, 2010, 11:31:43 PM »
I purchased the Unlimited edition the other day but I've never been able to crawl anymore than 1080 pages. My site is hundreds of thousands of pages long so I'm not quite sure of the problem.

Any suggestions?

Regards,
Mike Richardson
Re: Won't crawl the whole site
« Reply #1 on: June 02, 2010, 07:38:20 AM »
Hello,

could you please PM me your generator URL and an example URL that is not included in sitemap and how it can be reached from homepage?
Re: Won't crawl the whole site
« Reply #2 on: June 03, 2010, 04:28:50 PM »
Hi,
I have same problem with my own site, I have thousands of pages yet only get 20 indexed by my standalone generator. I have included below my generator URL and also an URL that is NOT currently being indexed so you can check if I am doing anything wrong?

Location of generator:
[ External links are visible to forum administrators only ]

2 Examples of URLS - Not currently being indexed:

[ External links are visible to forum administrators only ]

And...

[ External links are visible to forum administrators only ]

I hope you can assist in helping me with this issue many thanks.

Re: Won't crawl the whole site
« Reply #3 on: June 03, 2010, 06:50:08 PM »
Hello,

I will also need login/password to enter generator interface, please PM it to me.