Crawler give me nothing
« on: January 28, 2008, 03:04:22 AM »
Hi,

Just bought and installed Site Generator, setup config and proceeded to run the crawler manually.

I've noticed on this forum that many people have issues with only a limited number (sometimes 5) of pages are being indexed before an error or such. I don't even get a single page showing up as being indexed.
There are no Start, Stop or Resume buttons on my Crawler page. I just have a Run button with a checkbox for "running in the background".

Where are these other buttons supposed to be?

The script also bumps me back to the Configuration page after exactly 90 seconds. I've run it 5 times now and it's always 90 seconds before I'm bumped back to the Config page.

Server is dedicated with only one site running.

What's the go?
Re: Crawler give me nothing
« Reply #1 on: January 28, 2008, 12:16:37 PM »
More investigation reveals

The only requirement for CDSEO to be compatible with sitemap generators, is for the sitemap generator to respect the "base href" html tag.

I assume that XML Sitemap Generator does respect base href html tags.

As I am running CDSEO on my site, this may be applicable

Could someone confirm for me please
Re: Crawler gives me nothing
« Reply #2 on: January 29, 2008, 11:14:26 AM »
This forum appears deserted.

Not one response even from admins?

Well as an update if anyone's bothering to view this thread, I finally got the mapper to start crawling but after 6 hours it hung. Now I only have 1000 pages in my site but not having run a site mapper before I assumed that 6 hours would be more than enough time for a small site to be indexed.

However there are stil no sitemaps to view so again I assume nothings happened.

Is there anyone out there?????????????????
Re: Crawler give me nothing
« Reply #3 on: January 29, 2008, 09:55:13 PM »
Hello,

sorry for delay. Please let me know your generator URL in private message so that I can check that.