Sitemap Generator Forum

Hello,

running from the command line normally has less restrictions.
The command to run is shown on the Crawling page of sitemap generator.

1- I was more talking if I should put "php -a /path/runcrawl.php" or "php -f /path/runcrawl.php"

2- also, what is the recommended file permission for runcrawl.php?

3- Are you planing any fix for this issue?

4- I'll have one last question please: regarding the <lastmod> tag, it generates current date for dynamic pages, is there a way to be more accurate?

1. php -f /path/runcrawl.php

2. 644

3. In case if server configuration restricts scripts from running longer than certain amount of time, either resuming is required, or running in command line.

4. In case if server returns Lastmod time in http header, it will be also included in sitemap. Otherwise, current time is used.

OK, thank you.

Hi,

I'm back with one other question please.
I'm now running the crawl with command line and it works great.
But sometimes when my computer goes on saving mode, the crawl stops and doesn't resume. I quit the Terminal (MAC OSX) and when i run the commande again, it says <h4>Already in progress. Current process state is displayed:</h4>
so is there a way to completely interrupt the crawling?

Thanks a lot

Hello,

you need to  also remove generator/data/crawl_state.log file to reset it.

OK, thank you very much

I have about 20 000 files to be crawled, is it normal that it takes about 35 minutes to complete the job?

Hello,

The crawling time itself depends on the website page generation time mainly, since it crawls the site similar to search engine bots.
For instance, if it it takes 1 second to retrieve every page, then 1000 pages will be crawled in about 16 minutes.

Hello,

Good to know
Thousand thanks for your support.

Cheers

Hi again,

I have recently received an email informing me about a new version 7.1, i don't see the link to download it, where can I find it?

Thanks

You can see the red box at the top of this forum page, saying:
-----
Hello, you are a registered customer with the license assigned to email: ...
Your download link for the latest version of sitemap generator is: ...

I don't see this red box! I even searched within the source code the word "assigned" and didn't find it!! 

just updated to 7.1 and getting 500 error checked file permissions added htaccess with the one line
and memory_limit=512M
max_execution_time=50000

ok sorted there was a server hiccup at same time i installed new version just co-incidence

I'm glad it works now.