Sitemap Generator Forum

Hello,
last time i created the site map was 7 months ago and i added a lot of pages to my site because of this i decided to run the generator again.
the problem is that the generator stopped in the middle of the scan (this is the 3rd time happening) as you see in the attachment appear zero (before its stopped it was in level 4 and was 18k pages scanned from 40k).

what can be the problem?

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 

I have the same problem. What are the recommended settings?

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 

i spoke with the hosting and we applied with no limit for execution time and  512M for the memory.

i will run again the script now and will update you.

Thanks

Its not a good idea to make it unlimited. I think for example if there is bad code that gets itself into an infinite loop it my crash the site altogether. So my guy tells me to make it really high but dont make it unlimited.

Its not a good idea to make it unlimited. I think for example if there is bad code that gets itself into an infinite loop it my crash the site altogether. So my guy tells me to make it really high but dont make it unlimited.

i will return it to normal settings later.

Thx

Hello,

it looks like your server configuration doesn't allow to run the script long enough to create full sitemap. Please try to increase memory_limit and max_execution_time settings in php configuration at your host (php.ini file) or contact hosting support regarding this.
 

Hi
as i said above that i changed the settings, and after almost 4 days running the generator its reached around 90k scanned and 60k added to sitemap (level 4). unfortunately now i see that scan 300 pages level 1 
its look  like its stopped and start from the beginning and this is the 3rd time is happening.

i can't generate the sitemap.... pls help

Thx

Hello,
in this case I would recommend to run generator in command line if you have ssh access to your server.

Hello,
in this case I would recommend to run generator in command line if you have ssh access to your server.

I did this also before and i had the same problem...
i have VPS... maybe to run the script from this VPS to scan my site? this will work?

Hello,

what are the last lines of output you get when running in command line?

Hello,

i still waiting to finish the scan, and i hope this time will not stop
pls check the attachment its normal that scanned pages different from the pages added to sitemap?

Thanks

Hello,

yes, it means that some pages were not added (in case if robots meta tag blocks it or it's redirected or canonical meta tag points to a different URL).

Hello,

what are the last lines of output you get when running in command line?

below :

[root@host ~]# /usr/bin/php /home/mydomain/public_html/generator/runcrawl.php
Content-type: text/html

Resuming the last session (last updated: 2018-03-15 20:30:44)
88268 | 0 | 11,532,179.8 | 66:54:04 | 0:00:00 | 5 | 109,643.1 Kb | 58490 | 2793 | 109643
88269 | 19233 | 11,532,307.4 | 66:54:10 | 14:34:39 | 5 | 109,944.2 Kb | 58491 | 2793 | 109944
88271 | 19231 | 11,532,563.1 | 66:54:16 | 14:34:33 | 5 | 109,955.4 Kb | 58493 | 2793 | 109955
88273 | 19229 | 11,532,818.4 | 66:54:22 | 14:34:28 | 5 | 109,964.6 Kb | 58495 | 2793 | 109964
88275 | 19227 | 11,533,074.3 | 66:54:29 | 14:34:23 | 5 | 109,975.6 Kb | 58497 | 2794 | 109975
[root@host ~]#

hi
its look like stack in specific page, every time i am clicking Resume last session... nothing happening

@Admin  i PM you with my details, can you pleas have a look

Thanks
 

Replied to your PM.