Sitemap Generator Forum

If I use the online generator my site, including my PHP pages are spidered.

When I use the stand alone version, only the HTML pages are spidered.

I along with others, have found parts of the instructions very difficult.
Especially permissions.  I spent over 4 hrs to learn what permissions were as compared to attributes.

I have created a file called "permission.php"
These instructions were put inside and them I click run PHP.
<?php
chmod("/sitegenerate/config.inc.php", 0666);
chmod("/sitegenerate/data", 0777);

?>

I then re attempted a crawl, with the same results. The PHPs are not spidered in the stand alone program.

If I reload the files and start from the beginning and try to crawl my site, after about 2 minutes I get "page not found in the window.  We I go to recrawl, it shows maybe 200 files found. So I press continue. Again after 2 minutes, I get the same message. If I recrawl again, the system shows another 150 files added. Again after about 2 min "page not found" appears"

The problem is that in my PHP files there are approx 30,000 entries. I would have to reload crawl for the next month to pick up all of the files.

My Site is [ External links are visible to forum administrators only ]

There has got to be an easier way.

Ron

Hello Ron,

I've replied to your email: you specified the domain in initial URL with "www." prefix, while php pages have domain name without "www." in your case, which is considered to be different domain name.

I now, have about 15 hours into trying this program. I am still having problems. I want to eliminate any thing on my side. In the last post, I talked about permissions.php.

Is this the correct way to issue the proper commands to create the proper configurations?

Now, I have one more problem to track down. It appears that I have some terrible sites that can not be spidered, or I have a program that is not working properly?

I have used the 'Online Generator"  for 3 of my sites and in all cases, the site maps are incomplete.

This is what I expected.
1)  Download files to your computer.
2)  Create three folders on your host server
      sub folder (1) =  Generator
      sub folders in Generator  (2)    "pages" and "data"
3)   Upload the downloaded files with your FTP.
4)  Change the "permissions" with one of the examples below. In this case, showing a php,txt and HTML file.
5)  Go to your browser and input your "domain name"/generator/index.php
6)  Press Run!!

This whole project should have been over with, in less than 45 minutes.

Ron

Hello Ron,

I'm not sure if I understood you correctly, but re:

I have used the 'Online Generator"  for 3 of my sites and in all cases, the site maps are incomplete.

please let me know the starting URL you use and an example URLs that are not included in sitemap (please note that they *must* be from the same domain, including www. prefix and note that online sitemap generator crawls no more than 500 pages).

A raving testimonial to your for the online generator. You system found serios flaws in my site. I corrected them and the online generator works great.  Anybody would be a fool,  not to a least try the online generator for themselves. You might find  out why your pages are not getting spidered.

Now for the stand alone generator. After reading almost ever post, I have assured myself that I am not alone in the programming problems neccessary to make this program function properly.

Is there any place in the posts,  that show an example of an actual PHP.ini file that shows the scripts?
Is there any place in the posts, that show an example of an actual Preference file that show the scripts?
Is there any posts that show an example of a file with the scripts for memory etc.

I like many others, am not a programmer and never have been. Additional programming  is what this program requires to make it run. It is definitely not a one size fits all as shown by the many posts that have any number of problems installing the Stand Alone Generator.

If I pay for the installation, do you only upload the basic files to my host?  Or, do you include all of the necessary files mentioned above, to make this program work? 
Lastly, do you test to make sure it works or will I still have additional unforeseen problems?
Ron

Hello Ron,

thank you for your kind words!

re: installation
The corresponding lines in php.ini (usually root ssh access is required to modify php.ini at your host) look similar to:

max_execution_time = 180     ; Maximum execution time of each script, in seconds
memory_limit = 32M      ; Maximum amount of memory a script may consume (8MB)

The installation service includes uploading the files via ftp, setting corresponding permissions and cron job setup (in case if hosting control panel access is provided).
If you will provide root access as well, we'll check the php configuration at your host.