Error Generating Sitemap
« on: June 30, 2011, 08:10:25 PM »
I've ran the tool before and had nice results. However, I've reinstalled the script and now its not being so nice.

I've removed URL to avoid violating forum terms. Hopeful the info provided is helpful.

An error occured
There was an error while retrieving the URL specified: [ External links are visible to forum administrators only ]
HTTP headers follow:
date: Thu, 30 Jun 2011 18:59:45 GMT
server: Apache
content-length: 2196
location: [ External links are visible to forum administrators only ]
vary: Accept-Encoding
connection: close
content-type: text/html

HTTP output:

Main Generator Directory is set to 777

Starting URL:
[ External links are visible to forum administrators only ]

Save Sitemap to:
/home/mynickname/----.com/sitemap.xml   
NOTE permissions is set to 666 for all--
sitemap.xml, sitemap_images.xml, sitemap_video.xml

Your Sitemap URL:
[ External links are visible to forum administrators only ]'

HTML Sitemap Filename (full name with path:)
/home/mynickname/-------.com/gen/data/sitemap.html


Please note that I do run a PHP Script to protect my forum and several of my form pages, but it is NOT hooked into my main index.php page. I do know that people that are NOT whitelisted or found doing bad behavior will get looped to 127.0.0.1... However, the script is not hooked into my home page. I also did run xml sitemapper before with this same script, but my admin may have set something different in the ini file for the script.

Thank you,
M
Re: Error Generating Sitemap
« Reply #1 on: July 01, 2011, 12:50:43 AM »
I fixed it. I was my script indeed. I do have a question though. When it comes to PHPBB forums. What is your suggestion to keep the sitemap clean of clutter from the forum (even while using the PHPBB settings you offer it is still indexing a bit to much for my liking. I am interested in having the first post/replies page of each topic indexed. Do you happen to have a cheat sheet for configuring and tweaking the mapper even more for PHPBB. I guess what I am trying to say is I have an introduction area on my forum and I do not want that indexed. What would I add to exclude that section?  However I do want the tutorials section indexed, but only the first page of that post with its replies, and not all 10 pages of all the replies.  I apologize for my lack of terminology, but its is the best way I can describe it. THANKS!
« Last Edit: July 01, 2011, 01:25:50 AM by MicheleLee »
Re: Error Generating Sitemap
« Reply #2 on: July 01, 2011, 12:35:28 PM »
Hello,

there is a special "Exclusion preset" in generator configuration for phpBB forums that I recommend to use to avoid crawling of noise content.
Also, if you want to exclude specific section/subforum you need to add it in Exclude URLs too.
Re: Error Generating Sitemap
« Reply #3 on: July 01, 2011, 02:23:42 PM »
Thank you for your reply. Yes, I am aware of the "Exclusion preset" but to those of us whom are not familiar with what those strings, it is hard for us to know what to include and not to include.

Example, when I view my forum I can see:
viewforum.php?f=17 (which is the forum area of Tutorials)
viewforum.php?f=25 (which is the forum area of Introductions)

So, what I am curious about is-- if I wanted to exclude the Introductions area should I add viewforum.php?f=25 to the exclusion list? I am not fluent in PHP so your ideas are truly appreciated and I am sure other lurkers that seek this information today and in the future will as well.