Sitemap Generator Forum

...as header. My (dynamic) robots.txt file does NOT bar the new
sitemap file. I have also appended the Sitemap: fullpathURL. Is there
something I must now check or do you have a problem over there?
----best wishes, Robert

Please PM me your sitemap URL and robots.txt link so that I can reproduce the issue.

Oleg----
Done.
----best wishes, Robert

Oleg----
Tried using a non .gz flavour and also put the sitemap
into the root... as your 'test' [sic] tool seems to be
a facility more akin to the 'tail wagging the dog' in
terms of robustness. However even THIS trick did
not fix your tool which STILL says that my now
root-based sitemap.xml file is disallowed by the
robots.txt file (no it isn't). It's late g'night.
---best wishes, Robert

Hello,

I get no errors for your sitemap: [removed]

Oleg----
That is because I decided to help matters by placing the file
in the root despite the new area being not quite ready for
full production. [removed/with thanks]
----best wishes, Robert

[postedit by Robert: unnecessary text removed]

Yes, that is one of the requirements of xml sitemap protocol: [ External links are visible to logged in users only ]

Oleg----
You've already seen that I've moved my sitemap.xml file to the
root of the site (which has new and old model content areas as
described in another thread). However the initial problem is
STILL resident. Your tool inexplicably says that my robots.txt
file appearently disallows even the root-resident sitemap.xml
file. You know the URL of my site so please advise why your
tool seems to have got it so wrong. (My site URL should not
appear in live forum threads, I much prefer it that way).
I run a dynamic robots.txt file functionality. It's actually a
PHP file;~) Maybe it is this that is messing up your tool? It
shouldn't affect it but I mention this just in case.
----best wishes, Robert

Oleg----
Fixed it;~) My enforced requirement to move the sitemaps.xml
file to the root had repercussions. Already existing within the
disallowed section of the dynamic robots.txt file is a standing
no-no for /site as referring to a directory /site/foo.bah that
was previously in use. However the logic extends that into
disallowing the sitemap.xml file too... hence your tool's report.
Success - my thanks;~)
----best wishes, Robert

I'm glad it is sorted now

Oleg----
Agreed and thanks again for the help.
Now I have a mailbomber at the ramparts...
Still, looking at the bright side of it, I get a massive
list of broken or vulnerable sites to add to the filters;~/
May I ask how long it will be before the 'next update'
ie the one with the user-option to remove branding
remnant in the generated output? Days/weeks/months?
----best wishes, Robert