Sitemap Generator Forum

I would have posted this under the bug reports, but that forum refuses to allow me to post. In fact, the Unlimited Sitemap Generator forum refuses to allow me to report, and I am referring to the paid version, not the free version.

The Sitemap Generator reads content in <script> tags, and it should not. This is causing all sorts of bogus entries to get read in, and then my "Broken Links" page is loaded with a bajillion errors about pages that should never have been treated as URLs.

For example, if the page, [ External links are visible to forum administrators only ] has:
<script type="text/javascript">
  var somelink = 'a.html';
  function DoSomething() {
    document.getElementById.innerHTML = '<a href="' + somelink + '">text<\/a>';
  }
</script>

Then a ton of links to this appear:
[ External links are visible to forum administrators only ]" + somelink + "

This creates a TON of superfluous extra page hits that never should have happened in the first place.

Secondly, when I view the Broken Links tab, these URLs are not properly scrubbed, so the HTML of that link above comes out to:
<a href="[ External links are visible to forum administrators only ]"%20+%20somelink%20+%20""> ... </a>
Which is broken HTML and doesn't link to the fake link in question

Steps to fix:
1) The Sitemap Generator needs to ignore content inside of <script> tags
2) Scrub your content (with htmlentities($url); if nothing else) before you put it into the <a href="">

Hello,

you would need to use forum login/password that was sent to you after ordering generator license to be allowed to post on those sections.

Make sure that you have javascript entries added as:

<script type="text/javascript">
<!--
  var somelink = 'a.html';
  function DoSomething() {
    document.getElementById.innerHTML = '<a href="' + somelink + '">text<\/a>';
  }
//-->
</script>

I cannot make massive changes throughout 30+MB of website code to satisfy this condition, especially since it would cause problems elsewhere. That style of commenting JavaScript is effectively obsolete, due to XHTML.

We are using the XHTML method // <![CDATA[ .... // ]]> surrounding structure for our JS. Is it possible to make your parser <![CDATA[ aware so that it will not parse content inside of these regions?

What is an example page link where that code is located?

[ External links are visible to forum administrators only ]

View source. Line #449 is the "link" that is being picked up improperly.

Hello,

are you using standalone sitemap generator? You can add this in "Exclude URLs" setting:
all_link

Yes, I have the standalone sitemap generator. Adding that will fix my problem, but it does not fix the bug in the crawler.

You can easily fix this in the code with something as simple as this before crawling:
$content = preg_replace("<!\\[CDATA\\[.*?\\]\\]>", "", $content);
and then it will no longer be an issue.