Form > Crawler doesn't follow select dropdown links
« on: September 10, 2017, 06:35:27 AM »
Hi,

A question here:
I've got a SELECT dropdown & form on my homepage visitors need to use in order to select a town. The problem is the XML-sitemap doesn't crawl the different select options (all unique url's) but it should do that. I'm only getting back some general pages (links in footer / header menu).

Site: [ External links are visible to forum administrators only ]
OR in English: [ External links are visible to forum administrators only ]

Sitemap: [ External links are visible to forum administrators only ]

How can I make sure the sitemap is crawling through all the subpages...
The selection a user makes is stored in a cookie so he/she only needs to select a city once. Maybe that's a problem (but I don't see ANY cities in my xml sitemap)...

Anyone to the rescue?
Re: Form > Crawler doesn't follow select dropdown links
« Reply #1 on: September 11, 2017, 05:44:24 AM »
Hello,

all pages must be accessible via <a href=""> links and should be working with javascript disabled in the browser to allow crawler bots to find them.
Re: Form > Crawler doesn't follow select dropdown links
« Reply #2 on: September 11, 2017, 07:59:29 AM »
That did the trick indeed. Thanks for the quick reply.
Should all these links be visible or can I "display:none" them so they are only visible for bots?
Or will SEO punish me for this kind of trics.
Re: Form > Crawler doesn't follow select dropdown links
« Reply #3 on: September 11, 2017, 08:36:35 PM »
Those can be invisible for the sitemap generator bot, although it's not recommended for search engines bots.