https nginx + Apache 0 pages crawled
« on: April 07, 2017, 03:08:45 PM »
Hello,

I'm running Generator v.7.2. My server setup is nginx on the front end as a back proxy for Apache.  Nginx is serving static content only and redirecting all http requests to https (443).

In my configuration the starting URL is https://mydomain.com. However after starting crawling the process immediately stops saying 0 pages were crawled and no xml sitemap is created. I never had a problem earlier with the http site and Apache only on my dedicated server.

I have read some related topics but couldn't find out the reason. Could you please advise where to look?

Best Regards
Re: https nginx + Apache 0 pages crawled
« Reply #1 on: April 08, 2017, 06:48:11 AM »
Hello,

please let me know your generator URL/login in private message to check this.
Re: https nginx + Apache 0 pages crawled
« Reply #2 on: April 08, 2017, 07:06:29 AM »
Done. If you need any other information pls let me know.

Thanks
Re: https nginx + Apache 0 pages crawled
« Reply #4 on: April 08, 2017, 11:03:36 PM »
Thanks,

XML sitemap is created now after fixing robots.txt.

But why links to HTML and Text sitemaps on a "View Sitemap" tab are starting from http and not from https? It is not a problem since they are redirected to https by the server, however I would think that they should start from https as the link to XML Sitemap. Any suggestions?

BR
Re: https nginx + Apache 0 pages crawled
« Reply #5 on: April 09, 2017, 05:08:01 AM »
Hello,

you need to access generator using [ External links are visible to logged in users only ] as well (not via http://).
Re: https nginx + Apache 0 pages crawled
« Reply #6 on: April 09, 2017, 06:07:47 AM »
I access generator by [ External links are visible to forum administrators only ]

After crawling is finished the links to sitemaps are:

HTML SiteMap
[ External links are visible to forum administrators only ] shown as [ External links are visible to forum administrators only ]

Text SiteMap
[ External links are visible to forum administrators only ] shown as [ External links are visible to forum administrators only ]

1. XML SiteMap File
[ External links are visible to forum administrators only ] shown as [ External links are visible to forum administrators only ]

XML sitemap link and its text is ok. Two others need to look the same in my opinion - to avoid possible confusion.
« Last Edit: April 09, 2017, 06:14:49 AM by capricorn »