priority logic
« on: October 08, 2005, 09:15:30 PM »
I like the fact that you can exclude certain files etc. and filter out other elements.

How about building on top of that concept and allow you to set specific settings automatically for certain files, like index, sitemap etc. either by wildcard (displaycat.php?* or specifically displaycat.php?5) and allow you to set different settings such as priority, last updated and changefreq that is different from the catch all settings put on all files.

--Tone
Hardware Ranking
[ External links are visible to forum administrators only ]

- find Computer hardware reviews, news and shopping.
Discuss computer hardware reviews on .
Re: priority logic
« Reply #2 on: October 09, 2005, 12:09:53 AM »
No problems - I have a few more - how supported is this product? 

I just bought my copy today as I just found out about it and sofar it seems to work pretty good but it would be really nice to have the above feature request.

Also, how about a notification that google was notified successfully with a ping when a new map is created, and if no changes are made, does it still notify google or is there a check for that?

--Tone
Hardware Ranking
[ External links are visible to forum administrators only ]
- find Computer hardware reviews, news and shopping.
Discuss computer hardware reviews on .
Re: priority logic
« Reply #3 on: October 09, 2005, 06:19:51 AM »
one more thing - is there anyway to have it show you if there is a crawl in progress and the IP of the startup of that crawl or if it is a cron job?

Perhaps even stop you from starting another crawl if one is in progress right now, and offer to stop the one in progress just in case you have a runaway with 'do not interupt' set?  You got a pretty decent set of stats on the left side, that would be a great place to plop such info.

--Tone
Hardware Ranking
[ External links are visible to forum administrators only ]
- find Computer hardware reviews, news and shopping.
Discuss computer hardware reviews on .
Re: priority logic
« Reply #4 on: October 09, 2005, 07:25:39 PM »
Also, how about a notification that google was notified successfully with a ping when a new map is created, and if no changes are made, does it still notify google or is there a check for that?
Google is notified every time when sitemap is created, doesn't matter how many changes (if any) were made. Since Google does not differ sitemaps by the number of notifications, it is not a problem to do it as often as one like. :)
Quote
one more thing - is there anyway to have it show you if there is a crawl in progress and the IP of the startup of that crawl or if it is a cron job?

Perhaps even stop you from starting another crawl if one is in progress right now, and offer to stop the one in progress just in case you have a runaway with 'do not interupt' set?  You got a pretty decent set of stats on the left side, that would be a great place to plop such info.
If you will open the "Crawl" page of the script while crawler is currently running, you will see its progress state and the link to interrupt it.
Re: priority logic
« Reply #5 on: October 09, 2005, 07:36:57 PM »
oh i did not know that - now THAT should be in the docs... btw - why is the top header different on the doc page vs the rest of the program (minor gripe  8)).

Thank you though that is excellent that you can do that - it'd be great if a note in the side bar would show a crawl in progress too but if not, no biggie.

Is there an issue with a crawl being done when a cron'd job kicks in?  Also - what is the exact recommended cron command to run the cron?

Thanks,

--Tone
Hardware Ranking
[ External links are visible to forum administrators only ]
- find Computer hardware reviews, news and shopping.
Discuss computer hardware reviews on .
Re: priority logic
« Reply #6 on: October 09, 2005, 10:15:27 PM »
Is there an issue with a crawl being done when a cron'd job kicks in? 
There is no known issues with that.

Also - what is the exact recommended cron command to run the cron?
A recommended command line is displayed on the "Crawling" page in the "Cron job setup" block.
Re: priority logic
« Reply #7 on: October 16, 2005, 06:13:16 AM »
the priority feature request above - is that something you are considering adding or even perhaps working on now?

--Tone
- find Computer hardware reviews, news and shopping.
Discuss computer hardware reviews on .
Re: priority logic
« Reply #9 on: November 14, 2005, 06:24:02 AM »
Any further news on this?

--Tone
- find Computer hardware reviews, news and shopping.
Discuss computer hardware reviews on .