04/07/2016 at 12:19 pm
#372040
Participant
Thanks for your response Steve – However, I have to wait until whenever the bots decide to spider the site – a sitemap let’s me tell the search engines exactly when I have updated it – so there are significant SEO benefits.
Also another feature request would be to add a “NoIndex” or “Index” option on the various levels that the directory can be accessed – this way I can avoid duplicate content on the site.
If this were added to the Advanced settings tab against each of the permalinks, this would be awesome. We could then tell Google only to index one version of the directory entry, rather than the numerous ways it can be indexed just now.
Do you know if anyone has created a custom robots.txt file to handle the duplicate content issue?
Thanks
