Viewing 2 posts - 1 through 2 (of 2 total)
Richard

Robots.txt

Having a look at robots.txt file Does this look right so SE's can see the site? In advance, many thanks... User-agent: * Disallow: /cgi-bin Disallow: /wp-admin Disallow: /wp-includes Disallow: /wp-content/plugins Disallow: /wp-content/cache Disallow: /wp-content/themes Disallow: /category Disallow: /tag Disallow: /author Disallow: /trackback Disallow: /*trackback Disallow: /*trackback* Disallow: /*/trackback Disallow: /*?* Disallow: /*.html/$ Disallow: /*feed* # Google Image User-agent: Googlebot-Image Disallow: Allow: /* Sitemap: https://site.com/sitemap.xml RewriteCond %{HTTP_USER_AGENT} libwww-perl.* RewriteRule .* ? [F,L]

Created: September 21, 2015 at 3:05 am

Copyright © 2010-2020 PremiumPress Limited.

secure payments