Ask the community

This is a community support forum where you can ask
questions and interact with other PremiumPress Customers.

Robots.txt

  • Richard
    Richard
    Super Guru 971 points
    September 21, 2015 at 3:05 am

    Having a look at robots.txt file

    Does this look right so SE’s can see the site?

    In advance, many thanks…

    User-agent: *
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /category
    Disallow: /tag
    Disallow: /author
    Disallow: /trackback
    Disallow: /*trackback
    Disallow: /*trackback*
    Disallow: /*/trackback
    Disallow: /*?*
    Disallow: /*.html/$
    Disallow: /*feed*

    # Google Image
    User-agent: Googlebot-Image
    Disallow:
    Allow: /*

    Sitemap: http://site.com/sitemap.xml

    RewriteCond %{HTTP_USER_AGENT} libwww-perl.*
    RewriteRule .* ? [F,L]

  • Mark Fail
    Mark Fail
    Super Guru 12,822 points
    September 21, 2015 at 3:48 am

    i suggest testing it against your Google webmaster tools,

    https://support.google.com/webmasters/answer/6062598?hl=en

Viewing 2 posts - 1 through 2 (of 2 total)