User-agent: * Allow: / Allow: /archives/ Allow: /about/ Disallow: /vendors/ Disallow: /js/ Disallow: /css/ Disallow: /vendor/ Disallow: /content.json Disallow: /404.html