深圳SEO公司
欢迎您光临浏览!

SEO案例robots写法

Robots是什么
robots可以理解为搜索引擎的协议,当蜘蛛抓取网站时第一时间是检查网站根目录下有没robots,如有会优先访问robots里的内容。
Robots 的作用是什么
robots的作用 是告诉搜索引擎哪些页面是允许抓取和哪些页面是不允许抓取,比如网站的数据库,后台,脚本等等可以禁止蜘蛛抓取。说一个高级点的知识,robots还可以用作屏蔽一些重复性页面。
Robots写法
User-agent: * 意思为允许或许搜索引擎抓取
Allow: 允许抓取
Disallow: 禁止抓取
*:表示全部或所有
$:结束符
#:用来注释
举例:
Disallow:/ 表示禁止抓取整站页面
Allow:/ 允许抓取整站内容
常用网站程序Robots.txt写法
织梦Robots.txt写法
User-agent: *
Disallow: /plus/ad_js.php
Disallow: /plus/advancedsearch.php
Disallow: /plus/car.php
Disallow: /plus/carbuyaction.php
Disallow: /plus/shops_buyaction.php
Disallow: /plus/erraddsave.php
Disallow: /plus/posttocar.php
Disallow: /plus/disdls.php
Disallow: /plus/feedback_js.php
Disallow: /plus/mytag_js.php
Disallow: /plus/rss.php
Disallow: /plus/search.php
Disallow: /plus/recommend.php
Disallow: /plus/stow.php
Disallow: /plus/count.php
Disallow: /include
Disallow: /templets
Disallow: /*?*
Disallow: /*list*
Disallow: /tags.php
Disallow: /plus/flink_add.php
Disallow: /plus/flink.php
sitemap: http://www.smiseo.com/sitemap1.html
WordPress的Robots.txt写法
User-Agent: *
Disallow: /*#comments
Disallow: /?s=
Disallow: /?author=1
Disallow: /zm/
sitemap: http://www.smiseo.com/sitemap1.html
Robots禁止抓取案例
淘宝:由于该网站的robots.txt文件存在限制指令,系统无法提供该页面的内容描述
淘宝Robots写法
User-agent: Baiduspider
Disallow: /
User-agent: baiduspider
Disallow: /
Robots注意事项
Disallow的写法一定要谨慎,少了一个/会出现很多问题或多了一个*也会产生很多问题
通匹符的用法一定要正确。
本文出自深圳SEO公司,未经允许不得转载:深圳SEO-微笑SEO服务公司 » SEO案例robots写法
分享到: 更多 (0)

评论 2

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
  1. #1

    I pay a quick visit everyday some web pages and sites to read articles or reviews, except this web site presents quality based writing.

    vardenafil generic8个月前 (04-03)回复
  2. #2

    viagra online polskabuy generic viagra in canadareview for viagrahow does viagra work in pulmonary hypertensionjelly viagra buycomprar viagra en bogota colombiaget viagra from gpviagra online overnight deliveryviagra dosingviagra to buy ukcomprar viagra en jerezrecreational viagra use sex storieswhat is viagrapremio nobel viagrabest generic viagra canadaoda al viagra
    silagra viagra generic
    viagra online without where can i buy viagra
    buy generic viagra online uk next day delivery
    can you drink viagrahay similar de viagrapros and cons of generic viagrakan man ta 2 viagrapeople who bought viagra onlineviagra shops in singaporeviagra does it work for everyonequanto tempo demora viagra fazer efeitoviagra calciumviagra schon in jungen jahrenis cialis better than viagravirtual viagra tumblrwalgreens viagra couponschi dinh viagraviagra with prescriptionwhere can i get viagra in australiaviagra causa necrose

    Coreyson5个月前 (06-08)回复