#linkedin User-agent: * disallow: /partners/ #scrapy User-agent: Scrapy disallow: / #sitemap Sitemap: https://www.chiletrabajos.cl/sitemap.xml User-agent:* Disallow: