## robots.txt for Hostplus PDS # Sitemap: https://pds.hostplus.com.au/sitemap.xml # Crawl Delay to keep the traffic at a steady stream Crawl-delay: 21 # Pages to disallow User-agent: *