# Allow all User-agent: * Disallow: # robots.txt for pc-freak.net # Slow down bots # User-agent: * # mass download User-agent: wget User-agent: webzip User-agent: webmirror User-agent: webcopy User-agent: netants User-agent: getright User-agent: webcapture # User-agent: Libwww-perl # Disallow: /files/ Disallow: /pictures/ Sitemap: https://www.pc-freak.net/sitemap.xml # https://megaindex.com/crawler User-agent: MegaIndex.ru Disallow: / User-agent: megaindex.com Disallow: /