• #indieweb 2017-04-30
  • Prev
    Next
  • #indieweb
  • #dev
  • #wordpress
  • #meta
  • #stream
  • #microformats
  • #known
  • #events
#indieweb ≡
  • ←
  • →
2017-04-30 UTC
# 00:15
gRegorLove
Internet Archive looking to ignore robots.txt more broadly: http://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/