• #indieweb 2017-05-03
  • Prev
    Next
  • #indieweb
  • #dev
  • #wordpress
  • #meta
  • #stream
  • #microformats
  • #known
  • #events
#indieweb ≡
  • ←
  • →
2017-05-03 UTC
# 18:01
gRegorLove
Tangentially, IA is looking to ignore robots.txt more broadly: http://blog.archive.org/2017/04/17/robots-txt-meant-for-search-engines-dont-work-well-for-web-archives/