alex [they, il] to Technology@midwest.socialEnglish · 8 months agoThe rise and fall of robots.txtwww.theverge.comexternal-linkmessage-square0fedilinkarrow-up11arrow-down10cross-posted to: technology@beehaw.orgtechnology@lemmy.ziptechnology@lemmy.world
arrow-up11arrow-down1external-linkThe rise and fall of robots.txtwww.theverge.comalex [they, il] to Technology@midwest.socialEnglish · 8 months agomessage-square0fedilinkcross-posted to: technology@beehaw.orgtechnology@lemmy.ziptechnology@lemmy.world
