alex [they, il] to Technology@midwest.socialEnglish · 2 years agoThe rise and fall of robots.txtwww.theverge.comexternal-linkmessage-square0linkfedilinkarrow-up11arrow-down10cross-posted to: technology@beehaw.orgtechnology@lemmy.ziptechnology@lemmy.world
arrow-up11arrow-down1external-linkThe rise and fall of robots.txtwww.theverge.comalex [they, il] to Technology@midwest.socialEnglish · 2 years agomessage-square0linkfedilinkcross-posted to: technology@beehaw.orgtechnology@lemmy.ziptechnology@lemmy.world
