#dev 2021-07-16

2021-07-16 UTC

[KevinMarks] a common pattern is reference counting, so you do `d[url]=d.get(url,0)+1` then you have a dict with url and number of reference to it, which you can then maybe sort by count to decide which to crawl first