The Disadvantages of Speed: Finding Exact Match Domains in Drop Lists

I recently wrote a post on the advantages of speed specifically dealing with the ability to find exact match domains. One of the disadvantages of speed is that of the classic hammer problem. If you have a hammer, everything looks like a nail. Because lookup speeds are very fast, I made the assumption that I could just pound away. Eventually, though, that led to some insurmountable speed problems and would force more horizontal scaling. Because the lookups were so fast, I assumed that the number of lookups could be egregiously large without greatly damaging performance. I. Was. Wrong. It hit me over New Years Eve night that I had been looking at the problem all wrong. The lookup data was structured in a way that required the massive lookups. Subsequently,...