Wasn't the theory not that milk provides protection from smallpox, but proximity to cows transmits cowpox, which itself provides protection from small pox?

Then again, in a world where >95% of people are farmers and there's no refrigeration (so milk has to be consumed fast), I'm not sure the difference between handling cows and drinking milk is much of a difference.