Also, making decisions based on wishy-washy difficult-to-quantify human traits like "genius" and "sense of taste" seems unnecessarily risky, compared to making them based on measurements from repeatable tests.

I'm not advocating design by committee. I'm advocating justification with data. The designers should propose candidate designs. But the decision about which one to go with (or whether to redesign at all) should be supported by whether going with it furthers the business's goals.

You point out the need to pick the right measurement, and I agree, but it could be as simple as "What's the effect on conversion?" It has to be something! You can't just say "We're re-designing because I'm a HMI genius."

When a company is capable of replicating those black-box functions as an algorithm or business process, then it will be ready to launch the era of automated design.

Until then, the best known way to design something nice is to hire a talented designer and trust their instincts.