> How do you differentiate code that is using the wrong data structure from the one that is solving a hard problem?

You don't until you have to. At the limit, every piece of code is using the wrong data structure; the hardest problems get addressed by building custom hardware to maximize speed by stripping away every other possible concern in the transformation of the symbols and building as many of those as you possibly can force to work in parallel.

Almost nothing is that important, so when your program is too slow, you search for what is taking the most time and do the cost/benefit analysis of re-representing the problem in a way that matches the existing hardware better (which encompasses everything from requiring fewer steps in the computation to caching the right data to caching the right data in the right location, i.e. L0-L1 cache performance, all the way to, sometimes, pushing the computation onto a GPU or out into multiple parallel execution nodes in a cluster if the problem demands it).

None of this is black-and-white and there are degrees. You can certainly, during the design phase, look at a problem and go "That's probably going to want a cluster." But the point of Pike's 1 and 2 is "If you don't have a firm reason to believe a more complicated approach should be taken, err on the side of the simpler approach because it's easier to grow a simpler approach than move a complicated approach."

> At the limit, every piece of code is using the wrong data structure

So, at the example you commented earlier, you inspect every single function, sorted by execution time until the program was fast enough?

> do the cost/benefit analysis of re-representing the problem

How do you do that without knowing the speed you can get with another representation? You rewrite it on every possible way and test?

> But the point of Pike's 1 and 2 is "If you don't have a firm reason to believe a more complicated approach should be taken, err on the side of the simpler approach

Yet we are what, 5 messages down a thread where people vehemently deny you can't have reason to believe anything.

> Yet we are what, 5 messages down a thread where people vehemently deny you can't have reason to believe anything.

No, nobody's denied that at all. https://news.ycombinator.com/newsguidelines.html "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith."

The argument made by eschneider and reenforced by myself is "People usually guess wrong when they try to add performance optimization at the design phase," not "epistemology is dead." It's a soft rule from experience, not a black-and-white hard rule; most everyone who's been programming for years has a story about the time they built a complicated gadget that ended up being a waste of time because it either wasn't on the critical path where the bottleneck showed up or users didn't choose to use the software in a way that aggravated that bottleneck anyway.

I'll share my own. Way back in the day I hammered for weeks on getting the physics model right for a nascent game engine. Started from an open-source option and then beat the hell out of it because I knew physics was likely to scale badly, was easy for developers to "hold wrong," and needed to be plug-simple to use and understand.

In the end, 95% of our users only used the game engine as a rendering architecture and never turned the physics engine on. All of that optimization time was zero-business-value.