What does it matter if it's "just physics" or "just engineering" or "just math"? Anything you can say about AI, you can say about the atomic bomb. AI involves a quite a bit of engineering as well. This argument is silly.
At the end of the day we are still building a powerful new technology. To assume it will be completely harmless is naive. No technology is completely harmless. But at least our previous inventions didn't sprout minds of their own. And have desires and plans independent of their creators.
The maths used to discover nuclear weapons weren't dangerous. What was dangerous was that the maths happened to align with our physical world.