Physical gold still needs to be mined, smelted, transported, and guarded. When you buy it and move it around energy is spent too. So if you really want an apples-to-apples comparison you'd have to count the energy needed to mine each oz. of gold.
The most succinct critique of Bitcoin is, in my opinion, that it's rediscovering the drawbacks of a 100% gold standard vs paper money. It's just like a pure abstraction of the waste of digging stuff out of the ground just so we can count things. It's inexpressibly sophomoric.
The large majority of gold traded for bullion or investment isn't actually moved. It's just the rights to claim X tr. oz. of gold in the vault of XYZ company. Also value of my gold doesn't depend on the transport of all the gold out there so rolling those costs in at the individual level doesn't make much sense. BTC on the otherhand is useless without mining because the existence and transfer of BTC depends on the miners. Even post lightning mining is still critical to opening new channels.
The large majority of Bitcoin isn't actually moved every block either. And those systems you describe to reassign rights to gold also use energy. As block rewards fall, profit will come from transaction fees only, and only the most efficient miners (best equipment or energy access) will remain. Sure, Bitcoin uses more energy now than a database sitting on a server, but it's solving a different problem.
It is solving a different problem but that doesn't change the fact that the whole network's mining work is required to make it work. There are some energy costs to tracking the rights of stored gold but they're vanishingly small compared to BTC.
We're getting sucked down into the weeds here and losing sight of the original point which is BTC as designed requires the use of massive amount of energy to validate and secure the transaction ledger and that ledger and the things it enables is the whole value of BTC.
I think gold is flawed in essentially the same way as BTC. Mining BTC is called mining because it's a neat abstraction of the old fashioned process. The reason fiat currency took over historically is because there's a tremendous benefit to eliminating the dead weight of mining if you can evolve a more subtle way of limiting supply.