There's a vanilla 2TB plan for $10/month, the plan with 2TB and Gemini is $20/month. You could say that Gemini is $10/month then, but if you don't actually need 2TB or the other benefits then you're effectively paying $20/month because you can't unbundle Gemini from everything else.
Wait, where's this $10/month plan? I don't see it. I see it at https://one.google.com/about, but I can't actually choose that.
Edit: I found the answer on Reddit, as usual. You need to go to your plan settings and only then is the plan available under one.google.com/settings. That's the only way to downgrade.
