I don't think that there is any particular problem in the code base. Its design, however, has been made with 1970s constraints in mind.
A lot of people have thought about rewriting (parts) of TeX, but the amount of legacy is so high and you have to keep compatibility with lots of existing user code.
So you don't think there is a problem in the code base, but you say: " legacy code is so high and you have to keep compatibility with lots of existing user code". That's a contradiction.
I mean that the code works perfectly in the sense that the behaviour conforms to the specification, but it is hard to modify it. So, there's a problem with the code base, just one that is not visible if you don't want to extend the program (one can argue that it's the worst kind of problem).
A lot of people have thought about rewriting (parts) of TeX, but the amount of legacy is so high and you have to keep compatibility with lots of existing user code.