I tried this multiple times over the last few years and it always didn't matter. It's not making a measurable difference but it looks less pleasant and I rather look at pretty things.
I also don't find the syntax coloring to make a big difference for me. What does though is the semantic feedback/navigation in the IDE.
When I joined a Java shop for the first time I was all gung-ho about sticking to my 80x24 terminal and emacs and to heck with the point-and-click crowd who couldn't do their work without a graphical UI. Then my new manager got me to try Eclipse for a week. I humored him and discovered that I was an order of magnitude more productive with Eclipse's semantic features like on-the-fly compile error underlining, code autocomplete, and navigation. It really was a humbling experience.