Could anyone recommend a starting point on Neural Networks for the uninitiated? The parts of this I understood were fascinating, but I quickly realized I was looking up every third word, and not really absorbing much.
If I could only read one thing to gain the technical grounding for this history, what should it be?
It introduces you to some of the underlying principles which haven't changed much over time. I highly recommend it if you want to get deeper intuitions on the principles of CNN, LSTM/RNN, Restricted Boltzmann Machines etc. Also, Hinton's Coursera lectures, though not sure if you can access it anymore.
The other suggestions in this thread are quite good. I'll add "Machine Learning" by Murphy. It's not strictly about neural networks but it's an ML classic and a rigorous introduction to the subject that will give you a principled understanding of the statistical fundamentals. For actual NN implementation the Karpathy and Nielson sources are excellent.
Definitely not the starting point for someone who has no clue. Murphy is probably the very last thing one would read before becoming an expert, and start publishing ML papers.
If I could only read one thing to gain the technical grounding for this history, what should it be?