I have a love-hate relationship with the advice "Don’t reinvent solutions to common problems." In one sense, it's obviously a good idea because it's (generally) bad to repeat yourself. When you repeat yourself, the places where you can make a mistake increase, you violate single point of truth, yadda, yadda, yadda.
But in the same sense, for me, reinventing things has included some of the most enriching things I've ever done as a programmer. For example, the first big project I ever tackled (more than 10 years ago) was an open source message board heavily inspired by vBulletin. I really didn't solve any problem that hadn't been solved before, and the last thing the world needed was another message board. But holy hell, I really learned a lot! And it was maddeningly fun. I would never want to deprive someone of that experience just because I think that DRY is a Good Thing.
On the flip side, just a short while ago, I tried writing a package that handles a cluster of remote peers[1]. Want to know what I learned? That I knew a lot less about networking than I thought I did, and I'd have to read a lot more literature before I could get to where I wanted to go.
And yes, this can't always be the case, because you'd never get anything done otherwise.
(It occurs to me that maybe I'm talking about personal enrichment while the OP is talking about solving ML problems. Toe-may-toe, toe-mah-toe.)
> I have a love-hate relationship with the advice "Don’t reinvent solutions to common problems."
Yes: it's difficult for me to reconcile this with the next piece of advice: "Don't Ignore the Math".
If you haven't understood the common problems well enough to code your own solutions, it seems to me that you wouldn't have a secure enough understanding of the math to make any valuable improvements.
When experts give advice, they often forget how little most people know. Contradictory advice probably means they know which cases it applies in, and it should be obvious if you know what you are doing.
Ultimately, most of the advice will boil down to "be an expert". I guess that's what you're in for when you read a five point blog post on something which is really quite advanced.
Alan Kay said, "To a first approximation, you should never write your own software. To a second approximation, you should always write your own software."
I've had good experience with reinventing the most important part of the wheel. If I try to solve a common problem myself first, I will gain insight into the pitfalls, so that I can make a better decision which pre-made solution to use.
I completely agree but I think there's a nice explanation for this.
It is _absolutely_ fine to violate DRY and repeat work of others for _learning_ purposes.
However, if you're already past that stage and want to create code/library/binary/whatever for other people to actually (re)use, then DRY and be orthogonal.
One nasty aspect of the advice "Don't reinvent solutions to common problems" is it doesn't really provide a way to tell when your problem is really, truly a common problem, which can be a lot more subtle than you might think.
Sometimes the simplest single requirement change - say, works in low memory environment, or handles high latency, or utilizes a GPU, or has a good public API, or has low battery usage, or can save its entire state to disk, or is skinnable, or supports other random feature X - can instantly take all common solutions to a problem and invalidate them. But just as often, it doesn't. Or, just as often, you have to just give up on those requirements for pragmatic reasons.
The author does specifically point out that his advice is for people starting with machine learning. I think your bulletin board example wouldn't have worked out as well if you hadn't used any online bulletin boards before building your own.
It's true I had used bulletin boards before, but I hadn't done much programming before that outside of a high school classroom.
In any case, I could pose a litany of other examples where I started in a subject precisely by "resolving" some common problems. For instance, when starting with machine learning, I implemented a simple naive Bayes text classifier myself. It was fun and illuminating.
Yes, this is definitely a learning style. Yes, people's tastes differ. But, I think that's my point...
On the flip side, just a short while ago, I tried writing a package that handles a cluster of remote peers[1]. Want to know what I learned?
I'm having flashbacks to the last level of the Stripe CTF :) Have you tried it? It's a little late now but it's still up I think. It has discussion of distributed consensus and distributed logs (related to your solution above) and links to a few interesting articles/implementations, including one in Go (go-raft). Quite educational if you're a beginner at that sort of thing (as I am) - you might find it interesting.
Sometimes there can be pragmatic reasons to re implement existing software especially if your needs are simpler than the problem the library had in mind.
It can be a choice between "Write 200 lines of code that I understand and know does exactly what I need and no more" or "import 5000 lines of unknown code that might be abandoned and left to bit-rot by the original author, use 5% of it's functionality and hope I can shoehorn it in without too many gotchas"
Only try this if you are sure you understand the problem domain and are very comfortable dealing with all its pitfalls. Many times (especially when dealing with numeric code) 4500 of those 5000 lines are for dealing with special corner cases and obscure gotchas that people have spent month agonizing over how to fix and that has been battle tested by experts over years. Stuff which won't affect you 99% of the time, but when it does will leave you having to redo all their hard debugging work.
But in the same sense, for me, reinventing things has included some of the most enriching things I've ever done as a programmer. For example, the first big project I ever tackled (more than 10 years ago) was an open source message board heavily inspired by vBulletin. I really didn't solve any problem that hadn't been solved before, and the last thing the world needed was another message board. But holy hell, I really learned a lot! And it was maddeningly fun. I would never want to deprive someone of that experience just because I think that DRY is a Good Thing.
On the flip side, just a short while ago, I tried writing a package that handles a cluster of remote peers[1]. Want to know what I learned? That I knew a lot less about networking than I thought I did, and I'd have to read a lot more literature before I could get to where I wanted to go.
And yes, this can't always be the case, because you'd never get anything done otherwise.
(It occurs to me that maybe I'm talking about personal enrichment while the OP is talking about solving ML problems. Toe-may-toe, toe-mah-toe.)
[1] - https://github.com/BurntSushi/cluster