For those of you who use event sourcing: how do you scale horizontally? Do you have a single process that handles all events sequentially? Maybe I'm overthinking this but I'm worried this might become a bottleneck eventually. Otherwise, how do you deal with concurrency issues (e.g. 2 different visitors who signup with the same username concurrently)?
The first rule of eventsourcing is don't eventsource EVERYTHING.
Eventsourcing requires really, really detailed analysis of your boundaries (and no small bit of experimentation) to get right, even when you've done it before. It should really only be applied to the areas of your application that are going to see a real benefit from it. My proxy for measuring this is "how much money does this group of features make for me?" If the answer is some version of "none" or "none, but its a requirement to have it anyway" then I don't try to apply ES to that area of the application. I never use ES on user records anymore, for example.
I'm sidestepping your actual question, but mostly because your use-case is something I consider to be a bit of a canard. To answer it though, as others noted, "it depends"—I model projections with actors, which means all events DO come through a single process handling events sequentially (and this is a very common way to do it.) But because I've got tons of these actors (one per stream that's actively running), there's no real bottleneck—this is also one way you guarantee ordering within a stream.
Higher up in the stack, at the eventstore level, you'll probably have a pool of processes handling writes and routing events to the correct handlers/projections, but this is an implementation detail that can vary widely. The advice to not get bogged down in the details early on is good—the thing i see almost every developer new to ES do is get mired in the basic "how do i implement this" question, and that's really not what's going to kill your project. Not understanding nearly as much about your business and the software boundaries in your system is where most projects go wrong.
> The first rule of eventsourcing is don't eventsource EVERYTHING.
I've heard this rule so often now, but never what exactly it is that you _can_ safely event source. I'm afraid it's something you can only get a feeling for by doing it a few times and making mistakes, but that way of learning is hard to justify in real projects.
> I've heard this rule so often now, but never what exactly it is that you _can_ safely event source. I'm afraid it's something you can only get a feeling for by doing it a few times and making mistakes, but that way of learning is hard to justify in real projects.
see above:
>> My proxy for measuring this is "how much money does this group of features make for me?"
That said, i think you're also correct—there's a lot of trial and error in developing your instincts for this architecture. That's also true of monoliths, web MVC, or really any architectural style. Experience, as they say, is that thing you only get AFTER you needed it.
I've done both. My advice: use a well-maintained OSS library or server to handle the event store and subscriptions, because its easy to get bogged down in solving those problems yourself, and there's very little value in doing so.
My own is called Replay[1] but I wouldn't recommend using it for most people—I wrote it when I was first learning how to apply ES in Ruby back when the only material being written about it was for .Net. (I think I might have given the first-ever talk on ES at a RailsConf back in 2012.) Replay makes some assumptions I wouldn't make building something like that today, such as a very entity-focused perspective (e.g. in its original incarnation, all events were published from a projection by necessity), and it lacks some features I think a modern evented framework ought to have like projection snapshotting. I've got a lot of private changes to it to evolve it some, but you're better off going with something like Eventide[2] if you're working in Ruby today.
I've also used some of Ben Smith's libraries in Elixir[3]. If you're in .Net, the Orleans framework has some really interesting ideas going on, and in Javaland, Akka and friends are what I'd start looking at.
Very good question. The answer is, of course, 'it depends'.
What is depends on is how much horizontal scale you expect to have. At a small scale, you can use something like postgres as a centralized broker of sequence. A single postgres cluster can sequence a lot of records in a hurry, no problem.
At a larger scale you might need to decompose your domain into multiple, independently ordered aggregates. Now you have multiple choke point brokers, each of which can handle a lot of records, but which don't block each other.
Step it up another notch, you can use a consensus algorithm (paxos or raft) to get a cluster of machines to agree on the state of the sequence, and perform optimizations such as assigning blocks of sequence by shard and node.
I recommend not over building your event storage architecture until you measure your actual needs. There are cheap ways and huge expensive ways, and building too much is an easy way to make your project fail. Also, one of the nice things about a sequenced set of events is that it is pretty easy to replay into your new, faster event storage later once you've had the happy problem of too much success.
By having a good partitioning strategy, where you partition based on unique keys like usernames, you can very easily scale horizontally, because each consumer only consumes one partition. And it's still sequential for events that has to be sequential.
