Thanks for the ideas, voidlogic. We do want to capture server statistics and have that as an issue in Github [1]. I am particularly interested in capturing CPU and I/O utilization because in spot checks, we've observed some frameworks do not fully saturate the 8 HT cores on our i7 hardware, suggesting lock or resource contention.
As for a variety of other hardware and VM environments, the data would be interesting. Related: we plan to migrate the charts and tables to a stand-alone page. Right now, the blog entries are hard-coded to fetch two specific results.json files for rendering the charts/tables. But when we build a stand-alone page, I would like to enhance the script to allow selection of one or two results.json files from a menu for comparing side-by-side. And to your point, the community could then contribute their own results files. Imagine being able to compare EC2 large vs xlarge or vs Xeon E5s or ...
Right now, as you noticed, the latency is only displayed at 256 concurrency. I'll make a note to myself to include a chart for latency versus concurrency when we move to a stand-alone page [2].
As for a variety of other hardware and VM environments, the data would be interesting. Related: we plan to migrate the charts and tables to a stand-alone page. Right now, the blog entries are hard-coded to fetch two specific results.json files for rendering the charts/tables. But when we build a stand-alone page, I would like to enhance the script to allow selection of one or two results.json files from a menu for comparing side-by-side. And to your point, the community could then contribute their own results files. Imagine being able to compare EC2 large vs xlarge or vs Xeon E5s or ...
Right now, as you noticed, the latency is only displayed at 256 concurrency. I'll make a note to myself to include a chart for latency versus concurrency when we move to a stand-alone page [2].
[1] https://github.com/TechEmpower/FrameworkBenchmarks/issues/10...
[2] https://github.com/TechEmpower/FrameworkBenchmarks/issues/14...