The big problem with bundle exec (or Ruby startup times in general), is the ludicrous amount filesystem access because of searching through a ridiculous large number of files for each "require". E.g. if you have a ton of gem's installed, most "require" calls will look for the files you require relative to the root of every gem....
Much more extensive use of require_relative, and fewer search paths can fix that entirely.
Try an "strace 2>&1 -e last64,stat64,lstat,stat bundle exec [yourcommand] | less" and be prepared to be shocked at the waste.
(EDIT: This of course assumes you have strace; on most Linux distro's that's just a package install away - I don't know about OS X, and I've got no idea how to make dtrace do the same)
you can also attach to a PID. If you are a developer on OS X it is good to know DTrace and what it is capable of. There are a lot of default scripts installed in OS X and you can write your own (if you have ever run iosnoop or execsnoop then you have already used DTrace scripts), see:
Thanks. I don't actually run much stuff on OS X - I use it as my work desktop, but all my "real work" is done via ssh to Linux boxes, so the OS X box is rarely running more than a browser, iTerm 2 and Thunderbird, all maximized. I only "see" OS X when something crashes....
EDIT: Completely unrelated, I just noticed who you are. We met in Mike's house when I came over for the launch of Edgeio. And I just realized how long ago that is.
It is fairly trivial if you're willing to acknowledge the real problem:
You can't pretend to not know about paths.
You can make the current case faster by optimizing this and that, but it boils down to reducing the number of stat calls, and the ways to do that are:
- Minimize the number of paths in the $LOAD_PATH. Ideally there should be one path there, but that might not be practical.
- require_relative everything when you know where it is.
As a bonus you get substantial less risk of breakage because of accidental filename clashes (yes, I've had filename clashes happen several times).
Now, this isn't quick, because it'd mean making people used to require 'gem-name/file' and have gem/bundle ensure that the default load path contains a symlink to the real current include path for every gem, but this doesn't even need a single interpreter change.
The problem here is not the Ruby implementations, but the gem ecosystem: This only becomes difficult if one wants the interpreter to automagically find files you've dropped all over the filesystem.
The biggest problem here is RubyGems which uses "load the specification for every gem you have installed"-approach. 300 gems installed = 300 files read at startup.
300 files is nothing. I just strace'd an app I'm playing with that does 145,000 stat calls on startup.... It's in the process of being rewritten to do require_relative whenever possible. As for the gems, it'll probably end up with a hack to "precompile" a list of absolute paths after installation...
Another option is to build an index of the various parts of $LOAD_PATH. I monkeypatched (JRuby) to do this at runtime (that is, re-index every launch), and I saw a modest speed up in Rails boot time. If there were a standard way to build a semi-permanent index (updated by 'gem install', etc.), I'd expect the index to give even further gains. Of course, this is more invasive than your suggestions.
Yeah, that's at least in part because of the stat()/lstat()-blowup.
After the first run most of the stat data will be cached in-memory anyway, but tens of thousands of unnecessary system calls hurts even with the actual data in memory...
The big problem with bundle exec (or Ruby startup times in general), is the ludicrous amount filesystem access because of searching through a ridiculous large number of files for each "require". E.g. if you have a ton of gem's installed, most "require" calls will look for the files you require relative to the root of every gem....
Much more extensive use of require_relative, and fewer search paths can fix that entirely.
Try an "strace 2>&1 -e last64,stat64,lstat,stat bundle exec [yourcommand] | less" and be prepared to be shocked at the waste.
(EDIT: This of course assumes you have strace; on most Linux distro's that's just a package install away - I don't know about OS X, and I've got no idea how to make dtrace do the same)