Package managers seem to be an increasingly popular attack vector. It's only luck that none of the attacks have been particularly malicious yet. Considering how many package manager downloads go to a server in a datacenter, a widely distributed malicious package could control a botnet with extremely high throughput, or wreak havoc on any databases it comes into contact with.
It's only a matter of time before something like this happens. A big part of the problem is that application package managers, like pip or npm, are far less sophisticated than those of operating systems, like aptitude or yum. It needs to be easy for developers to open source their code, and to mark dependencies with precise commit hashes, but the download also needs to be secure and verifiable. There are many difficult tradeoffs to consider in terms of usability, centralization, security and trust.
This is why we are working on integrating TUF (The Update Framework) signing into the OCaml OPAM package manager. See https://github.com/hannesm/conex-paper/blob/master/paper.pdf for the talk from last year. There's one more iteration required on the implementation before we're happy with it, but we are aiming to get this live on the OPAMv2 package repository some time in 2018 for all the publicly available OCaml packages.
OPAMv2 also exposes sufficient hooks during the build process for using OS sandboxing during builds, and disconnecting network access/etc. It would be nice to factor this out to be more OS independent (e.g. for all the `unshare` tricks on Linux, or the sexp-format for sandboxing on OSX) in the future.
Another fun fact to consider is that with many package formats, you can execute arbitrary code at install time so if a malicious package can get into a repository, it's very likely to start compromising systems quickly.
Whilst a package manager repo. compromise would be the biggest bang in terms of attack, compromising the credentials of the developers of popualar libraries would be an easier attack (and indeed is already happening https://twitter.com/chrispederick/status/892768218162487300)
I was recently very surprised to see an expensive security scanner which builds apps as the service account apparently without sandboxing. A package manager which executes code or which is exploitable would give you access to what is very likely an interesting account & data, and in at least some cases might not leave many clues behind. PyPI, etc. at least has centralization and immutable versions but who knows what's serving some random repo, tarball, etc. which might not even be a direct dependency?
With Python you can execute arbitrary code at import time... you don't even have to get to the install process. I've seen packages on Pypi that try to "sudo apt-get install ..." when the setup.py file is imported.
Also, even without sudo there's absolutely nothing stopping you (for example) downloading a cryptocurrency miner, or DDOS tool, or something, and starting it up to run in the background.
I almost never see this. Even on systems that are only running a single python project, I only ever see folks use virtualenv. The only time I ever see things installed with sudo is when the package is being installed in a docker container.
I'm always fascinated by the amount of trust being exhibited by the developers of some node projects I've seen. Their projects have an order of magnitude more dependencies than I'm used to - and at the other end of each one is someone publishing some small module to npm with an unknown amount of review. I feel safe(r) installing dependencies from apt because I know the processes the Debian community follows before packages are included in the official repos.
More needs to be done by package managers to warn end users.
One scenario that worries me is where apps age and use popular trusted dependancies (e.g. gems on Github).
When those gems stop being maintained but need to be updated to work (say with latest OSX) - it's common to quickly look at the latest forks available and select the one that now works correctly - but without a detailed inspection of the new code it's potentially kryptonite for a production datacenter.
It always worries me when I install a well-known or large package from npm and it ends up downloading dozens of dependencies maintained by disparate and unaccountable github users.
Maybe most of those 500,000 packages simply shouldn't be trusted.
There's a precedent for curated subsets of package ecosystems. Stackage for Haskell is an example, although it doesn't have security as the primary goal.
I don't think we should focus on actual audits of packages. Just checking that packages seem basically credible seems like a better approach because it's doable.
Credibility is an easier check but still tricky. Many of the package are uploaded by anonymous or pseudonymous authors, so there's no easy way to even tie that to an IRL identity, let alone check for credibility.
I'd agree that a curated small package repository would be a better way to address the problem, but the market doesn't seem very interested in that as a solution.
I think it might just not have happened yet. The npm community can be pretty creative and enthusiastic!
I don't think IRL identities are necessary for what I imagine. It's more like establishing a basic set of packages that have been around, have communities of committers, reverse dependencies, etc.
Maybe we would even make a starting assumption that the transitive closure of dependencies originating with a set of high profile packages are "approved".
I'm thinking aloud but I think there could be a reasonably pragmatic way to get this started...
and yet, I don't see substantial movements relating to package security and trustability in these repo's. To be clear I'm not suggesting these two are any worse than others, they're just large repo's who have had incidents in the past.
The problem here (to my view) is that increasing the security of package repo's will slow down releases (additional checks take time) and cost money (additional security, hosting etc) and until there's a market demand for those service, they won't happen.
I'm skeptical too. But if I think like a sci-fi writer I can vaguely imagine ways for it to actually happen. That open source maintenance happens at all is pretty remarkable, so I think this thing, with an appropriate concept and some good tools (with emojis in their command line output), is at least vaguely plausible...
I don't see any reason it can't be done, either, in theory, and that's with the manual approach. Fancier ideas are viable too, but 500k is still relatively tiny and a manually tractable number. Incremental reviews starting today by many coordinating groups in the node community would take a while to complete, maybe a few years, but with some sensible ordering heuristics like e.g. the most downloaded first, or the most suspicious names first, some value could be produced quickly. But it won't happen, package vetting isn't really a value in these communities. (And that might not really be a bad thing, at least for now...)
Those service already exit, e.g. https://www.sourceclear.com/ whilst I hope they're doing well, I don't think they've made significant in-roads into the volume of people using open source software library repo's.
Yes, Conda [0] is a package manager designed by Continuum Analytics (now Anaconda, Inc.) to support their Anaconda Distribution [1]. The distribution is free, and the Conda client is open source. However, Anaconda sells several enterprise products, including an on-premise Conda server ("Anaconda Repository").
In general, Conda does more package verification than pip, and the packages in the Anaconda distribution are more thoroughly vetted than PyPi. Conda-Forge [2] provides an escape hatch for less-vetted community code.
Five hundred thousand packages, am I reading it right? I dont believe top ten OS package managers combined would reach that number. Either this is a typo or it's crazy.
There's nothing especially awful about left-pad being its own package, the disaster was because a huge number of developers were betting on npm to somehow be highly available (despite being donated by its admins at no cost and with no committed SLA) rather than vendoring their deps.
One can both vendor and use the package manager to fetch updates. Just add the node-modules directory to your VCS.
The thing with node is that AFAIK it requires you to have libraries for what in most languages would be in the standard library. Maybe someone should start a "stdnode" project where the most popular / successful libraries for generic tasks are integrated into a dependable, maintained de-facto standard library, with an eye on quality and sanity, and community / Joyent funding.
It's only a matter of time before something like this happens. A big part of the problem is that application package managers, like pip or npm, are far less sophisticated than those of operating systems, like aptitude or yum. It needs to be easy for developers to open source their code, and to mark dependencies with precise commit hashes, but the download also needs to be secure and verifiable. There are many difficult tradeoffs to consider in terms of usability, centralization, security and trust.