On March 22, 2016, Azer Koçulu, a
software engineer, removed a package from
npm (a
JavaScript
package manager) he had published titled left-pad
. Koçulu deleted the package following a dispute with
Kik, in which the company forcibly took control of his package name kik
. As a result, thousands of software projects that utilized left-pad
as a
dependency, including the
Babel
transcompiler and the
React
web framework, were unable to be
built or installed.
Technology corporations including
Meta Platforms,
PayPal,
Netflix and
Spotify were potentially affected after the removal of left-pad
as their software products utilized the package in some form. Several hours after the package was made publicly inaccessible on npm, the company behind the platform manually restored the package due to the widespread disruption caused by the incident.
In response to the aftermath of the disruption, npm disabled the removal of a module if more than 24 hours have elapsed since its publishing date and at least one other project requires it as a dependency. The incident also drew widespread media attention and reactions from individuals in the
software industry. The removal of left-pad
has prompted discussion regarding the intentional self-sabotage of software to promote
social justice and brought attention to the elevated possibility of
supply chain attacks in
modular programming.
left-pad
was a
free and open-source
JavaScript
package published by Azer Koçulu, an independent
software engineer based in
Oakland, California.
[1] The package repetitively prepends
characters to a
string using a
loop.
[1] left-pad
has been characterized as being extremely simple, consisting of only 11
lines of code (when empty lines are discounted) in the final version authored by Koçulu.
[2]
[3]
Koçulu published left-pad
on
npm, the default
package manager for
Node.js, a JavaScript
runtime environment.
[4]
[2] Despite its relative obscurity, left-pad
was heavily utilized; the package was used as a
dependency by thousands of other software projects and reached over 15 million downloads prior to its removal.
[5]
[6] Some of the projects that required left-pad
to function were critical to the JavaScript ecosystem at the time. This included
Babel, a
transcompiler that enables backwards-compatible JavaScript code,
Webpack, a module bundling system, and both
React and
React Native, which are
frameworks widely used for the development of
websites and
mobile apps, respectively.
[7]
[8]
[1]
In addition to left-pad
, Koçulu also owned kik
on npm, which was a tool that allowed developers to set up templates for their projects.
[1] On March 11, 2016, Kik Interactive, a Canadian company owning the
instant messaging platform
Kik Messenger, contacted Koçulu, requesting that he relinquish control of the kik
package due to the company's ownership of the "Kik"
trademark.
[9] Part of the correspondence included the following message from Kik:
We don’t mean to be a dick about [the
kik
package], but it’s a registered Trademark in most countries around the world and if you actually release an open source project called kik, our trademark lawyers are going to be banging on your door and taking down your accounts and stuff like that — and we’d have no choice but to do all that because you have to enforce trademarks or you lose them. Can we not come to some sort of a compromise to get you to change the name without involving lawyers? Is there something we could do for you in compensation to get you to change the name? [3]
Koçulu responded shortly after, refusing to change the name of his project:
hahah, you’re actually being a dick. so, fuck you. don’t e-mail me back. [3]
Koçulu also requested US$30,000 as compensation "for the hassle of giving up with my pet project for [
sic] bunch of corporate dicks".
[1] On March 18, 2016, Isaac Z. Schlueter, the
chief executive officer of npm, Inc., wrote to both Kik Interactive and Koçulu, stating that the ownership of the kik
package would be manually transferred to Kik Interactive.
[1]
After expressing his disappointment with npm, Inc.'s decision and stating that he no longer wished be part of the platform, Schlueter provided Koçulu with a command to delete all 273 modules that he had registered.
[9] Koçulu executed the command on March 22, 2016, removing every package he had previously released.
[1] left-pad
was one of the packages that was "unpublished", rendering it no longer publicly accessible on npm.
[5] The left-pad
software project and contents remained available on
GitHub.
[9]
Users attempting to
build or install any JavaScript project which utilized left-pad
as a dependency (including dependents such as Babel or Webpack) received a
404 error which caused the process to fail.
[1] Several notable software technology corporations including
Meta Platforms,
PayPal,
Netflix and
Spotify were potentially affected by the incident, as their products depended on the package.
[8] Ironically, Kik Interactive's developers were also affected and faced build issues as a result of the package's removal.
[1]
A flood of complaints, reactions, and workarounds from other software developers were immediately posted on the project's Git issue tracking system in response to the package's removal from npm. [7] [1] Maintainers of open-source projects including Babel also released hotfixes to remove the dependencies that Koçulu had unpublished. [7] Several of Koçulu's other package names were quickly taken over by newly published packages. [3] An hour after the packages had been deleted, Koçulu published a post on Medium titled "I've Just Liberated My Modules" where he explained that he had unpublished all his software projects from npm as a form of protest against corporate interests in free and open-source software. [1]
To address the software issues, another developer recreated the left-pad
package. However, as the developer released his package with version 1.0.0 while Koçulu published his as version 0.0.3, users continued to encounter issues.
[3] Around two hours after the original left-pad
package was removed, npm manually "un-un-published" the original 0.0.3 version by restoring a backup, effectively resolving the disruption.
[1] Regarding the restoration of the left-pad
the package, Laurie Voss,
chief technology officer of npm, wrote that the company "picked the needs of the many" despite internal disagreements about whether the action was "the right call".
[10]
To further prevent a similar issue from occurring, npm also published a new policy regarding the removal of published packages to prevent deletion if more than 24 hours have elapsed since its release date and at least one other project requires it as a dependency. [11] On behalf of npm, Ashley Williams apologized for the disruption caused by the incident, stating that the platform "[failed] to protect the community". [11] Kik Interactive also apologized for the incident, with the company's Head of Messaging Mike Roberts publishing the email chain with Koçulu on Medium and characterizing his interaction as a "polite request". [8] Roberts wrote that they had initially reached out to Koçulu because they wished to publish an open-source package on npm with the name Koçulu was using. [5] Koçulu stated that he was sorry for disrupting other's work, but he believed he did it "for the benefit of the community in [ sic] long term". [2]
The incident drew varied reactions from users on Twitter, GitHub, Reddit and Hacker News, with many claiming that it briefly "broke the Internet". [2] [8] [9] [1] Many commented on the " move fast and break things" culture of JavaScript development, the unpredictable nature of open-source software, and a perceived over-reliance on modular programming. [2] [8] [3] Users also expressed disappointment regarding npm's decision to forcefully transfer Koçulu's package to Kik Interactive over a legal threat. [1]
The potential for the disruption of a npm package to lead to a
supply chain attack was also highlighted from this incident. In addition to the widely publicized left-pad
incident, a number of individuals had immediately hijacked Koçulu's other packages with unknown code after they were removed.
[7] npm released a new policy to prevent malicious takeovers in similar disputes,
[3] but the left-pad
incident is still cited as an example of over-reliance on external contributors leading to an increased
attack surface for software products.
[12] Koçulu's intentional self-sabotage of left-pad
to highlight a
social issue has also been described as a precursor to incidences of
protestware being published on platforms like npm.
[6]