The unprecedented surge in demand from AI data centers is causing hard drive and storage costs to skyrocket. This trend is making it more expensive and challenging for digital archivists, academics, Wikipedia, and even hobby data hoarders to save and archive internet data.
Specific drives favored by prominent organizations like the Internet Archive have become significantly more expensive or are simply difficult to procure. Over the past several months, prices for both consumer-level and enterprise solid-state drives (SSDs), hard disk drives (HDDs), and other storage types have seen universal increases. For instance, a 2TB external Samsung SSD that cost $159 last fall now sells for $575. Data from PC Part Picker, a website tracking drive prices, indicates a widespread increase starting around October last year, with many tracked drives doubling or increasing by over 150 percent. Some stores are reporting SSD and HDD stockouts, and a secondary market has emerged where these drives are being scalped.
Brewster Kahle, founder of the Internet Archive and the Wayback Machine, described the escalating storage costs as “a very real issue costing us time and money.” He noted that their preferred 28-30TB drives are “just not available or at very high price.” The Internet Archive collects over 100 terabytes of new material daily and maintains over 210 petabytes of archived data, requiring continuous upgrades and maintenance, hence a constant need for new hard drives. Kahle mentioned they are fortunate to have an active community of donors and are seeking assistance from hard drive manufacturers to navigate these difficult times, though they currently have workarounds in place.
The Wikimedia Foundation, which operates Wikipedia and other projects like Wikimedia Commons, also expressed concern regarding storage costs. A spokesperson for the foundation stated that with over 65 million articles on Wikipedia alone, access to server and storage capacity is vital. They have observed price increases since late 2023, which are a concern for them as much as for any other industry player. The primary impact is felt in memory and hard drive purchases, as well as in server delivery lead times and the capacity to place future orders. As a non-profit organization, Wikimedia carefully considers budget allocation and maintains its own data centers to serve global users, implementing workarounds by intelligently prioritizing investment.