It seems bizarre to me that they'd have accepted such a high cost (150GB+ installation size!) without entirely verifying that it was necessary!
I expect it's a story that'll never get told in enough detail to satisfy curiosity, but it certainly seems strange from the outside for this optimisation to be both possible and acceptable.
Because it's a recent 20TB HDD it has read speeds that approach 250MB/s and I've also specifically partitioned it at the beginning of the disk just for games so that it can sustain those 250MB/s without files falling into the slower tracks. It's honestly fine for the vast majority of games.
On phone, I bet you see some more effort.
Users might be more hesitant to switch to another game if it means uninstalling yours and reinstalling is a big pain in the backside due to long download times.
Sure they may loose some sales but I have never seen many numbers on how much it really impacted sales.
Also on the disk side, I can't say I have ever looked at how much space is required for a game before buying it. If I need to clear out some stuff I will. Especially with it not being uncommon for a game to be in the 100gb realm already.
That all being said, I am actually surprised by the 11% using mechanical hard drives. I figured that NVME would be a lower percentage and many are using SSD's... but I figured the percent with machines capable of running modern games in the first place with mechanical drives would be far lower.
I do wonder how long it will be until we see games just saying they are not compatible with mechanical drives.
You should look at COD install sizes and almost weekly ridiculously huge "updates". 150gb for a first install is almost generous considering most AAA games.
In other words, the game studio externalised an eight-figure hardware cost onto their users, to avoid a five-to-six-figure engineering cost on their side.
Data duplication can't just be banned by Steam, because it's a legitimate optimisation in some cases. The only safeguard against this sort of waste is a company culture which values software quality. I'm glad the developers fixed this bug, but it should never have been released to users in the first place.
It's also a title that shows you can have a really good game without the latest tech.
Because offhand, I know you could do things like cute optimizations of redundant data to minimize seek time on optical media, but with HDDs, you get no promises about layout to optimize around...
The only thing I can think of is if it was literally something as inane as checking the "store deduplicated by hash" option in the build, on a tree with copies of assets scattered everywhere, and it was just nobody had ever checked if the fear around the option was based on outcomes.
(I know they said in the original blog post that it was based around fears of client performance impact, but the whole reason I'm staring at that is that if it's just a deduplication table at storage time, the client shouldn't...care? It's not writing to the game data archives, it's just looking stuff up either way...)
They realised, after a lot of players asking, that it wasn't necessary, and probably had less of an impact than they thought.
They removed the duplicates, and drastically cut the install size. I updated last night, and the update alone was larger than the entire game after this deduplication run, so I'll be opting in to the Beta ASAP.
It's been almost a decade since I ran spinning rust in a desktop, and while I admire their efforts to support shitty hardware, who's playing this on a machine good enough to play but can't afford £60 for a basic SSD for their game storage?
Let’s say you have UI textures that you always need, common player models and textures, the battle music, but world geometry and monsters change per stage. Create an archive file (pak, wad, …) for each stage, duplicating UI, player and battle music assets into each archive. This makes it so that you fully utilize HDD pages (some small config file won’t fill 4kb filesystem pages or even the smaller disk sectors). All the data of one stage will be read into disk cache in one fell swoop as well.
On optical media like CDs one would even put some data closer to the middle or on the outer edge of the disc because the reading speed is different due to the linear velocity.
This is an optimization for bandwidth at the cost of size (which often wasn’t a problem because the medium wasn’t filled anyway)
HDDs also have to real with fragmentation, I wonder what the odds that you get to write 150 GBs (and then regular updates in the 30GB range) without breaking it into fragments...
In this case, I don't think it was forgetfulness; unlike us, they have an excuse and they were trying to optimise for disk seek times.
Anyway, I've got a half-dozen cloud accounts I need to go check for unused resources waves.
Think of it as I have two packs for levels.
Creek.level and roboplanet.level
Both use the cyborg enemies, by duplicating the cyborg enemy model and texture data across both files, Only the level file needs to be opened to get all nessecary data for a match.
Because modern OS will allow you to preallocate contiguous segments and have auto defrag, you can have it read this level file at max speed, rather than having to stop and seek to go find cyborg.model file because it was referenced by the spawn pool. Engine limitations may prevent other optimisations you think up as a thought exercise after reading this.
It's similar to how crash bandicoot packed their level data to handle the slow speed of the ps1 disc drive.
As to why they had a HDD optimisation in 2024... Shrugs
A filesystem is by itself just one big "file" acting like a file archive.
> 3-D Hardware Accelerator (with 16MB VRAM with full OpenGL® support; Pentium® II 400 Mhz processor or Athlon® processor; English version of Windows® 2000/XP Operating System; 128 MB RAM; 16-bit high color video mode; 800 MB of uncompressed hard disk space for game files (Minimum Install), plus 300 MB for the Windows swap file […]
* https://store.steampowered.com/app/9010/Return_to_Castle_Wol...
* https://en.wikipedia.org/wiki/Return_to_Castle_Wolfenstein
Even older games would be even smaller:
* https://www.oldgames.sk/en/game/ultima-vi-the-false-prophet/...
* https://en.wikipedia.org/wiki/Ultima_VI:_The_False_Prophet
It's currently over 100GB because of duplicated assets, so this is a game-changer (pun intended).
EDIT: Just checked; 157GB on my SSD.
EDIT2: 26GB after updating with a 9.7GB download, 5.18GB of shaders before and after.