Manufacturers have been playing this game with DWPD/TBW numbers too --- by reducing the retention spec, they can advertise a drive as having a higher endurance with the exact same flash. But if you compare the numbers over the years, it's clear that NAND flash has gotten significantly worse; the only thing that has gone up, multiplicatively, is capacity, while endurance and rentention have both gone down by a few orders of magnitude.
For a long time, 10 years after 100K cycles was the gold standard of SLC flash.
Now we are down to several months after less than 1K cycles for QLC.
It turns out that a few extra bytes can turn a 1 year endurance into a 100 year endurance.
For instance, on Linux there is par2cmdline. For all my backups, I create pax archives, which are then compressed, then encrypted, then expanded with par2create, then aggregated again in a single pax file (the legacy tar file formats are not good for faithfully storing all metadata of modern file systems and each kind of tar program may have proprietary non-portable extensions to handle this, therefore I use only the pax file format).
Besides that, important data should be replicated and stored on 2 or even 3 SSDs/HDDs/tapes, which should preferably be stored themselves in different locations.
The issue being discussed is random blocks, yes?
If your entire drive is bricked, that is an entirely different issue.
If it decides, by some arbitrary measurement, as defined by some logic within its black box firmware, that it should stop returning all blocks, then it will do so, and you have almost no recourse.
This is a very common failure mode of SSDs. As a consequence of some failed blocks (likely exceeding a number of failed blocks, or perhaps the controller’s own storage failed), drives will commonly brick themselves.
Perhaps you haven’t seen it happen, or your SSD doesn’t do this, or perhaps certain models or firmwares don’t, but some certainly do, both from my own experience, and countless accounts I’ve read elsewhere, so this is more common than you might realise.
It's ridiculous (IMO) that SSD firmware is doing so much transparent work just to keep the illusion that the drive is actually spinning metal with similar sector write performance.
That work can't be done because there is no free blocks. However, no space can be freed up because every spare writable block is bad or is in some other unusable state.
The drive is therefore dead - it will enumerate, but neither read nor write anything.
You can sidestep this by bypassing the controller on a test bench though. Pinning wires to the chips. At that point it’s no longer an SSD.
At least in the ‘bricked’ case it’s a trivial RMA - corrupt blocks tend to be a harder fight. And since ‘bricked’ is such a trivial RMA, manufacturers have more of an incentive to fix it or go broke, or avoid it in the first place.
This is why backups are important now; and always have been.
SSDs know which blocks have been written to a lot, have been giving a lot of read errors before etc., and often even have heterogeneous storages (such as a bit of SLC for burst writing next to a bunch of MLC for density).
They can spend ECC bits much more efficiently with that information than a file system ever could, which usually sees the storage as a flat, linear array of blocks.
For any important data you should have your own file hashes, for corruption detection, and you should add some form of redundancy for file repair, either with a specialized tool or simply by duplicating the file on separate storage media.
A database with file hashes can also serve other purposes than corruption detection, e.g. it can be used to find duplicate data without physically accessing the archival storage media.
There's a lot of potential for errors when the storage controller processes and turns the data into analog magic to transmit it.
In practice, this is a solved problem, but only until someone makes a mistake, then there will be a lot of trouble debugging it between the manufacturer certainly denying their mistake and people getting caught up on the usual suspects.
Doing all the ECC stuff right on the CPU gives you all the benefits against bitrot and resilience against all errors in transmission for free.
And if all things go just right we might even be getting better instruction support for ECC stuff. That'd be a nice bonus
That's a physical layer, and as such should obviously have end-to-end ECC appropriate to the task. But the error distribution shape is probably very different from that of bytes in NAND data at rest, which is different from that of DRAM and PCI again.
For the same reason, IP does not do error correction, but rather relies on lower layers to present error-free datagram semantics to it: Ethernet, Wi-Fi, and (managed-spectrum) 5G all have dramatically different properties that higher layers have no business worrying about. And sticking with that example, once it becomes TCP's job to handle packet loss due to transmission errors (instead of just congestion), things go south pretty quickly.
Outside of wireless links (where FEC of some degree is necessary regardless) this is mostly because TCP’s checksum is so weak. QUIC for example handles this much better, since the packet’s authenticated encryption doubles as a robust error detecting code. And unlike TLS over TCP, the connection is resilient to these failures: a TCP packet that is corrupted but passes the TCP checksum will kill the TLS connection on top of it instead of retransmitting.
This is why newer TCP variants that use different congestion signals can deal with networks that violate that assumption better, such as e.g. Starlink: https://blog.apnic.net/2024/05/17/a-transport-protocols-view...
Other than that, I didn't realize that TLS has no way of just retransmitting broken data without breaking the entire connection (and a potentially expensive request or response with it)! Makes sense at that layer, but I never thought about it in detail. Good to know, thank you.
There are potentially ways a filesystem could use heirarchical ECC to just store a small percentage extra, but it would be far from theoretically optimal and rely on the fact just a few logical blocks of the drive become unreadable, and those logical blocks aren't correlated in write time (which I imagine isn't true for most ssd firmware).
BTW, byte size during the history varied from 4 to 24 bit! Even now, based on interpretation, you can say 16 bit bytes do exist.
Char type can be 16 bit on some DSP systems.
I was curious, so I checked. Before this comment, I only knew about 7 bit bytes.
It would not be hard for a COW file system to use them, but it can easily get out of control paranoia wise. Ideally you’d need them for every bit of data, including metadata.
That said, I did have a computer that randomly bit flipped when writing to storage sometimes (eventually traced it to an iffy power supply), and PAR (a type of reed solomon coding forward error correction library) worked great for getting a working backup off the machine. Every other thing I tried would end up with at least a couple bit flip errors per GB, which make it impossible.
ZFS has "copies=2", but iirc there are no filesystems with support for single disk erasure codes, which is a huge shame because these can be several orders of magnitude more robust compared to a simple copy for the same space.
Those QLC NAND chips? Pretty much all of them have an "SLC mode", which treats each cell as 1 bit, and increases both write speeds and reliability massively. But who wants to have 4 times less capacity for the same price?
Plenty of people would be willing to pay for SLC mode. There is an unofficial firmware hack that enables it: https://news.ycombinator.com/item?id=40405578
1TB QLC SSDs are <$100 now. If the industry was sane, we would have 1TB SLC SSDs for less than $400, or 256GB ones for <$100, and in fact SLC requires less ECC and can function with simpler (cheaper, less buggy, faster) firmware and controllers.
But why won't the manufacturers let you choose? The real answer is clearly planned obsolescence.
I have an old SLC USB drive which is only 512MB, but it's nearly 20 years old and some of the very first files I wrote to it are still intact (I last checked several months ago, and don't expect it's changed since then.) It has probably had a few hundred full-drive-writes over the years --- well worn-out by modern QLC/TLC standards, but barely-broken-in for SLC.
Very few people have the technical understanding required to make such a choice. And of those, fewer people still would actually pick SLC over QLC.
At the same time: a lot of people would, if facing a choice between a $50 1TB SSD and a $40 1TB SSD, pick the latter. So there's a big incentive to optimize on cost, and not a lot of incentive to optimize on anything else.
This "SLC only" mode exists in the firmware for the sake of a few very specific customers with very specific needs - the few B2B customers that are actually willing to pay that fee. And they don't get the $50 1TB SSD with a settings bit flipped - they pay a lot more, and with that, they get better QC, a better grade of NAND flash chips, extended thermal envelopes, performance guarantees, etc.
Most drives out there just use this "SLC" mode for caches, "hot spot" data and internal needs.
There was a period of time when you could still by consumer SLC drives and pay a premium for them. I still have one.
Anyone assuming the manufacturers are missing out on a golden market opportunity of hidden SLC drive demand is missing the fact that they already offered these. They know how well (or rather, how poorly) they sell.
Even if consumers had full technical knowledge to make decisions, most would pick the TLC and QLC anyway. Some of these comments are talking about optimizing 20 year old drives for being used again two decades later, but ignoring the fact that a 20 year old drive is nearly useless and could be replaced by a superior option for $20 on eBay.
The only thing that would change, practically speaking, is that people looking for really old files on drives they haven’t powered up for 20 years wouldn’t be surprised that the were missing.
The rest of us will do just fine with our TLC drives and actual backups to backup services or backup mediums.
I’ll happily upgrade my SSD every 4-5 years and enjoy the extra capacity over SLC while still coming out money ahead and not losing data.
No, it's not. The real answer is that customers (Even B2B) are extremely price sensitive.
Look, I know the prevailing view is that lower quality is some evil corporate plan to get you to purchase replacements on a more frequent basis, but the real truth is that consumers are price sensitive, short sighted, and often purchasing without full knowledge. There's a race to the bottom on price, which means quality suffers. You put your typical customer in front of two blenders at the appliance store, one is $20 and the other is $50, most customers will pick the $20 one, even when armed with the knowledge that the $50 version will last longer.
When it comes to QLC vs SLC, buyers don't care. They just want the maximum storage for the smallest price.
Having built computers casually for some time, I never recall being told by the marketing department or retailer that one kind of SSD was more reliable than another. The only thing that is ever advertised blatantly is speed and capacity. I saw the kind of SSD sometimes, but it was never explained what that meant to a consumer (the same way SMR hard drives were never advertised as having slow reads)
If I saw "this kind of SSD is reliable for 10 years and the other one is reliable for 2" then I may have made a decision based on that.
Yes.
QLC SSDs are reliable enough for my day-to-day use, but even QLC storage is quite expensive and I wouldn't want to pay 4x (or realistically, way more than 4x) to get 2TB SLC M.2 drives instead of 2TB QLC M.2 drives.
Though what make me wonder is that some reviews of modern SSDs certainly mention that that pSCL is somewhat less than 25% of capacity, like 400GB pSLC cache for 2TB SSD:
https://www.tomshardware.com/pc-components/ssds/crucial-p310...
So you get more like 20% of SLC capacity at least on some SSDs
If the original drive has sufficient reliability, then yes I do want that.
And the majority of consumers do, too.
Chasing absolute extreme highest powered off durability is not a priority for 99% of people when the drives work properly for typical use cases. I have 5 year old SSDs where the wear data is still in the single digit percentages despite what I consider moderately heavy use.
> I have an old SLC USB drive which is only 512MB, but it's nearly 20 years old and some of the very first files I wrote to it are still intact (I last checked several months ago, and don't expect it's changed since then.) It has probably had a few hundred full-drive-writes over the years --- well worn-out by modern QLC/TLC standards, but barely-broken-in for SLC.
Barely broken in, but also only 512MB, very slow, and virtually useless by modern standards. The only positive is that the files are still intact on that old drive you dusted off.
This is why the market doesn’t care and why manufacturers are shipping TLC and QLC: They aren’t doing a planned obsolescence conspiracy. They know that 20 years from now or even 10 years from now that drive is going to be so outdated that you can get a faster, bigger new one for pocket change.
It's not about age of drive. It's how much time it spent without power.
Industry is sane in both the common and capitalist sense.
The year 2025 and people still buy 256Tb USB thumbdrives for $30, because nobody cares except for the price.
I suspect that 2035 years time, hardware from 2010 will work, while that from 2020 will be less reliable.
CRTs from 1994 and 2002 still going strong. LCD tvs from 2012 and 2022 just went kaput for no reason.
Old hardware rocks.
Most likely bad capacitors. The https://en.wikipedia.org/wiki/Capacitor_plague may have passed, but electrolytic capacitors are still the major life-limiting component in electronics.
They still degrade with time, but in a very predictable way.
That makes it possible to build a version of your design with all capacitors '50 year aged' and check it still works.
Sadly no engineering firm I know does this, despite it being very cheap and easy to do.
The only MLC I use today are Samsungs best industrial drives and they work sort of... but no promises. And SanDisc SD cards that if you buy the cheapest ones last a surprising amount of time. 32GB lasted 11-12 years for me. Now I mostly install 500GB-1TB ones (recently = only been running for 2-3 years) after installing some 200-400GB ones that work still after 7 years.
As a counter anecdote, I have a lot of SSDs from the late 2010s that are still going strong, but I lost some early SSD drives to mysterious and unexpected failures (not near the wear-out level).
For logic and DRAM the biggest factors are how far they're being pushed with voltage and heat, which is a thing that trends back and forth over the years. So I could see that go either way.
Re-freezing is also critical, the container should contain no humid air when it goes into the freezer, because the water will condense and freeze as the container cools. A tightly wrapped bag, desiccant and/or purging the container with dry gas would prevent that.
Color changing desiccant is a good idea to monitor it.
There are ways to keep water out of frozen/re-frozen items, of course, but if you mess up you have water everywhere.
I much rather have 64GB of SLC at 100K WpB than 4TB of MLC at less than 10K WpB.
The spread functions that move bits around to even the writes or caches will also fail.
The best compromise is of course to use both kinds for different purposes: SLC for small main OS (that will inevitably have logs and other writes) and MLC for slowly changing large data like a user database or files.
The problem is now you cannot choose because the factories/machines that make SLC are all gone.
You can still get pure SLC flash in smaller sizes, or use TLC/QLC in SLC mode.
I much rather have 64GB of SLC at 100K WpB than 4TB of MLC at less than 10K WpB.
It's more like 1TB of SLC vs. 3TB of TLC or 4TB of QLC. All three take the same die area, but the SLC will last a few orders of magnitude longer.
So literally put your data in cold storage.
Like does a SSD do some sort of refresh on power-on, or every N hours, or you have to access the specific block, or...? What if you interrupt the process, eg, having a NVMe in an external case that you just plug once a month for a few minutes to just use it as a huge flash drive, is that a problem?
What about the unused space, is a 4 TB drive used to transport 1 GB of stuff going to suffer anything from the unused space decaying?
It's all very unclear about what all of this means in practice and how's an user supposed to manage it.
Generally, the data refresh will all happen in the background when the system is powered (depending on the power state). Performance is probably throttled during those operations, so you just see a slightly slower copy while this is happening behind the scenes.
The unused space decaying is probably not an issue, since the internal filesystem data is typically stored on a more robust area of media (an SLC location) which is less susceptible to data loss over time.
As far as how a user is supposed to manage it, maybe do an fsck every month or something? Using an SSD like that is probably ok most of the time, but might not be super great as a cold storage backup.
(As a note: I do have a 4TB USB SSD which did sit in a drawer without being touched for a couple of years. The data was all fine when I plugged it back in. Of course, this was a new drive with very low write cycles and stored climate controlled. Older worn out drive would probably have been an issue.) Just wondering how long I should keep it plugged in if I ever have a situation like that so I can "reset the fade clock" per se.
sudo pv -X /dev/sda
sudo cat /dev/sda >/dev/null
for copy-on-write filesystems that store checksums along with the data, you can request proper integrity checks and also get the nicely formatted report about how well that went.
for btrfs:
sudo btrfs scrub start -B /
sudo zpool scrub -a -w
sudo tar -cf - / | cat >/dev/null
> sudo dd if=/dev/sdX of=/dev/null bs=4M status=progress iflag=direct
https://www.man7.org/linux/man-pages/man1/dd.1.html
I have no idea if forcing a read is good / the right way. I'm just answering how to do it.
How does the SSD know when to run the refresh job? AFAIK SSDs don't have an internal clock so it can't tell how long it's been powered off. Moreover does doing a read generate some sort of telemetry to the controller indicating how strong/weak the signal is, thereby informing whether it should refresh? Or does it blindly refresh on some sort of timer?
There are several layers of data integrity that are increasingly expensive to run. Once the drive tries to read something that requires recovery, it marks that block as requiring a refresh and rewrites it in the background.
samsung fix was aggressive scanning and rewriting in the background
The case an average user is worried about is where they have an external SSD that they back stuff up to on a relatively infrequent schedule. In that situation, the question is whether just plugging it and copying some stuff to it is enough to ensure that all the data on the drive is refreshed, or if there's some explicit kind of "maintenance" that needs to be done.
Isn't that what periodic "scrub" operations are on modern fs like ZFS/BTRFS/BCacheFS?
> the data refresh will all happen in the background when the system is powered
This confused me. If it happens in the background, what's the manual fsck supposed to be for?
dd if=/dev/sdX of=/dev/null bs=1M status=progress
Edit: found this below: "Powering the SSD on isn't enough. You need to read every bit occasionally in order to recharge the cell."
Hm, so does the firmware have a "read bits to refersh them" logic?
NAND flash is freakishly unreliable, and it's up to the controller to keep this fact concealed from the rest of the system.
Modern controllers have a good idea how healthy the flash is. They will move data around to compensate for weakness. They're doing far more to detect and correct errors than a file system ever will, at least at the single-device level.
It's hard to get away from the basic question, though -- when is the data going to go "poof!" and disappear?
That is when your restore system will be tested.
Maybe as a debug feature some registers can be set up adjust the threshold up and down and the same data reread many times to get an idea of how close certain bits are to flipping, but it certainly isn't normal practice for every read.
That depends on the SSD controller implementation, specifically whether it proactively moves stuff from the SLC cache to the TLC/QLC area. I expect most controllers to do this, given that if they don't, the drive will quickly lose performance as it fills up. There's basically no reason not proactively move stuff over.
The more interesting thing to note from those standards is that the required retention period differs between "Client" and "Enterprise" category.
Enterprise category only has power-off retention requirement of 3 months.
Client category has power-off retention requirement of 1 year.
Of course there are two sides to every story...
Enterprise category standard has a power-on active use of 24 hours/day, but Client category only intended for 8 hours/day.
As with many things in tech.... its up to the user to pick which side they compromise on.
[1]https://files.futurememorystorage.com/proceedings/2011/20110...
Specifically in JEDEC JESD218. (Write endurance in JESD219.)
While if the endurance testing would exceed 1000 hours an extrapolated approach can be used to stress below the TBW but using accelerated techniques (including capping the max writable blocks to increase wear on the same areas).
Which is less dramatic than the retention values seem at first and than what gets communicated in articles I've seen. Even in the OP's linked article it takes a comment to also highlight this, while the article itself only cites its own articles that contain no outside links or citations.
[1] https://www.jedec.org/sites/default/files/Alvin_Cox%20%5BCom...
We treated NVMe drives like digital stone tablets. A year later, we tried to restore a critical snapshot and checksums failed everywhere. We now have a policy to power-cycle our cold storage drives every 6 months just to refresh the charge traps.
It's terrifying how ephemeral "permanent" storage actually is. Tape is annoying to manage, but at least it doesn't leak electrons just sitting on a shelf.
I wish there was archival storage something long-lasting and large capacity. That would make a good complement to flash for writing backups and long-lasting data.
I've seen this topic come up a lot on a datahoarder forum but there was never any consensus on what it would take to assure the data was refreshed because no one really knew how it worked. People would assume anything from a short power up to simply rewriting all data as possibilities.
The firmware engineer responding up thread is actually the best information I've seen on this. And it kind of confirmed my suspicions. He works on enterprise drives and they run a refresh cycle as background cycle that seems to start on a random timer. I didn't see any response (yet) on how long this process takes but I think we can infer that it takes a while based on it being a background process. Probably hours or more. And this is all for an enterprise drive, I wouldn't be surprised if consumer drives have a less robust or even non-existent refresh function.
I'm of the opinion that that based on the little information about some implementations of this function, the logical conclusion is you should just rewrite all of the data over again on your cold backups for the cycle. Powering it on isn't enough, even powering it on for a day might not do anything. As you said this is a pretty big drawback for uses flash as a cold backup.
The theory is that operating system files, which rarely change, are written and almost never re-written. So the charges begin to decay over time and while they might not be unreadable, reads for these blocks require additional error correction, which reduces performance.
There have been a significant number of (anecdotal) reports that a full rewrite of the drive, which does put wear on the cells, greatly increases the overall performance. I haven't personally experienced this yet, but I do think a "every other year" refresh of data on SSDs makes sense.
Eg. Data structures used for page mapping getting fragmented and therefore to access a single page written a long time ago requires checking hundreds of versions of mapping tables.
My desktop computer is generally powered except when there is a power failure, but among the million+ files on its SSD there are certainly some that I do not read or write for years.
Does the SSD controller automatically look for used blocks that need to have their charge refreshed and do so, or do I need to periodically do something like "find / -type f -print0 | xargs -0 cat > /dev/null" to make sure every file gets read occasionally?
I wonder if there's some easy way to measure power consumed by a device - to detect whether it's doing housekeeping.
2) even if your OS supports it, you may have difficulty using it for your root volume, so partitioning is probably required.
2a) in your case you may not want to use it on your boot volume which would negate the SSD benefit for you.
3) it is recommended that you have ECC RAM due to the checksums. This isn’t a hard and fast requirement, but it does make you more resilient to bitflips.
4) it isn’t the absolute fastest file system. But it’s not super slow. There are caching options for read and write that benefit from SSDs, but you’re just adding costs here to get speed increases.
I only use it on servers or NASs. The extra hassles of using it on a workstation keep me from running it on a laptop. Unless you want to use FreeBSD that is… then you’d be fine (and FreeBSD is pretty usable as a daily driver). Realistically, I’m not sure how practical it is for most home users. But it is an example of what a filesystem can offer when it is well designed.
I also think the popularity of FreeNAS especially contributed to the popularity of ZFS.
[0] I still look at XFS skeptically after a crash I suffered nearly 20 years ago. It’s not a rational fear, but it’s still there.
Still something I should look into? Thank you!
Still, unless you want to tinker with something new I can’t really recommend it. Would it work? Yes. Do you need it? No. You’re probably fine with whatever FS you currently have running. ZFS works on Debian, but it’s not first-party support (due to licensing). Do I think you’d have issues if you wanted to try it? Probably not. I’m just conservative in what I’d recommended for a daily use machine. I prioritize working over everything else, so I’d hate for you to try it and end up with a non working system.
Here’s what I’d recommend instead - try it in a VM first. See if you can get it to work in a system setup like yours. See if it’s something that you like. If you want to use it on your primary machine, then you’ll be able to make a more informed decision.
If you can/are willing to use UEFI, ZFSBootMenu is a Linux oriented solution that replicates the power of FreeBSD's bootloader, so you can manage snapshots and boot environments and rollback checkpoints all at boot without having to use recovery media (that used to be required when doing ZFS on Linux). Definitely worth looking into: https://zfsbootmenu.org/
This article just seems to link to a series of other xda articles with no primary source. I wouldn't ever trust any single piece of hardware to store my data forever but this feels like clickbait- At one point they even state "...but you shouldn't really worry about it..."
There was a guy on reddit that took about 20 cheap USB flash drives and checked 1 every 6 months. I think after 3 years nothing was bad yet.
I've copied OS ISO images to USB flash drives and I know they sat for at least 2 years unused. Then I used it to install the OS and it worked perfectly fine with no errors reported.
I still have 3 copies of all data and 1 of those copies is offsite but this scare about SSDs losing data is something that I've never actually seen.
do I just plug it in and let the computer on for a few minutes? does it needs to stay on for hours?
do I need to run a special command or TRIM it?
The problem is the test will take years, be out of date by the time its released and new controllers will be out with potentially different needs/algorithms.
https://www.tomshardware.com/pc-components/storage/unpowered...
The data on this SSD, which hadn't been used or powered up for two years, was 100% good on initial inspection. All the data hashes verified, but it was noted that the verification time took a smidgen longer than two years previously. HD Sentinel tests also showed good, consistent performance for a SATA SSD.
Digging deeper, all isn't well, though. Firing up Crystal Disk Info, HTWingNut noted that this SSD had a Hardware ECC Recovered value of over 400. In other words, the disk's error correction had to step in to fix hundreds of data-based parity bits.
...
As the worn SSD's data was being verified, there were already signs of performance degradation. The hashing audit eventually revealed that four files were corrupt (hash not matching). Looking at the elapsed time, it was observed that this operation astonishingly took over 4x longer, up from 10 minutes and 3 seconds to 42 minutes and 43 seconds.
Further investigations in HD Sentinel showed that three out of 10,000 sectors were bad and performance was 'spiky.' Returning to Crystal Disk Info, things look even worse. HTWingNut notes that the uncorrectable sectors count went from 0 to 12 on this drive, and the hardware ECC recovered value went from 11,745 before to 201,273 after tests on the day.However, this is implementation detail in the SSD FW. For Linux UBI devices, this will suffice.
No idea if that's enough, but it seems like a reasonable place to start.
You just can't trust the hardware to know how to do this, need backup software with multiple backup locations, it will know how to recheck integrity
Should I pop them in an old server? Is there an appliance that just supplies power? Is there a self-hosted thing I can monitor disks which I have 0 access usage for and don't want connected to anything but want to keep "live"
I have a HDD that was 17+ years since it last powered on. I dug it out recently to re-establish some memories, and discovered that it still reads. But of course you need to take care of them well, put them in an Anti-Static Bag or something similar, and make sure the storage environment is dry.
It's not prefect, but at least you don't have to struggle that much maintaining SSDs.
I use them for working with old unmounted hard drives or for cloning drives for family members before swapping them. But they would probably work for just supplying power too?
The one I use the most is an 18 year old Rosewill RCW-608.
I don't know if the firmware/controller would do what it needs to do with only power connected. I wonder if there's some way to use SMART value tracking to tell? Like if power on hours increments surely the controller was doing the things it needs to do?
If not, that feels like a substantial hole in the market. Non-flash durable storage tend to be annoying or impractical for day to day use. I want to be able to find a 25 year old SD card hiding in some crevice and unearth an unintentional time capsule, much like how one can pick up 20+ year old MiniDiscs and be able to play the last thing their former owners recorded to them perfectly.
In general, though, whether NAND or NOR, the fundamental way that flash works is by creating an isolated voltage charge for each bit (or several bits, for TLC), making it effectively a vast grid of very tiny batteries.
Like all batteries, no matter how well-stored, they will eventually leak energy to the point where the voltage levels change enough to matter.
Further, it's not enough to simply make power available to this grid since the refresh of the cells requires active management by a nand controller chip and its associated software stack.
How often does it need to run? If it could be solar powered you could probably avoid a whole bunch of complexity per unit longevity.
Long thread here:
https://www.techpowerup.com/forums/threads/samsung-870-evo-b...
I may not have noticed had fsck not alerted me something was wrong.
It will trigger reads in random areas in flash, and try ti correct any errors found.
Without it, the same issue as in the original article will happen (even if the device is powered on): areas in the NAND were not read for long time will have more and more errors, causing them to be non recoverable.
The difference between slc and mlc is just that mlc has four different program voltages instead of two, so reading back the data you have to distinguish between charge levels that are closer together. Same basic cell design. Honestly I can’t quite believe mlc works at all, let alone qlc. I do wonder why there’s no way to operate qlc as if it were mlc, other than the manufacturer not wanting to allow it.
~ Terry Pratchett's Disklessworld, book 8. (Re: Any sufficiently advanced technology is indistinguishable from magic)
You can run an error-correcting code on top of the regular blocks of memory, storing, for example (really an example; I don’t know how large the ‘blocks’ that you can erase are in flash memory), 4096 bits in every 8192 bits of memory, and recovering those 4096 bits from each block of 8192 bits that you read in the disk driver. I think that would be better than a simple “map low levels to 0, high levels to 1” scheme.
There is a way to turn QLC into SLC: https://news.ycombinator.com/item?id=40405578
Loads of drives do this(or SLC) internally. Though it would be handy if a physical format could change the provisioning at the kernel accessible layer.
Manufacturers often do sell such pMLC or pSLC (p = pseudo) cells as "high endurance" flash.
tlc/qlc works just fine, it's really difficult to consume the erase cycles unless you really are writing 24/7 to the disk at hundred of megabytes a second
Um. Backups seem like exactly why I might have data on an unpowered SSD.
I use HDDs right now because they're cheaper, but that might not be true some day. Also, I would expect someone less technically inclined than I am to just use whatever they have lying around, which may well be an SSD.
OptiNAND is a "SSHD" and thus has the same concerns with retention as an SSD. https://en.wikipedia.org/wiki/Hybrid_drive
the real issue here is QLC in which the flash cell's margins are being squeezed enthusiastically...
Flash storage is apparently cheaper (especially for smaller production runs) and/or higher density these days, so these cartridges just use that and make it appear ROM-like via a controller.
How many people have a device that they may only power up ever few years, like on vacation. In fact, I have a device that I've only used on rare occasions these days (an arcade machine) that now I suspect I'll have to reinstall since It's been 2 or 3 years since I've last used it.
This is a pretty big deal that they don't put on the box.
zfs in these filesystem-specific parity-raid implementations also auto-repairs corrupted data whenever read, and the scrub utility provides an additional tool for recognizing and correcting such issues proactively.
This applies to both HDDs and SSDs. So, a good option for just about any archival use case.
In a raidz1, you save one of the n drives' worth of space to store parity data. As long as you don't lose that same piece of data on more than one drive, you can reconstruct it when it's brought back online.
And, since the odds of losing the same piece of data on more than one drive is much lower than the odds of losing any piece of data at all, it's safer. Upping it to two drives worth of data, and you can even suffer a complete drive failure, in addition to sporadic data loss.
The odds of losing the same piece of data on multiple drives is much lower than losing any piece of data at all.
Had to replace them.
I'm unsure if dd if=/the/disk of=/dev/null does the read function.
so it's as if the data... rusts, a little bit at a time
dd if=$1 of=/dev/null iflag=direct bs=16M status="progress"
smartctl -a $1
Furthermore, replication isn't a backup.
Even if you are willing to spend that small fortune, good luck actually getting all the parts together without enterprise contracts.
[Edit: LOL, I see someone else posted literally the same example within the same minute. Funny coincidences.]
That said, they could also be storing relatively small amounts. For example, I back up to Backblaze B2, advertised at $6/TB/month, so ~300 MB at rest will be a "couple" bucks.
If I have enough data to need multiple SSDs (more than 8TB) then the cloud cost is not going to be substantially less. B2 is going to be above $500 a year.
I can manage to plug a backup SSD into a phone charger a couple times a year, or leave it plugged into one when it's not in my computer being updated. Even if I rate that handful of minutes of labor per year at a gratuitous $100, I'm still saving money well before the 18 month mark.
One concern I have is B2's downloading costs means verifying remote snapshots could get expensive. I suppose I could use `restic check --read-data-subset X` to do a random spot-check of smaller portions of the data, but I'm not sure how valuable that would be.
I like how it resembles LUKS encryption, where I can have one key for the automated backup process, and a separate memorize-only passphrase for if things go Very Very Wrong.
[0] https://restic.readthedocs.io/en/latest/080_examples.html#ba...
Long enough to experience data rot to a small degree but realistically what proportion of users have archived things away for 10+ years then audited the fidelity of their data on retrieval after fetching it from Glacier
I rotate religiously my offline SSDs and HDDs (I store backups on both SSDs and HDDs): something like four at home (offline onsite) and two (one SSD, one HDD) in a safe at the bank (offline offsite).
Every week or so I rsync (a bit more advanced than rsync in that I wrap rsync in a script that detects potential bitrot using a combination of an rsync "dry-run" and known good cryptographic checksums before doing the actual rsync [1]) to the offline disks at home and then every month or so I rotate by swapping the SSD and HDD at the bank with those at home.
Maybe I should add to the process, for SSDs, once every six months:
... $ dd if=/dev/sda | xxhsum
Note that I'm already using random sampling on files containing checksums in their name, so I'm already verifying x% of the files anyway. So I'd probably be detecting a fading SSD quite easily.
Additionally I've also got a server with ZFS in mirroring so this, too, helps keep a good copy of the data.
FWIW I still have most of the personal files from my MS-DOS days so I must be doing something correctly when it comes to backing up data.
But yeah: adding a "dd to xxhsum" of the entire disks once every six months in my backup'ing script seems like a nice little addition. Heck, I may go hack that feature now.
[1] otherwise rsync shall happily trash good files with bitrotten ones
This is somewhat confused writing. Consumer SSDs usually do not have a data retention spec, even in this very detailed Micron datasheet you won't find it: https://advdownload.advantech.com/productfile/PIS/96FD25-S2T... Meanwhile the data retention spec for enterprise SSDs is at the end of their rated life, which is usually a DPWD/TBW intensity you won't reach in actual use anyway - that's where numbers like "3 months @ 50 °C" or whatever come from.
In practice, SSDs don't tend to loose data over realistic time frames. Don't hope for a "guaranteed by design" spec on that though, some pieces of silicon are more equal than others.
> Component Design Life 5 years
> TBW 14 PB for 7.68 TB drives
> Data Retention 3 months
And then 2.7.3 explains that this number applies for 40 °C ambient, not the operating/non-operating range (up to 85 °C).