Sure, Rust is hardly the first language to include something like that and adoption of such systems tends to be ... spotty. But if it was reliable enough and had a better interface (that preferably allowed the rest of your program to sill have panics) this might be very useful for writing correct software.
From the article and only vague background Rust knowledge, I'm under the impression that the opposite is true: the compiler does not prove that. Hence why it's "assert_unchecked" - you are informing the compiler that you know more than it does.
You do get panics during debug, which is great for checking your assumptions, but that relies on you having adequate tests.
#[cfg(any(target_pointer_width = "32", target_pointer_width = "64"))]
#[inline(always)]
const fn usize_to_u32(x: usize) -> u32 {
x as u32
}
and this way you can just call this function and you'll get a compile-time error (no such function) if you're on a 16-bit platform.This wasn't a concern for me but I could also imagine some sort of linting being used to ensure that potentially lossy casts aren't done, and while it presumably could be manually suppressed, that also would just add to the noisiness.
From tools like "spin" and "tla+" to proof assistants like Coq to full languages like Idris and Agda.
Some of the stronger-typed languages already give us some of those benefits (Haskell, OCaml) and with restricted effects (like Haskell) we can even make the compiler do much of this work without it leaking into other parts of the program if we don't want it to.
Of course this doesn't prove that the assert will happen. You would have to execute the code for that. But you can treat the fact that the optimizer couldn't eliminate your assert as failure, showing that either your code violates the assert, your preconditions in combination with the code aren't enough to show that the assert isn't violated, or the whole thing was too complicated for the optimizer to figure out and you have to restructure some code
The big question is how much real-world code the optimizer would be capable of "solving" in this way.
I wonder if most algorithms would eventually be solvable if you keep breaking them down into smaller pieces. Or if some would have some step of irreducible complexity that the optimizer cannot figure out, now matter how much you break it down.
edit: There is https://github.com/dtolnay/no-panic
So you’re probably going to have to protect this with a cfg_attr to only apply the no_panic in release only.
That or forcing compilation to always target release.
If you have an assumption that gives unhelpful information, the optimizer will emit panic code. Worse, if the assumption is incorrect, then the compiler can easily miscompile the code (both in terms of UB because of an incorrectly omitted panic path AND because it can miscompile surprising deductions you didn’t think of that your assumption enables).
I would use the assume crate for this before this got standardized but very carefully in carefully profiled hotspots. Wrapping it in a safe call as in this article would have been unthinkable - the unsafe needs to live exactly where you are making the assumption, there’s no safety provided by the wrapper. Indeed I see this a lot where the safety is spuriously added at the function call boundary instead of making the safety the responsibility of the caller when your function wrapper doesn’t actually guarantee any of the safety invariants hold.
I want to be sure I understand your meaning. In your analysis, if the check_invariant function was marked unsafe, would the code be acceptable in your eyes?
This was most shocking to me in some of the Rust code Mozilla had integrated into Firefox (the CSS styling code). There was some font cache shenanigans that was causing their font loading to work only semi-consistently, and that would outright crash this subsystem, and tofu-ify CJK text entirely as a result.
And the underlying panic was totally recoverable in theory if you looked at the call stack! Just people had decided to not Result-ify a bunch of falliable code.
This is why Result (or Maybe, or runExceptT, and so on in other languages) is a perfectly safe way of handling unexpected or invalid data. As long as you enforce your invariants in pure code (code without side effects) then failure is safe.
This is also why effects should ideally be restricted and traceable by the compiler, which, unfortunately, Rust, ML, and that chain of the evolution tree didn't quite stretch to encompass.
In a good software architecture (imo) panics and other hard failure mechanisms are there for splitting E into E1 and E2, where E1 is the set of errors that can happen due to the caller screwing up and E2 being the set of errors that the caller screwed up. The caller shouldn't have to reason about the callee possibly being incorrect!
Functional programming doesn't really come into the discussion here - oftentimes this crops up in imperative or object oriented code where function signatures are lossy because code relies on side effects or state that the type system can't/won't capture (for example, a database or file persisted somewhere). Thats where you'll drop an assert or panic - not as a routine part of error handling.
Ideally, you can constrain the set of inputs to only valid ones by leveraging types. But if that's not possible and a truly invalid input is passed, then you should panic. At least that's the mental model that Rust is going with.
You do lose out on the ability to "catch" programming errors in subcomponents of your program. For example, it's extremely useful to catch exceptions related to programming errors for called code in response to a web request, and return a 500 in those cases. One could imagine a "try" "catch" for panics.
The thing is, it takes a lot of discipline by authors to not riddle their code with panics/exceptions when the language provides a try/catch mechanism (see C# and Java), even when a sensible error as value could be returned. So Rust opts to not introduce the footgun and extra complexity, at the expense of ungraceful handling of programming errors.
But how can the caller know what is "a truly invalid input"? The article has an example: "we unfortunately cannot rely on panic annotations in API documentation to determine a priori whether some Rust code is no-panic or not."
It means that calling a function is like a lottery: some input values may panic and some may not panic. The only way to ensure that it doesn't panic is to test it with all possible input values, but that is impossible for complex functions.
It would be better to always return an error and let the caller decide how to handle it. Many Rust libraries have a policy that if the library panics, then it is a bug in the library. It's sad that the Rust standard library doesn't take the same approach. For println!(), it would mean returning an error instead of panicking.
The best way to handle this is to crash the program. If you need constant uptime, then restart the program. If you absolutely need to keep things running then, yeah try to recover then. The last option isn't as bad for something like an http server where one request caused it to error and you just handle that error and keep the other threads running.
But for something like a 3D video game. If you arrive at erroneous state, man. Don't try to keep that thing going. Kill it now.
True: a program can't fix an internal assertion error, and the failed component might not be recoverable without a reset. But that doesn't means that the whole program is doomed. If the component was optional the program might still work although with reduced functionality. Consider this: way you do not stop the whole computer if a program aborts.
As I mentioned elsethread, in unsafe languages an assertion error might, with high probabilty, be due to the whole runtime being compromised, so a process abort is the safest option.
No, your program correctly determined that user input was invalid.
Or your parser backtracked from parsing a Bool and decided to try to parse an Int instead.
Say user input is a number and can never exceed 5. If the user input exceeds 5 your program should handle that gracefully. This is not invalid state. It is handling invalid input while remaining in valid state.
Let say it does exceed 5 and You forget to check that it should never exceeds 5 and this leads to a division by zero further down your program. You never intended for this to happen and you don’t k ow why this happened. This is invalid state.
Now either this division by zero leads to an error value or it can throw an exception. It doesn’t matter. Do you know how to recover? You don’t even know where the bug is. A lot of times your don’t even know what to do. This error can bubble up all the way to main and now what do you do with it?
You crash the program. Or you can put in a fail safe before the error bubbles up to main and do something else but now (if your program retains and mutates that state) has invalid values in it and a known bug as well.
Imagine that input number represented enumeration values for movement of some robot. You now have one movement that doesn’t exist or was supposed to be something else. Thus if you keep your program running the robot ends up in an unexpected and erroneous place. That’s why I’m saying a program should crash. It should not be allowed to continue running with invalid state.
Like imagine if this was a mission critical auto pilot and you detect negative 45 meters for altitude. Bro crash and restart. Don’t try to keep that program running by doing something crazy in attempt to make the altitude positive and correct. Reset it and hope it never goes to the invalid state again.
No thank you.
> Or you can put in a fail safe before the error bubbles up to main and do something else
In other words,
>> your parser backtracked from parsing a Bool and decided to try to parse an Int instead
> but now (if your program retains and mutates that state) has invalid values in it and a known bug as well.
Unless,
>>> The thing with functional programming (specifically, immutable data,) is that as long as the invalid state is immutable, you can just back up to some previous caller, and they can figure out whether to deal with it or whether to reject up the its previous caller.
Look at my auto pilot example. You have a bug in your program you don’t know about. Altitude reads -9999 meters your heading and direction is reading at the speed of light. Your program sees these values and recognizes invalid state. There is a BUG in your program. Your program WAS never designed to go here.
You want to try to recover your autopilot program? How the fuck are you gonna do that? You don’t even know where the bug is. Does your recovery routine involve patching and debugging? Or are you gonna let your autopilot keep operating the plane with those nonsense values? That autopilot might hit the wind breaks to try to slow down the plane and end up crashing the thing.
You don’t recover this. You restart the program and pray it doesn’t hit that state again then when you are on the ground you debug it.
>>> The thing with functional programming (specifically, immutable data,) is that as long as the invalid state is immutable, you can just back up to some previous caller, and they can figure out whether to deal with it or whether to reject up the its previous caller.
This makes no fucking sense. Where the hell will you back up to? Your error came from somewhere but you don’t know where. Return an error value in the child function the parent function receives the error and returns an error itself and this keeps going until you bubble up to the main function because you don’t know where that error came from.
Now you have an error in the main function. Wtf are you gonna do? How do you handle an error in the main function that you have no idea where it came from? Here’s an idea. You have the main function restart the program loop. See a similarity here? It’s called crashing and restarting the program. Same effect!
This isn’t a functional programming versus non functional thing. It’s a concept that you’re not seeing.
They are great for handling expected errors that make sense to handle explicitly.
If you try to wrap up any possible error that could ever happen in them you will generate horrendous code, always having to unwrap things, everything is a Maybe. No thanks.
I know it is tempting to think "I will write the perfect program and handle all possible errors and it will never crash" but that just results in overly complex code that ends up having more bugs and makes debugging harder. Let it crash. At the point where the error happened, don't just kick the bucket down the road. Just log the the problem and call it a day. Exceptions are an amazing tool to have for things that are.. exceptions.
I actually disagree with this. Use the maybe type religiously, even without the monad because it prevents errors via exhaustive matching.
The exception should only be used if your program detects a bug.
I understand your point in general, I just find that if you're writing a program that is running on unconstrained environments, not panic'ing (or at least not doing it so bluntly at a low level) can at the very least help with debugging.
At least have the courtesy to put the panic at a higher level to provide context beyond "key not found!"!
Either the library should have enforced the invariant of the key existing (and returned an equivalent error, or handled it internally), or documented the preconditions at a higher level function that you could see.
[1] even safe languages have unsafe escape hatches and safe abstractions sometimes have bugs, but on most cases you can assume that the runtime is not compromised.
Even if you are no_std, core has tons of APIs like unwrap(), index slicing, etc. that can panic if you violate the preconditions. It's not practical to grep for all of them.
No-panic looks nifty: it appears to be reliable, which is great. I wish there was an easy way to automatically apply this annotation to every single function in a given file or crate.
https://doc.rust-lang.org/beta/src/std/io/stdio.rs.html#674-...
The implementation calls indeed a panicking API, OnceLock::get_or_init:
https://doc.rust-lang.org/beta/std/sync/struct.OnceLock.html...
But it only panicks if it is being used in a wrong way, which it isn't. The usage is contained within the implementation of std::io::stdout, so it's an implementation detail.
It's a shame that there are no better ways to eliminate panics in case they are impossible to trigger. The article shows some tricks, but I think the language is missing still some expressability around this, and the stdlib should also thrive harder to actually get rid of hard-to-optimize links to panic runtime in case of APIs that don't actually panic.
The main reason getting panic-free stdout is hard, is that the STDOUT initializers both call allocating constructors, and LineWriter doesn't have non-allocating constructors.
Solving this elegantly seems hard.
I 100% agree with your last paragraph. I would love if the language and stdlib made it easier to make panic-free binaries.
Still miffed, but we'll get there.
But I would assume that for mozilla their entire CSS subsystem is pulled in as a git (hg?) submodule or something anyways.
Essentially an error means that the caller failed in a way that's expected. A panic means the caller broke some contract that wasn't expressed in the arguments.
A good example of this is array indexing. If you're using it you're saying that the caller (whoever is indexing into the array) has already agreed not to access out of bounds. But we still have to double check if that's the case.
And if you were to say that hey, that implies that the checks and branches should just be elided - you can! But not in safe rust, because safe code can't invoke undefined behavior.
Obviously context-free this is very hand wave-y, but would you want Firefox to crash every time a website prematurely closes its connection to your browser for whatever reason? No, right? You would want Firefox to fail gracefully. That is what I wanted.
Is fundamentally unsound `check_invariant` needs to be unsafe as it doesn't actually check the invariant but tells the compiler to blindly assume they hold. Should probably also be named `assume_invariant_holds()` instead of `check_invariant()`.
Panics really seem bad for composability. And relying on the optimzer here seems like a fragile approach.
(And how is there no -nopanic compiler flag?)
Trying to enforce no-panics without a proof system helping out is just not a very practical approach to programming. Consider code like
some_queue.push_back("new_value");
process(some_queue.pop_front().unwrap());
This code is obviously correct. It never panics. There's no better way to write it. The optimizer will instantly see that and remove the panicing branch. The language itself doesn't want to be in the business of trying to see things like that.Or consider code like
let mut count: usize = 0;
for item in some_vec {
// Do some stuff with item
if some_cond() {
count += 1;
}
}
This code never panics. Integer arithmetic contains a hidden panic path on overflow, but that can't occur here because the length of a vector is always less than usize::MAX.Or so on.
Basically every practical language has some form of "this should never happen" root. Rust's is panics. C's is undefined behavior. Java's is exceptions.
Finally consider that this same mechanism is used for things like stack overflows, which can't be statically guaranteed to not occur short of rejecting recursion and knowledge of the runtime environment that rustc does not have.
---
Proof systems on top of rust like creusot or kani do tend to try to prove the absence of panics, because they don't have the same compunctions about not approving code today that they aren't absolutely sure they will approve tomorrow as well.
It doesn't panic within the code you typed, but it absolutely still can panic on OOM. Which is sort of the problem with "no panic"-style code in any language - you start hitting fundamental constructs that can can't be treated as infallible.
> Basically every practical language has some form of "this should never happen" root.
99% of "unrecoverable failures" like this, in pretty much every language, are because we treat memory allocation as infallible when it actually isn't. It feels like there is room in the language design space for one that treats allocation as a first-class construct, with suitable error-handling behaviour...
It's not really what this is about IMV. The vast majority of unrecoverable errors are simply bugs.
A context free example many will be familiar with is a deadlock condition. The programmer's mental model of the program was incomplete or they were otherwise ignorant. You can't statically eliminate deadlocks in an arbitrary program without introducing more expensive problems. In practice programmers employ a variety of heuristics to avoid them and just fix the bugs when they are detected.
The same article written a couple years in the future would look very different
In other environments, like embedded or safety-critical devices, I would need a guarantee that even heap allocation failure can not cause a panic.
panics are very much designed to be recoverable at some well defined boundaries (e.g. the request handler of a web server, a thread in a thread pool etc.)
this is where most of it's overhead comes from
you can use panic=abort setting to abort on panics and there is a funny (but unpractical) hack with which somewhat can make sure that no not-dead-code-eliminated code path can hit a panic (you link the panic->abort handler to a invalid symbol)
However, since it is still possible to have them in a place where the exiting the process is not okay, it was beneficial to add a way to recover from them. It does not mean that they are designed to be recoverable.
> this is where most of it's overhead comes from
Overhead comes from the cleaning process. If you don't clean properly, you might leak information or allocate more resources than you should.
no it's them being recoverable at well defined boundaries is a _fundamental_ design aspect of rust
> Overhead comes from the cleaning process. If you don't clean properly, you might leak information or allocate more resources than you should.
and that same cleanup process makes it recoverable
https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-...
so that e.g. you web server doesn't fall over just because one request handler panics
and it still is fundamental required that any code is safe in context of panic recovery (UnwindSafe is misleading named and actually not about safety, anything has to be "safe" in context of unwind, UnwindSafe just indicates that it's guaranteed to have sensible instead of just sound behavior in rust)
people over obsessing with panic should be unrecoverable is currently IMHO on of the biggest problems in rust not in line with any of it's original design goals or how panics are in the end design to work or how rust is used in many places in production
yes they are not "exceptions" in the sense that you aren't supposed to have fine grained recoverablility, but recoverability anyway
without recoverable panics you wouldn't be able to write (web and similar) servers in a reasonable robust way in rust without impl some king of CGI like pattern, which would be really dump IMHO and is one of the more widely used rust use cases
But as long as you don't do anything unusual you are basically introducing a (potentially huge) availability risk to your service for no reason but except not liking panics.
Like it's now enough for there to be a single subtle bug causing a panic to have a potentially pretty trivially exploitable and cheap DoS attack vector. Worse this might even happen accidentally. Turning a situation where some endpoints are unavailable due to a bug into one where your servers are constantly crashing.
Sure you might gain some performance, but for many use cases this performance is to small to reason in favor of this decisions.
Now if you only have very short lived calls, and not too many parallel calls in any point in time, and anyway spread scaling across many very very small nodes it might not matter that you might kill other requests, but once that isn't the case it seems to most times be a bad decision.
It also doesn't really add security benefits, maybe outside of you having very complicated in memory state or similar which isn't shared across multiple nodes through some form of db (if it is, you anyway have state crossing panics, or in your case service restarts).
Tearing down the entire process tends to be a pretty safe alternative.
yes and that's most times the better design decision
> operating on an &mut T, while state is invalid.
but you normally don't and for many use cases is in my experiment a non-issue
panic recovery isn't fine grained and passing `&mut T` data across recovery boundaries is bothersome
at the same time most of the cross request handler in-memory state is stuff like caches and similar which as long as you don't hand role them should work just fine with panic recovery
At the same time deciding to not recover some kinds of shared state and recreate them isn't hard at all, at least assuming no spaghetti code with ton's of `static` globals are used.
And sure there are some use cases where you might prefer to tear down the process, but most web-server use cases aren't anywhere close to it in my experience. I can't remember having had a single bug because of panic recovery in the last like ~8?? years of using it in production (yes a company I previous worked for started using it in production around it's 1.0 release).
EDIT: Actually I correct myself 2 bugs. One due to Muxtex poison which wouldn't have been a problem if the Muxtex didn't had poison. And another due to a certain SQL library insisting of not fixing a long standing bug by insisting on reexport a old version of a crate known to be broken which had been fixed upstream because they didn't like the way it was fixed and both not documenting that you can't use it with panic recovery and closing all related issues because "if you use panics you are dump".
But both of them where like 6-8 years ago.
EDIT - shockingly, reader mode also fails completely after the page reloads itself
Note that on Android process separation is not usually as good, so a crashing iframe can bring down the whole page.
Here’s a archived link:
https://web.archive.org/web/20250204050500/https://blog.reve...
I can't replicate the crash at all on my Linux cloud VM though. Usually the only difference there is that advertisers tend to not buy ads for clients on cloud IPs.
Specifically, how does it handle recursion? Consider, for example, the following function, which decrements a number until it reaches zero. At each step, it asserts that the number is nonzero before recursing:
fn recursive_countdown(n: u32) -> u32 { assert!(n > 0, "n should always be positive"); if n == 1 { return 1; } recursive_countdown(n - 1) }
Can the compiler prove that the assertion always holds and possibly optimize it away? Or does the presence of recursion introduce limitations in its ability to reason about the program?
Basically, the promise here "We formally promise not to promise anything other than the fact that optimized code should have 'broadly' the same effects and outputs as non-optimized code, and if you want to dig into exactly what 'broadly' means prepare to spend a lot of time on it". Not only are there no promises about complexity, there's no promises that it will work the same on the same code in later versions, nor that any given optimization will continue firing the same way as you add code.
You can program this way. Another semi-common example is taking something like Javascript code and carefully twiddling with it such that a particular JIT will optimize it in a particular way, or if you're some combination of insane and lucky, multiple JITs (including multiple versions) will do some critical optimization. But it's the sort of programming I try very, very hard to avoid. It is a path of pain to depend on programming like this and there better be some darned good reason to start down that path which will, yes, forever dominate that code's destiny.
Is there some context I'm missing here? Is this to be used from non-Rust applications for example?
Typically, safety-related processes are set up in two phases. First they set up, then they indicate readiness and perform their safe operation. A robot, for example, may have some process checking the position of the robot against a virtual fence. If the probability for passing through that fence passes some limit, this requires the process to engage breaks. The fence will need to be loaded from a configuration, communication with the position sensors will need to be established, the fence will generally need to be transformed into coordinates that can be guaranteed to be checked safely, taking momentum and today's breaking performance in account, for example. The breaks itself may need to be checked. All that is fine to do in an unsafe state with panics that don't just abort but unroll and full std. Then that process indicates readiness to the higher-level robot control process.
Once that readiness has been established, the software must be restricted to a much simpler set of functions. If libraries can guarantee that they won't call panic!, that's one item off our checklist that we can still use them in that state.
Using this flag one than can use `panic_abort`. This will eliminate the unwinding part but would still give a "nice" printout on a panic itself. This reduces, in most cases, the mention bloat by a lot. Though nice printouts also cost binary space. For eliminating that `panic_immidiate_abort` exists.
But yeah the above is only about bloat and not the core goal to eliminate potential path's in your program, that would lead to a panic condition itself.
Also currently building the std library yourself needs a nightly compiler. There is afaik work on bringing this to a stable compiler but how exactly is still work in progress.
Something like: Given a function, rewrite its signature to return a Result if it doesn't already, rewrite each non-Resulty return site to a Some(), add a ? to every function call, then recurse into each called function and do the same.
[1] https://doc.rust-lang.org/std/panic/fn.catch_unwind.html
You really want to avoid sharing mutable objects across a catch_unwind() boundary, and also avoid using it on a regular basis. Aside from memory leaks, panicking runs the thread's panic hook, which by default prints a stacktrace. You can override the panic hook to be a no-op, but then you won't see anything for actual panics.
Instead of serializing data (to disk, not the network), it would be much faster if Rust allowed us to allocate datastructures directly in an mmapped file, and allowed us to read back the data (basically patching the pointers so they become valid if the base address changed).
Wouldn’t panicking asap make debugging easier?
"We'd much rather like to make library to corrupt the memory of the rest of the application and generally make the demons fly out of the users' noses, as it does when written in C"?
I believe implementations of C stdio also can abort on program startup if somehow the pthreads' locking mechanism is broken (or if e.g. fcntl(2)/open(2) keeps returning -1), Rust is not that unique in this regard.
On the same page are also the options for controlling debug assertions and overflow checks, which would get rid of the "panics in debug, but not release", if that behavior bugs you
1: https://doc.rust-lang.org/cargo/reference/profiles.html#pani...
``` [profile.release] panic = 'abort' ```
Setting `panic = abort` disables unwinding and this means leaking memory, not closing file descriptors, not unlocking mutexes and not rolling back database transactions on panic. It's fine for some applications to leave this to the operating system on process exit but I would argue that the default unwinding behavior is better for typical userspace applications.
The whole of modern operating systems are already very familiar with the idea of programs not being able to exit gracefully, and there’s already a well understood category of things that happen automatically even if your program crashes ungracefully. Whole systems are designed around this (databases issuing rollbacks when the client disconnects, being a perfect example.) The best thing to do is embrace this and never, ever rely on a Drop trait being executed for correctness. Always assume you could be SIGKILLed at any time (which you always can. Someone can issue a kill -9, or you could get OOM killed, etc.)
But there are still cases where you would like to fsync your mmaps, print out a warning message or just make sure your #[should_panic] negative tests don't trigger false positives in your tooling (like leak detectors or GPU validators) or abort the whole test run.
It's not perfect by any means but it's better than potentially corrupting your data when a trivial assert fires or making negative tests spew warnings in ci runs.
It's very easy to opt out from, and I don't consider the price of panic handlers and unwinding very expensive for most use cases.
But I tend to lament the overall tendency for people to write cleanup code in general for this kind of thing. It’s one of those “lies programmers believe about X” kinds of scenarios. Your program will crash, and you will hit situations where your cleanup code will not run. You could get OOM killed. The user can force quit you. Hell, the power could go out! (Or the battery could go dead, etc.)
Nobody should ever write code that is only correct if they are given the opportunity to perfectly clean up after any failure that happens.
I see this all the time: CLI apps that trap Ctrl-C and tell you you can’t quit (yes I bloody well can, kill -9 is a thing), apps which don’t bother double checking that the files they left behind on a previous invocation are actually still used (stale pid files!!!), coworkers writing gobs of cleanup code that could have been avoided by simply doing nothing, etc etc.
It also feels like most of the pains on avoiding panics centers around allocations which, though a bit unfortunate, makes sense; it was an intentional design choice to make allocations panic instead of return Results, because most users of the language would probably crash on allocation fails anyways and it would introduce a lot of clutter. There was some effort some while ago on having better fallible allocations, but I'm not sure what happened over there.