Whenever I see such things, I immediately think "whatever the resulting order is, it had better not matter"; and if it does, which is definitely true for Java classpaths, I consider it a bug that needs to be fixed ASAP, before it causes what happened in the article.
Maybe there should be a mount options to randomize directory order that people can use in their staging environments.
The behavior I've witnessed suggests that the the order is based on inode numbering, which is initially sequential from creation time, and drifts semi-randomly as inodes are unlinked and reused. I don't know this for a fact, but it makes enough sense. Directory ordering should be assumed to be random in all cases, as you suggest.
Since duplicates on the classpath don't cause problems, a quick & dirty fix is to manually list versioned jars first, in order, then the jars/* argument.
> up until the node image update, we “fortunately” had node images with directory hash seeds ordering “jdk15” or “jdk18” before “jdk14”.
So the actual bug is that something needing jdk15+ should either retry or be deterministically fed a valid file, right? And this whole article is figuring out why the filesystem coincidentally masked it by accidentally always happening to hand it a file with what it needed?
Actually, no, that "15" refers to Java 1.5, aka Java 5, released 2004. Bouncy Castle has some funky variants, specially for Java 1.1, 1.2, 1.3, 1.4, 5, 6, 7, 8. All you actually need is the Bouncy Castle for Java 8 onwards, which is pretty much all versions of Java in use today.
The bug is that multiple providers of Bouncy Castle don't cleanly work when in the classpath together. The authors of Bouncy Castle aren't changing that, because they're like "use our software correctly, please". It's not Java's fault, you can only make classes that don't work on old versions of the JDK, you can't make new Java somehow notice you've included a jar written specifically for an old version of the JVM.
Java did introduce the ability to create multi-release jar files, where you can have JDK-version-specific classes/resources in one jar file... but only from Java 9 onwards. All this mixing and matching by filename that Bouncy Castle uses is for Java 1.1 - Java 1.8 only.
You can also mix and match and cause failure by using one of the Bouncy Castle JCE provider variants with the wrong corresponding "pkix", "util", "mail" jars (extra jars for all the things you might want to do with cryptography that _aren't_ part of the standardised Java Cryptography Extensions API that the main "provider" jar implements). And you can also mess up by mixing FIPS-approved BC with FIPS-not-approved BC.
You only need one set of jars:
* If you don't need FIPS approval: bcprov-jdk18on, bcutil-jdk18on, bcpkix-jdk18on, ...
* If you do: bc-fips, bcutil-fils, bcpkix-fips, ...
If you read files in the same order they are on disk (often, the order in which they were written, which readdir on modern filesystems should choose to produce), I/O is much faster.
> the title is a cheeky reference to something at the front page of the orange site today
Yes, that's what I'm asking. Why do people refer to HN as "the orange site"?
I've seen it typically (though not universally) used seemingly dismissively, so I've always assumed it was a euphemism. People very commonly refrain from naming a thing directly if they disapprove.
Disclaimer: I'm no mind reader.
So if order matters, always sort.
(Luckily in most situations where dir iter order matters, the performance impact from sorting is acceptable or even outright irrelevant.)
That doesn't tell you anything; the parent's dentries could have been cached days ago and still present, meaning it didn't actually access the disk or cause it to be spun up (if it wasn't) at all.
When doing any kind of repeatable measurement or experimentation on disks you will want to drop the page cache every time first:
# echo 3 >/proc/sys/vm/drop_cachesI have a folder with 5500 subfolders. Doing "ls -U" in that folder (after dropping the page cache like above) takes 50 seconds (!) And the dir entries appear all at once, i.e. not in a streaming way.
Its parent folder only contains 6 subfolders. Doing a cache drop followed by "ls -U" gives immediate results.
How to investigate this further? (Using an Ubuntu 18.04 system)
Dropping the cache and doing "time ls -U" gives:
real 0m51,204s
user 0m0,116s
sys 0m0,718s
So -cp /jars/ instead of -cp /jars/*?
Another fix can be to use a fat jar (containing your software and all its dependencies), but this doesn't work for Bouncy Castle, because Cryptography Is Special(TM), and Java won't load cryptography providers unless their jars are signed, and including the cryptography provider jar in the far jar means it loses its signature.
I doubt anyone is doing that manually, that’s probably done by mvn/gradle/sbt/whatever the cool Java kids use these days. Do the build tools not know about this problem and just make a mess?
But Bouncy Castle - and almost nothing else - adds another dimension across its artifact names. This is not standard! You now have to watch your dependency trees like a hawk to see that some other artifact doesn't bring in <artifactId>bcprov-jdk14</artifactId> to fuck with your <artifactId>bcprov-jdk18on</artifactId>, and if they do, you need to slap an <exclusion> on that dependency's dependency.
The reason Bouncy Castle does this is because it chooses to support some very old versions of Java, that predate JDK 9 introducing multi-release jars (https://docs.oracle.com/en/java/javase/21/docs/specs/jar/jar...) which removed the need for different named jars for different JDKs (...but only from JDK 9 onwards)
So, in general, the Java tools have this solved, unless you're Bouncy Castle.
I'm feeling like an old man now but who the hell calls a tool "buildah"? Especially with its ugly dog logo. You can almost assume the dog wants to say "builder" but the extra flaps of skin makes the sound distorted
> Since I’m relatively new to the world of containers and images, I was excited to learn about the Buildah tool. Especially since I’m a native New Englander and it’s a clever play on how we say Builder in these parts. [0]
[0] https://buildah.io/blogs/2017/06/22/introducing-buildah.html
Kinda like Buildly or Buildr. It’s cool until it’s your turn to be old. Then you look back and wince.
That's purely a familiarity effect; it's a self-solving problem.
Is it, or it's just a niche just like people who write 5 digit years, putting a 0 in front?
It's still very rare to encounter any of those.
Bostonians? :P
All that said, probably wouldn't have been my choice either.
It's weird. I personally wouldn't want quite such a silly name for that particular kind of tool, but that is a funny thing for me to say because I was never one of the people who wanted to remove the swear words from the kernel because "professional impression". Don't ask me to explain it.
people who seem to have done a pretty good job
I mean branding logo for a this kind of tool really doesn't matter and if so why should you hire a graphic designer to do that for you if you already have something which is passable.
You can read it as build-ah, ah is in some languages the word for the sound people make when they have a insight/light bulb moment. It might also just be a coincidence, idk.
But most importantly it's nicely searchable word, it's memorable too, it's pronounceable and it's somewhat related to what it does (a "build" tool).
So in all the metrics which matter it's a good name.