The article nor the talk appear to reference the XML standard that EN 16931 is built upon: Universal Business Language, https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=... - which is freely available. Examples can be found here: https://github.com/Tradeshift/tradeshift-ubl-examples/tree/m... . It is a good standard and yes it's complex, but it is not complicated by accident. I would any day recommend UBL over IDOC, Tradacom, EDIFACT and the likes.
> How would you digitally sign a Json document and embed the signature in the document?
You would not, because that's exactly how you get these bugs. Fortunately serialization mechanisms, whether JSON or Protobuf or XML or anything else, turn structured data into strings of bytes, and signature schemes operate on strings of bytes, so you'll have a great time signing data _after_ serializing it.
> It does not matter whether your serialization is canonical or not if you don't need to parse the document before you've verified the signature on it.
It most certainly does. First or last duplicate key?
Hash: SHA1
> How would you digitally sign a Json document and embed the signature in the document?
Embedding a signature into the same file is easy enough.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.7 (GNU/Linux)
iEYEARECAAYFAjdYCQoACgkQJ9S6ULt1dqz6IwCfQ7wP6i/i8HhbcOSKF4ELyQB1
oCoAoOuqpRqEzr4kOkQqHRLE/b8/Rw2k =y6kj
-----END PGP SIGNATURE-----
This avoids JSON-inside-JSOn and allows to pretty-print the original object with the signature.
Pretty significant catch if interoperability is a concern at all. Whitespace is easy enough to handle but how do dict keys get ordered? Are unquoted numbers with high precision output as-is or truncated to floats/JS Numbers? Is scientific notation ever used and if so when?
These are non-trivial issues that, thankfully, some very smart and/or experienced people have usually handled for us. However, they still frequently lead to all sorts of vulnerabilities. "Stuffing" attacks sometimes rely on these issues, as have several major crypto incidents.
Or at all.
> How would you digitally sign a Json document and embed the signature in the document?
Preferrably you wouldn't because that's a really bad idea.
That said, this type of support-every-conceivable-idea design-by-committee systems would be equally bad built on json or anything else. That much is true.
There's probably no silver bullet here. But that is still not an excuse for XML-Sig.
Presumably the same way you accomplish the thing in xml:
{ “signature”: “…”, “payload”: … } invoice.inv (zip archive)
└- payload.json
└- signature.asc
It's effectively what the Java jar is.
It helps to have a well defined expected structure in the archive.
This specific exploit is one that only exists when you are extracting a zip on windows.
1. Standard C memory vulnerabilities
2. Unsafe file traversal while unzipping
The entire second class is avoided in a fixed file format. The first class of vulnerabilities plague everything. A quick look at libxml2 CVEs shows that.
but yeah the first class of vulns is why we have advice like don’t run untrusted input, which is not dissimilar to “don’t unzip untrusted payloads”.
For example, in the USA https://www.rcfp.org/briefs-comments/astm-v-upcodes-inc/
This is an especially hot topic in the EU in medical device regulations: https://www.bsigroup.com/en-GB/insights-and-media/insights/b...
But yes, for commercial offers, presumption of conformity mean you have to pay for norms to adhere to law. Big fail.
Especially since non-commercial but persistent and public, not "for profit", is still surmised in e.g. warranty laws. (E.g. geschäftsmäßige Nutzung / usage with said two terms, even for F/LOSS)
You mean the government that’s already the tax authority for which you create and report invoices? I can promise you it’s a good thing. Electronic invoices are a lot easier for us (obviously since it’s just a click instead of a whole PDF operation). It also removes a whole bunch of possibilities for mistakes.
> you have to pay to see the entire standard
The article is wrong about that though, all technical docs are available via the europa portal and reference implementation examples.
That said, I actually agree with you - it's crazy that we need to pay for a stupid standard document.
This XML will usually not be read by the companies that pay the invoice. For instance, in France by the end of 2027, every business will have to send e-invoices, but never directly to the real recipient. The business sends the invoice to a registered go-between, which will ask a national platform for the address of the recipient's go-between, etc. So, only those official go-between companies will have to securely parse the XML.
BTW, in 2022 when the French government decided to make e-invoicing mandatory, it announced that it would develop a national unique go-between platform. Two years later, it dropped that part of the project and announced that there would be an official list of private platforms. So, by the end of 2026 or 2027, every French business will have to select one of the 112 platforms and buy a subscription. It give the State more control, but for small businesses it means higher costs and complexity.
Actually, most governmental agencies (which have required e-invoices for many years), only accept the pure XML files. Zugferd is a bit of an interim solution until everybody is familiar with an e-invoice visualizer. There are already many free (as in beer and source) solutions available for that. Zugferd (X-Factur) is a problematic standard, because there can be discrepancies between the PDF and the embedded file and its unclear, which is authorative. Concerning the platforms: I don't know about France, but in Germany, sending e-invoices by e-mail is explicitly permitted by the government.
Github: https://github.com/VladSez/easy-invoice-pdf
App: https://easyinvoicepdf.com/?template=stripe
I’m planning to use this package to generate e-invoice: https://github.com/gflohr/e-invoice-eu
UPD: issue to follow the progress https://github.com/VladSez/easy-invoice-pdf/issues/121
If you have any feedback or suggestions please feel free to reach out to me :)
The invoicing standard is an attempt to mitigate reverse charge fraud by gathering more machine-readable data. Some countries even demand that b2b invoices are sent to the country, which then dispatches a copy to the recipient.
Knowing this background, it's pretty clear why the EU is making it mandatory.
Personally, in the abstract I like the idea to mandate the use of an open standard, I think we have way too many inefficiencies from treating many things as text documents that could be data structures. I don't like this particular standard though, it's bloated and the result of a typical top-down process.
I much prefer it when there are competing standards for a while, and one or a couple of winner emerge on technical merits. THEN I have no objections to a regulatory body picking a standard and mandating it.
Besides, many standards have been created over the past 20 years, yet most invoices are still only sent as PDF.
[1] https://www.ibm.com/docs/en/cobol-zos/6.4.0?topic=arithmetic...
People invent their own standard to make their own lives easier at the cost of making everyone else's lives miserable which is exactly what the European Committee for Standardization was intended to prevent.
If tidiness and neatness are not a good enough argument to mandate this taxpayer savings, time efficiency, and better software should be.
Companies who insist on being precious about their favored invoice format can invest their own time and money on conversion tools that let them convert invoices they get into whatever format they like for their own internal records and convert them to meet the standard again when sending invoices out. That leaves them free to use what they want without making everyone else deal with their mess.
Scale is a much bigger deal than the complexity of any one invoice. When you're dealing with hundreds of thousands if not millions of invoices from all over the place it makes sense to have it standardized so that software can be developed to do most your work with those invoices automatically and consistently.
I've worked on automating high volume document processing from a much smaller number of companies (mainly just from those within the US), just one or two outliers can massively expand your codebase and when those companies are free to change their formats on a whim in whatever why suits them it can break everything in ways that can be immediately catastrophic or very subtle but no less disastrous.
- It's a barrier for small businesses, or those which seldomly invoice, such as craft and hobby businesses (particularly remote online businesses).
- Large companies see eInvoicing as a cost saving method and force it upon their vendors. This reduces the need to make it mandatory and provides a financial incentive for companies to adopt eInvoicing (i.e. more carrot, less stick.)
The EU has a solid trend of finding ways to self-harm when introducing reforms. This self-harm story segue's into how the EU is considering implementing an Australian-style social media restriction for children:
Quote from abc.net.au below:
European Commission president Ursula von der Leyen told the audience she had been "inspired" by Australia's "bold" move to introduce the ban.
"As a mother of seven children and grandmother of five, I share their view," she said.
The European Parliament has since passed a non-legislative report that would set a minimum age of 16 for social media, while allowing those aged 13 to 15 with parental consent.
-- end quote --
Here the EU is walking down the path of another bad implementation.
Limiting the age for social media only works if it's mandatory for all children, otherwise kids will just pester their parents for access. In the EU's plan the parents become the "bad guy" in that arrangement, the home becomes the battleground for obtaining access to social media.
The EU's plan also means that social media remains relevant for young people, where access may be needed for arranging social activities and sports, and those which don't have it are either inconvenienced or miss out. Meanwhile the Australian implementation removes that purpose as no kids are allowed on the platform, thus there are no "haves" and "have nots" kids.
Finally, and probably most importantly, advertisers, data brokers, and bad actors will still continue to target children through social media networks, since they will still be there in useful numbers.
In this scenario we can anticipate that business practices will shift to the common standard over time, and that would include the accounting software used by new businesses: resulting in a phased conversion with minimal disruptions to running a business.
You must be new to the internet /s
A company does not gain anything by sending "better" invoices that follow a standard. Only if they receive standardized invoices, but usually not enough to pay extra for it. The fact that standardized invoices haven't happened yet without legislation should be proof of that
In some member states, like Germany, the EDIFACT format, when compliant with the EN 16931 data model, is accepted as a valid e-invoice format.
EN 16931 defines what information needs to be in an invoice (the data model), while EDIFACT INVOIC defines how that information is structured and formatted for electronic transmission (the syntax).
But also let me get this straight, there is an actual EU standard for invoices? Why the does nobody follow this and I have to keep asking people to put the fucking VAT ID onto it like I'm a broken record?