Permission Systems for Enterprise That Scale
87 points
14 hours ago
| 7 comments
| eliocapella.com
| HN
tekkk
11 hours ago
[-]
Strange the article proposes itself for "Enterprise" yet has no mention of Google's Zanzibar and how it compares to the other approaches. AFAIK it doesn't use pre-computed values but just queries really fast (using Spanner so there's that)
reply
jschorr
5 hours ago
[-]
Google's Zanzibar actually does both: for the vast majority of queries, it uses significant levels of caching and a permitted amount of staleness [1], allowing Spanner to return a (somewhat stale) copy of the relationship data from local nodes, rather than having to wait or coordinate with the other nodes.

However, some deeply recursive or wide relations can still be slow, so Zanzibar also has a pre-computation cache called Leopard that is used for a very specific subset of these relations [2]. For SpiceDB, we called our version of this cache Materialize and it is designed expressly for handling "Enterprise" levels of scale in a similar fashion, as sometimes it is simply too slow to walk these deep graphs in real-time.

[1]: https://zanzibar.tech/24uQOiQnVi:1T:4S [2]: https://zanzibar.tech/21tieegnDR:0.H1AowI3SG:2O

reply
eliocs
9 hours ago
[-]
Can you let me know how would you for example query all accessible resources for a user using Google's Zanzibar?
reply
jschorr
5 hours ago
[-]
In SpiceDB, this is known as the LookupResources [1] API, which returns all resources (of a particular type) that a particular subject (user in this case) has a particular permission on.

We have a guide on doing ACL-aware filtering and listing [2] with this API and describing other approaches for larger Enterprise scales

Disclaimer: I'm the co-founder and CTO of AuthZed, we develop SpiceDB, and I wrote our most recent implementation of LookupResources

[1]: https://buf.build/authzed/api/docs/main:authzed.api.v1#authz... [2]: https://authzed.com/docs/spicedb/modeling/protecting-a-list-...

reply
phrotoma
8 hours ago
[-]
Related: if anyone has a method of achieving this query against GCP resources I'd be keen to learn that as well.
reply
jschorr
5 hours ago
[-]
We actually have users that synchronize their resources from various sources (AWS, Kubernetes, etc) into SpiceDB, explicitly so they can perform these kinds of queries!

One of the major benefits of a centralized authorization system is allowing for permissions queries across resources and subjects from multiple different services/sources (of course, with the need to synchronize the data in)

Happy to expand on how some users do so, if you're curious.

reply
svaha1728
6 hours ago
[-]
If you are interested in Zanzibar and Relationship-Based Access Control (ReBAC) it’s worth taking a look at OpenFGA https://openfga.dev/
reply
mirzap
6 hours ago
[-]
There are quite a few OSS Zanzibar-inspired authorization services/servers:

  - SpiceDB (https://github.com/authzed/spicedb)
  - Permify (https://github.com/Permify/permify)
  - Warrant (https://github.com/warrant-dev/warrant)
  - Ory Keto (https://github.com/ory/keto)
reply
smarx007
10 hours ago
[-]
reply
Xmd5a
9 hours ago
[-]
https://docs.feldera.com/use_cases/fine_grained_authorizatio...

Fine-grained authorization as an incremental computation problem

reply
eliocs
9 hours ago
[-]
How would you achieve fast list queries of accessible resources with this approach?
reply
gz09
8 hours ago
[-]
feldera has a way to run ad-hoc/list queries on materialized views. Alternatively, you can send the result somewhere where you can query it.
reply
gneray
8 hours ago
[-]
Yes we've implemented this at Oso.
reply
bencyoung
12 hours ago
[-]
If you're using Postgres then using the ltree module is great for permission systems. Available in RDS too
reply
calderwoodra
7 hours ago
[-]
Agreed, specifically for the file structure use-case, we were able to solve this with ltree.
reply
amcvitty
3 hours ago
[-]
About to embark on a similar project. Would love to hear any insights you can share!
reply
casper14
10 hours ago
[-]
Could you explain why this is great over alternatives?
reply
nh2
10 hours ago
[-]
Do you have an article about that?
reply
the_arun
5 hours ago
[-]
Isn’t Open Policy Agent (OPA) and Zanzibar not good enough to be in the article or author talking about specific permission controls?
reply
charcircuit
12 hours ago
[-]
>We added a point of failure, as the permissions table can get out of sync with the actual data.

>The main risk with pre-computed permissions is data getting out of sync.

It would make sense to have permissions be a first class concept for databases and to ensure such a desync could never happen. Data being only read or written from specific users is a very common thing for data so it would be worth having first class support for it.

reply
eliocs
9 hours ago
[-]
Lot of 'new' databases are basing their moat on this and sync engines. Eg: supabase, zero.dev, jazzdb, etc.
reply
valiant55
10 hours ago
[-]
I'm struggling to understand what the issue that the author is getting at. The point of a database is that it's ACID compliant, wrap insets/updates/deletes in a transaction and no such drift would occur. What am I missing?
reply
charcircuit
9 hours ago
[-]
I don't think you are missing anything. I think he is just pointing out that technically nothing is enforcing this synchronization, so if someone forgets to wrap things in a transaction, it could get out of sync.
reply
ahsisibssbx
5 hours ago
[-]
Depending on your DBMS and isolation level, using a transaction might not fix things. That being said I don’t think (at least for Postgres) most people are using an isolation level that could cause this.

Much more likely I think is that you can’t use the db to prevent invalid states here (unique constraint, etc) and you’re dependent on other areas of the code correctly implementing concurrency controls. Race condition in resource A causes problems in your permissions table now.

And just from a general engineering perspective, you should assume things are going to fail and assess what your path forward looks like when they do. Recovery script sounds like a good idea for a critical area.

reply
eliocs
9 hours ago
[-]
I just want to point out you have to take care about that, yes you can have a trigger or a transaction to make sure it happens but it isn't there out of the box
reply
jeffbee
4 hours ago
[-]
Why is it a useful property that everything is always "in sync"? I propose this is not possible anyway. These systems are always asynchronous, and the time of check is always before the time of use, and it is always possible that a revocation occurs between them, and this problem cannot be eliminated.
reply
ExoticPearTree
6 hours ago
[-]
Another approach to complex requirements without spending a lot of time querying databases is to use bitmaps. A set of permissions can be expressed through a bitmap and all you need to do in code is to "decode" that to what you actually let the user do.

The downside to this approach is that it requires some planning and to maintain in code what mask retrieves what permission(s).

reply
bitweis
5 hours ago
[-]
Permit.io

Scales both on the tech, and on the human side - e.g. your product manager can add roles (with CI approval) without requiring engineering involvement.

(I'm biased but still true)

reply
afiori
2 hours ago
[-]
I only did a quick read of permit.io offering but iirc they don't focus on hierarchical data. If having access to a resource cannot grant access to unbounded number of other independent resources (eg sharing a folder) then almost all issues of the article disappear
reply