Ty: A fast Python type checker and language server
868 points
1 day ago
| 37 comments
| github.com
| HN
zanie
1 day ago
[-]
:wave:

Looks like you found the not-so-secret repository we're using to prepare for a broader announcement :)

Please be aware this is pre-alpha software. The current version is 0.0.0a6 and the releases so far are all in service of validating our release process. We're excited to get this in people's hands, but want to set the expectation that we still have a lot of work left to do before this is production ready.

Stay tuned for more for news in the near future!

(... I work at Astral)

reply
BewareTheYiga
1 day ago
[-]
For pre-alpha software it's working fantastic for my project. I thought I type annotated it well, but Ty had quite a lot of feedback for me. Great job and I can't wait until this is released.
reply
IshKebab
1 day ago
[-]
Had you checked it with Pyright previously?
reply
BewareTheYiga
1 day ago
[-]
I've only used Pylance standard type checking in vscode. I have not used pyright as a stand alone package.
reply
ddorian43
1 day ago
[-]
Use Pyright or basedpyright for now.
reply
likeon
1 day ago
[-]
pylance is pyright
reply
taldo
1 hour ago
[-]
pylance is pyright + proprietary stuff on top, see https://docs.basedpyright.com/latest/ for a fork with some of pylance's features added.
reply
12_throw_away
1 day ago
[-]
If you can say - are there any thoughts about implementing plugins / extension capabilities to keep type checking working even with libraries that aren't otherwise typecheckable?

(where "not otherwise typecheckable" means types that can't be expressed with stubs - e.g., Django, dataclasses pre-PEP-681, pytest fixtures, etc.)

reply
dcreager
1 day ago
[-]
At least for the moment, we aren't planning on a plugin architecture. We do recognize that there are some popular libraries and code patterns that aren't easily (or at all) typeable with the current state of the typing spec. We feel it would be more useful to help drive changes to the typing spec where we can, so that other type checkers can also benefit; and/or implement workarounds for the most popular libraries directly in ty, so that a library author doesn't have to rely on their downstream consumers installing a particular set of plugins to get good type-checker results.

(It's also more difficult to support plugins effectively in a type checker like ty, than in a linter like ruff, since any non-trivial use case would likely require deep changes to how we represent types and how we implement type inference. That's not something that lends itself to a couple of simple hook extension points.)

reply
tyrion
1 day ago
[-]
Helping improve the spec and all is great, but being 100% honest, as a user, I would rather have a type checker I can bend to my needs. As you said, some code patterns in a dynamic language like Python are difficult, or even impossible, to type-check without custom code. Type checkers are becoming more popular than ever, and this implicitly means that these code patterns are are going to be discouraged. On one hand, I believe the dynamism of Python is core to the language. On the other, I would never want to write any collaborative piece of software without a type checker anymore. Therefore, to get the benefits of a type checker, I am occasionally forced to write worse code just to please it.

Considering how fast uv and ruff took off, I am sure you are aware of the impact your project could have. I understand that supporting plugins is hard. However, if you are considering adding support for some popular libraries, IMHO, it would be really beneficial for the community if you could evaluate the feasibility of implementing things in a somewhat generic way, which could be then maybe leveraged by third-party authors.

In any case, thanks for all the amazing work.

reply
jez
1 day ago
[-]
Out of curiosity, do you have experience with other languages that have type system plugins that you’d hope be used as inspiration for something in Python?

I don’t have any such experience (short of a macro system, which requires code generation or runtime support) and it always makes me curious when people ask for type system plugins whether this is a standard feature in a type system I’ve never used.

reply
dcreager
1 day ago
[-]
To add to the complexity, you have to worry about not just which language you're analyzing, but also which language the type-checker is implemented in.

So if we were to do this for ty, we would have to carefully design the internal data types and algorithms that we use to model Python code, so that they're extensible in a robust way.

But we would also have to decide what kind of Rust plugin architecture to use. (Embed a Lua interpreter? dlopen plugins at runtime? Sidecar process communication over stdin/stdout?)

Solvable problems, to be sure, but it adds to the amount of work that's needed to support this well — which in turn affects our decisions about whether/when to prioritize this relative to other features.

reply
badmintonbaseba
1 day ago
[-]
Isn't mypy extensible with plugins?
reply
throwaway17_17
1 day ago
[-]
Can you either give some additional details on the code patterns you’re talking about, or link to some ‘typical’ examples? I do appreciate the flexibility of being able to just write code and not particularly be overly sensitive to jumping through typing hoops, but I can’t think of any place I’ve actually used algorithms or specific code patterns that rely on untyped-ness to actually work at run time. I’d be very interested in trying to work through what is actually required to consider these code patterns as well-typed.
reply
renmillar
1 day ago
[-]
IMO creating custom rules is problematic - when projects import external code, rule conflicts become inevitable. C++'s type system might be complex, but at least there's consistency across header files within a project.

Regarding type checkers: while I don't love optimizing code just to make them run faster, most Python patterns can be implemented in statically checkable ways without much compromise. The benefits typically outweigh the costs. Python's dynamic features are powerful but rarely essential for everyday tasks.

reply
danlamanna
1 day ago
[-]
reply
collinmanderson
1 day ago
[-]
https://pypi.org/project/django-types/ is compatible with pyright without plugins, so it should theoretically work with ty. It might not check as much as mypy though especially with values() querysets.
reply
davedx
1 day ago
[-]
What might, possibly, redeem Python in my eyes as a potential language for making production applications (something that today, it is most certainly not) would be if the type checker worked across the broader ecosystem of common Python packages.

For example, as my recent struggles showed, SQLAlchemy breaks `pyright` in all kinds of ways. Compared with how other 'dynamic' ORMs like Prisma interact with types, it's just a disaster and makes type checking applications that use it almost pointless.

How does Ty play with SQLAlchemy?

reply
reubenmorais
1 day ago
[-]
This is a weakness of the Python typing system and not necessarily of individual typecheckers. Pyright has a policy of only implementing what's standardized, and the Python type system is simply inadequate to annotate most real world Python code out there. It's been years now and something as basic as properly typing kwargs is still not supported.

Ty could solve this if they rebel and decide to ignore the Python typing standards, which I honestly would appreciate, but if they take the sensible approach and follow the standards, it won't change anything.

reply
9dev
1 day ago
[-]
Python code feels like back in the day when JavaScript was typed using JSDoc comments, and libraries would use all kinds of fantastical object shapes for their option parameters, so users could pass "just about anything" and it would work. You would never know how to configure an Express app without digging through the documentation, for example.

I loathe the Python convention of just using kwargs instead of clearly annotated parameters; most libraries don't even have doc comments in the code, so you're really required to look up the documentation, hope that it actually describes the method you're interested in and contains more than stuff like "foo: the foo to use"—or fall back to rummaging in the library intestines to figure out how it works.

It's pathetic.

reply
cadu2
1 day ago
[-]
I'm not sure into what kind of industry you're in, but having most functions as (args, *kwargs) is not the way I deal with most of my code and the libraries I work at all (backend development). Everything is typed fully.

Maybe you're in a niche spot, or using scientist-based code. I've seen plenty of trainwrecks in 'conda-only' ""libraries"" done by scientists. Maybe that's the niche you're at?

reply
oxcabe
1 day ago
[-]
Sometimes, though, you may get lucky, and find some tests for the code you want to use!

On a more serious note, I can't even blame library devs as long as they try. Type "hints" often are anything but _just_ hints. Some are expected to be statically checked; some may alter runtime behavior (e.g. the @overload decorator). It's like the anti-pattern of TypeScript's enums laid out here and there, and it's even harder to notice such side-effects in Python.

reply
Spivak
1 day ago
[-]
> properly typing kwargs is still not supported.

I've been typing them with TypedDict for a while now and it's been fine. What can't you do?

reply
reubenmorais
1 day ago
[-]
reply
jakewins
1 day ago
[-]
My experience is this is nearly impossible, the solution is new packages written after typing was introduced.

I don’t know about SQLAlchemy, but for libraries like pandas I just don’t see how it can be done, and so people are actively replacing them with modern typed alternatives

reply
davedx
1 day ago
[-]
Ha. I just finished a huge rewrite at work from sync SQLAlchemy to async SQLAlchemy, because the async version uses a totally different API (core queries) to sync. So this implies if I want type checking I need to use a different ORM and start again?

I love how Python makes me so much faster due to its dynamic nature! Move fast, break things!

reply
mapcars
1 day ago
[-]
I don't agree that dynamic nature makes things necessarily faster, if you compare Python to C or Java it is true, but if you compare to Typescript it is not. With a decent typing system and a good editor that makes use of it (and AI-assistants nowadays) the prototyping can actually be both faster and more stable.
reply
davedx
23 hours ago
[-]
Yes, I 100% agree. My career has been Java/C++ -> php/JavaScript -> typescript/python. Types are a godsend
reply
networked
23 hours ago
[-]
I think davedx was being sarcastic. Python's dynamic nature cost them time.
reply
melodyogonna
1 day ago
[-]
What version of SQLAlchemy? SQLAlchemy v2 is built with with type-hinting support, I didn't have any issues with it when I used it few months ago
reply
bootsmann
1 day ago
[-]
Only tangentially related but does anyone else here get very bothered when looking at the SQLAlchemy documentation? It seems so hard to find what kind of magic incantation you need to do in which order when trying to do a somewhat non-trivial query and I often just write the SQL I want and then tell chatGPT to rewrite it to SQLAlchemy operations but thats not really a sustainable solution.
reply
WD-42
23 hours ago
[-]
Yes. It’s super opaque. SqlAlchemy is one of those libs I want to like but the docs just make it too difficult.
reply
globular-toast
1 day ago
[-]
Have you sat down and read the SQLAlchemy docs properly? It made a lot more sense to me once I'd set aside an hour or two to work through the Unified Tutorial.[0] I feel like these days people just want quick answers to do very specific things but that's a very inefficient way to learn something like SQLAlchemy.

If you know the SQL you want it's just a matter of writing it in SQLAlchemy's query language which is quite close to SQL. Should just be a matter of practice to become fluent in it. "Complex queries" usually turn up when you're doing something like rendering a table or report or something. You don't need the ORM for this kind of thing, just write a query.

An ORM is useful when you want to write domain logic to do read/write operations against domain entities and persist them back to a database. IMO people get hung up on ORMs and think if they're using one then they have to use it for everything then do the most horrible contortions that should have just been db queries. SQLAlchemy allows you to use the ORM judiciously.

[0] https://docs.sqlalchemy.org/en/20/tutorial/index.html

reply
9dev
1 day ago
[-]
> I feel like these days people just want quick answers to do very specific things but that's a very inefficient way to learn something like SQLAlchemy.

Good documentation should absolutely provide a usable reference to quickly look up common ways to solve common problems. Even the PHP docs got that right twenty years ago.

Also, I disagree: A library should be as self-evident and incrementally understandable as possible, not require reading a full tome and grow a grey beard before being accessible.

> "Complex queries" usually turn up when you're doing something like rendering a table or report or something. You don't need the ORM for this kind of thing, just write a query.

Or, when building generic filtering/sorting/pagination logic for a bog-standard CRUD app. Or to do full-text search. Or when doing lateral joins to minimize queries. Or to iterate over a huge table. There's lots of cases where I want the ergonomics and malleability of ORM query instances even when working with complex queries.

reply
johnfn
22 hours ago
[-]
> I feel like these days people just want quick answers to do very specific things but that's a very inefficient way to learn something like SQLAlchemy.

In defense of OP, a particular frustration I have with SQLAlchemy is that I understand SQL just fine, but the ways in which I translate my SQL knowledge into SQLAlchemy incantations is often pretty obscure. I think I deserve "quick answers to do very specific things" because I already have the exact form of the SQL solution in my head. That it then takes 20 minutes of digging through docs or ChatGPT is annoying.

reply
tinodb
1 day ago
[-]
Exactly the reason I stay away from it. I prefer just SQL and something like aiosql to load it.
reply
ukblewis
1 day ago
[-]
Have you tried SQLModel?
reply
triyambakam
1 day ago
[-]
I've tried using it but it's still so immature and poorly documented. I wish it were different because I love the idea of it.
reply
wpeterw
1 day ago
[-]
You mean Python is not a language for production applications ?
reply
pjmlp
1 day ago
[-]
Only when performance doesn't matter, then it becomes a DSL for C and C++ libraries.
reply
pjmlp
1 day ago
[-]
For me, until it gets a production quality JIT, or PyPy and GraalPY get more community love, it remains a scripting language for learning on how to program, automating OS and applications tasks.
reply
Spivak
1 day ago
[-]
Instagram I think would like a word with you on its viability for production.
reply
pjmlp
1 day ago
[-]
Maybe you should first investigate all the gimmicks they had to do, between amount of servers they had to ramp up burning needless budget, rewriting code into C and C++ libraries, Go or whatever else they ended up adding, before doing such statements.

https://stackshare.io/instagram/instagram

reply
Nab443
1 day ago
[-]
Any other link to share about that? The stackshare url does not even mention anything related.
reply
pjmlp
22 hours ago
[-]
It surely does, it is quite simple to correlate how many of those technologies are actually implemented in Python.

Pure Python that is.

reply
digdugdirk
1 day ago
[-]
Cool! Out of curiosity, what's the bedrock that's used to determine what the fundamental python AST objects are? I'm wondering what the "single source of truth" is, if you will.

Is this all based off a spec that python provides? If so, what does that look like?

Or do you "recode" the python language in rust, then use rust features to parse the python files?

Regardless of how it's done - This is a really fascinating project, and I'm really glad you guys are doing it!

reply
dcreager
1 day ago
[-]
There is a formal grammar defined in the CPython repo, implemented in a language called ASDL:

https://github.com/python/cpython/blob/main/Parser/Python.as...

ty uses the same AST and parser as ruff. We don't use the ASDL grammar directly, because we store a few syntax nodes differently internally than how they're represented upstream. Our parser is hand-written in Rust. At first, our AST was also entirely hand-written, though we're moving in the direction of auto-generating more of it from a declarative grammar.

https://github.com/astral-sh/ruff/issues/15655

https://github.com/astral-sh/ruff/tree/main/crates/ruff_pyth...

https://github.com/astral-sh/ruff/blob/main/crates/ruff_pyth...

reply
zanie
1 day ago
[-]
ditto! but we gave impressively non-overlapping answers
reply
zanie
1 day ago
[-]
As in, how are we parsing the Python code into an AST?

CPython uses a generated parser. The grammar is defined in https://github.com/python/cpython/blob/main/Grammar/python.g... which is used to generate the specification at https://docs.python.org/3/reference/grammar.html#full-gramma...

We use a hand-written parser, in Rust, based on the specification. We've written that previously at https://astral.sh/blog/ruff-v0.4.0#a-hand-written-parser

reply
theLiminator
1 day ago
[-]
Curious if this means it'll be released as a separate binary than ruff? I personally feel like having it within ruff is much nicer for ensuring that we have a consistent set of dependencies that play nicely with each other. Though I guess because a type checker doesn't mutate the files maybe that's not a real concern (vs formatting/linting with --fix).
reply
zanie
1 day ago
[-]
It'll be separate (at least to start) — we want to be able to iterate on it rapidly. Long-term, a consistent toolchain is definitely important and something we're thinking about.
reply
skwashd
1 day ago
[-]
+1 for (eventually) baking it ty into ruff. In my mind static type checking is a form of linting.

For years I pushed black for formatting code. Once formatting was baked into ruff I ditched black. Having fewer dependencies to track and update simplifies my life and shortens my dependabot queue.

reply
opem
1 day ago
[-]
Finally the missing puzzle piece from the astral toolchain is here! <3
reply
ZiiS
1 day ago
[-]
Pointlessmy anal; but 0.0.0a6 is very strongly indicative of the sixth alpha release. Pre-alpha are much better as .dev releases.
reply
usr9012809
1 day ago
[-]
> Pre-alpha are much better as .dev releases.

No, they are correctly using semantic versioning to indicate pre-alpha releases. https://github.com/astral-sh/ty/releases https://semver.org/

reply
kstrauser
1 day ago
[-]
Python doesn’t use plain semver: https://peps.python.org/pep-0440/
reply
zahlman
1 day ago
[-]
The reference Python implementation, written in C, doesn't use semver. But other projects in the Python ecosystem are generally assumed to unless stated otherwise. For example, Setuptools does (but not pip: https://pip.pypa.io/en/stable/development/release-process/).
reply
fastball
1 day ago
[-]
It is the sixth alpha release. They haven't yet released a stable version – this is their sixth alpha release before that.

What am I missing here?

reply
ZiiS
1 day ago
[-]
I was replying to a post that said "Please be aware this is pre-alpha software." presumably trying to make a distinction from "alpha software".
reply
ngoldbaum
1 day ago
[-]
I gave away the “ty” project name on pypi to Astral a week or so ago. I wanted to use it for a joke a few years ago but this is a much better use for a two letter project name. They agreed to make a donation to the PSF to demonstrate their gratefulness.
reply
_carljm
1 day ago
[-]
Yes, thank you for your graciousness and generosity, very much appreciated.
reply
Celeo
1 day ago
[-]
I love this outcome; kudos to you and Astral both!
reply
swyx
1 day ago
[-]
thanks for not charging obnoxious amounts for package names!
reply
smitty1e
1 day ago
[-]
ty--thank you
reply
rrszynka
1 day ago
[-]
nice! what was the planned joke?
reply
ngoldbaum
23 hours ago
[-]
Either something about beanie babies or something riffing on "thank you". Couldn't ever make up my mind then basically forgot about it.
reply
aleksanb
1 day ago
[-]
The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.

The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django that properly manages to type check its crazy runtime generated methods.

I wish more python tooling took the TS approach of "what's in the wild IS the language", as opposed to a "we only typecheck the constructs we think you SHOULD be using".

reply
mjr00
1 day ago
[-]
> The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.

Or in this case, writing it in Rust...

mypy is written in Python. People have forgotten that Python is really, really slow for CPU-intensive operations. Python's performance may not matter when you're writing web service code and the bottlenecks are database I/O and network calls, but for a tool that's loading up files, parsing into an AST, etc, it's no surprise that Rust/C/even Go would be an order of magnitude or two faster than Python.

uv and ruff have been fantastic for me. ty is definitely not production ready (I see several bizarre issues on a test codebase, such as claiming `datetime.UTC` doesn't exist) but I trust that Astral will match the "crazy reality" of real Python (which I agree, is very crazy).

reply
dcreager
1 day ago
[-]
> such as claiming `datetime.UTC` doesn't exist)

This is a known issue — we're currently defaulting to a conservative Python version, and `datetime.UTC` really doesn't exist until Python 3.11!

https://docs.python.org/3/library/datetime.html#datetime.UTC

We will probably change the default to "most recent supported Python version", but as mentioned elsewhere, this is very early and we're still working out these kinds of kinks!

reply
zo1
1 day ago
[-]
You should be doing this dynamically based on the version of python you are running against, so that you don't have to hardcode or make such "conservative" choices by hand.
reply
dcreager
1 day ago
[-]
Note that we're not ever spinning up a Python interpreter to run your code, or monitoring an existing running Python process. So we do need some kind of metadata.

But yes, if you have a Python version specified in pyproject.toml, we respect that, and if you have a virtualenv, we can see the Python version that was used to create that. And that's what we use to type-check your code.

The default being discussed here is what we fall back on if that project metadata isn't available.

reply
lacasito25
1 day ago
[-]
I think they probably know that, this is alpha software, no need to be condescending.
reply
lionkor
1 day ago
[-]
They said they will default to some newer version, which indicates they are not planning to do this dynamically.
reply
johnisgood
1 day ago
[-]
How is it condescending in any way? I found it to be a constructive criticism; i.e. useful help.
reply
mahogany
22 hours ago
[-]
I don't necessarily read it as condescending, but I do read it as presumptuous. What someone "should" do depends on many things. Maybe, because this is software in alpha stage, they should _not_ focus on this part of the code if it is minor compared to other obligations. Or maybe there are other reasons they've chosen not to do this (as was explained in an above comment).

IMO, a less presumptuous criticism would be phrased like "if you did X then benefits Y would happen", or "if you haven't, consider X", or even (the least presumptuous - make it a conversation!) "have you considered X?", rather than "you should do X".

reply
johnisgood
18 hours ago
[-]
I see what you mean. Perhaps it was just a "poor" choice of words for whatever reasons. I am sure we can assume he intended it in a way of "have you considered X?".
reply
zarathustreal
1 day ago
[-]
Criticism isn’t necessarily condescending. “You should be doing x because y” is just a plain assertion, it doesn’t imply any moral judgement or opinion of the author.
reply
_carljm
1 day ago
[-]
(ty developer here)

Currently we default to our oldest supported Python version, in which `datetime.UTC` really doesn't exist! Use `--python-version 3.12` on the CLI, or add a `ty.toml` with e.g.

``` [environment] python-version = "3.12" ```

And we'll find `datetime.UTC`.

We've discussed that this is probably the wrong default, and plan to change it.

reply
miki123211
1 day ago
[-]
I realize this might be hard from a technical / architecture standpoint, but it would be great if "does not exist" and "does not exist in this version of Python" were two different errors.

If I saw something like "datetime.UTC doesn't exist", I'd immediately think "wait, was that datetime.utc", not "ooh it got added in 3.11, I need to change my Python version"

reply
_carljm
1 day ago
[-]
I agree that would be nice; probably not near the top of our list right now (and not trivial to implement), but it makes sense. Thanks for the suggestion.
reply
jychang
1 day ago
[-]
Nontrivial way to do it is dynamically scan the python 3.12 namespace, and add these warnings.

Is there any big downside to do it the boring way, hardcode a list and compare the error to the list?

reply
_carljm
19 hours ago
[-]
This information is already maintained via `if sys.version_info >= (...):` conditionals in typeshed stubs. I don't think this is important enough to justify maintaining the same information in a duplicate way.
reply
mjr00
1 day ago
[-]
aha makes sense! Yeah it'd be nice if you could divine the intended python version from the uv configuration/`.python-version`. Thanks for all your hard work, looking forward to the full release!
reply
HelloNurse
1 day ago
[-]
Defaulting is wrong: what is checked is the aggregate of actual user code, standard library for a given Python version and installed packages. It has to be the same environment as when the program is run, leaving conservative approximations (checking types with the oldest supported library versions and hoping newer ones are OK) to the user.
reply
dcreager
1 day ago
[-]
Yes, if you have a Python version specifed in pyproject.toml, for instance, we respect that, and that's what we use to type-check your code. The default being discussed here is what we fall back on if that project metadata isn't available.
reply
0xffff2
21 hours ago
[-]
Could you check what version of `python` is in the PATH and use that as the default?
reply
miki123211
1 day ago
[-]
Python is slow for some CPU-intensive operations.

There are some extremely CPU-intensive low-level operations that you can easily write in C and expose as a Python API, like what Numpy and Pandas do. You can then write really efficient algorithms in pure Python. As long as those low-level operations are fast, those Python-only algorithms will also be fast.

I don't think this is necessarily "cheating" or "just calling disguised C functions." As an example, you can write an efficient linear regression algorithm with Numpy, even though there's nothing in Numpy that supports linear regression specifically, it's just one of the ways a Python programmer can arrange Numpy's low-level primitives. If you invent some new numerical algorithm to solve some esoteric problem in chemistry, you may be able to implement it efficiently in Python too, even if you're literally the first person ever writing it in any language.

The actual problem is that it's hard for people to get an intuition of which Python operations can be made fast and which can't, AST and file manipulation are sadly in the latter group.

reply
fastball
1 day ago
[-]
That is a confusing way to look at it. Python is slow, C is fast. If your python code is calling functions that were not written in Python (even if it is indirectly thru a library you are using), that is not "pure python".
reply
francasso
1 day ago
[-]
That works in numerical libraries because you can encapsulate the loops into basic operations that you then lower to C. In a domain like type checking it's not nearly as easy/doable.
reply
maleldil
20 hours ago
[-]
> As long as those low-level operations are fast, those Python-only algorithms will also be fast.

Only if you spend more time on the C implementations than on Python. If you have pure Python loops, you'll be slow. You need quite high-level components and minimal Python glue for it to be fast.

reply
shiandow
1 day ago
[-]
CPU intensive is not quite the right metric. What python is slow at is all the extra administration that comes with basic stuff like accessing attributes and function calls.

This gives somewhat counterintuitive results where declaring and summing a whole list of integers in memory can be faster than a simple for loop with an iterator.

But yeah writing stuff in a different (compiled) language is often better if that means the python interpreter doesn't need to go through as many steps.

reply
davidfstr
1 day ago
[-]
mypy is compiled using mypyc. It does not run as Python code.
reply
mzl
1 day ago
[-]
The semantics of Python makes it problematic to run at speed, it is not just about interpreted vs compiled code. Give the high levels of dynamic behaviors that are allowed, a Jit (like pypy) has a higher chance of getting decent performance if the code has an underlying behavior that can be extracted.
reply
Sinidir
19 hours ago
[-]
mypy is also written in a style conducive to speed ups when compiling with mypyc
reply
johnfn
1 day ago
[-]
In defense of mypy et al, Typescript had some of the greatest minds of our generation working for a decade+ on properly typing every insane form found in every random Javascript file. Microsoft has funded a team of great developers to hammer away at every obscure edge case imaginable. No other python checker can compare to the resources that TS had.
reply
dathinab
1 day ago
[-]
It's even worse (for python).

TS might transpile to JS and can always be split into a js and type annotation file but is it's own language developed in tandem with the type check based on a holistisch approach to find how to type check then and then put it into the syntax and type checker.

Thats not true for python at all.

Python types where added as annotations to the language many years ago, but not in a holistic approach but in simplistic approach only adding some fundamental support and then extended bit by bit over the years (and not always consistently).

Furthermore this annotations are not limited to type checking which can confuse a type checker (through Annotated helps a lot, but is also verbose, wonder how long until there is a "Annotated" short syntax e.g. by impl @ on type or similar).

Basically what the type annotation feature was initially intended to be and what it is now differ quite a bit (dump example `list` vs. `List`, `Annotated` etc.).

This is made worse that a bunch of "magic" is deeply rooted in python, e.g. sub-classing `Enum`. Sure you have that in JS too, and it also doesn't work that well in TS (if you don't add annotation on the dynamically produced type).

Lastly TS is structurally typed, which allows handling a bunch of dynamic typing edge cases, while Python is, well, in-between. Duck-typing is (simplified) structural typing but `isinstance` is a common thing in Python and is nominal typing...

So yeah it's a mess in python and to make it worse there bunch of annoyances related to ambiguity or to many ways how to do a thing (e.g. re-exports+private modules, you can do that common coding pattern, but it sucks badly).

reply
networked
1 day ago
[-]
Why do you say that duck typing is simplified structural typing? Its relationship with structural typing is on a different axis. Duck typing is its dynamic-typing counterpart.

Python does support structural typing through protocols introduced in version 3.8. They are documented in https://typing.python.org/en/latest/spec/protocol.html.

As a demo, here is part of https://www.typescriptlang.org/play/typescript/language/stru... translated to Python:

  from dataclasses import dataclass
  from typing import Protocol

  class Globular(Protocol):
      diameter: float

  class Spherical(Protocol):
      diameter: float

  # In Python, we need to define concrete classes that implement the protocols.
  @dataclass
  class Ball:
      diameter: float

  @dataclass
  class Sphere:
      diameter: float

  ball: Globular = Ball(diameter=10)
  sphere: Spherical = Sphere(diameter=20)

  # These assignments work because both types structurally conform to the protocols.
  sphere = ball
  ball = sphere

  class Tubular(Protocol):
      diameter: float
      length: float

  @dataclass
  class Tube:
      diameter: float
      length: float

  tube: Tubular = Tube(diameter=12, length=3)

  tube = ball  # Fail type check.
  ball = tube  # Passes.
This is what Pyright says about it:

  Found 1 error.
  /scratch/structural.py
    /scratch/structural.py:37:8 - error: Type "Ball" is not assignable to declared type "Tubular"
      "Ball" is incompatible with protocol "Tubular"
        "length" is not present (reportAssignmentType)
  1 error, 0 warnings, 0 informations
Edit: And this is ty:

  error: lint:invalid-assignment: Object of type `Ball` is not assignable to `Tubular`
    --> structural.py:37:1
     |
  35 | tube: Tubular = Tube(diameter=12, length=3)
  36 |
  37 | tube = ball  # Fail type check.
     | ^^^^
  38 | ball = tube  # Passes.
     |
  info: `lint:invalid-assignment` is enabled by default
  
  Found 1 diagnostic
reply
bunderbunder
22 hours ago
[-]
I'd go a step further and say that duck typing is more than just structural typing's dynamic counterpart. Because, again, that's confounding two different axes. Dynamic vs static describes when type checking happens and whether types are associated with names or with values. But it doesn't necessarily describe the definition of "type".

The real difference between structural typing and duck typing is that structural typing requires all of a type's declared members to be present for an object to be considered compatible. Duck typing only requires the members that are actually being accessed to be present.

This is definitely more common in dynamic languages, but I'm not aware of any particular reason why that kind of checking couldn't also be done statically.

reply
maleldil
20 hours ago
[-]
If I understand correctly, defining the protocol like this forces the implementation classes to have the members as proper fields and disallows properties. If you define `diameter` as a property in the protocol, it supports both:

    from dataclasses import dataclass
    from typing import Protocol

    class Field(Protocol):
        diameter: float


    class Property(Protocol):
        @property
        def diameter(self) -> float: ...

    class Ball:
        @property
        def diameter(self) -> float:
            return 1

    @dataclass
    class Sphere:
        diameter: float

    ball_field: Field = Ball()
    sphere_field: Field = Sphere(diameter=20)

    ball_prop: Property = Ball()
    sphere_prop: Property = Sphere(diameter=20)

Pyright output:

    /Users/italo/dev/paper-hypergraph/t.py
      /Users/italo/dev/paper-hypergraph/t.py:27:21 - error: Type "Ball" is not assignable to declared type "Field"
        "Ball" is incompatible with protocol "Field"
          "diameter" is invariant because it is mutable
          "diameter" is an incompatible type
            "property" is not assignable to "float" (reportAssignmentType)
    1 error, 0 warnings, 0 information 
That is to say, I find Python's support for structural typing to be limited in practice.
reply
spooky_action
1 day ago
[-]
It's not perfect, but runtime_checkable is a thing https://docs.python.org/3/library/typing.html#typing.runtime...

It doesn't actually preserve typing on the protocol's methods though

reply
surfingdino
1 day ago
[-]
What annoys me is that every programmers who wish their favourite language / feature was as popular as Python and they choose to implement it in Python to make Python "better". Python was created as a dynamically typed language. If you want a language with type checking, there are plenty of others available.

Rust devs in particular are on a bend to replace all other languages by stealth, which is both obviously visible and annoying, because they ignore what they don't know about the ecosystem they choose to target. As cool as some of the tools written for Python in Rust are (ruff, uv) they are not a replacement for Python. They don't even solve some annoying problems that we have workarounds for. Sometimes they create new ones. Case in point is uv, which offers custom Docker images. Hello? A package manager is not supposed to determine the base Docker image or Python version for the project. It's a tool, not even an essential one since we have others, so know your place. As much as I appreciate some of the performance gains I do not appreciate the false narratives spread by some Rust devs about the end of Python/JavaScript/Golang based on the fact that Rust allowed them to introduce faster build tools into other programming languages' build chains. Rust community is quickly evolving into the friends you are embarrassed to have, a bit like any JVM-based language that suddenly has a bunch of Enterprise Java guys showing up to a Kotlin party and telling everyone "we can be like Python too...".

reply
Spivak
1 day ago
[-]
This argument doesn't make a whole lot of sense because nothing about type annotations constrains Python code at all. In fact because they're designed to be introspectable they make Python even more dynamic and you can do even crazier stuff than you could before. Type checkers are working very hard to handle the weird code.

Pydantic being so fast because it's written in Rust is a good thing, you can do crazy dynamic (de-)serializations everywhere with very little performance penalty.

reply
maleldil
20 hours ago
[-]
> nothing about type annotations constrains Python code at all

Sorry, but this is just not true. Don't get me wrong, I write typed Python 99% of the time (pyright in strict mode, to be precise), but you can't type check every possible construct in the language. By choosing to write typed Python, you're limiting how much of the language you can use. I don't think that's a bad thing, but it can be a problem for untyped codebases trying to adopt typing.

reply
Sinidir
19 hours ago
[-]
It is literally true. You don't need to run a type checker.
reply
smithkl42
1 day ago
[-]
And in the process, they ended up creating an extremely powerful type system that ~nobody outside of that original team can (fully) understand.
reply
johnfn
22 hours ago
[-]
How many people understand the intricacies of any complex language or type system? Though I think one of the great things about TS is that you need to understand none of it in order to do `npm install @types/lodash` and get all the benefits.
reply
dathinab
1 day ago
[-]
IMHO they created type annotations, not a type system

and how you use the type annotations to indicate a type system is inconsistent and incomplete (e.g. NoneType vs. None for inconsistency and a lot of mess related to mataclasses (e.g. Enum) and supporting type annotations for them for incomplete)

the fact that even today something as fundamental as enums have issues with type checking _which are not just type checker incompetence_ is I think a good way to highlight what mess it is

or that `Annotated[]` was only added in 3.9 and has a ton of visual overhead even through its essential for a lot of clean definitions in modern python code (where for backwards compatibility there is often some other way, which can be de-facto wrongly typed but shouldn't be type linted, have fun type checkers).

reply
simonw
1 day ago
[-]
Parent comment was talking about TypeScript, not Python.
reply
bvrmn
1 day ago
[-]
It's not mypy issue. Comparing to TS python typehints (spec wise) are a joke. It's started as bolted on adhoc solution and evolved quite chaotically. For example [1]. TS doesn't require a special decorator (sic!) to make your custom classes to be picked up by type checkers.

Or how make a wrapper function with args and kwargs to pass through?

[1]: https://docs.python.org/3/library/typing.html#typing.datacla...

reply
maleldil
20 hours ago
[-]
The dataclass decorator isn't there to make the type checkers understand the class. Its main purpose is to automatically implement trivial methods like the constructor, equality, repr, etc. The type hints make this more convenient, but something similar already existed with attrs.
reply
bvrmn
7 hours ago
[-]
It's literally stated in the PEP: https://peps.python.org/pep-0681/

Also: https://github.com/python/mypy/blob/501a07b45af8e44eda665e53...

Also did you know mypy ignores typing of class decorators? You simply can't return a different type other than type[thisclass].

reply
TheTaytay
1 day ago
[-]
Even Typescript is rewriting their compiler in Go. I think that the bottleneck is _actually_ the language sometimes.

(And uv and ruff have basically proved that at this point)

reply
dathinab
1 day ago
[-]
through algorithmic improvements can also go a long way and if you are one of the first type checker which have to figure out the mess the python type annotation system is you will vast a lot of time on figuring that out instead of refactoring it's architecture to allow for algorithmic improvements

which brings us to another python issue, python is quite bad at such huge refactoring even with type checkers

but yeah python is by far the slowest widely used language, and for some use cases you can side step it by placing most hot code in C++/Rust extension modules, (or don't care because you are much much more network latency bound) but a type checker probably doesn't belong into that category

reply
amelius
1 day ago
[-]
> I wish more python tooling

And not directly related, but I wish more python modules did proper checks with Valgrind before shipping.

reply
lyu07282
1 day ago
[-]
The CPython API is such a dumpster fire, even writing very simple modules the reference counting is very difficult to do correctly. The majority of python modules written in C are probably leaking memory somewhere but nobody knows.
reply
amelius
1 day ago
[-]
My problem is that debugging a segfault in a Python system is impossible because of all the noise generated by Python modules that never bothered to clean up their Valgrind output.
reply
badmintonbaseba
1 day ago
[-]
Some tricks that worked for me in the past:

1. use rr for debugging your binary wheel, you can set up watchpoints, and reverse step/continue from the segfault.

2. compile and run your wheel with sanitizers (ASAN, UBSAN).

I rarely use valgrind so I can't comment on that.

reply
collinmanderson
23 hours ago
[-]
I also mentioned this up-thread, but https://pypi.org/project/django-types/ is compatible with pyright without plugins, so it should theoretically work with ty. It's not quite as good as the mypy-django plugin but it still catches a lot.
reply
davedx
1 day ago
[-]
Can mypy type check SQLAlchemy somehow? That's what caused me to give up on Python type checking recently
reply
lukaslalinsky
1 day ago
[-]
SQLAlchemy 2.x has direct support for mypy, it works out of the box, no longer needing mypy plugins. Many things in SQLAlchemy as are still dynamic and can't be type checked, but the native support works great where it can.
reply
HideousKojima
1 day ago
[-]
>The way these type checkers get fast is usually by not supporting the crazy rich reality of realworld python code.

Nah, that's just part of the parade of excuses that comes out any time existing software solutions get smoked by a newcomer in performance, or when existing software gets more slow and bloated.

Here's one of many examples:

https://m.youtube.com/watch?v=GC-0tCy4P1U&pp=0gcJCdgAo7VqN5t...

reply
Redoubts
1 day ago
[-]
Sure, but so far this has been a true criticism of every python type checker that isnt mypy that is production ready today
reply
dathinab
1 day ago
[-]
the thing is most (all) of the type checkers including e.g. mypy _do not_ support most crazy python ...

not because they don't want to or because it's to slow

but because it's not really viable without fully executing module loading in a sandbox, which might seem viable until you realize that you still need to type check `__main__` modules etc. and that its a common trend in python to do configs by loading python modules and grabbing the module locals as keys of the config or loading some things might actually idk. initialize a GPU driver :sob: So it's kinda 100% guaranteed not possible to do fully correct type checking for all project :smh:

But also python is one of the slowest popular languages (and with a large margin to any not also "one of slowest" languages). Only by moving hot code into C++/Rust is it fast, which often is good enough, but a type checker is exactly this kind of software where this approach stops working.

reply
renmillar
1 day ago
[-]
Python's static checking capabilities could significantly improve both tracing and compilation efficiency. The language features that currently limit type checkers are likely the same ones making efficient compilation difficult. Perhaps we'll eventually see a Python 3.40 with complete JIT compilation, functioning similarly to Julia but retaining Python's extensive ecosystem that makes it essential in certain domains.
reply
tmvphil
1 day ago
[-]
Just compared the time to check on a fairly large project:

- mypy (warm cache) 18s

- ty: 0.5s (and found 3500 errors)

They've done it again.

reply
_carljm
1 day ago
[-]
(ty developer here)

This is an early preview of a pre-alpha tool, so I would expect a good chunk of those 3500 errors to be wrong at at this point :) Bug reports welcome!

reply
joshdavham
1 day ago
[-]
Any rough estimates of how much faster you expect ty to be compared to mypy? I'd be super curious to know!

I was also one of those people who, when first trying Ruff, assumed that it didn't work the first time I ran it because of how fast it executed!

reply
_carljm
1 day ago
[-]
We're looking forward to hearing what your experience is! There's a certain amount of roughly-constant overhead (e.g. reading all the files), so generally ty will look relatively faster the larger the project is. For very large projects we've seen up to 50-60x faster or more. We haven't really put a lot of work into targeted optimization yet, so we aim for it to get faster in the future.

It will certainly be slower than Ruff, just because multi-file type analysis more complex and less embarrassingly parallel than single-file linting.

reply
mil22
1 day ago
[-]
Great, but how does it compare to Pyright on the utility / performance curve? Pyright is mature and already very fast.

https://github.com/microsoft/pyright

reply
rtpg
1 day ago
[-]
I don't get why so many people go to bat for pyright, my experience with it has been pretty miserable. Open enough instances of it and you're in OOM city. It works, but often gets confused... and of course the absolute audacity of MSFT to say "let's go over to pyright, and by the way we're going to carve up some stuff and put it into pylance instead", meaning that it's totally not within the actual spirit of open source.

I would like to just not use it, but the existence of pyright as a _barely_ functional alternative really sucks the air out of other attempts' continued existence. Real "extend/extinguish" behavior from MSFT.

reply
mil22
15 hours ago
[-]
I tried all the type checkers available as of ~1 year ago, and Pyright worked the best for me. It's not perfect, but it's better than any of the pure Python checkers. Memory is cheap (unless you're buying it from Apple I guess...). Would I take a faster type checker with better memory footprint? Heck yes, assuming equal or superior functionality.
reply
lemontheme
1 day ago
[-]
If you haven’t checked it out already, basedpyright is pyright with all the arbitrarily carved out functionality put back in – plus some extra features that you may or may not find useful depending on how strict you like your typing.

Can’t recommend it enough

reply
rtpg
7 hours ago
[-]
To be honest I can't respect a project that names itself like that. I am a working professional.
reply
mil22
1 day ago
[-]
I tested it side-by-side on my ~100Kloc codebase.

Ty: 2.5 seconds, 1599 diagnostics, almost all of which are false positives

Pyright: 13.6 seconds, 10 errors, all of which are actually real errors

There's plenty of potential here, but Ty's type inference is just not as sophisticated as Pyright's at this time. That's not surprising given it hasn't even been released yet.

Whether Ty will still perform so much faster once all of Pyright's type inference abilities have been matched or implemented - well, that remains to be seen.

Pyright runs on Node, so I would expect it to be a little slower than Ty, but perhaps not by very much, since modern JS engines are already quite fast and perform within a factor of ~2-3x of Rust. That said, I'm rooting for Ty here, since even a 2-3x performance boost would be useful.

reply
the_duke
1 day ago
[-]
Compilation / type checking depends on a lot of trees of typed data, and operating on those tree nodes. That's something where a statically typed language with custom data structures that allows for optimised representations makes a big difference, and where a lot of the fancy optimisations in v8 don't work so well.

There is a reason Typescript moved to a typed language.

reply
mil22
15 hours ago
[-]
Let's hope you're right and that translates to even higher performance for Ty compared to Pyright. There are of course many variables and gotchas with these sorts of things.
reply
bjourne
1 day ago
[-]
Pyright is only for type-checking and it lacks many features you'd expected from a modern LSP (I forgot which). Hence, it was forked and someone created basedpyright to fix it: https://github.com/DetachHead/basedpyright
reply
dathinab
1 day ago
[-]
To extend on this:

In python it's pretty common to have LSP separate from type checking separate from linting (e.g. ruff+mypy+ide_specific_lsp).

Which to be fair sucks (as it limits what the LSP can do, can lead to confusing mismatches in error/no-error and on one recent project I had issues with the default LSP run by vscode starting to fall apart and failing to propose auto imports for some trivial things for part of the project....)

But it's the stack where pyright fits in.

reply
geekraver
1 day ago
[-]
The pylance team has started exploring this, namely whether it makes sense to have an API for type checkers that is not the LSP, as language servers have a somewhat different goal in which type checking/inference is an enabling technology. This could allow multiple different language servers to be built on top of different type checkers (and the type checkers can run out-of-proc, so implementation languages can be different). https://github.com/microsoft/pylance-release/discussions/718...
reply
nine_k
1 day ago
[-]
Pyright is good, but it's quite a memory hog. (Yes, I have plenty of RAM on my machine. No, it has other uses during development, too.)
reply
0xFF0123
1 day ago
[-]
Pyright is incredibly slow in my experience, I've seen it take over a minute on complex codebases
reply
insane_dreamer
1 day ago
[-]
in my experience pyright is unable to infer many inherited object types (compared to PyCharm's type inference)
reply
lemontheme
1 day ago
[-]
PyCharm definitely excels on more ‘dynamic’ code but the number of times I’ve pulled in code written by colleagues using PyCharm only to get a rainbow of type errors from Pyright is too damn high.

The PyCharm checker seems to miss really, really obvious things, e.g. allowing a call site to expect a string while the function returns bytes or none.

Maybe my colleagues just have it configured wrong but there’s several of them and the config isn’t shared.

reply
js2
1 day ago
[-]
I have no doubt that it will be faster than mypy, but:

> This project is still in development and is not ready for production use.

reply
rybosome
1 day ago
[-]
> They’ve done it again.

Indeed they have. Similar improvement in performance on my side.

It is so fast that I thought it must have failed and not actually checked my whole project.

reply
Handprint4469
1 day ago
[-]
If you have uv installed, you can test it without installing by running:

  uvx ty check
reply
simonw
1 day ago
[-]
Here's what I got against one of my larger open source projects:

  cd /tmp
  git clone https://github.com/simonw/sqlite-utils
  cd sqlite-utils
  uvx ty check
Here's the output: https://gist.github.com/simonw/a13e1720b03e23783ae668eca7f6f...

Adding "time uvx ty check" shows it took:

  uvx ty check  0.18s user 0.07s system 228% cpu 0.109 total
reply
diggan
1 day ago
[-]
I'm not sure if it's using your environment correctly, or are you expecting ~150 errors? Lots of import errors, and I'm guessing most of the other ones are errors because it couldn't infer what was imported.
reply
alexmolas
1 day ago
[-]
how does it compare against mypy? is it much faster?
reply
IshKebab
1 day ago
[-]
How does it compare against Pyright. Pyright is the gold standard of Python type checking currently. Mypy is slower and buggier.
reply
mil22
1 day ago
[-]
I tested it side-by-side on my ~100Kloc codebase.

Ty: 2.5 seconds, 1599 diagnostics, almost all of which are false positives

Pyright: 13.6 seconds, 10 errors, all of which are actually real errors

There's plenty of potential here, but Ty's type inference is just not as sophisticated as Pyright's at this time. That's not surprising given it hasn't even been released yet.

Whether Ty will still perform so much faster once all of Pyright's type inference abilities have been matched or implemented - well, that remains to be seen.

Pyright runs on Node, so I would expect it to be a little slower than Ty, but perhaps not by very much, since modern JS engines are already quite fast and perform within a factor of ~2-3x of Rust. That said, I'm rooting for Ty here, since even a 2-3x performance boost would be useful.

reply
scosman
1 day ago
[-]
0.2s on ty compared to 4.7s on pyright. Not even close.
reply
Redoubts
1 day ago
[-]
Mypy is also able to check things that are inexpressible with stubs, among other things
reply
simonw
1 day ago
[-]
In the same folder:

    time uvx mypy .    
Result:

    uvx mypy .  0.46s user 0.09s system 74% cpu 0.740 total
So ty is about 7x faster - but remember ty is still in development and may not catch the same errors / report false errors, so it's not a fair comparison yet.
reply
hauntsaninja
1 day ago
[-]
Note that `uvx mypy` may give you inaccurate timings on macOS. The antivirus in macOS goes a little crazy the first time it executes a mypyc compiled program.
reply
wdroz
1 day ago
[-]
You can also install it "globally" for your user with:

  uv tool install ty
Then you can use it anywhere

  ty check
reply
heavyset_go
1 day ago
[-]
Can also do

   uv tool run ty
If your $PATH sucks
reply
scosman
1 day ago
[-]
Found 275 diagnostics

0.56s user 0.14s system 302% cpu 0.231 total

Damn that's fast. And zero issues in this project with pyright so curious that these are...

reply
_carljm
1 day ago
[-]
There's a good chance many of the errors we emit are incorrect, at this stage. Lots to do still!
reply
dcreager
1 day ago
[-]
Wait wait wait

Carl, you did not tell me that the errors we emit need to be correct!!

I think I need to go revisit some of my PRs...

reply
_carljm
1 day ago
[-]
I knew we left something important out of that onboarding document
reply
renmillar
1 day ago
[-]
Why don't you just run ty on the ty codebase and let it tell you which errors are correct and which are erroneous?
reply
hamandcheese
1 day ago
[-]
Not the most fun question, but as I see Astral taking over the python ecosystem, I can't help but wonder: how do y'all plan to make money? It seems like you've taken VC funding, so monetization is inevitable.
reply
zanie
19 hours ago
[-]
The plan remains the same as when the company was announced (https://astral.sh/blog/announcing-astral-the-company-behind-...) — paid services that build on and integrate with the free tooling.
reply
renmillar
1 day ago
[-]
Yes, are they going for enterprise licensing or something similar to JetBrains' approach?
reply
blibble
1 day ago
[-]
prior to astral appearing, python's tooling has been beyond terrible, compared to say, Java's

astral have now replaced the awful pip with the fantastic uv

various awful linters with with the fantastic ruff

and now hopefully replacing the terrible type checkers (e.g. mypy) with a good one!

I hope they have the pypi backend on their list too, my kingdom for Maven Central in python!

reply
kokada
1 day ago
[-]
> prior to astral appearing, python's tooling has been beyond terrible, compared to say, Java's

I would concur with you if you said Go, Rust, Ruby, or even heck, PHP, but Java is probably the only language that I know that is in a situation even as bad as Python or even worse (at least for me definitely worse, because at least I understand Python tooling enough even when using it only for hobby projects, while I still don't understand Java tooling enough even after working professionally with JVM languages for 7+ years).

Java is the only language that I know except Python that has multiple project/package managers (Maven, Gradle, probably even more). It also has no concept of lock files in at least Maven/Gradle, and while resolution dependency in Maven/Gradle is supposed to be deterministic, from my experience it is anything but: just a few weeks ago we had a deployment that failed but worked locally/CI because of dependency resolution somehow pulled different versions of the same library.

Fighting dependency hell because different dependencies pull different version constraints is a pain (all Java/JVM projects that I ever worked had some manually pinned dependencies to either fix security issues or to fix broken dependency resolution), and don't even get me in the concept of Uber JARs (that we had to use in previous job because it was the only way to ensure that the dependency tree would be solved correctly; yes maybe it was by incompetence of the team that maintained our shared libraries, but the fact that we even got at that situation is unacceptable).

Oh, and also Gradle is "so fun": it is a DSL that has zero discovery (I had IntelliJ IDEA Ultimate and I could still not get it to auto-complete 60% of the time), so I would just blindly try to discover what where the inputs of the functions. The documentation didn't help because the DSL was so dynamic and every project would use it slightly different, so it was really difficult to discover a way to make it work for that specific project (the examples that I would find would be enough different from my current project that 90% of time it wouldn't work without changing something). Python at least has `pyproject.toml` nowadays, and the documentation from PyPA is good enough that you can understand what you want to do after reading it for 10 minutes.

reply
screye
1 day ago
[-]
Even after all the praise, I'd say they're underrated.

Modular.ai raised $100 million to solve tangentially similar problems with python. Astral has already had a much larger impact, while providing better integration with less than 10% of that money.

reply
melodyogonna
1 day ago
[-]
Modular raised $100 million to build heterogeneous AI infrastructure; building a language is just a subset of their goals.
reply
swyx
1 day ago
[-]
disagree. modular has a different focus, to turn python into a systems language.

you could even say that astral and modular focus on two extreme ends of the developer experience spectrum - just making python tooling faster, vs making python-ish code faster.

reply
screye
1 day ago
[-]
To be clear, I wasn't dunking on Modular. I meant it more as praise for what Astral has achieved. Lattner is a veteran and Mojo seems to be coming along at a good pace. Mojo is admittedly more of a Rust-competitor. Couldn't think of other contemporary "we are trying fix python's problems" companies. So went for the low hanging fruit.

Either way, Python's irritants are the 'stick' that motivates devs to try alternatives. Astral is patching python's problems at lightning speed. Soon, there may not be enough incentive left to migrate off python. I'm assuming Mojo's target customer is an application dev who uses python and not a seasoned system dev looking for a more aesthetic language.

fwiw, I hope Mojo succeeds.

reply
sakesun
1 day ago
[-]
Credit should also go to Rye project by Armin Ronacher.
reply
lemontheme
1 day ago
[-]
Yeah, iirc he was the first to go ‘what if we had cargo for python?’
reply
danlamanna
1 day ago
[-]
> I hope they have the pypi backend on their list too

IIRC they have floated the idea of private registries as a commercial offering in the past.

reply
zahlman
1 day ago
[-]
What issues do you have with the PyPI backend?
reply
rexledesma
1 day ago
[-]
Very excited to have a new fully featured Python language server working in both vscode and vscode forks (e.g. Windsurf, Cursor).

Pylance is borked on these forked distributions, so having a new solid alternative here that doesn't involve adopting yet another forked Pyright implementation (BasedPyright, Cursor Pyright, Windsurf Pyright, ...) sounds great to me.

reply
maxloh
1 day ago
[-]
You should try basedpyright: https://docs.basedpyright.com/latest/

> basedpyright re-implements many features exclusive to pylance - microsoft's closed-source extension that can't be used outside of vscode.

reply
rexledesma
1 day ago
[-]
I mentioned it in my initial comment. Ideally, I would like to use the same type checker/LSP for a language everywhere (any of my local editors, CI/CD).

This is especially important when working in a team setting.

It doesn't feel great to use a forked type checker/LSP that's not enforced in your org's CI/CD. And it also doesn't feel great to force the forked type checker onto the entire organization when only a subset of folks may be using a forked vscode editor.

reply
krupan
1 day ago
[-]
Have these guys figured out how to make money yet?
reply
dcreager
1 day ago
[-]
We're going to set up a lemonade stand in the main hall at PyCon next week
reply
all2
1 day ago
[-]
This might actually be a decent business model. Sell hard goods to fund your habit of making excellent software tools.
reply
digdugdirk
1 day ago
[-]
Fingers crossed this isn't a joke.
reply
dcreager
1 day ago
[-]
Well, it is a joke...but that said, we're hosting a happy hour which is kinda similar!

https://partiful.com/e/Dcrv6XA8PjWTK5Zhw8yr

reply
ensignavenger
1 day ago
[-]
But are you going to actually make money from it? I am quite leery of companies releasing awesome open source code, growing and taking on massive financing (based largely on that growth, which they achieved because of being open source) then after becoming entrenched in the market, bitching and whining about not "capturing enough value" from their open source code. (And often, switching to an non open source license, while simultaneously declaring their love for open source!)

Now, I know you folks haven't ever done that, but it is such a pattern lately that I almost expect it from any for profit company whose primary product is open source. I don't want to judge you based on the actions of others... but the pattern is so well established now that I have to exercise caution.

Now, in the end, I simply switch to whatever open source fork or alternative crops up (or already exists). But it makes it hard for me to go "Hey everyone, you should be using Astral's X because they are awesome and X is awesome and X is open source. And do 'y'[code contributions, financial sponsorship, buy support from them- whatever it might be] to make sure to support Astral and their development of the open source X code!" ... because I am anticipating the rug pull, taking the trademark and brand down the tube with it. Yet, I really, really, really want to support you folks. I want to tell everyone about your great products. I want to encourage them to use your product and to support you financially.

reply
bb88
1 day ago
[-]
I'm looking forward to talking with you guys.
reply
__MatrixMan__
1 day ago
[-]
If they're making our lives better, perhaps the question is whether we've figured out how to pay them yet.
reply
codr7
1 day ago
[-]
From my experience it's VERY rare for people to pay for anything unless they're forced, I do the same thing even though I'm very aware of the need for compensating creators.

Talk is cheap, and people talk a lot about supporting projects.

Maybe if we could make some kind of statistics over the number of projects that were abandoned because maintainers didn't feel like working and dealing with random people for free anymore. Make the consequences of freeloading visible somehow.

reply
__MatrixMan__
1 day ago
[-]
That has been my experience as well. I consider it a bug. I'm not proposing that we all just try harder to be altruistic, but rather that we craft some institution for rewarding people who have solved problems for many without encumbering those solutions with a monetization scheme.

Make like, a week long "holiday" where you either verify that your company has made an adequate donation to the OSS maintainers that make their products possible, or we all just go on strike for that week. Or... something. I'm sure somebody has a better idea than mine, lets get creative.

reply
bdzr
1 day ago
[-]
> I'm not proposing that we all just try harder to be altruistic, but rather that we craft some institution for rewarding people who have solved problems for many without encumbering those solutions with a monetization scheme.

> I'm sure somebody has a better idea than mine, lets get creative.

Every creative scheme I've seen someone try to come up with fails to do what charging money for a product can. Charge money for stuff, have a free tier, enjoy sustainable software.

reply
__MatrixMan__
1 day ago
[-]
That's fine for software that can still scratch the itch after it has been turned into a product, but I think there's a lot of unexplored space outside that category.

There's also a bunch of cases where adding tiers and payment flows blows the complexity budget and now what used to be a good idea is no longer worth it.

reply
codr7
1 day ago
[-]
Yeah, I'm not that interested in selling stuff, it shifts the focus too much for me.

But I wouldn't mind being compensated for sharing what I create by those who find it useful.

reply
joshdavham
1 day ago
[-]
I'm curious to see what Astral will cook up! I assume they'll probably eventually create some sort of paid devtool service.

With that being said, the worst case scenario is that they go caput, but that still leaves the python community with a set of incredible new rust-based tools. So definitely a net win for the python community either way!

reply
never_inline
1 day ago
[-]
They might pivot to some enterprise value added services. Probably around SAST - think sonarqube.
reply
nickagliano
1 day ago
[-]
Interesting to see astral come out with this right around facebook’s release of “Pyrefly, a faster Python type checker written in Rust”.

Not making any sort of ethical statement, just interesting that rust keeps eating the python and JS tooling worlds.

reply
masklinn
1 day ago
[-]
> Interesting to see astral come out with this right around facebook’s release

Astral announced they were building a typechecker back in January: https://x.com/charliermarsh/status/1884651482009477368

reply
emptysea
1 day ago
[-]
Also interesting, pyrefly uses Ruff’s parser!

https://github.com/facebook/pyrefly/blob/a8626110da034f8e513...

reply
tasn
1 day ago
[-]
Astral announced it last year I think, so it's been a long time coming.
reply
kmacdough
1 day ago
[-]
Yes, because this is precisely what rust has proven it is exceptional for: tooling that needs to be very correct and strives to be fast.
reply
ipsum2
1 day ago
[-]
Pyrefly is a rewrite of Pyre, a Python typechecker which has been around for 4-5 years. Pyre is the strictest type checker I've used, compared to mypy, but its kind of a pain to set up.
reply
zem
1 day ago
[-]
language tooling is a sweet spot for rust, for sure.
reply
akdor1154
1 day ago
[-]
Yeah.. Have either of the ty / pyrefly teams reached out to the other? I feel like the community does not really need two fast python type checkers.

(However, vc-backed astral probably need control over theirs to keep monetization options open, and Facebook probably need control over theirs so it can be targeted at Facebook's internal cool-but-non-standard python habits... Sigh.

Why do we have nice things? Money. Why can't we have nice things? Also money.)

reply
_carljm
1 day ago
[-]
Yes, we've talked; I know a number of the pyrefly devs well. Ty had already been months in development when pyrefly development started. We discussed collaboration, but they decided they needed to do their own thing in order to move quickly and ensure it would serve their needs, which is totally reasonable.
reply
digdugdirk
1 day ago
[-]
It looks like both projects seem to use Ruff at their core, is there any documentation that describes the differences in each project's approach?
reply
theLiminator
21 hours ago
[-]
I believe the only sharing is parsing/ast.
reply
cristea
1 day ago
[-]
Will it support Django stubs? Only blocker for my company to switch
reply
pamelafox
1 day ago
[-]
I am literally checking HackerNews while I wait for mypy to finish running, so I am excited to hear a faster type checker is on the way! Hope the error messages are also helpful.
reply
dcreager
1 day ago
[-]
We're definitely thinking hard about the ergonomics of our error messages! We're drawing inspiration from rustc and miette for the diagnostic model, and are aiming for a quality bar on par with rustc for their content.
reply
tiltowait
1 day ago
[-]
I've been looking forward to this since the original announcement (and before, really).

On the modest codebase I tried it on (14k LOC across 126 files), it runs in 149ms compared to 1.66s in pyright (both run via uvx <tool>). I couldn't get it to play nicely with a poetry project, but it works fine (obviously) in a uv project.

Definitely some false-positives, as expected. Interestingly, it seems to hate the `dict()` initializer (e.g. `dict(foo="bar")`).

reply
SOLAR_FIELDS
1 day ago
[-]
Only one order of magnitude? I thought it would be 2. Isn’t ruff 400x faster than the fastest Python alternative?
reply
Philpax
1 day ago
[-]
A type checker is much more algorithmically bound / difficult to parallelise, but I'm sure there are still wins to be had in the future.
reply
throwaway63467
1 day ago
[-]
I was wondering when someone would write a type checker in Rust, seemed like an obvious thing to do given how slow mypy is.
reply
joshdavham
1 day ago
[-]
There's Pyrefly if you wanna check it out: https://github.com/facebook/pyrefly
reply
no_time
1 day ago
[-]
Awesome work. What is the business model for these astral tools? It’s a bit of a “waiting for the other shoe to drop” feeling after seeing the VC backing on the company page.
reply
lemontheme
1 day ago
[-]
From what I’ve gathered (because I had similar concerns), the code is properly open source. In the very worst case, should there ever come a rug pull, it can be forked.
reply
dcreager
1 day ago
[-]
The code is definitely open source from a licensing perspective, but we are also trying to ensure that a healthy community forms around our tools as well. We've been developing ty in the open for the last year or so, and it already includes significant work from external contributors. This is definitely not a project where only Astral-paid engineers can contribute.
reply
Tryk
20 hours ago
[-]
That's cool. So can you elaborate on how your long-term profitability is projected? Of course there would not be too much point in contributing to a project that gets aggressively monetized in the near future.
reply
urbandw311er
1 day ago
[-]
It’s like when we hit a new month the quota of “talk about Rust” credits is renewed.
reply
andenacitelli
1 day ago
[-]
Say what you want, Astral ships impressively fast and their stuff works well. Python has been looking for better tooling for a long time.
reply
rc00
1 day ago
[-]
The timing of the recent batch of propaganda makes it hard to believe it's not coordinated. I wouldn't suggest paid actors but maybe just an attempt to counter some fairly visible and negative recent takes. The amount of "I love Rust but" comments make it hard to take the commentary seriously too.
reply
Philpax
1 day ago
[-]
Is it really that hard to believe that people like the language and the problems it solves for them?
reply
urbandw311er
1 day ago
[-]
I think what stands out (and becomes quite tedious) is that most posts on HN don’t tend to include the language choice in the headline.

So a typical HN post might be “A new widget that saves times rendering Python code”. Whereas we get this constant barrage of “A new widget that saves times rendering Python code in Rust” with Rust appended to it.

reply
urbandw311er
1 day ago
[-]
PS: just to explain, the original post title was like this but it has now been modified to remove Rust.
reply
Sarios
1 day ago
[-]
Perhaps a silly question. Will ty be usable for getting semantical completions / suggestions. Similar to using pyright to get completions based on what's being written.
reply
dcreager
1 day ago
[-]
We are planning on shipping an LSP front end, and the goal is for that to include code completions. Though to set expectations, they will probably not be that sophisticated on day one. There's a lot of interesting work that we could do here, but it will take time!
reply
Sarios
21 hours ago
[-]
You guys take your time and enjoy yourselves :). You have a proven record of making good tools, so if this part takes a while, let it take a while.
reply
pizza
1 day ago
[-]
Probably not a top priority but it would be really really cool if this thing had solid t-string support from the jump, to the extent that it’s feasible without actually executing code
reply
The-Ludwig
1 day ago
[-]
If this will be only 50% as awesome as ruff or uv, it will be a future must-have for me.
reply
cyounkins
1 day ago
[-]
How does Astral plan on making money?
reply
preciousoo
1 day ago
[-]
CI/CD products most likely, or something more futuristic in that line
reply
nindalf
1 day ago
[-]
CI/CD, private repositories, providing hosting. These are the options used by similar companies.

But I like that they’re focussing on creating something useful before chasing revenue. Once they’ve got a single tool that provides a consistent dev experience for Python developers and it’s widely adopted they should be able to pursue monetisation easily.

reply
Philpax
1 day ago
[-]
I think that's a bit optimistic; that's the path every VC-funded tooling company tries to take, and it often doesn't end well; restrictive licenses, hostile forks, early deaths, etc. You need to have some kind of plan ahead of time.
reply
anentropic
1 day ago
[-]
"Step 3: profit!"
reply
Affric
1 day ago
[-]
See the thing about astral is that they get why Python has been successful in the first place:

When it was released it might have been one of the easiest to use languages.

The focus on tooling and making the tooling fast has been sharp. Seeing people recommend using non-astral tooling seems nuts at this point.

reply
kodablah
1 day ago
[-]
Fingers crossed this is/becomes extensible. Pyright and MyPy both suffer from lack of extensibility IMO (Pyright doesn't consider the use case and MyPy plugins come across as an afterthought with limited capabilities). There are many things that can be built on the back of type-checked AST.
reply
notatallshaw
1 day ago
[-]
Charlie already said in a podcast (https://www.youtube.com/watch?v=XVwpL_cAvrw) that they are not looking to make it extensible. That it's considered a feature that type checking works interchangeably across tools and projects.

Ruff's linting and formatting is more likely to get plugin/extension support at some point in the future.

reply
WhyNotHugo
1 day ago
[-]
I wonder how they'll handle situations like Django, which don't seem expressible via stubs.
reply
anentropic
1 day ago
[-]
I love Django, but maybe Django should change more from their side? e.g. Django 6?
reply
dathinab
1 day ago
[-]
not specific to just your answer but why do people mention ruff?

Ruff is a linter which (intentionally) does close to no type checking.

So you pretty much have to pair it up with a type check to get any even just half way decent static code analysis.

reply
andenacitelli
1 day ago
[-]
I’ve had this same thought. Ruff doesn’t support extensions / custom lint rules that I’m aware of, so maybe don’t get your hopes up.
reply
the_mitsuhiko
1 day ago
[-]
Not supporting plugins for a type checker to me is a plus. It’s quite frustrating that some Python projects only typecheck if you have plugins. That is a major source of frustration.
reply
zem
1 day ago
[-]
python packages that do a lot of metaprot can only be properly type checked if you replicate that metaprogramming at the type level. e.g. if dataclasses were not part of the standard library they would need a plug-in to be handled correctly.
reply
Redoubts
1 day ago
[-]
I mean that kind of code exists; things like attrs are too magical otherwise
reply
joshdavham
1 day ago
[-]
Any plans to create an official ty github action? I've been loving the ruff github action.
reply
simlevesque
1 day ago
[-]
I installed it in VSCode and removed Mypy, I haven't looked back: https://marketplace.visualstudio.com/items/?itemName=astral-...
reply
tayo42
1 day ago
[-]
Curious why so many people want to implement type checkers for python? What problems are being solved that aren't covered already?
reply
dathinab
1 day ago
[-]
1. complete type checking

in python eco system you have linters like ruff which do hardly any type checking and type checkers like mypy which do try to approach complete type checking, but still are absurdly full of holes

2. speed

any of the "established" type checkers either are supper slow (e.g. mypy) so you only run it like once before commit instead of "life" or do fail to properly type check so many things that if you have a requirement for "adequate static code analysis" they reliably fail that requirement (which might result in a legal liability, but even if not is supper bad for reliable code and long term maintenance)

also probably priorities are switched with 1st speed then closing holes as the later part is really hard due to how a mess python typing is (but in many code bases most code won't run into this holes so it's okay, well except if you idk. use pyalchemy as "ORM" subclassing base model (just don't terrible idea)).

reply
mvieira38
1 day ago
[-]
The ones that are around are slow, and working with untyped Python is a pain in large codebases
reply
silverwind
12 hours ago
[-]
Existing python typecheckers are bad, slow and their communities are fragmented. If the community would agree on a single good and fast type checker, everyone will benefit.
reply
simonw
1 day ago
[-]
Speed.
reply
alexwaygood
1 day ago
[-]
That's a big part of it, but there are also several areas where we're trying to innovate on functionality as well as speed. I'd personally be pretty disappointed if the only thing we had to offer at the end of all this was a type checker where the only value add was speed. We've also got first-class support for intersection types, and quite a different model to other type checkers regarding when and whether redefinitions are allowed, for example. We believe there are significant areas where typing can be made more usable and easily adoptable than it is today.
reply
briandw
1 day ago
[-]
Looks good but it has the same issues that i have with mypy. Packages that don't include the type hints blow-up my process. In mypy i've come to terms with strategically ignoring packages or finding a package of type hints. Mypy is runs cleanly on my project but I get >800 errors with TY, mostly things like:

lint:unresolved-import: Cannot resolve imported module `pydantic` --> vartia/usr_id.py:4:6 | 2 | from typing import Optional, Any 3 | from enum import Enum 4 | from pydantic import BaseModel, ConfigDict

looking forward to the release version.

reply
_carljm
1 day ago
[-]
The current version can handle importing pydantic without error just fine, but it probably can't find your virtualenv, so it doesn't know what third-party dependencies you have installed. Ty will discover your venv if it is in `.venv` in the project directory; otherwise you can help it out with the `--python` CLI flag.
reply
alexwaygood
1 day ago
[-]
We'll also discover your venv if you: - Activate it manually (`source .venv/bin/activate`, etc.) - Set the `VIRTUAL_ENV` environment variable - Or use a command such as `uv run` or the equivalent from pdm/poetry/hatch to run ty (these project managers usually implicitly set the `VIRTUAL_ENV` variable to point to the project's virtual environment before executing any commands)
reply
f311a
1 day ago
[-]
Does it support go to definition and other lsp features?
reply
dcreager
1 day ago
[-]
We do plan to provide an LSP server and VS Code plugin, which will support GTD etc. Though as several others have pointed out (e.g. https://news.ycombinator.com/item?id=43919354), it's still very early days for ty, so we don't have concrete release announcements for that yet.
reply
darkteflon
1 day ago
[-]
The VS Code extension is linked in one of the above comments. Accepting that it’s early days and all, but … if someone _really_ hated Pylance, could they replace it with ty for hobby projects and get most of the same headline functionality?

Thanks for all your great work! Love ruff, rye/uv.

reply
_carljm
1 day ago
[-]
Not the same headline functionality yet, no. At the moment all we support is diagnostics (some of which may be wrong) and go-to-type-definition. More will come.
reply
darkteflon
1 day ago
[-]
Thanks for the indulgence. My question was unreasonable, upon reflection, in light of the other information you’ve already provided in the thread.
reply
Hasnep
1 day ago
[-]
Depending on the reason you hate Pylance, you could try basedpyright instead https://docs.basedpyright.com/v1.18.4/installation/ides/
reply
ljouhet
1 day ago
[-]
uv is an incredible tool ; ty will be also. It's insanely fast

For now, I have some false negative warnings :

'global' variables are flagged as undefined `int:unresolved-reference: Name ... used when not defined` (yeah, it's bad, I know)

f(*args) flagged as missing arguments `lint:missing-argument: No arguments provided for required parameters ...`

reply
pt_PT_guy
1 day ago
[-]
don't forget ruff checker and formatter
reply
robertwt7
1 day ago
[-]
This will be similar to Typescript I assume? If so I can’t wait to use it!! I cant count how many times I’ve searched for “TS like in Python” since I’ve started working on Python codebase. TS is so awesome that I use it 100% on new projects. Ruff is also very good, but with this, large code base Python will be a breeze to work with
reply
IshKebab
1 day ago
[-]
You can already use static type annotations in Python and check them with Pyright. This will just make it faster.

Also currently the Python IDE support (autocompletion, refactoring, etc.) in VSCode is provided by Pylance which is closed source, so this would provide an open source alternative to that.

reply
sestep
1 day ago
[-]
Is this the same thing as Red Knot?
reply
_carljm
1 day ago
[-]
Yes, red knot was the internal development code name; ty is the actual name.
reply
drcongo
1 day ago
[-]
I've been looking forward to this for what seems like an age.
reply
codydkdc
1 day ago
[-]
how long is an age?
reply
amelius
1 day ago
[-]
We're living in the information age. Not sure when or what the next age will be, but you get the idea.
reply
arijun
1 day ago
[-]
Pretty sure we’ve already made it to the disinformation age.
reply
GuinansEyebrows
1 day ago
[-]
in the tolkien legendarium they may be over 3000 years.
reply
tough
1 day ago
[-]
a year (probably)?
reply
drewcoo
1 day ago
[-]
An age is a long amount of time.

"An age" is probably an attempted cleaning-up of "a coon's age."

https://grammarist.com/usage/coons-age/

reply
dhruv3006
22 hours ago
[-]
Great job!
reply
TeeMassive
23 hours ago
[-]
Glad to see that we have the type-checking equivalent of Ruff!
reply
rowanG077
1 day ago
[-]
Recently I started a python project and I wanted to do it the "proper" way. mypy + pylint. But even on this small 15-20kloc program these tools are way to slow to do anything in realtime. It takes double digit seconds to have feedback. Way to long for an LSP. I'm honestly appalled the state of affairs is this bad. What the hell do people do with moderately or even large sized code bases?
reply
mil22
1 day ago
[-]
Pyright + Pylance + Ruff has been rock solid for me on my 100Kloc codebase for more than a year now. I use the VS Code extensions, and Pyright and Ruff are integrated into my pre-commit.
reply
Hasnep
1 day ago
[-]
They probably use (based)pyright which is much faster than mypy and ruff which is much much much faster than pylint.
reply
joejoo
1 day ago
[-]
Astral killing it with the Python tooling.
reply
ivanbelenky
1 day ago
[-]
holy shit this is happening
reply
canterburry
1 day ago
[-]
How about we just stop creating non type safe languages. Would save everyone so much hassle.

[bring on the downvotes]

reply
Hasnep
1 day ago
[-]
It's a bit late now that python has existed for a couple of decades
reply
canterburry
1 day ago
[-]
How about put it on the roadmap for v4?
reply
codr7
1 day ago
[-]
You're begging for it.

Not every situation calls for type safe languages, you're projecting a preference.

reply