Show HN: Wat – Deep inspection of Python objects
393 points
2 months ago
| 26 comments
| github.com
| HN
skeledrew
2 months ago
[-]
Waaat :D. This is so nice. I used to use python-ls[0] for similar, but something about it that I can't recall broke for me and it's no longer maintained. Adding to my debugging arsenal which primarily consists of snoop[1] and pdbpp. Only thing I'd like for wat now is maybe an ipy widget for even easier object exploration in Jupyter.

I'm also really appreciating the base64 exec hack. All my years in Python I never thought of or came across it until now. I'll totally be using it for some things :).

[0] https://github.com/gabrielcnr/python-ls [1] https://pypi.org/project/snoop/

reply
randomfool
2 months ago
[-]
For object inspection in Jupyter if doing more numpy type things, check out penzai: https://github.com/google-deepmind/penzai
reply
nyoomboom
2 months ago
[-]
Seems like if you add more "a"s to W[a..]t you slide a scale from disgusted to awed to comical.
reply
benrutter
2 months ago
[-]
Ah this looks fun! I use "dir" all the time with python, and find it more useful than official documentation in some cases where documentation is not great.

Surprised there isn't more innovation and new tools like this around python's interactive shell given it's one of the real strong points the language has.

reply
pjot
2 months ago
[-]
There’s the help() function as well! Super helpful!
reply
notpushkin
2 months ago
[-]
And if you use IPython, you can use the ? shorthand, too:

    In [1]: int?
    Init signature: int(self, /, *args, **kwargs)
    Docstring:
    int([x]) -> integer
    int(x, base=10) -> integer

    Convert a number or string to an integer, or return 0 if no arguments
    are given.  If x is a number, return x.__int__().  For floating point
    numbers, this truncates towards zero.

    [...]
reply
hughesjj
2 months ago
[-]
Favorite way to develop python, so far:

1. Write the program skeleton

2. Put an import pdb; pdb.set_trace() whereever I need to fill out next

3. Once I hit the breakpoint, type 'interact'

4. Start coding in the debugger.

4a. For small functions write in a scratch pad, test against another python console, then paste into the current debugger.

4b. For network calls, save the inputs and outputs. Toss a decorator on some functions to log the inputs and outputs to files, and cool we already have some proto test data.

4c. For documentation, call help() on the variable or method or module.

reply
shabble
2 months ago
[-]
the double questionmark `foo??` is sometimes also useful, since it'll show full source for the function/method/thing if you do it to a function.
reply
bobuk
2 months ago
[-]
Looks like a fancy version of good old icecream. https://github.com/gruns/icecream

If you never heard about it, scroll to the bottom to https://github.com/gruns/icecream#icecream-in-other-language...

reply
creichenbach
2 months ago
[-]
Neat. I built something similar but web-based for Java years ago: https://scg.unibe.ch/wiki/projects/DoodleDebug
reply
fermigier
2 months ago
[-]
These kinds of tools are useful.

Twenty years ago I wrote an object introspector for Zope ;)

Nowadays, I'm using devtools daily, and icecream and q occasionally. I'll give wat a try.

reply
enriquto
2 months ago
[-]
> from wat import wat

Given the cool nature of this project, I'm surprised they don't offer simply "import wat" with identical usage syntax. Thus inviting curious users to wat/wat in order to discover the trick...

reply
igrek51
2 months ago
[-]
"import wat" would be great, but Python has some restrictions about modules not being callable. That's why I ended up with longer `from wat import wat`. Not sure but maybe this would be more convenient `import wat; wat.wat / object`
reply
Rhapso
2 months ago
[-]
Python has code execution at import time. Just grab the global context, and overwrite the module with a callable.
reply
skitter
2 months ago
[-]
Afaik the import sets the module in the importing module's context only after the code in the imported module is run.

Edit: Oh, you don't mean in the importing module's globals, you mean `sys.modules`. Yeah that works!

reply
Rhapso
2 months ago
[-]
actually, no you were right about what I meant but i'll claim credit for helping you realize there was a better way XD
reply
scrollaway
2 months ago
[-]
Check out the module “q”! It’s callable, its author talked about how great it would be to have a module level __call__ because the way it was made callable is super wonky.
reply
enriquto
2 months ago
[-]
> the way it was made callable is super wonky.

why do you say it's wonky? it's just

    sys.modules[__name__] = wat
According to the official python documentation [0] of sys.modules:

> This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks.

Thus, the modules dictionary is being used as intended, and it has the desired effect. Nothing wonky about that.

Of course, a __call__ method would be better because you could still keep the other functions inside the module. But for a single-function import like "wat" it seems quite natural and sane.

[0] https://docs.python.org/fr/3/library/sys.html

reply
hprotagonist
2 months ago
[-]
reply
igrek51
2 months ago
[-]
Thank you all, I learned from you that it's possible to have a callable module. Although `sys.modules[__name__] = wat` looks like a black magic and I'm a bit afraid of locking out to other importable things in this package, I think I'll go for it.
reply
notpushkin
2 months ago
[-]
It is kinda black magic, and I think it's not something people would expect. You can import other things just fine though:

    import sys

    def wat(): print("wat")
    def bar(): print("bar")

    wat.wat = wat
    wat.bar = bar

    sys.modules[__name__] = wat
...

    >>> import wat
    >>> wat()
    wat
    >>> from wat import bar
    >>> bar()
    bar
Edit: I think a better way would be to instead add __call__ to the current module somehow though.
reply
skeledrew
2 months ago
[-]
See snoop[0], which does just this. It also has an install function that registers it in builtins, making it universally available.

[0] https://pypi.org/project/snoop/

reply
jkrubin
2 months ago
[-]
I wrote this a long while ago: https://pypi.org/project/funkify/
reply
heyts
2 months ago
[-]
This looks super useful, but I was wondering if I'm the only one bothered by this recent trend of overloading completely unrelated operators (here the `/` operator) in the name of legibility.
reply
wrboyce
2 months ago
[-]
I do agree overloading / is an odd choice in this instance, but it’s a shame “is” can’t be overloaded! Realistically wat(foo) would be fine though.
reply
098799
2 months ago
[-]
One might suggest adding:

  try:
      from wat import wat
  except ImportError:
      pass
to your $PYTHONSTARTUP file to avoid the cumbersome import.
reply
BiteCode_dev
2 months ago
[-]
You can even add the inline importer in base64 that is pretty neat.

But I eventually printed that output, and put it in a dir I point my PYTHONPATH to so that I will always have it available.

Let's see if it sticks.

reply
AcerbicZero
2 months ago
[-]
Wow, this is the kind of tool that would have been a game changer when I was learning python; being able to see what is happening under the covers is part of the critical path for me to learn a language, and native python debugging is underwhelming, at best.

Instead I just installed pry and became a rabid ruby fanboy, but this might get me to give it another go.

reply
fuzztester
2 months ago
[-]
the author is using the Python inspect module of the stdlib under the hood to provide the functionality, of course with a lot of value-add.

see inspection.py in the wat module.

it has this on line 2:

import inspect as std_inspect

reply
fuzztester
2 months ago
[-]
reply
nikvdp
2 months ago
[-]
> If you want to quickly debug something, you can use this inspector without installing anything, in the same session.

> Load it on the fly by pasting this snippet to your Python interpreter

The idea of having a project's readme include a full copy of the project itself as base64'd compressed data is pretty ingenious!

especially for a project like this where you may not have had the foresight to preload it into the environment where you most need it

reply
blueboo
2 months ago
[-]
rich.inspect has been my go-to for this. Comes pre-installed in a vanilla Colab kernel too
reply
emmanueloga_
2 months ago
[-]
Right! I usually use rich's print [1] for this kind of thing too, which I chose after testing a few other libraries that do pretty printing.

--

1: https://github.com/Textualize/rich?tab=readme-ov-file#rich-p...

reply
ilyagr
2 months ago
[-]
I wonder if there's something like this for Lua. It doesn't have Python's built-in conveniences for introspection like `help()`.
reply
nython
2 months ago
[-]
There were various internal Quality of Life modules during my time at amzn, but nothing public. I suspect that given enough time working with Lua, each team will create an ad hoc, informally-specified, bug-ridden, slow implementation of half of python.
reply
ilyagr
2 months ago
[-]
I found https://github.com/kikito/inspect.lua, haven't tried it yet.
reply
lofaszvanitt
2 months ago
[-]
Python's biggest weakness is the lack of a function that shows you how exactly a variable looks like. the var_dump (php, ha!) works like wonders. The best library ever created for Python ;). and now wat is also looking to be a close contender for best helping hand.
reply
fuzztester
2 months ago
[-]
>Python's biggest weakness is the lack of a function that shows you how exactly a variable looks like.

Doesn't the repr() built-in function do what you want?

It can also be customized by defining the __repr__() method of a class, IIRC.

Commonly done.

reply
lofaszvanitt
2 months ago
[-]
Nop.
reply
fuzztester
2 months ago
[-]
reply
chis
2 months ago
[-]
This is incredible
reply
blackbear_
2 months ago
[-]
Looks great! And it would make for a great addition to Jupyter lab. I actually can't believe there is still no decent variable explorer bundled in
reply
revskill
2 months ago
[-]
Is there similar tool for typescript/javascript ?
reply
CornCobs
2 months ago
[-]
Isn’t the browser inspector more than sufficient? You get the interactive object inspector in the console, live expressions, log points etc.

For node you it’s possible to hook it up with the browser inspector as well

reply
jampekka
2 months ago
[-]
Don't know why you're downvoted, but browser devtools have the best object inspection around. I miss them all the time when coding Python.

Hooking devtools with Node is possible but could be a lot nicer.

reply
ur-whale
2 months ago
[-]
Given the natural built-in introspection capabilities of the language, I never understood why something like that wasn't also a built-in part Python.

I mean the number of times I've bumped on json not being able to dump the content of an object is just infuriating given how flexible Python is.

I'm definitely giving this a whirl.

reply
fuzztester
2 months ago
[-]
>Given the natural built-in introspection capabilities of the language, I never understood why something like that wasn't also a built-in part Python.

Somthing like that does exist, and has, from Python 2.x; the inspect module, apart from help() and dir():

It is somewhat lower-level than wat, which builds upon it. But it is powerful.

https://news.ycombinator.com/item?id=41072878

Or did you mean the inspect module itself?

reply
frays
2 months ago
[-]
dir(), help() and now adding wat to my toolbox to help debug!
reply
kseistrup
2 months ago
[-]
reply
maxmcd
2 months ago
[-]
Is there something similar for Javascript?
reply
wantsanagent
2 months ago
[-]
Re: Insta-load

Please don't do this. Exec'ing arbitrary, obfuscated code you just copied from the internet directly in a running environment is such a bad idea it shouldn't even be suggested.

At the very lease please put huge warnings around this section to let people know that it is a horrendously bad idea and they do it at their own peril.

reply
oreilles
2 months ago
[-]
How is it any different than installing the package via pip ? Not only most people won't check the source before running the code, but there is also no way to be sure that the code shipped by pip is the one you read on GitHub...
reply
__MatrixMan__
2 months ago
[-]
gp has a leg to stand on only if they regularly audit the contents of their site packages. Otherwise you're totally right.
reply
tantalor
2 months ago
[-]
This is so bad it should be nsfw-blurred and you have to click 2 buttons to even see it.
reply
erinaceousjones
2 months ago
[-]
Yes, the big warning disclaimer in that part of the docs is definitely required here.

That idea of exec'ing arbitrary obfuscated (compressed) shell code that's easy to copy-paste into a python shell is very helpful, mind.

I've had to debug issues in production with silently hanging I/O threads and my only access is via `kubectl exec`.

This wat tool and that "insta-load" idea pairs nicely with pyrasite for that very useful !!!DO NOT DO THIS, YOUR CONTAINERS SHOULD BE LOCKED DOWN SO YOU CANNOT DO THIS!!! step of copying gdb and the pyrasite library into a running container when all you have is a python shell to play with.

(This almost feels like an exploit, running shellcode after getting RCE :))

https://pypi.org/project/pyrasite/

reply
slt2021
2 months ago
[-]
the gzipped payload is static, you are free to inspect the payload before eval()
reply
timdorr
2 months ago
[-]
I'm not a Python dev. Why would they do this? This is giving vibes of malware embedded into npm packages.
reply
skeledrew
2 months ago
[-]
Convenient way to quickly add extra debugging capability without rerunning. It isn't much different from the many `curl example.com/install.sh |bash` you see around. It's up to the user to check things out before running.
reply
SOLAR_FIELDS
2 months ago
[-]
Yeah, the piping to bash is a tried and true method for various installers. People make a fuss about it, but we don’t see people getting owned that way often. I think with bash installers though it’s pretty trivial to just visit the link and read through the 100 lines of bash. So anything installed this way should be as simple as visiting the link and reading a short amount of code imo
reply
jampekka
2 months ago
[-]
The same people seem to be more OK with installing random .debs and .rpms which can of course arbitrarily run any code with root.
reply
etbebl
2 months ago
[-]
Can't you just install it in your environment in another terminal and then import?
reply
skeledrew
2 months ago
[-]
Sure, but it's a few extra steps. It's been proven(tm) that many people prefer a single, simple thing they can just copy, paste and run, so they can get back to their main concern.

And some may not want it actually installed for whatever reason. Such as when there's no proper separation between dev and prod deps. (I'm mostly just guessing at this point though...)

reply
CornCobs
2 months ago
[-]
Not everywhere that python is run has access to pip. Sshing into some locked down remote machine and needing to debug some script is a use case that comes to mind.
reply
jampekka
2 months ago
[-]
Pip install allows arbitrary code execution.
reply
timhh
2 months ago
[-]
Maybe it's just me but I just use a proper debugger. Debugpy and VSCode work fantastically.

In a previous company I set things up so that when there's an uncaught exception it will automatically start a VSCode debug session and connect to it.

Here's the extension: https://github.com/Timmmm/autodebug/

Unfortunately the Python part of that is not open source but it was only a few lines of code - should be easy to recreate. That repo does contain a C library that does a similar thing.

You might just say "why not just run your program directly with the debugger?" and yeah that is better when you can do it, but I'm working with complicated silicon verification flows that are often several layers of Python and Make, followed by a simulator (e.g. Questa or VCS) that itself loads the library you want to debug. Very difficult to make debugging work through all those layers, but it's quite easy to write some code at the bottom of the stack that says to a debugger "I'm here, debug me!".

reply
alkh
2 months ago
[-]
It important to remind everyone that you can already do something similar via the build-in function help(). For example, if I run help({1}) I get the documentation for a set, running help(set.add) gives the documentation for the add() method, etc. You can even preview objects that you haven't imported by using strings(ex. running help('numpy.ndarray') will correctly give you the documentation, provided numpy is installed in your current python environment). It's pretty helpful and doesn't require installing anything extra
reply
igrek51
2 months ago
[-]
You're completely right. `dir()` does the same in terms of functionality. In fact, my tool makes use of `dir` under the hood. I just wanted to make it more readable, and to combine `dir`, `type`, `repr`, `len`, `inspect`, etc. into one easily accessible place.
reply
fuzztester
2 months ago
[-]
even just

  print(name.__doc__)
works for many kinds of names in Python.
reply
guruparan18
2 months ago
[-]
Also most advanced IDE's do this better and lot more to troubleshoot/debug.
reply
agumonkey
2 months ago
[-]
help says too much, usually you'd like a quick overview slightly formatted and well classified, which is what wat seems to afford here
reply
orbisvicis
2 months ago
[-]
And vars().
reply
fuzztester
2 months ago
[-]
and the dir() built-in Python function.
reply
Retr0id
2 months ago
[-]
I'm normally a big fan of python operator overload hacks, but why are we doing `wat / foo` instead of just `wat(foo)` ?
reply
JulianChastain
2 months ago
[-]
`wat(foo)` also works. The author says he implmented the `wat / foo` syntax to allow you to more quickly use wat by avoiding parentheses
reply
contravariant
2 months ago
[-]
Probably so you can run a big complicated formula in the REPL and easily prepend `wat /` to it to inspect what's going on.

Same reason jupyter notebooks allow you to prepend '?' or '??' to inspect a variable (though not expressions, in that respect this syntax is better)

reply
jawns
2 months ago
[-]
I get the reasoning behind it, but the choice of the division operator is still a little weird.

I believe it should be equally possible to overload the bitwise OR operator (|), which, for people used to piping in Unix, is probably more intuitive.

reply
Tomte
2 months ago
[-]
Wrong direction: you‘d „pipe“ wat through the object. Although TiddlyWiki users might feel right at home.
reply
igrek51
2 months ago
[-]
Right, that would be a wrong pipe direction. Fun fact: It's already possible to do "wat | foo" or "wat << foo" (if you're more familiar with C++ iostream), it has the same effect as "wat / foo"
reply
auscompgeek
2 months ago
[-]
It could override the reverse bitwise or magic method `__ror__`.
reply
Aeveus
2 months ago
[-]
Looks like a great tool. Will start playing with it whenever I have to dive into an existing project again.

One thing I'd like to note, though, is that most engineers (at least around and including myself) would be triggered by the "Insta-load" example of executing base64 encoded (and thus obfuscated) code.

reply
Spivak
2 months ago
[-]
The insta-load is fine. You're supposed to save the snippet, decode and uncompress it, look at it to see what it does and that it's completely self-contained uncontroversial Python code and copy-paste your version that you bookmarked. It's super short.
reply
fuzztester
2 months ago
[-]
just so that it is clear for everyone, including newbies here, does the insta-load code snippet work like this:

the author has encoded the wat module's source code into base64. the snippet shown decodes that back into python code, and then executes it on the fly, thus having the same effect as importing the wat module in the normal python way?

reply
igrek51
2 months ago
[-]
Yes, that has the same effect as installing the package. So if you feel uncomfortable, you can either install the pip package (and of course review the installed code) or review the decoded string before executing it. It's not that obfuscated anyways, it's still quite readable.
reply
fuzztester
2 months ago
[-]
got it, thanks.
reply
tacoooooooo
2 months ago
[-]
not hard to see it

with open('wat_source.py', 'w') as f: f.write(zlib.decompress(base64.b64decode(code.encode())).decode())

reply
kajecounterhack
2 months ago
[-]
I was just about to post that it freaked me out a little, but wasn't sure if I was going to be the only one.
reply
languagehacker
2 months ago
[-]
This is interesting. I'm wondering what compelled the author to use the division magic method for arguments instead of the more intuitive and commonly used approach to passing parameters.
reply
enragedcacti
2 months ago
[-]
I think it makes a lot of sense in the context of a live interpreter. Wrapping and object in help() or dir() is an annoying set of movements vs. just pressing 5 keys with no modifiers (Home, w,a,t,/). It makes its feel much more like a magic command without actually needing it to be magic. The dot modifiers are also convenient compared to passing kwargs imo.
reply
lakshayg
2 months ago
[-]
Interesting choice indeed. It looks like it supports the traditional approach as well

> wat object can quickly inspect things by using the division operator (to avoid typing parentheses). A short, no-parentheses syntax wat / object is equivalent to wat(object).

reply
igrek51
2 months ago
[-]
It's the parentheses that drove me crazy. As people already noted, it's for faster typing, at the cost of the division magic as you noted. If it's more familiar to you, it works with `wat(object)` syntax as well.
reply
wnolens
2 months ago
[-]
I love it
reply
pvg
2 months ago
[-]
The README suggests for easier typing in interactive use.
reply