Finally. The Big Bear has landed.
@beartype 0.19.0 release candidate 1 – the second (and better-be final) beta pre-prelease of the zombified @beartype 0.19.0 release cycle that I should have released three months ago – lands in your lap with a suspicious quacking noise. You're startled. You shriek in dismay! And that's when you hear it. A knock on the door. Muffled voices from the hallway. Disconcertingly familiar, you feel like you almost understand what's going on:
Code Lieutenant: I think we can handle one little beta release. I sent two unit tests. They're bringing it down now.
Agent Leycec: No, Code Lieutenant. Your bugs are already dead.
@beartype 0.19.0rc1 politely hiccups. Yet the struggle for your codebase has only just begun:
pip install --upgrade --pre beartype # <-- sound of bugs panicking distantly heard@beartype 0.19.0rc1 is brought to you by Ultros, a purple octopus in dire need of dental work.
your bugs are up the creek without a paddle! and @beartype isn't going to let them through! does that make @beartype a good Bear?
Okay. Okay. Ultros didn't do nuffin' except lie about on a raft all day and try to eat heroes. Let's try again.
@beartype 0.19.0rc1 is actually brought to you by...
GitHub Sponsors: When You Befriend the Bear, You've got a Bear for Life
This release comes courtesy these proud GitHub Sponsors, without whom @leycec's cats would currently be eating grasshoppers:
- @sesco-llc (SESCO Enterprises), "The Power of Innovation in Trading": this inspires me to get out of the house and do something
https://sescollc.com - @DylanModesitt (Dylan Modesitt), quantitative strategies energy trading associate: ...wikipedia, don't fail me now!
https://dylanmodesitt.com - @tactile-metrology (Tactile Metrology), "Software and hardware that you can touch." When I want to be touched by software and hardware, I call @tactile-metrology:
https://metrolo.gy imagine if this domain actually worked. how cool would that be!?
Thanks so much, masters of fintech and metrology.
The Masters of Fintech and Metrology. That's who.
infer_hint(): Because you hate type hints, make @beartype write them all for you
Type hints are the most compact description of the internal structure of your objects. If you know the type hint for an object, you know the object better than the object knows itself. Type hints are thus the optimal documentation. Unlike docstrings, type hints never lie or @beartype breaks your app. Type hints are both human-readable and machine-readable. They're literally the only thing that is.
Many type hints are trivial to write. All of us can sling around breazy list[int] | None type hints while yawning. It's not impressive. My cats can write that type hint with one sleepless eye open. Seriously. Why do cats sleep with one eye open, anyway? Doesn't that kinda defeat the purpose of... I dunno, sleeping? Must suck to be a paranoid cat. Uhhh. Back to the discussion.
Some type hints, however, are non-trivial. You can't write them. Nobody can. They contain more square brackets than the final dungeon of 80's ASCII BOFH terminal darling Rogue. When your type hint looks like this, the end of code maintainability cannot be far:
that smiling face is you hammering the commit button despite CI failures from @beartype.
Moreover, you don't even know the internal structure of most objects. Somebody else wrote those objects. They forgot how those objects worked a hot minute after clocking out at 4:12AM seven days deep into a crunch-time death march last January. They documented how those objects worked, but their documentation doesn't make sense and lies about everything. Now, nobody knows how those objects work.
But what if somebody did know how those objects work? What if somebody knew Python better than Python knew itself? Introducing... that somebody.
infer_hint(): Deep Introspection for the Deep Code Diver
Let @beartype ease your weary burden, traveller. It is dangerous to go alone:
# Crazy object you could understand. But... ain't nobody got that kinda time.
>>> from pygments.lexers import PythonLexer
>>> root_tokens = PythonLexer().tokens["root"]
# I've got a crazy object here, @beartype. What's the crazy type hint that
# matches my crazy object? This is gonna really suck. I can *FEEL* it coming
# through my monitor tonight.
>>> from beartype.door import infer_hint
>>> infer_hint(root_tokens)
list[ # <-- what could possibly go wrong?
typing.Union[ # <-- sucky stuff starts
tuple[str | collections.abc.Callable[typing.Concatenate[object, object, ...], object], ...], # <-- sucky stuff intensifies
tuple[str | pygments.token._TokenType[str], ...], # <-- so much sucky stuff
typing.Annotated[collections.abc.Collection[str], beartype.vale.IsInstance[pygments.lexer.include]] # <-- go to heck, typing!
]
] # <-- i have no idea and neither do you...uhh. If you say so, @beartype. I guess? </weeps_in_square_bracket_hell>
what it feels like to live in Canada, friends.
BeartypeAI™: Even a Broken Algorithm is Right Twice a Commit
beartype.door.infer_hint() (i.e., the algorithm hereafter known simply as BeartypeAI™) knows all about deep introspection of arbitrarily complex objects. Here's what BeartypeAI™ knows:
BeartypeAI™: "I know that your type hints kinda suck. Wait... where are your type hints? I'm panicking. I'm panicking."
you: "Tell my team something my team don't know, BeartypeAI™. We hate type hints. So we don't do type hints. We code instead. You should try it sometime. But... aren't you a type-checking bear? Can you even code with paws?"
BeartypeAI™: "Hold my QA beer."
BeartypeAI™ is here to tell you something you don't know and wouldn't care about even if you did.
BeartypeAI™ is here to write your type hints for you. Why? Because you hate type hints. Just:
- Feed
beartype.door.infer_hint()arbitrarily complex objects. - Annotate those objects with the type hints it regurgitates.
It's impossible to unpack how much madness is happening inside BeartypeAI™. Let's try anyway.
son of a submariner bounces on the hot sand like bugs across your git log.
Homebrew Collection Classes: Annotate the Unannotatable
We've all been there. Some "genius"-tier wise guy devbro invented his own homebrew pure-Python collection called WeirdoCustomList without subclassing a standard collections.abc abstract base class. Sure, they could have just subclassed collections.abc.MutableSequence to write their special-needs alternative to the builtin list type. That would have been too easy. They're a masochist, so they wrote everything from scratch.
Homebrew collections are more common than you think. They're friggin' everywhere! They're multiplying like meat flies! The cat's choking on bloody homebrow collections! Look above, for example. See that nasty typing.Annotated[collections.abc.Collection[str], IsInstance[pygments.lexer.include]] type hint that BeartypeAI™ wrote for you? Yeah.
That's right. The public pygments.lexer.include type is actually a homebrew collection. They thought they were being smart. Sadly, they were being dumb. Because they wrote a homebrew collection from scratch, their collection type is unsubscriptable. You can't subscript it with child type hints like with list[str], so you can't use their collection type as a type hint factory to write type hints, so you can't actually validate their homebrew collection. In fact, you can't validate any homebrew collections.
...until now. BeartypeAI™ knows all about homebrew collections. BeartypeAI™ knows that you can actually validate homebrew collections – but only if you write a custom beartype validator leveraging the PEP 593-compliant typing.Annotated[...] type hint factory in concert with the @beartype-specific beartype.vale.IsInstance[...] validator. Please don't do this manually. You value your precious life force that is leaking all over your keyboard as we speak.
That's what the typing.Annotated[collections.abc.Collection[str], IsInstance[pygments.lexer.include]] type hint is all about. BeartypeAI™ correctly detected that this homebrew collection is actually just an instance of the pygments.lexer.include class that is externally usable as a collection of strings.
Let's get more explicit. Nobody understands pygments – not even pygments. Instead, consider...
this is for @Moosems!
Exhibit (A) – Weirdo Custom List
It's... it's hideous!
from beartype.door import infer_hint
from collections.abc import Iterable, Iterator
# Define a weirdo custom list.
class WeirdoCustomList(object):
'''
Weirdo custom list. The one. The only.
Weirdo custom list pretends to mean no harm. Weirdo custom list is
lonely at night and only wants to be snuggle-friends. *How could you
refuse weirdo custom list in its example of need?*
'''
def __init__(self, items: list) -> None: self._items = items
def __contains__(self, item: object) -> bool: return item in self._items
def __iter__(self) -> Iterator: return iter(self._items)
def __len__(self) -> int: return len(self._items)
def __getitem__(self, index: int) -> object: return self._items[index]
def __reversed__(self) -> Iterator: return reversed(self._items)
def count(self, item: object) -> int: return self._items.count(item)
def index(self, *args, **kwargs) -> int:
return self._items.index(*args, **kwargs)
def __delitem__(self, index: int) -> None: del self._items[index]
def __setitem__(self, index: int, item: object) -> None:
self._items[index] = item
def __iadd__(self, item: object) -> object: self._items += item
def append(self, item: object) -> None: self._items.append(item)
def clear(self) -> None: self._items.clear()
def extend(self, items: Iterable) -> None: self._items.extend(items)
def insert(self, index: int, item: object) -> None:
self._items.insert(index, item)
def pop(self, *args, **kwargs) -> object:
return self._items.pop(*args, **kwargs)
def remove(self, item: object) -> None: self._items.remove(item)
def reverse(self) -> None: self._items.reverse()
# Infer the type hint for a weirdo custom list of strings.
print(infer_hint(WeirdoCustomList([
'No way,', '@beartype.', 'No.', "Friggin'.", 'Way.'])))...which prints:
typing.Annotated[collections.abc.MutableSequence[str], IsInstance[WeirdoCustomList]] "What's so hot about that?", you may now be thinking. Allow me to now pontificate boringly.
WeirdoCustomList isn't subscriptable. It's not a type hint factory. Moreover, despite being a mutable sequence, WeirdoCustomList doesn't actually subclass the standard collections.abc.MutableSequence protocol. Yet, @beartype correctly detected that this particular weirdo custom list is a mutable sequence of strings. How? It's best not to ask weirdo custom list these questions. 🤣
a lone dev seeks the calm in the midst of the bug storm, yet finds only a rainbow
Callable[...] Type Hints: Gods, They Suck.
Annotating callables (especially callbacks) with PEP-compliant Callable[...] type hints is basically impossible. Personally, I've never gotten a single Callable[...] type hint to work right. They never match the callables they're supposed to when I write them myself. mypy and pyright always vomit all over themselves and then me. I mostly just give up now and use the unsubscripted collections.abc.Callable abstract base class instead of full-blown Callable[...] type hints...
...until now. BeartypeAI™ knows literally everything there is to know about annotating callables. What doesn't BeartypeAI™ know? Well, friends:
-
BeartypeAI™ knows that a
lambdafunction accepting no parameters is annotated as...>>> infer_hint(lambda: "No. Friggin'. Way.") collections.abc.Callable[[], object] # <-- woah
-
BeartypeAI™ knows that a
lambdafunction accepting multiple parameters is annotated as...>>> infer_hint(lambda you, will, believe: "@beartype, I am your code father.") collections.abc.Callable[[object, object, object], object] # <-- i don't know what's happening here, but i like it
-
BeartypeAI™ knows that a normal function accepting two mandatory annotated parameters and one optional annotated parameter is "best" annotated with a PEP 612-compliant
typing.Concatenate[...]subscription as...>>> def i_am_tired(this: float, so: int, boring: str = "y u so boring, @beartype!?"): ... >>> infer_hint(i_am_tired) collections.abc.Callable[typing.Concatenate[float, int, ...], object] # <-- don't ask, just accept.
-
BeartypeAI™ knows that a decorator wrapper function accepting a PEP 612-compliant parameter specification is annotated as...
>>> from typing import ParamSpec >>> P = ParamSpec('P') >>> def so_param_so_spec(*args: P.args, **kwargs: P.kwargs): ... >>> infer_hint(so_param_so_spec) collections.abc.Callable[~P, object] # <-- yer frickin' blowin' mah mind here, yo
-
BeartypeAI™ knows that a decorator wrapper function accepting two mandatory annotated parameters followed by a PEP 612-compliant parameter specification is annotated with a PEP 612-compliant
typing.Concatenate[...]subscription as...>>> from typing import ParamSpec >>> P = ParamSpec('P') >>> def more_param_more_spec( ... go_crazy: int, ... dont_mind_if_i_do: str, ... *args: P.args, ... **kwargs: P.kwargs ... ): ... >>> infer_hint(more_param_more_spec) collections.abc.Callable[typing.Concatenate[int, str, ~P], object] # <-- pretty sure the universe just exploded
What I'm trying to say here is that BeartypeAI™ knows all and sees all and doesn't like it what it sees, but is still doing it's best for everybody. It knows more than me. It probably knows more than even you, even though you know everything. That's how much BeartypeAI™ knows.
@beartype casts Meteor Suplex on the runaway train that is your codebase
Tensors Type Hints: Gods, They Suck Too.
Tensor type hints really suck. So your team wants to annotate NumPy, JAX, PyTorch, or TensorFlow arrays, huh? That's a perfectly reasonable request. Too bad, though. Because tensor type hints suck.
Tensor type hints suck so bad you have to use third-party packages like jaxtyping just to make them work, despite the fact that both NumPy and JAX ship type hint-centric subpackages like numpy.typing and jax.typing that are supposed to make tensor type hints "just work." Of course, tensor type hints don't "just work." They don't even work...
...until now. BeartypeAI™ knows literally everything there is to know about annotating tensor type hints. Actually, that's a lie. I really wanted BeartypeAI™ to know literally everything there is to know about annotating tensor type hints in time for @beartype 0.19.0rc1. Sadly, I played video games instead. I only got around to implementing BeartypeAI™ support for inferring NumPy tensor type hints.
Still, NumPy is better than nothing. One out of four ain't bad. Right? ...anybody? 😮💨
# Define the greatest NumPy array that has ever existed.
>>> from numpy import asarray
>>> best_array_is_best = asarray((1, 0, 3, 5, 2, 6, 4, 9, 2, 3, 8, 4, 1, 3, 7, 7, 5, 0,))
# Create a type hint validating that array, @beartype! Look. Just do it.
>>> from beartype.door import infer_hint
>>> infer_hint(best_array_is_best)
typing.Annotated[numpy.NDArray[int], beartype.vale.IsAttr['ndim', beartype.vale.IsEqual[1]]] # <-- wtf, @beartypeAnd... that's the type hint. That type hint requires no third-party dependencies. It's all @beartype, all one-liner. Nobody's writing that sort of gruelling bracket hell on their own. Not even @leycec. Now, just let somebody else do all the suffering for you. That somebody is @beartype. Who knew?
don't be that one guy who falls off the floating island. just... don't
BeartypeAI™: So It Does Structural Similarity Too Now, Huh?
The new beartype.door.infer_hint() function tidily combines with the existing beartype.door.is_bearable() tester to create a new monstrous form of Python object test: structural similarity. But let's back up. In the beginning, there was:
# The "is" operator. Test whether two objects are literally identical.
>>> "I like big bugs and I cannot lie." is "I like big bugs and I cannot lie."
True
# The "==" operator. Test whether two objects are semantically identical.
>>> ['Other', 'devbros', 'may', 'deny',] == ['Other', 'devbros', 'may', 'deny',]
TrueBut what if you want to test whether two objects are structurally similar (i.e., have a similar internal structure but are neither literally nor semantically identical)? Without BeartypeAI™, you can't do that. But you have BeartypeAI™. You no longer have to accept the mouldy table scraps that the standard Python library has left you. Why? Because you have harnessed the perfidious power of beartype.door.infer_hint() + beartype.door.is_bearable(). Arise, a new darkness!
>>> from beartype.door import infer_hint, is_bearable # <-- boring stuff
# Excitement builds. Declare data structures for great glory of your code.
>>> awesome_data_structure = [{"uhh...": int, "wat!?!": [lambda: None]}, 0xFEEEEEEED]
>>> baleful_data_structure = [0xBABEEEE, {"ohgod": [lambda: True], "NOOOO!": object}]
# Describe the internal structure of your great glory.
>>> infer_hint(awesome_data_structure)
list[dict[str, type[int] | list[collections.abc.Callable[[], object]]] | int] # <-- ok
>>> infer_hint(baleful_data_structure)
list[dict[str, type[int] | list[collections.abc.Callable[[], object]]] | int] # <-- i don't know. sure, i guess?
# Do these two objects have a similar internal structure?
>>> is_bearable(baleful_data_structure, infer_hint(awesome_data_structure))
True # <----- WTF-F-F-F-F-Structural similarity: when you care about what your objects care about.
so tired. so very, very tired.
One Ping to Rule Them All and in the GitHub Announcement Summon Them
@pablovela5620, @thiswillbeyourgithub, @WeepingClown13, @JWCS, @Logan-Pageler, @knyazer, @Moosems, @frrad, @minmax, @jaanli, @jonnyhyman, @f-fuchs, @jennydaman, @denballakh, @bionicles