Just in time Python

I recently watched TJ DeVries give a fascinating talk on his just-in-time Lua program. Intrigued by the concept, I decided to build my own Python implementation: joshtyf/jit-python.

Bypassing the Agent Chat

Suppose you need a function to iterate through a list of numbers and print only the odd ones. The typical AI-assisted workflow involves opening an IDE's agent chat, typing a prompt, and waiting for code generation.

jit-python bypasses this context switch entirely by allowing you to define functionality simply by invoking a method that doesn't yet exist:

Note that print_odd_numbers_in_list is not a pre-defined function within the object class. Instead, it is generated entirely at runtime.

In essence, jit-python acts as a shortcut for the agentic workflows software developers use today. There is no need to switch contexts to an agent chat; you just define the functionality by its invocation. It isn't meant to replace standard AI workflows entirely, but rather to streamline them for specific use cases.

If the generated output isn't quite right, you can refine it using improve_method:

How it works under the hood

Because Python is a dynamically typed, interpreted language, it resolves method calls via attribute lookups. When you call a method, Python first checks if the function exists as an attribute of the class. This lookup relies on two key dunder methods:

1. __getattribute__: Called unconditionally to implement attribute access for instances of the class.
2. __getattr__: Called as a fallback when an attribute lookup fails to find the attribute in the usual places (i.e., it is neither an instance attribute nor found in the class tree).

When we call print_odd_numbers_in_list, the interpreter first invokes __getattribute__ and subsequently falls back to __getattr__. By default, since the attribute doesn't exist, an AttributeError is raised.

By overriding __getattr__, we can intercept this failure. Instead of raising an error, we generate the function on the fly, use setattr to bind the newly generated method to the instance, and return it. The Python interpreter then seamlessly invokes this returned method with the original arguments.

Here is the custom __getattr__ implementation that overwrites the default behavior:

Notice that _generate_method returns the method's code as a string. We then use Python's built-in exec function to dynamically execute and load this string into our namespace.

The _generate_method itself is simply an invocation of an AI agent, powered by Google's Agent Development Kit under the hood.

WARNING: Combining exec with AI-generated code introduces significant security vulnerabilities. In a standard agentic workflow, the developer reviews the code before execution. jit-python removes that safety net. Use this with extreme caution!

If a method name alone is insufficient to describe the desired logic, you can guide the generation process by passing a __description parameter:

Closing thoughts

While I always knew that Python's dynamic nature allowed for functions to be defined at runtime, building jit-python illuminated the finer nuances of the attribute resolution chain.

More broadly, this experiment offers a compelling perspective on AI integration. Rather than treating AI as an external chat assistant or building complex harnesses for simple tasks, embedding it directly into the runtime presents a powerful alternative. Native, just-in-time code generation could be the next logical step in removing developer friction.

The implications, however, extend far beyond developers. Imagine a future where laymen interact directly with AI-embedded devices, simply stating their needs. Instead of relying on rigid, pre-packaged applications, these devices could dynamically assemble custom features and workflows on the fly. It suggests a formidable paradigm shift: software will no longer be a static product we download, but a fluid utility that writes itself into existence exactly when—and exactly how—we need it.