Python 3.11 is Making a Gamble in Memory Over Speed! Is it Worth it?

Python 3.11 is the first release to benefit from a project called Faster CPython!

The Python programming language releases new versions yearly, with a feature-locked beta release in the first half of the year and the final release toward the end of the year. The feature set for Python 3.11 has just been finalized, with a beta version available for testing. Developers are encouraged to try out this latest version on non-production code, both to verify that it works with your programs and to get an idea of whether your code will benefit from its performance enhancements.

Let's look into the features in Python 3.11

Typing improvements

Python's type-hinting features make larger codebases easier to manage and analyze and have increased significantly with each revision since Python 3.5. Python 3.11 brings in several new type-hinting additions.

The Self type

Class methods that return self previously required obtuse and verbose annotations to be useful. typing. Self lets you annotate the return value of a class method as, simply, Self. You get useful and predictable results from your analysis tools for such methods.

CPython Optimizations

CPython is the reference implementation of the Python programming language. Written in C and Python, CPython is the default and most widely used implementation of the Python language. In version 3.11, the CPython interpreter is much more optimized and much faster than in version 3.10. CPython 3.11 is on average 1.22x faster than CPython 3.10 when measured with the performance benchmark suite, and compiled with GCC on Ubuntu Linux. Depending on your workload, the speedup could be up to 10–60% faster. In Python 3.11, the developers have mostly focused on faster startup and faster runtime as has been stated in the documentation.

Arbitrary string literal type

Previously, type annotations had no way to indicate a given variable needed to be a string literal—that is, a string defined in source code. The new typing.LiteralString annotation fixes that. Using the new annotation, linters can test for a variable is either a string defined in the source or a new string composed of only source-defined strings.

Python 3.11: I am Speed

Every new version comes with lots of improvements and the same goes with Python 3.11. One of the features (Speed) that every developer was waiting for is finally here. Since an object's type rarely changes, the interpreter now attempts to analyze running code and replace general bytecodes with type-specific ones. For instance, binary operations (add, subtract, etc.) can be replaced with specialized versions for integers, floats, and strings.

How Python 3.11 is gaining performance?

Python function calls also require less overhead in Python 3.11. Stack frames for function call now use less memory and are more efficiently designed. Also, while recursive calls aren't tail-optimized (which probably isn't possible in Python, anyway), they are more efficient than in previous versions. The Python interpreter itself also starts faster, and core modules needed for the Python runtime are stored and loaded more efficiently.

According to the official Python benchmark suite, Python 3.11 runs around 1.25 times faster than version 3.10. Note that this speedup is an aggregate measure: some things are much faster, but many others are only slightly faster or about the same. Still, the best part about these improvements is that they come for free. You don't need to make any code changes for Python programs to take advantage of 3.11's speedups.

Python 3.11 is the first release to benefit from a project called Faster CPython, where CPython is the standard version of the interpreter. Faster CPython is a project funded by Microsoft, whose members include Python inventor Guido van Rossum, Microsoft senior software engineer Eric Snow, and Mark Shannon – who is under contract to Microsoft as tech lead for the project.

A session scheduled for the EuroPython event to be held in Dublin in July centers on some of the changes that enable the speed-up. Shannon will describe the "adaptive specializing interpreter" in Python 3.11, which is PEP (Python Enhancement Proposal) 659. This describes a technique called specialization which, Shannon explains, "is typically done in the context of a JIT [just in time] compiler, but research shows specialization in an interpreter can boost performance significantly."

The interpreter identifies code that can benefit from specialization and "once an instruction in a code object has executed enough times, that instruction will be "specialized" by replacing it with a new instruction that is expected to execute faster for that operation," states the PEP. The speed-up can be "up to 50 percent."