PyPy (Posts about sponsors)

PyPy for low-latency systems

Antonio Cuni — Thu, 03 Jan 2019 14:21:00 GMT

PyPy for low-latency systems

Recently I have merged the gc-disable branch, introducing a couple of features which are useful when you need to respond to certain events with the lowest possible latency. This work has been kindly sponsored by Gambit Research (which, by the way, is a very cool and geeky place where to work, in case you are interested). Note also that this is a very specialized use case, so these features might not be useful for the average PyPy user, unless you have the same problems as described here.

The PyPy VM manages memory using a generational, moving Garbage Collector. Periodically, the GC scans the whole heap to find unreachable objects and frees the corresponding memory. Although at a first look this strategy might sound expensive, in practice the total cost of memory management is far less than e.g. on CPython, which is based on reference counting. While maybe counter-intuitive, the main advantage of a non-refcount strategy is that allocation is very fast (especially compared to malloc-based allocators), and deallocation of objects which die young is basically for free. More information about the PyPy GC is available here.

As we said, the total cost of memory managment is less on PyPy than on CPython, and it's one of the reasons why PyPy is so fast. However, one big disadvantage is that while on CPython the cost of memory management is spread all over the execution of the program, on PyPy it is concentrated into GC runs, causing observable pauses which interrupt the execution of the user program.
To avoid excessively long pauses, the PyPy GC has been using an incremental strategy since 2013. The GC runs as a series of "steps", letting the user program to progress between each step.

The following chart shows the behavior of a real-world, long-running process:

The orange line shows the total memory used by the program, which increases linearly while the program progresses. Every ~5 minutes, the GC kicks in and the memory usage drops from ~5.2GB to ~2.8GB (this ratio is controlled by the PYPY_GC_MAJOR_COLLECT env variable).
The purple line shows aggregated data about the GC timing: the whole collection takes ~1400 individual steps over the course of ~1 minute: each point represent the maximum time a single step took during the past 10 seconds. Most steps take ~10-20 ms, although we see a horrible peak of ~100 ms towards the end. We have not investigated yet what it is caused by, but we suspect it is related to the deallocation of raw objects.

These multi-millesecond pauses are a problem for systems where it is important to respond to certain events with a latency which is both low and consistent. If the GC kicks in at the wrong time, it might causes unacceptable pauses during the collection cycle.

Let's look again at our real-world example. This is a system which continuously monitors an external stream; when a certain event occurs, we want to take an action. The following chart shows the maximum time it takes to complete one of such actions, aggregated every minute:

You can clearly see that the baseline response time is around ~20-30 ms. However, we can also see periodic spikes around ~50-100 ms, with peaks up to ~350-450 ms! After a bit of investigation, we concluded that most (although not all) of the spikes were caused by the GC kicking in at the wrong time.

The work I did in the gc-disable branch aims to fix this problem by introducing two new features to the gc module:

gc.disable(), which previously only inhibited the execution of finalizers without actually touching the GC, now disables the GC major collections. After a call to it, you will see the memory usage grow indefinitely.

gc.collect_step() is a new function which you can use to manually execute a single incremental GC collection step.

It is worth to specify that gc.disable() disables only the major collections, while minor collections still runs. Moreover, thanks to the JIT's virtuals, many objects with a short and predictable lifetime are not allocated at all. The end result is that most objects with short lifetime are still collected as usual, so the impact of gc.disable() on memory growth is not as bad as it could sound.

Combining these two functions, it is possible to take control of the GC to make sure it runs only when it is acceptable to do so. For an example of usage, you can look at the implementation of a custom GC inside pypytools. The peculiarity is that it also defines a "with nogc():" context manager which you can use to mark performance-critical sections where the GC is not allowed to run.

The following chart compares the behavior of the default PyPy GC and the new custom GC, after a careful placing of nogc() sections:

The yellow line is the same as before, while the purple line shows the new system: almost all spikes have gone, and the baseline performance is about 10% better. There is still one spike towards the end, but after some investigation we concluded that it was not caused by the GC.

Note that this does not mean that the whole program became magically faster: we simply moved the GC pauses in some other place which is not shown in the graph: in this specific use case this technique was useful because it allowed us to shift the GC work in places where pauses are more acceptable.

All in all, a pretty big success, I think. These functionalities are already available in the nightly builds of PyPy, and will be included in the next release: take this as a New Year present :)

Antonio Cuni and the PyPy team

PyPy v5.8 released

mattip — Thu, 08 Jun 2017 23:20:00 GMT

The PyPy team is proud to release both PyPy2.7 v5.8 (an interpreter supporting Python 2.7 syntax), and a beta-quality PyPy3.5 v5.8 (an interpreter for Python 3.5 syntax). The two releases are both based on much the same codebase, thus the dual release. Note that PyPy3.5 supports Linux 64bit only for now.

This new PyPy2.7 release includes the upstream stdlib version 2.7.13, and PyPy3.5 includes the upstream stdlib version 3.5.3.

We fixed critical bugs in the shadowstack rootfinder garbage collector strategy that crashed multithreaded programs and very rarely showed up even in single threaded programs.

We added native PyPy support to profile frames in the vmprof statistical profiler.

The struct module functions pack* and unpack* are now much faster, especially on raw buffers and bytearrays. Microbenchmarks show a 2x to 10x speedup. Thanks to Gambit Research for sponsoring this work.

This release adds (but disables by default) link-time optimization and profile guided optimization of the base interpreter, which may make unjitted code run faster. To use these, translate with appropriate options. Be aware of issues with gcc toolchains, though.

Please let us know if your use case is slow, we have ideas how to make things faster but need real-world examples (not micro-benchmarks) of problematic code.

Work sponsored by a Mozilla grant continues on PyPy3.5; numerous fixes from CPython were ported to PyPy and PEP 489 was fully implemented. Of course the bug fixes and performance enhancements mentioned above are part of both PyPy 2.7 and PyPy 3.5.

CFFI, which is part of the PyPy release, has been updated to an unreleased 1.10.1, improving an already great package for interfacing with C.

Anyone using NumPy 1.13.0, must upgrade PyPy to this release since we implemented some previously missing C-API functionality. Many other c-extension modules now work with PyPy, let us know if yours does not.

As always, this release fixed many issues and bugs raised by the growing community of PyPy users. We strongly recommend updating.

You can download the v5.8 release here:

https://pypy.org/download.html

We would like to thank our donors and contributors, and encourage new people to join the project. PyPy has many layers and we need help with all of them: PyPy and RPython documentation improvements, tweaking popular modules to run on PyPy, or general help with making RPython’s JIT even better.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7 and CPython 3.5. It’s fast (PyPy and CPython 2.7.x performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
The PyPy 2.7 release supports:

x86 machines on most common operating systems (Linux 32/64 bits, Mac OS X 64 bits, Windows 32 bits, OpenBSD, FreeBSD)

newer ARM hardware (ARMv6 or ARMv7, with VFPv3) running Linux,

big- and little-endian variants of PPC64 running Linux,

s390x running Linux

What else is new?

PyPy 5.7 was released in March, 2017.

There are many incremental improvements to RPython and PyPy, the complete listing is here.

Please update, and continue to help us make PyPy better.

Cheers, The PyPy team

PyPy gets funding from Mozilla for Python 3.5 support

Armin Rigo — Tue, 09 Aug 2016 16:38:00 GMT

"Python 2.x versus Python 3.x": this is by now an old question. In the eyes of some people Python 2 is here to stay, and in the eyes of others Python has long been 3 only.

PyPy's own position is that PyPy will support Python 2.7 forever---the RPython language in which PyPy is written is a subset of 2.7, and we have no plan to upgrade that. But at the same time, we want to support 3.x. This is particularly true now: a relatively recent development is that Python 3.5 seems to attract more and more people. The "switch" to Python 3.x might be starting to happen.

Correspondingly, PyPy has been searching for a while for a way to support a larger-scale development effort. The goal is to support not just any old version of Python 3.x, but Python 3.5, as this seems to be the version that people are switching to. PyPy is close to supporting all of Python 3.3 now; but the list of what is new in Python 3.4 and 3.5 is far, far longer than anyone imagines. The long-term goal is also to get a version of "PyPy3" that is as good as "PyPy2" is, including its performance and its cpyext layer (CPython C API interoperability), for example.

So, the end result: Mozilla recently decided to award $200,000 to Baroque Software to work on PyPy as part of its Mozilla Open Source Support (MOSS) initiative. This money will be used to implement the Python 3.5 features in PyPy. Within the next year, we plan to use the money to pay four core PyPy developers half-time to work on the missing features and on some of the big performance and cpyext issues. This should speed up the progress of catching up with Python 3.x significantly. We are extremely thankful to Mozilla for supporting us in this way, and will keep you updated on the progress via this blog.

Couchbase contribution to PyPy

Maciej Fijalkowski — Tue, 14 Oct 2014 17:40:00 GMT

Hello everyone!

We always offer to put on the blog info about our sponsors who donate substantial amounts of money. So far most people decided to stay anonymous, so this is the first blog post describing our sponsor and his relationship to PyPy, hopefully not the last. We'll also publish a full blog post about the PSF-matched fundraiser soon. This is a guest post by Brent Woodruff from Couchbase.

Couchbase is a leading NoSQL document database that provides a flexible data model, high performance, scalability, and high availability. Couchbase is a commercially supported open source project. Visit us at https://www.couchbase.com and https://github.com/couchbase.

Couchbase Inc. donated $2000.00, and employees of Couchbase personally contributed a disclosed additional $230.00, towards Pypy progress during the September funding drive. These funds will see a match from the Python Software Foundation.

Pypy is primarily used by Couchbase employees to perform product analysis and troubleshooting using internally developed tools. Every customer of Couchbase benefits from the use of Pypy; both due to the rapid development provided by Python, and the speed of the resulting tools provided by the Pypy JIT interpreter.

“PyPy is great - it gave us a 4x speedup in our CPU-intensive internal application over CPython” -Dave Rigby and Daniel Owen, Couchbase Engineers

Additionally, Couchbase has a preliminary CFFI based Couchbase client available for Pypy users.

pygame_cffi: pygame on PyPy

Maciej Fijalkowski — Wed, 26 Mar 2014 16:28:00 GMT

The Raspberry Pi aims to be a low-cost educational tool that anyone can use to learn about electronics and programming. Python and pygame are included in the Pi's programming toolkit. And since last year, thanks in part to sponsorship from the Raspberry Pi Foundation, PyPy also works on the Pi (read more here).

With PyPy working on the Pi, game logic written in Python stands to gain an awesome performance boost. However, the original pygame is a Python C extension. This means it performs poorly on PyPy and negates any speedup in the Python parts of the game code.

One solution to making pygame games run faster on PyPy, and eventually on the Raspberry Pi, comes in the form of pygame_cffi. pygame_cffi uses CFFI to wrap the underlying SDL library instead of a C extension. A few months ago, the Raspberry Pi Foundation sponsored a Cape Town Python User Group hackathon to build a proof-of-concept pygame using CFFI. This hackathon was a success and it produced an early working version of pygame_cffi.

So for the last 5 weeks Raspberry Pi has been funding work on pygame_cffi. The goal was a complete implementation of the core modules. We also wanted benchmarks to illuminate performance differences between pygame_cffi on PyPy and pygame on CPython. We are happy to report that those goals were met. So without further ado, here's a rundown of what works.

Current functionality

Surfaces support all the usual flags for SDL and OpenGL rendering (more about OpenGL below).
The graphics-related modules color, display, font and image, and parts of draw and transform are mostly complete.
Events! No fastevent module yet, though.
Mouse and keyboard functionality, as provided by the mouse and key modules, is complete.
Sound functionality, as provided by the mixer and music modules, is complete.
Miscellaneous modules, cursors, rect, sprite and time are also complete.

Invention screenshot:

Mutable mamba screenshot:

With the above-mentioned functionality in place we could get 10+ of the pygame examples to work, and a number of PyWeek games. At the time of writing, if a game doesn't work it is most likely due to an unimplemented transform or draw function. That will be remedied soon.

Performance

In terms of performance, pygame_cffi on PyPy is showing a lot of promise. It beats pygame on CPython by a significant margin in our events processing and collision detection benchmarks, while blit and fill benchmarks perform similarly. The pygame examples we checked also perform better.

However, there is still work to be done to identify and eliminate bottlenecks. On the Raspberry Pi performance is markedly worse compared to pygame (barring collision detection). The PyWeek games we tested also performed slightly worse. Fortunately there is room for improvement in various places.

Invention & Mutable Mamba (x86)

Standard pygame examples (Raspberry Pi)

Here's a summary of some of the benchmarks. Relative speed refers to the frame rate obtained in pygame_cffi on PyPy relative to pygame on CPython.

Benchmark	Relative speed (pypy speedup)
Events (x86)	1.41
Events (Pi)	0.58
N² collision detection on 100 sprites (x86)	4.14
N² collision detection on 100 sprites (Pi)	1.01
Blit 100 surfaces (x86)	1.06
Blit 100 surfaces (Pi)	0.60
Invention (x86)	0.95
Mutable Mamba (x86)	0.72
stars example (x86)	1.95
stars example (Pi)	0.84

OpenGL

Some not-so-great news is that PyOpenGL performs poorly on PyPy since PyOpenGL uses ctypes. This translates into a nasty reduction in frame rate for games that use OpenGL surfaces. It might be worthwhile creating a CFFI-powered version of PyOpenGL as well.

Where to now?

Work on pygame_cffi is ongoing. Here are some things that are in the pipeline:

Get pygame_cffi on PyPy to a place where it is consistently faster than pygame on CPython.
Implement the remaining modules and functions, starting with draw and transform.
Improve test coverage.
Reduce the time it takes for CFFI to parse the cdef. This makes the initial pygame import slow.

If you want to contribute you can find pygame_cffi on Github. Feel free to find us on #pypy on freenode or post issues on github.

Cheers,
Rizmari Versfeld

PyPy 2.0 alpha for ARM

Maciej Fijalkowski — Tue, 07 May 2013 13:35:00 GMT

Hello.

We're pleased to announce an alpha release of PyPy 2.0 for ARM. This is mostly a technology preview, as we know the JIT is not yet stable enough for the full release. However please try your stuff on ARM and report back.

This is the first release that supports a range of ARM devices - anything with ARMv6 (like the Raspberry Pi) or ARMv7 (like Beagleboard, Chromebook, Cubieboard, etc.) that supports VFPv3 should work. We provide builds with support for both ARM EABI variants: hard-float and some older operating systems soft-float.

This release comes with a list of limitations, consider it alpha quality, not suitable for production:

stackless support is missing.
assembler produced is not always correct, but we successfully managed to run large parts of our extensive benchmark suite, so most stuff should work.

You can download the PyPy 2.0 alpha ARM release here (including a deb for raspbian):

https://pypy.org/download.html

Part of the work was sponsored by the Raspberry Pi foundation.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7.3. It's fast due to its integrated tracing JIT compiler.

This release supports ARM machines running Linux 32bit. Both hard-float armhf and soft-float armel builds are provided. armhf builds are created using the Raspberry Pi custom cross-compilation toolchain based on gcc-arm-linux-gnueabihf and should work on ARMv6 and ARMv7 devices running at least debian or ubuntu. armel builds are built using gcc-arm-linux-gnuebi toolchain provided by ubuntu and currently target ARMv7. If there is interest in other builds, such as gnueabi for ARMv6 or without requiring a VFP let us know in the comments or in IRC.

Benchmarks

Everybody loves benchmarks. Here is a table of our benchmark suite (for ARM we don't provide it yet on https://speed.pypy.org, unfortunately).

This is a comparison of Cortex A9 processor with 4M cache and Xeon W3580 with 8M of L3 cache. The set of benchmarks is a subset of what we run for https://speed.pypy.org that finishes in reasonable time. The ARM machine was provided by Calxeda. Columns are respectively:

benchmark name
PyPy speedup over CPython on ARM (Cortex A9)
PyPy speedup over CPython on x86 (Xeon)
speedup on Xeon vs Cortex A9, as measured on CPython
speedup on Xeon vs Cortex A9, as measured on PyPy
relative speedup (how much bigger the x86 speedup is over ARM speedup)

Benchmark	PyPy vs CPython (arm)	PyPy vs CPython (x86)	x86 vs arm (pypy)	x86 vs arm (cpython)	relative speedup
ai	3.61	3.16	7.70	8.82	0.87
bm_mako	3.41	2.11	8.56	13.82	0.62
chaos	21.82	17.80	6.93	8.50	0.82
crypto_pyaes	22.53	19.48	6.53	7.56	0.86
django	13.43	11.16	7.90	9.51	0.83
eparse	1.43	1.17	6.61	8.12	0.81
fannkuch	6.22	5.36	6.18	7.16	0.86
float	5.22	6.00	9.68	8.43	1.15
go	4.72	3.34	5.91	8.37	0.71
hexiom2	8.70	7.00	7.69	9.56	0.80
html5lib	2.35	2.13	6.59	7.26	0.91
json_bench	1.12	0.93	7.19	8.68	0.83
meteor-contest	2.13	1.68	5.95	7.54	0.79
nbody_modified	8.19	7.78	6.08	6.40	0.95
pidigits	1.27	0.95	14.67	19.66	0.75
pyflate-fast	3.30	3.57	10.64	9.84	1.08
raytrace-simple	46.41	29.00	5.14	8.23	0.62
richards	31.48	28.51	6.95	7.68	0.91
slowspitfire	1.28	1.14	5.91	6.61	0.89
spambayes	1.93	1.27	4.15	6.30	0.66
sphinx	1.01	1.05	7.76	7.45	1.04
spitfire	1.55	1.58	5.62	5.49	1.02
spitfire_cstringio	9.61	5.74	5.43	9.09	0.60
sympy_expand	1.42	0.97	3.86	5.66	0.68
sympy_integrate	1.60	0.95	4.24	7.12	0.60
sympy_str	0.72	0.48	3.68	5.56	0.66
sympy_sum	1.99	1.19	3.83	6.38	0.60
telco	14.28	9.36	3.94	6.02	0.66
twisted_iteration	11.60	7.33	6.04	9.55	0.63
twisted_names	3.68	2.83	5.01	6.50	0.77
twisted_pb	4.94	3.02	5.10	8.34	0.61

It seems that Cortex A9, while significantly slower than Xeon, has higher slowdowns with a large interpreter (CPython) than a JIT compiler (PyPy). This comes as a surprise to me, especially that our ARM assembler is not nearly as polished as our x86 assembler. As for the causes, various people mentioned branch predictor, but I would not like to speculate without actually knowing.

How to use PyPy?

We suggest using PyPy from a virtualenv. Once you have a virtualenv installed, you can follow instructions from pypy documentation on how to proceed. This document also covers other installation schemes.

We would not recommend using in production PyPy on ARM just quite yet, however the day of a stable PyPy ARM release is not far off.

Cheers,
fijal, bivab, arigo and the whole PyPy team

A thank you to the PSF

Maciej Fijalkowski — Mon, 21 Mar 2011 23:50:00 GMT

This year's PyCon was an incredible time; several members of the PyPy team were there, and we'll be blogging more about our experiences in the coming days. However, we quickly wanted to extend a thank you to the Python Software Foundation (PSF).

As you may have heard, on Friday morning at PyCon Jesse Noller handed the PyPy team a check for $10,000, on behalf of the PSF. This was in recognition of our success over the past few years in bringing PyPy from a research project to a fast, compliant, production-ready Python implementation, and to allow us to continue our work on making it faster and more up-to-date with upstream version changes.

Beyond the large check, we're grateful for the endorsement this represents, not only of our work on PyPy, but also of all alternatve Python VMs. The PSF has shifted its focus from representing just CPython to representing the Python Language, reguardless of its implementation, something we are very appreciative of.

From left to right, PyPy people present at PyCon 2011: Maciej Fijałkowski, Armin Rigo, Alex Gaynor, Laura Creighton and Jacob Hallén

Thank you, PSF.