Porting Plone to Python 3

&tldr; Plone is running nicely on Python 3 but now we have to fix over 500 failing tests.

Since I wrote the proposal to [Port Plone to Python 3](https://github.com/plone/Products.CMFPlone/issues/2368), so much has happened that a status update is needed. Let's step back a little: The first steps towards Python 3 were taken during the sprint at the Plone conference in Barcelona. The epic PLIP to [update to Zope 4](https://github.com/plone/Products.CMFPlone/issues/1351) was merged and we started porting individual packages to Python 3 without being able to run any tests. With the help of [sixer](https://six.readthedocs.io) and [python-modernize](https://python-modernize.readthedocs.io/en/latest/) we tried to get the most obvious import and syntax issues out of the way. That work was continued at the [Alpine City Sprint in Innsbruck](https://alpinecity.tirol/2018) and in the end, we treated more than 150 packages like this. Some of this work is [documented in the Plone tracker](https://github.com/plone/Products.CMFPlone/issues/2217). Along the way I wrote a [best-practice guide](https://github.com/plone/Products.CMFPlone/issues/2184#issuecomment-359445243) on how to port Plone packages this way. As the PLIP states, there are several approaches to porting Plone to Python 3: 1. Migrate and test packages that have no dependency to CMFPlone and test them in Python 2 and 3. 1. Prepare packages for Python 3 without being able to test them in Python 3. 1. Start up Plone on Python 3 and fix whatever breaks. When start-up works, create a Plone Site and again, fix whatever breaks. 1. Port plone.testing and plone.app.testing to Python 3 and start running tests. Fix what breaks during the setup of the layers. 1. Run the tests with Python 3 and fix all broken tests. At the sprint in Innbruck I started with #3 and kept going after the sprint until I was able to create an instance. At the [Plone Tagung in Berlin](https://openspacer.org/186-plone-tagung/190-plone-tagung-2018/) I was able to demo the creation of a site but nothing was rendered yet. After that, I kept going and I was finally able to create a site and manage content, which is what a CMS is about. It looked a bit raw but I was able to add and edit some content - yay!  Work continued at an unsteady pace, and with important contributions from Michael Howitz, Alessandro Pisa and David Glick, things started to get better. Even the theme and js started working. Slowly broken features became the exception, not the rule. Last week at the [Zope 4 Welcome Sprint in Halle](https://www.meetup.com/de-DE/Zope-Sprint/events/248187180/) we removed all feature blockers that we had found so far. Plone with Python 3 looks and feels like it used to in Python 2. During the sprint there was also a lot of progress on: * the wsgi setup * logging and tracebacks when using wsgi * porting plone.testing * a new theme for the ZMI * beta releases for many packages. There was also some progress on the difficult issue of database migrations. It seems like [zodbupdate](https://github.com/zopefoundation/zodbupdate#migration-to-python-3) is the best tool to do that but there is probably a lot of work ahead. Read more about the sprints outcome in the [blogpost by Michael Howitz](https://blog.gocept.com/2018/05/24/zope-is-welcome-in-the-python-3-wonderland/). There is a [Jenkins job](https://jenkins.plone.org/view/PLIPs/job/plip-py3/) and as of today, it is running (not passing) tests for all packages (except Archetypes) with Python 3. At the moment we run **6594 tests from 115 packages with 257 failures and 315 errors** - not bad when you keep in mind that we still focus on features, not on tests. Tests for a couple of packages are green, including plone.api which is great since plone.api uses most of our stack one way or another. Jenkins runs every three hours and [here](https://jenkins.plone.org/view/PLIPs/job/plip-py3/138/testReport/history/) you can see the progress over time. Here is a screenshot from today that shows Plone doing some pretty non-trivial things: Editing a recurring Event and also Richtext with Images and Links:  Next steps: * Create a demo site running on Python 3 that people can use to find broken features. This will happen at the [Plonator Sprint in Munich](https://pads.ccc.de/vco1AVDkgj). * Fix all tests that look like they fail because of a broken feature. * Fix all remaining tests to pass in Python 2 and Python 3. Since there are a gazillion doctests that will take some time. * Port plone.app.robotframework to python 3 and fix robottests. * Experiment with porting a ZODB with Plone to Python 3. If you want to participate you can simply look at the failing tests and start fixing them. You can also try to fix one of the open [issues](https://github.com/plone/Products.CMFPlone/projects/4). The setup of a coredev environment with Python 3 is really simple and [documented](https://github.com/plone/buildout.coredev/blob/5.2/py3.cfg).