Plone finally supports Python 3!
Our favourite Open Source CMS has landed on Python 3. Here is what you need to know about migrating projects.
At the Plone Conference 2018 in Tokyo I gave a plenary talk in which I told the story how Plone got to support Python 3. Ten minutes before my talk Plone 5.2a1 was released. That release was the culmination of a 3-year long journey (or even 5 years if you start in Brazil). If you want to learn how we got there you should watch the talk.
Where are we now?
Plone 5.2 supports Python 3.7, 3.6 and 2.7. It is currently in alpha but the plan is to release a final version in February 2019 right after the Alpine City Sprint. You should seriously consider joining that sprint if you plan to migrate your own projects. Working together with the people who ported Plone to Python 3 will give you the know-how to succeed in your own migrations!
But in fact you should not wait until February to start working with Plone on Python 3 but start right now so you'll have enough time to migrate to Python 3 before January 1st 2020 (End of Life of Python 2).
To make testing and working with Python 3 easier some tools and addons were already working on Python 3.
The following essential development-tools are already ported:
Also the three addons that achieved the highest ranking during the latest poll at Ploneconf now work on Python 3:
You can also run the new frontend Volto with Plone on Python 3.
Open tasks
Before the final release of Plone 5.2 scheduled for February 2019 there are still a lot of things to do.
Here are the six most pressing issues:
- The upgrade- and porting-guide need to be completed.
- ZODB migration needs to be finished and documented.
- Much more addons need to be ported and released.
- We require performance tests.
- People who really need a replacement for FTP and/or WebDAV need to figure out how to do that.
There is still a lot to do. You can help Plone by coming to a sprint, testing it and fixing or reporting a bug.
Migrating to Python 3
From 1.1.2020 on you'll have to inform your clients that you are running their super-secure system on an unsupported version of Python. That might be ok if you're running a small website for a friend but for serious projects this simply will not do. Instead you should start to adopt Python 3 in your Plone projects now. You need to start early and plan ahead!
Here are the six steps you have to follow to upgrade to Python 3. Do not attempt to do all steps at once, instead you should work iteratively and even deploy to production in between steps.
1. Upgrade to Plone 5.2
You should still use Python 2.7 for this step. The changes between Plone 5.1 and Plone 5.2 are not huge but you will gain some great features including a new and much faster dropdown-navigation and a bootstrap-based ZMI.
The Upgrade-Guide for 5.2 still needs much more love. So if you encounter any issues please update that document and/or create a ticket in github.
2. Drop Archetypes and only use Dexterity
Archetypes is still supported on Plone 5.2 but only when running with Python 2.7. It is now officially deprecated and is not ported to Python 3. There has been a built-in migration for default types from Archetypes to Dexterity since 2013, and since 2015 there have been helpers and even a form that allows you to migrate your custom content.
3. Migrate your code to Python 3
Follow the documentation in to migrate your custom code and any addons that you need in your packages.
- Make sure your code works in Python 2 and Python 3. If you are certain that you will not publish that code as a addon or reuse it in another project you can also drop support for Python 2. That will make porting and testing easier.
- For most addons: Use python-modernize as described in the documentation.
- For small addons you could even simply try to startup on Python 3 and and fix whatever fails (e.g. relative imports or invalid syntax)
- Manually fix whatever python-modernize misses (e.g. relative imports)
- Do the same with tests. You'll need a test-setup with tox and a test-matrix in travis that tests different Python versions. The setup in collective.ifttt is almost done and might be a good example.
- If you have complex doctests consider migrating them to python tests since these are easier to get to pass in both Python 2 and 3. If you do want to keep them as doctests, change them so that the output for Python 3 is the default and use a Py23DocChecker to make the tests pass in Python 2 as well. See the example in plone.indexer.
- Once all seems to work you need to test every single feature manually because even a high test coverage is never a guarantee that your code actually does what it should.
4. Migrate Addons and Dependencies
Now migrate any addons you use to Python 3 as explained above. The main difference is that now everyone who uses this addon benefits. You shoud not drop support for Python 2 for those!
I listed the addons that already run on Python 3 above. I'm sure that list will be outdated very soon. Before you start migrating an addon you need to check if there already is a branch that supports Python 3 and check if there is a ticket like the one for collective.easyform. If not create such a ticket and start the process.
5. Migrate your Database
ZODB itself is compatible with Python 3 but a database that was created in Python 2.7 cannot be used in Python 3 without being modified first. I will not go into details about that. You should read the documentation and see David Glick's talk.
The migration will probably require some downtime and in rare cases you might have to write your own mappings to tell zodbupdate how to handle your data.
6. Deploy on Python 3
Plone 5.2 will use WSGI with waitress as the default WSGI server. Alternatively you can use uwsgi or gunicorn. ZServer can only be used with Python 2.7.
Some deployment tools may not ported yet, others like ZRS or RelStorage already support Python 3. You should test your production setup early. There may be changes that you don't want to struggle with a week before a scheduled launch.
Summing up
Start that whole process as early as possible because you may need to deal with unexpected problems and addons that you did not know still used Archetypes. 2019 might be much busier for you than you expect.
Do not wait for a final release of Plone 5.2! Start the process of planning and testing a migration. Consinder hiring experts to help you if necessary.
Again: You can and you should work iteratively. That means you do not have to do everything in one step and one night:
- Upgrade to 5.2 with Dexterity on Python 2 and deploy that early in 2019.
- Then take some time to migrate and test your code and addons.
- You can then deploy the Python 3-compatible code while still running Python 2.7.
- Then you can migrate your existing database and deploy on Python 3.
FAQ
Here are some questions you might have:
What about Python 4?
That will not be a big issue since 4.0 will be a upgrade like from 3.7 to 3.8. The Python community simply don't want to release a Python 3.10.
When will Plone drop support for Python 2?
Not before we have a good reason. Maybe Plone 7? It would allow us to remove and clean up some code but we would not gain a lot.
What about Zope 5?
There are no real plans for that yet.
Will Archetypes be migrated to Python 3?
Please don't try to postpone the inevitable. We are too small a community to maintain two Python versions, two frontends and two content type frameworks.
How do we replace FTP/WebDAV?
I don't know, for WebDAV you might be able to use a middleware like WsgiDAV.
Why did you make all that effort and not 'simply' migrate to Guillotina?
Guillotina does not aim to be a replacement for Plone. Also it has no equivalent addon ecosystem, feature equality or migration path.
Can I now use async/await?
You can but it won't really help you since Plone and Zope are still fundamentally synchronous.
Why doesn't the Plone Foundation simply buy support for Python 2.7 from RedHat/IBM for all of us?
Are you kidding me?
Does Python 3 make Plone run faster?
Maybe. We still lack performance tests but my guess is that Plone 5.2 will run faster on Python 3 than on 2.
Can I run the same Database in Python 2 and 3?
No.You can neither use a DB create with py2 in py3 nor use one created with py3 in py2.
Is addon xyz already ported?
Most likely not.
Plone supports the latest two major version. Does that extend to Python?
Plone's security update policy supports the latest two major versions (currently, that means the 5.x series and the 4.3.x series) with security updates. That guarantee does not extend to the programming language, the operating systen or the hardware you are running Plone on.
Will this work on Windows?
Very likely. Someone will have to test it.
I need help with migrating our projects to Python 3. Who can help me?
Talk to us at Starzel.de or ask other professionals with experience.
Warm words
Migrating Plone to Python 3 was quite a ride and a great community effort. I had to cope with some brain-melting changes in Zope, fight heartbreaking test-isolation issues and toil through an endless list of packages and tests. But I had the immense privilege and pleasure to do all of that together with some of the smartest and nicest people I know. Thanks to all of you!
Watch the full talk if you want to learn more about how Plone finally landed on Python 3.
Here are all videos from Ploneconf 2018: Day 1, Day 2, Day 3.