Python 2.7 to Python 3 is Trouble for Data Scientists but Not Coders! Why?

Coders have a field day with Python transformation but it is not a cup of tea for data scientists.

Even if you just started thinking about migrating to Python 3, there is one policy you should introduce into your code development right away: every new bit of code committed to your repository needs to be Python 3, at least in theory. It's a "best effort" type of deal here. If your product is under active development, following that principle alone will make the actual migration much smoother.

Data Scientists vs Coders

Now, if you are a coder, all these changes are no trouble at all. But if you are a data scientist, things might not be as easy as it seems when it comes to transferring from Python 2.7 to Python 3. The real problem lies in the process of refactoring. Since the shift from Python 2.7 to Python 3 brings in a lot of new features and improvements, anything written for Python 2.7 is required to be refactored to welcome the new changes. The process of refactoring is actually the way programmers can adjust the code base to respond to environmental changes, such as a change in the language version, or just to improve existing code in some form. Without refactoring, a shift from Python 2.7 to Python 3.0 often means the code for Python 2.7 just doesn't work that well anymore, or even at all. However, it is a difficult task for data scientists as there is a high chance of risk that performance will degrade and that bugs will creep in – sometimes only visible when an edge case appears. Small bugs become a major concern when Python code is used for critical, 24/7 purposes such as scientific analysis. Their inexperience in internal coding can also lead to unexpected performance degradation. Even if it's just a 5% performance hit, a poorly executed code update can quickly create much bigger bills on expensive pay-for-use HPC platforms.

Sticking to The Older Version is Not a Good Idea

If you think about the hard work and risks involved in adjusting code, it's no surprise that users often choose to just stick to older versions of Python. Running existing code on an outdated version of Python avoids quite a lot of challenges because you don't need to refactor: you're keeping your code just the way it was. It doesn't sound like a big problem, but relying on outdated, unsupported building blocks for your computing is a DevSecOps nightmare. New vulnerabilities will appear, and the needed patches just won't come. Relying on old versions of programming languages, therefore, introduces huge risks to your computing environment.

The responsible thing to do is to update the Python version when needed and to edit the code running on it but there just isn't a painless way to do it. Realistically, due to a lack of resources, refactoring often doesn't get done, with potentially costly consequences.