Work in progress to improve Python 3 support

Introduction

This document lists issues we detected related to compatibility between Iron Python 2.7 and CPython 3.8 (via Python.NET) engines. Next to each issue we provide a status of it:

• Fixed: The issue has been fixed. Either a release version is specified or it can be tested with our latest daily build.
• Planned: The issue is either being worked on or is planned for a future release.
• Not planned: The issue has been noted but has not been considered to be worked on.

We ask the community to provide feedback about these or other issues we might have overlooked.

Migrations from version 2 to 3 issues are purposely left out from this document, as they will be handled by a Migration Assistant which is currently in development. Documentation about it will be added later.

Iron Python and Python.NET

(If you are already aware of what these libraries are, you may skip this section, but if not, read on.)

Dynamo has supported Python execution through the Iron Python runtime for a long time, even though the Engine selector surfaced that only recently. As you may have heard, Iron Python currently supports Python 2, but it doesn't support Python 3. Something else you should know is that Iron Python does not use CPython, which is the standard Python runtime, but instead implements its own runtime over the .NET Framework. If you are not familiar with .NET, let's just say it is a framework to develop applications. It's pretty popular, even Dynamo and other tools you love are built on top of it 😄

The CPython engine, on the other hand, uses the standard Python runtime, also called CPython. Since Dynamo and its libraries are based on .NET, you need somethings that acts like a "glue" to make execution and data flow correctly between the two. This is achieved by using a library called Python.NET.

So if you can write Python and it will run in any case, you may ask yourself why we bring .NET to the discussion. Basically there are three reasons:

1. Some of the issues mentioned here deal with incompatibilities around .NET integration when comparing the two engines
2. It helps to understand what's going on behind the scenes: in one case everything runs in .NET, while in the other execution and data flows between CPython and .NET.
3. Most of your favourite applications here at Autodesk use the .NET framework, so if you want your Python node to be able to do things in these environments (such as talk to the Revit API) then your Python code needs a way to talk to them!

Issues

Python primitives are not .NET objects (Not planned)

Iron Python supports calling .NET functions on Python objects like this:

import clr

x = 1
OUT = x.ToString()

That works because for Iron Python x is actually a variable in .NET, which happens to have a type that implements ToString as expected.

The previous code does not work for native Python, because the Python type int does not have a ToString function. However, in Python you could achieve the same result by doing this:

x = 1
OUT = str(x)

Certain types cannot be returned from a Python node (Fixed in 2.8)

An easy way to reproduce this issue would be to write the following code in a Python node, with the CPython3 engine:

s = { 'hello', 'world' }
OUT = s

The previous code tried to return a Python set from a Python node. This type is not currently supported, and will also lead to a crash due to a bug. Support for Python types is being improved and the crash bug will be fixed soon.

TypeError : No method matches given arguments (Fixed in 2.8)

A TypeError is an error that might be seen from a .NET function call that receives Python types as arguments. One example that is reproducible in Dynamo 2.7 with the addition of CPython3 is:

import clr
clr.AddReference('ProtoGeometry')
from Autodesk.DesignScript.Geometry import *

point = Point.ByCoordinates(0,0,0)
OUT = point

In this case, the TypeError issue was caused by an inability of Python.NET to determine that an overload of Point.ByCoordinates should have been called. The overload Point.ByCoordinates(double x, double y, double z) could have been called by converting the int inputs with values of 0 to float inputs with values of 0.0. However, Python.NET did not attempt this and failed with the TypeError: No method matches given arguments message.

This specific problem for int to float conversions was fixed. Unfortunately, the message is quite generic and might describe other failed conversions or scenarios where the method was not found due to being misspelled, for instance. If you run into this type of error, please reach out to us with detailed information about the issue.

No line number on Python errors (Fixed in 2.8)

Most often than not, the CPython3 engine would omit the line number where an error occurred, making it harder to debug long Python scripts. Errors of type TypeError would never provide a line number, while others of type SyntaxError would.

This was a rather inconvenient omission in Python.NET that was fixed.

Python with blocks don't call Dispose for wrapped IDisposable .NET objects (Not planned)

The following example shows what this issue is about:

with ClassThatImplementsIDisposable() as something:
  pass

Iron Python would call Dispose() automatically for something when exiting the with block. This is not the case for Python.NET, so developers should be aware and call Dispose() explicitly if required.

Very big integers are not supported (Fixed in 2.8)

Python supports through its int type integer values of any size. However, taking that value from Python to .NET requires special considerations, because the target type needs to be able to support very large values as well. Iron Python convert to BigInteger automatically, so the following code example would work:

# That is bigger than what would fit in an Int64
x = 11111111111111111111
OUT = x

Python.NET did not support converting large integers to BigInteger out of the box, so additional work was required to make it work.

Custom Python objects cannot transcend a Python script node (Fixed in 2.8)

Iron Python allowed to return objects of custom classes defined in the same Python script node. An example of this would be the following:

class Dog:
  kind = 'canine'
  
  def __init__(self, name):
    self.name = name

OUT = Dog('Rex')

Even though the output from that node would not be of much use in Dynamo, it could be reused in another Python node, allowing certain degree of code reuse between nodes.

Conversion differences between engines for custom objects that are iterable (Fixed in 2.8)

Related to the previous issue, we have found examples of objects that are subject to conversion in the CPython3 engine, while they would be passed unconverted using the IronPython2 engine. To better illustrate this, here is an example of such a class:

class iterable:
    def __str__(self):
        return 'I want to participate in conversion'
    def __iter__(self):
        return iter([0,1,2,3])
    def __getitem__(self,key):
        return key

The IronPython2 engine would return this object as an IronPython.Runtime type, which can be passed around between Python nodes. On the other hand, for the CPython3 engine, this would be recognized as a collection and converted to a List. This is because the class implement __iter__ and __getitem__, which is something only the CPython3 engine checks for. This check is required to support a wide range of built-in collections, so simply removing it was not an option we cared for.

But what if the author of this class wants the old behaviour offered by the IronPython2 engine and intends to move this object between Python nodes? This is still possible using a Dynamo specific attribute we have introduced: __dynamoskipconversion__

class iterable:
    def __dynamoskipconversion__(self):
        pass
    def __str__(self):
        return 'I don't want to participate in conversion'
    def __iter__(self):
        return iter([0,1,2,3])
    def __getitem__(self,key):
        return key

By including this attribute, even with an empty implementation, Dynamo will return this object as-is, instead of attempting any conversion.