Advanced Python Features

Especially the SDF.data_model submodule makes use of some Python features that some Python programmers might not know.

Abstract Classes and Methods

Abstract classes inherit from abc.ABC (abstract base class) and can only be initialized (self.__init__(...)) if there are no abstract methods (decorated by @abstractmethod). Abstract methods therefore are methods that all classes inheriting from the abstract class need to implement themselves.

Attributes sadly cannot be abstract, so for the case of required attributes, the usage of properties is required.

>>> from abc import ABC, abstractmethod

>>> class Animal(ABC):
...     @property
...     @abstractmethod
...     def sound(self):
...         pass
...     def make_sound(self):
...         print(self.sound)

>>> class Cat(Animal):
...     @property
...     def sound(self):
...         return "meow"

>>> a = Animal()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class Animal with abstract methods sound
>>> b = Cat()
>>> b.make_sound()
meow

Comparison to other programming languages: Classes with only abstract methods are often called “Interfaces”.

Private Methods and Attributes

Class methods and attributes with names starting with two underscores (like __a) are only accessible inside the class definition. This can be used to hide implementation details from the user.

Private attributes __ATTR are internally renamed to _CLASSNAME__ATTR. This should not be used.

>>> class A:
...     def __init__(self):
...         self.__a = 1
...     def print_a(self):
...         print(self.__a)

>>> a = A()
>>> a.print_a()
1
>>> print(a.__a)  # error, there is no attribute __a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute '__a'
>>> print(a._A__a)  # don't use this
1

Comparison to other programming languages: Hiding implementation details from the user is often called “Encapsulation”.

Properties

Attributes in Python can usually be accessed and set in an unlimited way:

>>> class A:
...     def __init__(self):
...         self.a = 1

>>> a = A()
>>> print(a.a)
1
>>> a.a = 5
>>> print(a.a)
5

By using properties, access and modification can be redirected through functions:

>>> class A:
...     def __init__(self):
...         self.__a = 1
...     @property
...     def a(self):  # getter
...         return self.__a
...     @a.setter
...     def a(self, new_a):
...         if not isinstance(new_a, int):
...             raise TypeError("Attribute a must be an integer")
...         self.__a = new_a  # setter

These functions can do whatever you want, so you can check if new_a satisfies some requirements, raise exceptions, change something else, and so on. There does not even need to be a private attribute __a.

Properties can be used to make objects behave more intuitively and ensure users don’t shoot themselves in the foot by misusing your classes.

Comparison to other programming languages: This behavior is often implemented explicitly with getter and setter methods and has a strong relation to encapsulation.

Type Annotations

Python is an implicitly typed language, so we can use a = 1 instead of int a = 1 like in C. It also is dynamically typed, so after a = 1, we can use a = 1.5. This is a curse and a blessing: We can write short and elegant code, but we cannot be sure our programs do what we expected, unless we execute every single line with every possible combination of values: range(a) raises an exception if a is not an integer.

Since Python 3.5, we can optionally annotate objects with their types, like a: int = 1. These annotations are not checked, so a: int = 1.5 is valid code and runs until the float value 1.5 causes problems. Static analysis tools like mypy or the ones integrated into some editors and IDEs can warn about such problems before the code is executed.

Annotating every single variable inside of a function does usually not add much value, since function code is often short and it should not be necessary to read it, unless something breaks. But annotation higher-level code like function signatures and class attributes can cause a huge improvement in developer experience.

def a_or_b(a: int, b: int, use_a: bool = True) -> int:
    # the function can do lots of complicated things, but from reading the signature we know that
    # `a_or_b(1, 2)` and `a_or_b(1, 2, False)` are valid, but `a_or_b('a', 'b', 'c')` can cause problems
    ...

Genetic types can also be annotated:

  • List[int] is a list of integers

  • Dict[int, str] is a dictionary with integer keys and string values

  • Iterable[float] is any iterable object (like a list, tuple or array) that contains floats

  • Callable[[int, int, bool], int] is a function that takes two integers and a boolean parameter and returns an integer

Types like List, Dict, Iterable, Callable must be imported from typing. Since Python 3.9, built-in container types like list and dict allow such parametrization (list[int] instead of List[int]) and this features is accessible since Python 3.7 via from __future__ import annotations.

The annotation of class names inside the class definition is only possible since Python 3.7 via from __future__ import annotations. Up to 3.6, strings have to be used instead:

>>> class A:
...     @staticmethod
...     def make_a1() -> A:  # NameError: name 'A' is not defined
...         return A()
...     @staticmethod
...     def make_a2() -> "A":  # works
...         return A()

Static and Class Methods

“Normal” methods use the class instance self as implicit first argument and thus have access to the state of the object. Class methods use the class object itself as implicit first argument. Static methods don’t have an implicit first argument at all.

Both static and class methods are often used as “factory methods”:

>>> class A:
...     @staticmethod
...     def make_a() -> "A":
...         return A()  # always returns an instance of A
...     @classmethod
...     def make_c(cls) -> "A":
...         return cls()  # if called on subclasses of A, returns subclass instance

>>> class B(A):
...     pass

>>> A.make_a()
<A object at ...>
>>> A.make_c()
<A object at ...>
>>> B.make_a()
<A object at ...>
>>> B.make_c()
<B object at ...>

Besides the usage as factory methods, static methods are also often used for helper functions that only make sense in the context of the class.

Comparison to Java: Static attributes (“members”) in Java are equivalent to class attributes in Python. A static method in Java is equivalent to a static method in Python, if it does not access class attributes. If a static method in Java accesses static members, it is equivalent to a class method in Python.