Why don't format specifiers work with lists, dictionaries and other objects?

When you use the variable without any format specifier (print(f'{variable}')), internally its __str__ method is called. Considering your Test class, it already has this method:

class Test:
    def __init__(self, valor):
        self.valor = valor

    def __str__(self):
        print('calling __str__') # calling print just to show this method is called
        return f'Test({self.valor})'

t = Test(42)
print(f'{t}')

So the output is:

calling __str__
Test(42)

But what it's actually being called behind the scenes is the __format__ method.

According to the documentation, __format__ is called when the object is being evaluated inside a f-string, or when passed as an argument to str.format and the format built-in.

But my Test class didn't implement __format__, so it uses the method inherited from its superclass (object). And if we check its implementation in CPython source code - checked in 2020-08-11, comments removed):

static PyObject *
object___format___impl(PyObject *self, PyObject *format_spec)
{
    if (PyUnicode_GET_LENGTH(format_spec) > 0) {
        PyErr_Format(PyExc_TypeError,
                     "unsupported format string passed to %.200s.__format__",
                     Py_TYPE(self)->tp_name);
        return NULL;
    }
    return PyObject_Str(self);
}

This means that, when no format specifier is used (only f'{variable}'), it doesn't enter the if and returns PyObject_Str(self) (and according to the documentation, PyObject_Str(whatever) is equivalent to calling str(whatever) - which in turn calls whatever.__str__()).

But if I use some format specifier (such as f'{variable:>10}'), it enters the if and shows the error message ("unsupported format string etc").

The docs says that this behaviour of raising a TypeError when there's no format specifier was implemented in Python 3.4, and the call to str(self) was implemented in Python 3.7 (before that, it called format(str(self), ''), according to this commit).

Conclusion: if I want format specifiers to work with my class, I have to implement __format__:

class Test:
    def __init__(self, value):
        self.value = value

    def __str__(self):
        print('calling __str__')
        return f'Test({self.value})'

    def __format__(self, format_spec):
        print(f'calling __format__ with: "{format_spec}"')
        return f'{self.value:{format_spec}}'

t = Test(42)
print(f'{t}')
print(f'{t:>10}')

Now both print statements work (and I implemented __format__ in a way that it doesn't delegate to __str__, but I could've done it). The output is:

calling __format__ with: ""
42
calling __format__ with: ">10"
        42

Just reminding that I'll get the same output above using str.format:

print('{}'.format(t))
print('{:>10}'.format(t))

That's why format specifiers don't work with lists and dictionaries, because list e dict don't override __format__ and use the inherited implementation from object (which delegates to str when there are no format specifiers, and raises an error when there is - and that's why it works when I do just print(f'{some_list}')).

Unfortunately we can't implement __format__ in lists and dictionaries, because we can't add new methods in native classes, so the only solution seems to be to convert them to strings (using str, or iterating through its elements and manually building a string).

On the other hand, numbers and strings implement __format__ and work fine with format specifiers.

And, as already said, this works not only with f-strings, but also with str.format and format built-in:

t = Test(42) # using the last version above, with the implementation of __format__
# both lines below call Test.__format__
print('{:>10}'.format(t))
print(format(t, '>10'))

mylist = [1, 2]
# both lines below raise TypeError: unsupported format string passed to list.__format__
print('{:>10}'.format(mylist))
print(format(mylist, '>10'))

By the way, that's why it's possible to format datetime classes this way - such as f'{datetime.now():%d/%m/%Y}' - because those classes implement __format__, delegating to strftime.

Which means that I could also do something similar with my Test class, defining any custom format specifier I want:

class Test:
    # ... constructor, etc

    def __format__(self, format_spec):
        formats = { # custom formats (not the best examples)
          'whatever' : f'format whatever -> {self.value}',
          'another format' : f'another: {self.value}'
        }
        if format_spec in formats:
            return formats[format_spec]

        return f'{self.value:{format_spec}}'

t = Test(42)
print(f'{t:whatever}') # format whatever -> 42
print(f'{t:another format}') # another: 42
print(f'{t:>10}')

# or
#print('{:whatever}'.format(t))
#print('{:another format}'.format(t))
#print('{:>10}'.format(t))

Output:

format whatever -> 42
another: 42
        42

posted about 4 years ago

CC BY-SA 4.0

hkotsubo‭

5020 reputation 20 68 566 221