Notifications
Q&A

Why don't format specifiers work with lists, dictionaries and other objects?

+6
−0

When I want to print a number or a string, I can use f-strings (Python >= 3.6) or str.format, and I can use just the variable between braces, or use format specifiers. Ex:

num, text = 10, 'abc'

# passing just the variables
print(f'{num} {text}')
# or
#print('{} {}'.format(num, text))

# using format specifiers
# number left-aligned, with 6 spaces, text right-aligned with 10 spaces
print(f'{num:<6} {text:>10}')
# or
#print('{:<6} {:>12}'.format(num, text))

Output:

10 abc
10            abc

But if I do the same with lists or dictionaries, only the first option works:

mylist = [1, 2]
dic = {'a': 1}
# this works
print(f'{mylist} {dic}')
# or
#print('{} {}'.format(mylist, dic))

# this doesn't work
print(f'{mylist:<10} {dic:>15}')
# or
#print('{:<10} {:>15}'.format(mylist, dic))

The first print outputs:

[1, 2] {'a': 1}

But the second print (with the format specifiers <10 and >15) gives this error:

TypeError: unsupported format string passed to list.__format__

If I try the same thing with an instance of a class that I created, the same thing occurs. Ex:

class Test:
    def __init__(self, value):
        self.value = value

    def __str__(self):
        return f'Test({self.value})'

t = Test(42)
print(f't={t}') # this works
print(f'{t:>10}') # this doesn't work

The first print outputs:

t=Test(42)

The second print gives an error:

TypeError: unsupported format string passed to Test.__format__

My question is not about how to fix it (I could just convert the list/dictionary/object to string, either by using str, or by iterating through its elements or atributes and manually building the string, etc).

What I want to know is why this happens. Why it's not possible to use format specifiers with lists, dictionaries and instances of my own classes, but if I pass them without any specifiers, it works? Are there any internal details about those types, that make them behave differently to numbers and strings, when those are formatted?

Why should this post be closed?

0 comments

1 answer

+9
−0

When you use the variable without any format specifier (print(f'{variable}')), internally its __str__ method is called. Considering your Test class, it already has this method:

class Test:
    def __init__(self, valor):
        self.valor = valor

    def __str__(self):
        print('calling __str__') # calling print just to show this method is called
        return f'Test({self.valor})'

t = Test(42)
print(f'{t}')

So the output is:

calling __str__
Test(42)

But what it's actually being called behind the scenes is the __format__ method.

According to the documentation, __format__ is called when the object is being evaluated inside a f-string, or when passed as an argument to str.format and the format built-in.

But my Test class didn't implement __format__, so it uses the method inherited from its superclass (object). And if we check its implementation in CPython source code - checked in 2020-08-11, comments removed):

static PyObject *
object___format___impl(PyObject *self, PyObject *format_spec)
{
    if (PyUnicode_GET_LENGTH(format_spec) > 0) {
        PyErr_Format(PyExc_TypeError,
                     "unsupported format string passed to %.200s.__format__",
                     Py_TYPE(self)->tp_name);
        return NULL;
    }
    return PyObject_Str(self);
}

This means that, when no format specifier is used (only f'{variable}'), it doesn't enter the if and returns PyObject_Str(self) (and according to the documentation, PyObject_Str(whatever) is equivalent to calling str(whatever) - which in turn calls whatever.__str__()).

But if I use some format specifier (such as f'{variable:>10}'), it enters the if and shows the error message ("unsupported format string etc").

The docs says that this behaviour of raising a TypeError when there's no format specifier was implemented in Python 3.4, and the call to str(self) was implemented in Python 3.7 (before that, it called format(str(self), ''), according to this commit).

Conclusion: if I want format specifiers to work with my class, I have to implement __format__:

class Test:
    def __init__(self, value):
        self.value = value

    def __str__(self):
        print('calling __str__')
        return f'Test({self.value})'

    def __format__(self, format_spec):
        print(f'calling __format__ with: "{format_spec}"')
        return f'{self.value:{format_spec}}'

t = Test(42)
print(f'{t}')
print(f'{t:>10}')

Now both print statements work (and I implemented __format__ in a way that it doesn't delegate to __str__, but I could've done it). The output is:

calling __format__ with: ""
42
calling __format__ with: ">10"
        42

Just reminding that I'll get the same output above using str.format:

print('{}'.format(t))
print('{:>10}'.format(t))

That's why format specifiers don't work with lists and dictionaries, because list e dict don't override __format__ and use the inherited implementation from object (which delegates to str when there are no format specifiers, and raises an error when there is - and that's why it works when I do just print(f'{some_list}')).

Unfortunately we can't implement __format__ in lists and dictionaries, because we can't add new methods in native classes, so the only solution seems to be to convert them to strings (using str, or iterating through its elements and manually building a string).

On the other hand, numbers and strings implement __format__ and work fine with format specifiers.

And, as already said, this works not only with f-strings, but also with str.format and format built-in:

t = Test(42) # using the last version above, with the implementation of __format__
# both lines below call Test.__format__
print('{:>10}'.format(t))
print(format(t, '>10'))

mylist = [1, 2]
# both lines below raise TypeError: unsupported format string passed to list.__format__
print('{:>10}'.format(mylist))
print(format(mylist, '>10'))

By the way, that's why it's possible to format datetime classes this way - such as f'{datetime.now():%d/%m/%Y}' - because those classes implement __format__, delegating to strftime.

Which means that I could also do something similar with my Test class, defining any custom format specifier I want:

class Test:
    # ... constructor, etc

    def __format__(self, format_spec):
        formats = { # custom formats (not the best examples)
          'whatever' : f'format whatever -> {self.value}',
          'another format' : f'another: {self.value}'
        }
        if format_spec in formats:
            return formats[format_spec]

        return f'{self.value:{format_spec}}'

t = Test(42)
print(f'{t:whatever}') # format whatever -> 42
print(f'{t:another format}') # another: 42
print(f'{t:>10}')

# or
#print('{:whatever}'.format(t))
#print('{:another format}'.format(t))
#print('{:>10}'.format(t))

Output:

format whatever -> 42
another: 42
        42

0 comments

Sign up to answer this question »