Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Post History

75%
+4 −0
Q&A Understanding mutable default arguments in Python

Workarounds Avoiding mutation Because problems are only caused by actually mutating the default argument, the simplest way to avoid problems is to... not do that. Pythonic code obeys command-quer...

posted 8mo ago by Karl Knechtel‭  ·  edited 5mo ago by Karl Knechtel‭

Answer
#2: Post edited by user avatar Karl Knechtel‭ · 2023-12-01T23:28:04Z (5 months ago)
Clarify the status of PEP 671, and do some copyediting
  • ## Workarounds
  • ### Avoiding mutation
  • Because problems are only caused by actually mutating the default argument, the simplest way to avoid problems is to... not do that. Pythonic code obeys [command-query separation](https://en.wikipedia.org/wiki/Command%E2%80%93query_separation); a function should take its effect *either* by mutating one or more parameters (a "command") *or* by returning a useful value (a "query") - not both. To implement a "command" in this paradigm, there should always be an explicitly passed argument to mutate. Python doesn't support "out parameters" - everything passed to the function must actually exist before the function is called. If the default value were used, the calling code wouldn't be able to access it directly unless it were `return`ed, which would violate the command-query separation.
  • Therefore, if the code **needs to work by** modifying one of the provided arguments, the corresponding parameter **should not have a default value at all**.
  • On the other hand, sometimes carelessly written code simply modifies the provided arguments for convenience. **Avoid this; it leads to more subtle errors** even without using mutable default arguments. Keep in mind that the passed-in objects might be used in other places, which might not expect the modification made by your function. If you need to determine the result of, say, adding an element to a provided list, **make a new list**:
  • <section class="notice is-danger">
  • **This way can modify the caller's `x` list unnecessarily:**
  • ```
  • def join_lists(x=[], y=[]):
  • x.extend(y)
  • return x
  • ```
  • </section>
  • <section class="notice is-success">
  • **This way preserves the inputs:**
  • ```
  • def join_lists(x=(), y=()):
  • result = list(x)
  • x.extend(y)
  • return result
  • ```
  • </section>
  • ### Specifying immutable defaults
  • The above example also shows a useful safeguard: the default values for `x` and `y` are changed to empty *tuples*, which are immutable. This serves two purposes:
  • 1. Anyone who reads the code can deduce that the function is *not intended to* mutate the provided `x` and `y` values, and that other `tuple` values will be acceptable. (Of course, this can also be hinted using type annotations.)
  • 1. If the code is mistakenly written to try something like `x.append(y)`, a call that uses the default arguments will *raise an exception* rather than silently producing the wrong result. This makes it easier to debug the problem.
  • ### Sentinels as default arguments
  • The established practice in the Python community is to use the special value `None` for default arguments.
  • <section class="notice is-success">
  • **For example, this function can either make a tuple from two provided values, or take a single value and make a tuple that uses the same value twice:**
  • ```
  • def pair(x, y=None):
  • if y is None:
  • y = x
  • return (x, y)
  • ```
  • </section>
  • The special value `None` has its own type, and there is special logic to make sure that there can only be that one instance of the type. Therefore, by convention, the `is` operator is used to check for `None`, to guard against types implementing `__eq__` to make themselves "equal to" `None`.
  • <section class="notice is-warning">
  • **There are some problems with this approach, however.** For example, if `None` could be a legitimate value that is passed explicitly, then some other scheme will need to be used to create a sentinel value. There isn't a clear one-size-fits-all solution for this yet. The idea [has been discussed quite a bit](https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810) and there is a [draft PEP (661)](https://peps.python.org/pep-0661/), but no clear resolution so far.
  • More importantly, though, the idiom is arguably overused. If something like `()` or `''` makes sense as a default value, and provides the necessary functionality required for the algorithm, it would be better to use that rather than treating the default argument as a special case. As the Zen of Python says, "Simple is better than complex", and "special cases aren't special enough to break the rules".
  • </section>
  • ### PEP 671 (future)
  • [PEP 671](https://peps.python.org/pep-0671/) describes a new syntax for default arguments. Parameters that use a `=>` operator ("arrow" symbol instead of an equals sign) for a default argument will instead treat the code there as a *way to calculate* a default value, instead of creating one ahead of time.
  • <section class="notice is-warning">
  • **In the future (hopefully), this code will work, and repeated calls to `example() will show a single-element list each time.**
  • ```
  • def example(param=>[]):
  • param.append('test')
  • print('The list is now', param)
  • ```
  • </section>
  • This also allows default argument values to be defined in terms of other arguments that were provided:
  • <section class="notice is-warning">
  • **In the future (hopefully), this code will work, and `pair(3)` will result in `(3, 3)`.**
  • ```
  • def pair(x, y=>x):
  • return (x, y)
  • ```
  • </section>
  • Without PEP 671 implemented, implementing `pair` would require a sentinel value as described in the previous section.
  • Currently, PEP 671 is marked as "draft" status. It is supposed to have been targeted for inclusion in Python 3.12. However, that Python version is just about to be released (currently on release candidate 3), and PEP 671 is [not mentioned in the current "what's new" document](https://docs.python.org/3.12/whatsnew/3.12.html).
  • ## Workarounds
  • ### Avoiding mutation
  • Because problems are only caused by actually mutating the default argument, the simplest way to avoid problems is to... not do that. Pythonic code obeys [command-query separation](https://en.wikipedia.org/wiki/Command%E2%80%93query_separation); a function should take its effect *either* by mutating one or more parameters (a "command") *or* by returning a useful value (a "query") - not both. To implement a "command" in this paradigm, there should always be an explicitly passed argument to mutate. Python doesn't support "out parameters" - everything passed to the function must actually exist before the function is called. If the default value were used, the calling code wouldn't be able to access it directly unless it were `return`ed, which would violate the command-query separation.
  • Therefore, if the code **needs to work by** modifying one of the provided arguments, the corresponding parameter **should not have a default value at all**.
  • On the other hand, sometimes carelessly written code simply modifies the provided arguments for convenience. **Avoid this; it leads to more subtle errors** even without using mutable default arguments. Keep in mind that the passed-in objects might be used in other places, which might not expect the modification made by your function. If you need to determine the result of, say, adding an element to a provided list, **make a new list**:
  • <section class="notice is-danger">
  • **This way can modify the caller's `x` list unnecessarily:**
  • ```
  • def join_lists(x=[], y=[]):
  • x.extend(y)
  • return x
  • ```
  • </section>
  • <section class="notice is-success">
  • **This way preserves the inputs:**
  • ```
  • def join_lists(x=(), y=()):
  • result = list(x)
  • x.extend(y)
  • return result
  • ```
  • </section>
  • ### Specifying immutable defaults
  • The above example also shows a useful safeguard: the default values for `x` and `y` are changed to empty *tuples*, which are immutable. This serves two purposes:
  • 1. Anyone who reads the code can deduce that the function is *not intended to* mutate the provided `x` and `y` values, and that other `tuple` values will be acceptable. (Of course, this can also be hinted using type annotations.)
  • 1. If the code is mistakenly written to try something like `x.append(y)`, a call that uses the default arguments will *raise an exception* rather than silently producing the wrong result. This makes it easier to debug the problem.
  • ### Sentinels as default arguments
  • The established practice in the Python community is to use the special value `None` for default arguments.
  • <section class="notice is-success">
  • **For example, this function can either make a tuple from two provided values, or take a single value and make a tuple that uses the same value twice:**
  • ```
  • def pair(x, y=None):
  • if y is None:
  • y = x
  • return (x, y)
  • ```
  • </section>
  • The special value `None` has its own type, and there is special logic to make sure that there can only be that one instance of the type. Therefore, by convention, the `is` operator is used to check for `None`, to guard against types implementing `__eq__` to make themselves "equal to" `None`.
  • <section class="notice is-warning">
  • **There are some problems with this approach, however.** For example, if `None` could be a legitimate value that is passed explicitly, then some other scheme will need to be used to create a sentinel value. There isn't a clear one-size-fits-all solution for this yet. The idea [has been discussed quite a bit](https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810) and there is a [draft PEP (661)](https://peps.python.org/pep-0661/), but no clear resolution so far.
  • More importantly, though, the idiom is arguably overused. If something like `()` or `''` makes sense as a default value, and provides the necessary functionality required for the algorithm, it would be better to use that rather than treating the default argument as a special case. As the Zen of Python says, "Simple is better than complex", and "special cases aren't special enough to break the rules".
  • </section>
  • ### Possible future enhancement: PEP 671
  • [PEP 671](https://peps.python.org/pep-0671/) describes a new syntax for default arguments. Parameters that use a `=>` operator ("arrow" symbol instead of an equals sign) for a default argument will instead treat the code there as a *way to calculate* a default value, instead of creating one ahead of time.
  • <section class="notice is-warning">
  • **In the future (hopefully), this code will work, and repeated calls to `example()` will show a single-element list each time. However, this syntax was not implemented for 3.12 as hoped by the PEP author, and there is currently no indication that implementation is planned at all.**
  • ```
  • def example(param=>[]):
  • param.append('test')
  • print('The list is now', param)
  • ```
  • </section>
  • This also allows default argument values to be defined in terms of other arguments that were provided:
  • <section class="notice is-warning">
  • **In the future (hopefully), this code will work, and `pair(3)` will result in `(3, 3)`.**
  • ```
  • # Currently (without PEP 671), implementing `pair` requires
  • # a sentinel value, as described in the previous section.
  • def pair(x, y=>x):
  • return (x, y)
  • ```
  • </section>
#1: Initial revision by user avatar Karl Knechtel‭ · 2023-09-20T04:50:33Z (8 months ago)
## Workarounds

### Avoiding mutation

Because problems are only caused by actually mutating the default argument, the simplest way to avoid problems is to... not do that. Pythonic code obeys [command-query separation](https://en.wikipedia.org/wiki/Command%E2%80%93query_separation); a function should take its effect *either* by mutating one or more parameters (a "command") *or* by returning a useful value (a "query") - not both. To implement a "command" in this paradigm, there should always be an explicitly passed argument to mutate. Python doesn't support "out parameters" - everything passed to the function must actually exist before the function is called. If the default value were used, the calling code wouldn't be able to access it directly unless it were `return`ed, which would violate the command-query separation.

Therefore, if the code **needs to work by** modifying one of the provided arguments, the corresponding parameter **should not have a default value at all**.

On the other hand, sometimes carelessly written code simply modifies the provided arguments for convenience. **Avoid this; it leads to more subtle errors** even without using mutable default arguments. Keep in mind that the passed-in objects might be used in other places, which might not expect the modification made by your function. If you need to determine the result of, say, adding an element to a provided list, **make a new list**:

<section class="notice is-danger">

**This way can modify the caller's `x` list unnecessarily:**
```
def join_lists(x=[], y=[]):
    x.extend(y)
    return x
```
</section>

<section class="notice is-success">

**This way preserves the inputs:**
```
def join_lists(x=(), y=()):
    result = list(x)
    x.extend(y)
    return result
```
</section>

### Specifying immutable defaults

The above example also shows a useful safeguard: the default values for `x` and `y` are changed to empty *tuples*, which are immutable. This serves two purposes:

1. Anyone who reads the code can deduce that the function is *not intended to* mutate the provided `x` and `y` values, and that other `tuple` values will be acceptable. (Of course, this can also be hinted using type annotations.)

1. If the code is mistakenly written to try something like `x.append(y)`, a call that uses the default arguments will *raise an exception* rather than silently producing the wrong result. This makes it easier to debug the problem.

### Sentinels as default arguments

The established practice in the Python community is to use the special value `None` for default arguments. 

<section class="notice is-success">

**For example, this function can either make a tuple from two provided values, or take a single value and make a tuple that uses the same value twice:**
```
def pair(x, y=None):
    if y is None:
        y = x
    return (x, y)
```
</section>

The special value `None` has its own type, and there is special logic to make sure that there can only be that one instance of the type. Therefore, by convention, the `is` operator is used to check for `None`, to guard against types implementing `__eq__` to make themselves "equal to" `None`.

<section class="notice is-warning">

**There are some problems with this approach, however.** For example, if `None` could be a legitimate value that is passed explicitly, then some other scheme will need to be used to create a sentinel value. There isn't a clear one-size-fits-all solution for this yet. The idea [has been discussed quite a bit](https://discuss.python.org/t/sentinel-values-in-the-stdlib/8810) and there is a [draft PEP (661)](https://peps.python.org/pep-0661/), but no clear resolution so far.

More importantly, though, the idiom is arguably overused. If something like `()` or `''` makes sense as a default value, and provides the necessary functionality required for the algorithm, it would be better to use that rather than treating the default argument as a special case. As the Zen of Python says, "Simple is better than complex", and "special cases aren't special enough to break the rules".
</section>

### PEP 671 (future)

[PEP 671](https://peps.python.org/pep-0671/) describes a new syntax for default arguments. Parameters that use a `=>` operator ("arrow" symbol instead of an equals sign) for a default argument will instead treat the code there as a *way to calculate* a default value, instead of creating one ahead of time.

<section class="notice is-warning">

**In the future (hopefully), this code will work, and repeated calls to `example() will show a single-element list each time.**
```
def example(param=>[]):
    param.append('test')
    print('The list is now', param)
```
</section>

This also allows default argument values to be defined in terms of other arguments that were provided:

<section class="notice is-warning">

**In the future (hopefully), this code will work, and `pair(3)` will result in `(3, 3)`.**

```
def pair(x, y=>x):
    return (x, y)
```
</section>

Without PEP 671 implemented, implementing `pair` would require a sentinel value as described in the previous section.

Currently, PEP 671 is marked as "draft" status. It is supposed to have been targeted for inclusion in Python 3.12. However, that Python version is just about to be released (currently on release candidate 3), and PEP 671 is [not mentioned in the current "what's new" document](https://docs.python.org/3.12/whatsnew/3.12.html).