Can you solve these three (deceptively) simple problems in Python?

From the very beginning of my path as a software developer, I really loved digging into the insides of programming languages. I've always wondered how this or that construction works, how this or that command works, what's under the hood of syntactic sugar, etc. Recently, I came across an interesting article with examples of how mutable- and immutable-objects in Python do not always obviously work. In my opinion, the key is how the behavior of the code changes depending on the data type used, while maintaining the identical semantics and used language constructs. This is a great example of thinking not only when writing, but also when using. I invite everyone to familiarize themselves with the translation.







Try to solve these three problems, and then check the answers at the end of the article.



Tip : Problems have something in common, so brush up on the first problem when you move on to the second or third, it will be easier for you.

First task

There are several variables:

x = 1
y = 2
l = [x, y]
x += 5

a = [1]
b = [2]
s = [a, b]
a.append(5)

What will be displayed when printing land s?

Second task

Define a simple function:

def f(x, s=set()):
    s.add(x)
    print(s)

What happens if you call:

>>f(7)
>>f(6, {4, 5})
>>f(2)

Third task

Let's define two simple functions:

def f():
    l = [1]
    def inner(x):
        l.append(x)
        return l
    return inner

def g():
    y = 1
    def inner(x):
        y += x
        return y
    return inner

What do we get after executing these commands?

>>f_inner = f()
>>print(f_inner(2))

>>g_inner = g()
>>print(g_inner(2))

How confident are you in your answers? Let's check your case.

Solution of the first problem

>>print(l)
[1, 2]

>>print(s)
[[1, 5], [2]]

Why does the second list react to a change to its first item a.append(5), while the first list completely ignores the same change x+=5?

Solution of the second problem

Let's see what happens:

>>f(7)
{7}

>>f(6, {4, 5})
{4, 5, 6}

>>f(2)
{2, 7}

Wait, shouldn't the last result be {2}?

The solution to the third problem

The result will be like this:

>>f_inner = f()
>>print(f_inner(2))
[1, 2]

>>g_inner = g()
>>print(g_inner(2))
UnboundLocalError: local variable ‘y’ referenced before assignment

Why g_inner(2)didn’t she betray 3? Why f()does the inner function remember the outer scope, but the inner function g()does not? They are almost identical!

Explanation

What if I told you that all of these odd behaviors have to do with the difference between mutable and immutable objects in Python?



Modifiable objects, such as lists, sets, or dictionaries, can be modified locally. Immutable objects, such as numeric and string values, tuples, cannot be changed; their "change" will lead to the creation of new objects.

First task explanation

x = 1
y = 2
l = [x, y]
x += 5

a = [1]
b = [2]
s = [a, b]
a.append(5)

>>print(l)
[1, 2]

>>print(s)
[[1, 5], [2]]

Since it is ximmutable, the operation x+=5does not change the original object, but creates a new one. But the first item in the list still refers to the original object, so its value doesn't change.



Because a mutable object, then the command a.append(5)modifies the original object (rather than creating a new one), and the list s"sees" the changes.

Explanation of the second task

def f(x, s=set()):
    s.add(x)
    print(s)

>>f(7)
{7}

>>f(6, {4, 5})
{4, 5, 6}

>>f(2)
{2, 7}

With the first two results, everything is clear: the first value is 7added to the initially empty set and it turns out {7}; then the value is 6added to the set {4, 5}and obtained {4, 5, 6}.



And then the oddities begin. The value 2is not added to the empty set, but to {7}. Why? The initial value of the optional parameter is scalculated only once: on the first call, s will be initialized as an empty set. And since it is mutable, f(7)it will be changed in place after being called . The second call f(6, {4, 5})will not affect the default parameter: the set replaces it {4, 5}, that is, it is {4, 5}a different variable. The third call f(2)uses the same variablesthat was used during the first call, but it is not reinitialized as an empty set, but instead its previous value is taken {7}.



Therefore, you should not use mutable arguments as default arguments. In this case, the function needs to be changed:

def f(x, s=None):
    if s is None:
        s = set()
    s.add(x)
    print(s)

Explanation of the third task

def f():
   l = [1]
   def inner(x):
       l.append(x)
       return l
   return inner

def g():
   y = 1
   def inner(x):
       y += x
       return y
   return inner

>>f_inner = f()
>>print(f_inner(2))
[1, 2]

>>g_inner = g()
>>print(g_inner(2))
UnboundLocalError: local variable ‘y’ referenced before assignment

Here we are dealing with closures: internal functions remember how their external namespaces looked at the time of their definition. Or at least they should remember, but the second function makes the poker face and behaves as if it hadn't heard of its external namespace.



Why is this happening? When we execute l.append(x), the mutable object created when the function is defined changes. But the variable lstill refers to the old address in memory. However, trying to change an immutable variable in the second function y += xresults in y starting to refer to a different memory address: the original y will be forgotten, which will result in an UnboundLocalError.

Conclusion

The difference between mutable and immutable objects in Python is very important. Avoid the strange behavior described in this article. Especially:

• .
• - .