python – Multiple variables in a ‘with’ statement?

The Question :

429 people think this question is useful

Is it possible to declare more than one variable using a with statement in Python?

Something like:

from __future__ import with_statement

with open("out.txt","wt"), open("in.txt") as file_out, file_in:
    for line in file_in:
        file_out.write(line)

… or is cleaning up two resources at the same time the problem?

The Question Comments :
  • Maybe like this: with [expr1,expr2] as f: and then use f[0] and f[1].
  • Would have been nice because no need to import something…. but it doesn’t work AttributeError: ‘list’ object has no attribute ‘exit
  • If python just had closures, you wouldn’t need the with statement
  • You don’t need to use a with statement, right? You can just set file_out and file_in to None, then do a try/except/finally where you open them and process them in the try, and then in the finally close them if they are not None. No double-indentation needed for that.
  • Many of these answers don’t deal with the need for more than two with statements. Theoretically there may be applications that need to open tens of contexts, the nesting falls apart very quickly is any line length limitations are imposed.

The Answer 1

722 people think this answer is useful

It is possible in Python 3 since v3.1 and Python 2.7. The new with syntax supports multiple context managers:

with A() as a, B() as b, C() as c:
    doSomething(a,b,c)

Unlike the contextlib.nested, this guarantees that a and b will have their __exit__()‘s called even if C() or it’s __enter__() method raises an exception.

You can also use earlier variables in later definitions (h/t Ahmad below):

with A() as a, B(a) as b, C(a, b) as c:
    doSomething(a, c)

The Answer 2

56 people think this answer is useful

contextlib.nested supports this:

import contextlib

with contextlib.nested(open("out.txt","wt"), open("in.txt")) as (file_out, file_in):

   ...

Update:
To quote the documentation, regarding contextlib.nested:

Deprecated since version 2.7: The with-statement now supports this functionality directly (without the confusing error prone quirks).

See Rafał Dowgird’s answer for more information.

The Answer 3

47 people think this answer is useful

Note that if you split the variables into lines, you must use backslashes to wrap the newlines.

with A() as a, \
     B() as b, \
     C() as c:
    doSomething(a,b,c)

Parentheses don’t work, since Python creates a tuple instead.

with (A(),
      B(),
      C()):
    doSomething(a,b,c)

Since tuples lack a __enter__ attribute, you get an error (undescriptive and does not identify class type):

AttributeError: __enter__

If you try to use as within parentheses, Python catches the mistake at parse time:

with (A() as a,
      B() as b,
      C() as c):
    doSomething(a,b,c)

SyntaxError: invalid syntax

When will this be fixed?

This issue is tracked in https://bugs.python.org/issue12782.

Recently, Python announced in PEP 617 that they’ll be replacing the current parser with a new one. Because Python’s current parser is LL(1), it cannot distinguish between “multiple context managers” with (A(), B()): and “tuple of values” with (A(), B())[0]:.

The new parser can properly parse “multiple context managers” surrounded by tuples. The new parser will be enabled in 3.9, but this syntax will still be rejected until the old parser is removed in Python 3.10.

The Answer 4

18 people think this answer is useful

Since Python 3.3, you can use the class ExitStack from the contextlib module.

It can manage a dynamic number of context-aware objects, which means that it will prove especially useful if you don’t know how many files you are going to handle.

The canonical use-case that is mentioned in the documentation is managing a dynamic number of files.

with ExitStack() as stack:
    files = [stack.enter_context(open(fname)) for fname in filenames]
    # All opened files will automatically be closed at the end of
    # the with statement, even if attempts to open files later
    # in the list raise an exception

Here is a generic example:

from contextlib import ExitStack

class X:
    num = 1

    def __init__(self):
        self.num = X.num
        X.num += 1

    def __repr__(self):
        cls = type(self)
        return '{cls.__name__}{self.num}'.format(cls=cls, self=self)

    def __enter__(self):
        print('enter {!r}'.format(self))
        return self.num

    def __exit__(self, exc_type, exc_value, traceback):
        print('exit {!r}'.format(self))
        return True

xs = [X() for _ in range(3)]

with ExitStack() as stack:
    print(stack._exit_callbacks)
    nums = [stack.enter_context(x) for x in xs]
    print(stack._exit_callbacks)
print(stack._exit_callbacks)
print(nums)

Output:

deque([])
enter X1
enter X2
enter X3
deque([<function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f86158>, <function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f861e0>, <function ExitStack._push_cm_exit.<locals>._exit_wrapper at 0x7f5c95f86268>])
exit X3
exit X2
exit X1
deque([])
[1, 2, 3]

The Answer 5

15 people think this answer is useful

I think you want to do this instead:

from __future__ import with_statement

with open("out.txt","wt") as file_out:
    with open("in.txt") as file_in:
        for line in file_in:
            file_out.write(line)

The Answer 6

0 people think this answer is useful

In Python 3.1+ you can specify multiple context expressions, and they will be processed as if multiple with statements were nested:

with A() as a, B() as b:
    suite

is equivalent to

with A() as a:
    with B() as b:
        suite

This also means that you can use the alias from the first expression in the second (useful when working with db connections/cursors):

with get_conn() as conn, conn.cursor() as cursor:
    cursor.execute(sql)

The Answer 7

0 people think this answer is useful

You can also separate creating a context manager (the __init__ method) and entering the context (the __enter__ method) to increase readability. So instead of writing this code:

with Company(name, id) as company, Person(name, age, gender) as person, Vehicle(brand) as vehicle:
    pass

you can write this code:

company = Company(name, id)
person = Person(name, age, gender)
vehicle = Vehicle(brand)

with company, person, vehicle:
    pass

Note that creating the context manager outside of the with statement makes an impression that the created object can also be further used outside of the statement. If this is not true for your context manager, the false impression may counterpart the readability attempt.

The documentation says:

Most context managers are written in a way that means they can only be used effectively in a with statement once. These single use context managers must be created afresh each time they’re used – attempting to use them a second time will trigger an exception or otherwise not work correctly.

This common limitation means that it is generally advisable to create context managers directly in the header of the with statement where they are used.

Add a Comment