python – Named tuple and default values for optional keyword arguments

The Question :

328 people think this question is useful

I’m trying to convert a longish hollow “data” class into a named tuple. My class currently looks like this:

class Node(object):
    def __init__(self, val, left=None, right=None):
        self.val = val
        self.left = left
        self.right = right

After conversion to namedtuple it looks like:

from collections import namedtuple
Node = namedtuple('Node', 'val left right')

But there is a problem here. My original class allowed me to pass in just a value and took care of the default by using default values for the named/keyword arguments. Something like:

class BinaryTree(object):
    def __init__(self, val):
        self.root = Node(val)

But this doesn’t work in the case of my refactored named tuple since it expects me to pass all the fields. I can of course replace the occurrences of Node(val) to Node(val, None, None) but it isn’t to my liking.

So does there exist a good trick which can make my re-write successful without adding a lot of code complexity (metaprogramming) or should I just swallow the pill and go ahead with the “search and replace”? 🙂

The Question Comments :
  • Why do you want to make this conversion? I like your original Node class just the way it is. Why convert to named tuple?
  • I wanted to make this conversion because the current Node and other classes are simple data-holder value objects with a bunch of different fields (Node is just one of them). These class declarations are nothing much more than line noise IMHO hence wanted to trim them out. Why maintain something which isn’t required? 🙂
  • You don’t have any method functions on your classes at all? You don’t, for example, have a .debug_print() method that walks the tree and prints it?
  • Sure I do, but that’s for the BinaryTree class. Node and other data holders don’t require such special methods esp given that named tuples have a decent __str__ and __repr__ representation. 🙂
  • Okay, seems reasonable. And I think Ignacio Vazquez-Abrams has given you the answer: use a function that does the default values for your node.

The Answer 1

579 people think this answer is useful

Python 3.7

Use the defaults parameter.

>>> from collections import namedtuple
>>> fields = ('val', 'left', 'right')
>>> Node = namedtuple('Node', fields, defaults=(None,) * len(fields))
>>> Node()
Node(val=None, left=None, right=None)

Or better yet, use the new dataclasses library, which is much nicer than namedtuple.

>>> from dataclasses import dataclass
>>> from typing import Any
>>> @dataclass
... class Node:
...     val: Any = None
...     left: 'Node' = None
...     right: 'Node' = None
>>> Node()
Node(val=None, left=None, right=None)

Before Python 3.7

Set Node.__new__.__defaults__ to the default values.

>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.__defaults__ = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)

Before Python 2.6

Set Node.__new__.func_defaults to the default values.

>>> from collections import namedtuple
>>> Node = namedtuple('Node', 'val left right')
>>> Node.__new__.func_defaults = (None,) * len(Node._fields)
>>> Node()
Node(val=None, left=None, right=None)

Order

In all versions of Python, if you set fewer default values than exist in the namedtuple, the defaults are applied to the rightmost parameters. This allows you to keep some arguments as required arguments.

>>> Node.__new__.__defaults__ = (1,2)
>>> Node()
Traceback (most recent call last):
  ...
TypeError: __new__() missing 1 required positional argument: 'val'
>>> Node(3)
Node(val=3, left=1, right=2)

Wrapper for Python 2.6 to 3.6

Here’s a wrapper for you, which even lets you (optionally) set the default values to something other than None. This does not support required arguments.

import collections
def namedtuple_with_defaults(typename, field_names, default_values=()):
    T = collections.namedtuple(typename, field_names)
    T.__new__.__defaults__ = (None,) * len(T._fields)
    if isinstance(default_values, collections.Mapping):
        prototype = T(**default_values)
    else:
        prototype = T(*default_values)
    T.__new__.__defaults__ = tuple(prototype)
    return T

Example:

>>> Node = namedtuple_with_defaults('Node', 'val left right')
>>> Node()
Node(val=None, left=None, right=None)
>>> Node = namedtuple_with_defaults('Node', 'val left right', [1, 2, 3])
>>> Node()
Node(val=1, left=2, right=3)
>>> Node = namedtuple_with_defaults('Node', 'val left right', {'right':7})
>>> Node()
Node(val=None, left=None, right=7)
>>> Node(4)
Node(val=4, left=None, right=7)

The Answer 2

146 people think this answer is useful

I subclassed namedtuple and overrode the __new__ method:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, value, left=None, right=None):
        return super(Node, cls).__new__(cls, value, left, right)

This preserves an intuitive type hierarchy, which the creation of a factory function disguised as a class does not.

The Answer 3

96 people think this answer is useful

Wrap it in a function.

NodeT = namedtuple('Node', 'val left right')

def Node(val, left=None, right=None):
  return NodeT(val, left, right)

The Answer 4

77 people think this answer is useful

With typing.NamedTuple in Python 3.6.1+ you can provide both a default value and a type annotation to a NamedTuple field. Use typing.Any if you only need the former:

from typing import Any, NamedTuple


class Node(NamedTuple):
    val: Any
    left: 'Node' = None
    right: 'Node' = None

Usage:

>>> Node(1)
Node(val=1, left=None, right=None)
>>> n = Node(1)
>>> Node(2, left=n)
Node(val=2, left=Node(val=1, left=None, right=None), right=None)

Also, in case you need both default values and optional mutability, Python 3.7 is going to have data classes (PEP 557) that can in some (many?) cases replace namedtuples.


Sidenote: one quirk of the current specification of annotations (expressions after : for parameters and variables and after -> for functions) in Python is that they are evaluated at definition time*. So, since “class names become defined once the entire body of the class has been executed”, the annotations for 'Node' in the class fields above must be strings to avoid NameError.

This kind of type hints is called “forward reference” ([1], [2]), and with PEP 563 Python 3.7+ is going to have a __future__ import (to be enabled by default in 4.0) that will allow to use forward references without quotes, postponing their evaluation.

* AFAICT only local variable annotations are not evaluated at runtime. (source: PEP 526)

The Answer 5

21 people think this answer is useful

This is an example straight from the docs:

Default values can be implemented by using _replace() to customize a prototype instance:

>>> Account = namedtuple('Account', 'owner balance transaction_count')
>>> default_account = Account('<owner name>', 0.0, 0)
>>> johns_account = default_account._replace(owner='John')
>>> janes_account = default_account._replace(owner='Jane')

So, the OP’s example would be:

from collections import namedtuple
Node = namedtuple('Node', 'val left right')
default_node = Node(None, None, None)
example = default_node._replace(val="whut")

However, I like some of the other answers given here better. I just wanted to add this for completeness.

The Answer 6

19 people think this answer is useful

I’m not sure if there’s an easy way with just the built-in namedtuple. There’s a nice module called recordtype that has this functionality:

>>> from recordtype import recordtype
>>> Node = recordtype('Node', [('val', None), ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)

The Answer 7

14 people think this answer is useful

Here is a more compact version inspired by justinfay’s answer:

from collections import namedtuple
from functools import partial

Node = namedtuple('Node', ('val left right'))
Node.__new__ = partial(Node.__new__, left=None, right=None)

The Answer 8

13 people think this answer is useful

In python3.7+ there’s a brand new defaults= keyword argument.

defaults can be None or an iterable of default values. Since fields with a default value must come after any fields without a default, the defaults are applied to the rightmost parameters. For example, if the fieldnames are ['x', 'y', 'z'] and the defaults are (1, 2), then x will be a required argument, y will default to 1, and z will default to 2.

Example usage:

$ ./python
Python 3.7.0b1+ (heads/3.7:4d65430, Feb  1 2018, 09:28:35) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from collections import namedtuple
>>> nt = namedtuple('nt', ('a', 'b', 'c'), defaults=(1, 2))
>>> nt(0)
nt(a=0, b=1, c=2)
>>> nt(0, 3)  
nt(a=0, b=3, c=2)
>>> nt(0, c=3)
nt(a=0, b=1, c=3)

The Answer 9

7 people think this answer is useful

Short, simple, and doesn’t lead people to use isinstance improperly:

class Node(namedtuple('Node', ('val', 'left', 'right'))):
    @classmethod
    def make(cls, val, left=None, right=None):
        return cls(val, left, right)

# Example
x = Node.make(3)
x._replace(right=Node.make(4))

The Answer 10

5 people think this answer is useful

A slightly extended example to initialize all missing arguments with None:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, *args, **kwargs):
        # initialize missing kwargs with None
        all_kwargs = {key: kwargs.get(key) for key in cls._fields}
        return super(Node, cls).__new__(cls, *args, **all_kwargs)

The Answer 11

5 people think this answer is useful

Python 3.7: introduction of defaults param in namedtuple definition.

Example as shown in the documentation:

>>> Account = namedtuple('Account', ['type', 'balance'], defaults=[0])
>>> Account._fields_defaults
{'balance': 0}
>>> Account('premium')
Account(type='premium', balance=0)

Read more here.

The Answer 12

4 people think this answer is useful

You can also use this:

import inspect

def namedtuple_with_defaults(type, default_value=None, **kwargs):
    args_list = inspect.getargspec(type.__new__).args[1:]
    params = dict([(x, default_value) for x in args_list])
    params.update(kwargs)

    return type(**params)

This basically gives you the possibility to construct any named tuple with a default value and override just the parameters you need, for example:

import collections

Point = collections.namedtuple("Point", ["x", "y"])
namedtuple_with_defaults(Point)
>>> Point(x=None, y=None)

namedtuple_with_defaults(Point, x=1)
>>> Point(x=1, y=None)

The Answer 13

4 people think this answer is useful

I find this version easier to read:

from collections import namedtuple

def my_tuple(**kwargs):
    defaults = {
        'a': 2.0,
        'b': True,
        'c': "hello",
    }
    default_tuple = namedtuple('MY_TUPLE', ' '.join(defaults.keys()))(*defaults.values())
    return default_tuple._replace(**kwargs)

This is not as efficient as it requires creation of the object twice but you could change that by defining the default duple inside the module and just having the function do the replace line.

The Answer 14

4 people think this answer is useful

Combining approaches of @Denis and @Mark:

from collections import namedtuple
import inspect

class Node(namedtuple('Node', 'left right val')):
    __slots__ = ()
    def __new__(cls, *args, **kwargs):
        args_list = inspect.getargspec(super(Node, cls).__new__).args[len(args)+1:]
        params = {key: kwargs.get(key) for key in args_list + kwargs.keys()}
        return super(Node, cls).__new__(cls, *args, **params) 

That should support creating the tuple with positional arguments and also with mixed cases. Test cases:

>>> print Node()
Node(left=None, right=None, val=None)

>>> print Node(1,2,3)
Node(left=1, right=2, val=3)

>>> print Node(1, right=2)
Node(left=1, right=2, val=None)

>>> print Node(1, right=2, val=100)
Node(left=1, right=2, val=100)

>>> print Node(left=1, right=2, val=100)
Node(left=1, right=2, val=100)

>>> print Node(left=1, right=2)
Node(left=1, right=2, val=None)

but also support TypeError:

>>> Node(1, left=2)
TypeError: __new__() got multiple values for keyword argument 'left'

The Answer 15

3 people think this answer is useful

Since you are using namedtuple as a data class, you should be aware that python 3.7 will introduce a @dataclass decorator for this very purpose — and of course it has default values.

An example from the docs:

@dataclass
class C:
    a: int       # 'a' has no default value
    b: int = 0   # assign a default value for 'b'

Much cleaner, readable and usable than hacking namedtuple. It is not hard to predict that usage of namedtuples will drop with the adoption of 3.7.

The Answer 16

2 people think this answer is useful

Inspired by this answer to a different question, here is my proposed solution based on a metaclass and using super (to handle future subcalssing correctly). It is quite similar to justinfay’s answer.

from collections import namedtuple

NodeTuple = namedtuple("NodeTuple", ("val", "left", "right"))

class NodeMeta(type):
    def __call__(cls, val, left=None, right=None):
        return super(NodeMeta, cls).__call__(val, left, right)

class Node(NodeTuple, metaclass=NodeMeta):
    __slots__ = ()

Then:

>>> Node(1, Node(2, Node(4)),(Node(3, None, Node(5))))
Node(val=1, left=Node(val=2, left=Node(val=4, left=None, right=None), right=None), right=Node(val=3, left=None, right=Node(val=5, left=None, right=None)))

The Answer 17

2 people think this answer is useful

The answer by jterrace to use recordtype is great, but the author of the library recommends to use his namedlist project, which provides both mutable (namedlist) and immutable (namedtuple) implementations.

from namedlist import namedtuple
>>> Node = namedtuple('Node', ['val', ('left', None), ('right', None)])
>>> Node(3)
Node(val=3, left=None, right=None)
>>> Node(3, 'L')
Node(val=3, left=L, right=None)

The Answer 18

1 people think this answer is useful

Here’s a short, simple generic answer with a nice syntax for a named tuple with default arguments:

import collections

def dnamedtuple(typename, field_names, **defaults):
    fields = sorted(field_names.split(), key=lambda x: x in defaults)
    T = collections.namedtuple(typename, ' '.join(fields))
    T.__new__.__defaults__ = tuple(defaults[field] for field in fields[-len(defaults):])
    return T

Usage:

Test = dnamedtuple('Test', 'one two three', two=2)
Test(1, 3)  # Test(one=1, three=3, two=2)

Minified:

def dnamedtuple(tp, fs, **df):
    fs = sorted(fs.split(), key=df.__contains__)
    T = collections.namedtuple(tp, ' '.join(fs))
    T.__new__.__defaults__ = tuple(df[i] for i in fs[-len(df):])
    return T

The Answer 19

0 people think this answer is useful

Using the NamedTuple class from my Advanced Enum (aenum) library, and using the class syntax, this is quite simple:

from aenum import NamedTuple

class Node(NamedTuple):
    val = 0
    left = 1, 'previous Node', None
    right = 2, 'next Node', None

The one potential drawback is the requirement for a __doc__ string for any attribute with a default value (it’s optional for simple attributes). In use it looks like:

>>> Node()
Traceback (most recent call last):
  ...
TypeError: values not provided for field(s): val

>>> Node(3)
Node(val=3, left=None, right=None)

The advantages this has over justinfay's answer:

from collections import namedtuple

class Node(namedtuple('Node', ['value', 'left', 'right'])):
    __slots__ = ()
    def __new__(cls, value, left=None, right=None):
        return super(Node, cls).__new__(cls, value, left, right)

is simplicity, as well as being metaclass based instead of exec based.

The Answer 20

0 people think this answer is useful

Another solution:

import collections


def defaultargs(func, defaults):
    def wrapper(*args, **kwargs):
        for key, value in (x for x in defaults[len(args):] if len(x) == 2):
            kwargs.setdefault(key, value)
        return func(*args, **kwargs)
    return wrapper


def namedtuple(name, fields):
    NamedTuple = collections.namedtuple(name, [x[0] for x in fields])
    NamedTuple.__new__ = defaultargs(NamedTuple.__new__, [(NamedTuple,)] + fields)
    return NamedTuple

Usage:

>>> Node = namedtuple('Node', [
...     ('val',),
...     ('left', None),
...     ('right', None),
... ])
__main__.Node

>>> Node(1)
Node(val=1, left=None, right=None)

>>> Node(1, 2, right=3)
Node(val=1, left=2, right=3)

The Answer 21

-1 people think this answer is useful

Here’s a less flexible, but more concise version of Mark Lodato’s wrapper: It takes the fields and defaults as a dictionary.

import collections
def namedtuple_with_defaults(typename, fields_dict):
    T = collections.namedtuple(typename, ' '.join(fields_dict.keys()))
    T.__new__.__defaults__ = tuple(fields_dict.values())
    return T

Example:

In[1]: fields = {'val': 1, 'left': 2, 'right':3}

In[2]: Node = namedtuple_with_defaults('Node', fields)

In[3]: Node()
Out[3]: Node(val=1, left=2, right=3)

In[4]: Node(4,5,6)
Out[4]: Node(val=4, left=5, right=6)

In[5]: Node(val=10)
Out[5]: Node(val=10, left=2, right=3)

Add a Comment