Primer on Python Decorators β Real Python
In this tutorial on decorators, we’ll look at what they are and how to create and use them. Decorators provide a simple syntax for calling higher-order functions.
By definition, a decorator is a function that takes another function and extends the behavior of the latter function without explicitly modifying it.
This sounds confusing, but it’s really not, especially after you’ve seen a few examples of how decorators work. You can find all the examples from this article here.
Free Bonus: Click here to get access to a free "The Power of Python Decorators" guide that shows you 3 advanced decorator patterns and techniques you can use to write to cleaner and more Pythonic programs.
Updates:
- 08/22/2018: Major update adding more examples and more advanced decorators
- 01/12/2016: Updated examples to Python 3 (v3.5.1) syntax and added a new example
- 11/01/2015: Added a brief explanation on the
functools.wraps()decorator
Functions
Before you can understand decorators, you must first understand how functions work. For our purposes, a function returns a value based on the given arguments
>>> def add_one(number): ... return number + 1 >>> add_one(2) 3
In general, functions in Python may also have side effects rather than just turning an input into an output. The print() function is a basic example of this: it returns None while having the side effect of outputting something to the console. However, to understand decorators, it is enough to think about functions as something that turns given arguments into a value.
Note: In functional programming, you work (almost) only with pure functions without side effects. While not a purely functional language, Python supports many of the functional programming concepts, including functions as first-class objects.
First-Class Objects
In Python, functions are first-class objects. This means that functions can be passed around and used as arguments, just like any other object (string, int, float, list, and so on). Consider the following three functions:
def say_hello(name): return f"Hello {name}" def be_awesome(name): return f"Yo {name}, together we are the awesomest!" def greet_bob(greeter_func): return greeter_func("Bob")
Here, say_hello() and be_awesome() are regular functions that expect a name given as a string. The greet_bob() function however, expects a function as its argument. We can, for instance, pass it the say_hello() or the be_awesome() function:
>>> greet_bob(say_hello) 'Hello Bob' >>> greet_bob(be_awesome) 'Yo Bob, together we are the awesomest!'
Note that greet_bob(say_hello) refers to two functions, but in different ways: greet_bob() and say_hello. The say_hello function is named without parentheses. This means that only a reference to the function is passed. The function is not executed. The greet_bob() function, on the other hand, is written with parentheses, so it will be called as usual.
Inner Functions
It’s possible to define functions inside other functions. Such functions are called inner functions. Here’s an example of a function with two inner functions:
def parent(): print("Printing from the parent() function") def first_child(): print("Printing from the first_child() function") def second_child(): print("Printing from the second_child() function") second_child() first_child()
What happens when you call the parent() function? Think about this for a minute. The output will be as follows:
>>> parent() Printing from the parent() function Printing from the second_child() function Printing from the first_child() function
Note that the order in which the inner functions are defined does not matter. Like with any other functions, the printing only happens when the inner functions are executed.
Furthermore, the inner functions are not defined until the parent function is called. They are locally scoped to parent(): they only exist inside the parent() function as local variables. Try calling first_child(). You should get an error:
Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'first_child' is not defined
Whenever you call parent(), the inner functions first_child() and second_child() are also called. But because of their local scope, they aren’t available outside of the parent() function.
Returning Functions From Functions
Python also allows you to use functions as return values. The following example returns one of the inner functions from the outer parent() function:
def parent(num): def first_child(): return "Hi, I am Emma" def second_child(): return "Call me Liam" if num == 1: return first_child else: return second_child
Note that you are returning first_child without the parentheses. Recall that this means that you are returning a reference to the function first_child. In contrast first_child() with parentheses refers to the result of evaluating the function. This can be seen in the following example:
>>> first = parent(1) >>> second = parent(2) >>> first <function parent.<locals>.first_child at 0x7f599f1e2e18> >>> second <function parent.<locals>.second_child at 0x7f599dad5268>
The somewhat cryptic output simply means that the first variable refers to the local first_child() function inside of parent(), while second points to second_child().
You can now use first and second as if they are regular functions, even though the functions they point to can’t be accessed directly:
>>> first() 'Hi, I am Emma' >>> second() 'Call me Liam'
Finally, note that in the earlier example you executed the inner functions within the parent function, for instance first_child(). However, in this last example, you did not add parentheses to the inner functions—first_child—upon returning. That way, you got a reference to each function that you could call in the future. Make sense?
Simple Decorators
Now that you’ve seen that functions are just like any other object in Python, you’re ready to move on and see the magical beast that is the Python decorator. Let’s start with an example:
def my_decorator(func): def wrapper(): print("Something is happening before the function is called.") func() print("Something is happening after the function is called.") return wrapper def say_whee(): print("Whee!") say_whee = my_decorator(say_whee)
Can you guess what happens when you call say_whee()? Try it:
>>> say_whee() Something is happening before the function is called. Whee! Something is happening after the function is called.
To understand what’s going on here, look back at the previous examples. We are literally just applying everything you have learned so far.
The so-called decoration happens at the following line:
say_whee = my_decorator(say_whee)
In effect, the name say_whee now points to the wrapper() inner function. Remember that you return wrapper as a function when you call my_decorator(say_whee):
>>> say_whee <function my_decorator.<locals>.wrapper at 0x7f3c5dfd42f0>
However, wrapper() has a reference to the original say_whee() as func, and calls that function between the two calls to print().
Put simply: decorators wrap a function, modifying its behavior.
Before moving on, let’s have a look at a second example. Because wrapper() is a regular Python function, the way a decorator modifies a function can change dynamically. So as not to disturb your neighbors, the following example will only run the decorated code during the day:
from datetime import datetime def not_during_the_night(func): def wrapper(): if 7 <= datetime.now().hour < 22: func() else: pass # Hush, the neighbors are asleep return wrapper def say_whee(): print("Whee!") say_whee = not_during_the_night(say_whee)
If you try to call say_whee() after bedtime, nothing will happen:
>>> say_whee() >>>
Syntactic Sugar!
The way you decorated say_whee() above is a little clunky. First of all, you end up typing the name say_whee three times. In addition, the decoration gets a bit hidden away below the definition of the function.
Instead, Python allows you to use decorators in a simpler way with the @ symbol, sometimes called the “pie” syntax. The following example does the exact same thing as the first decorator example:
def my_decorator(func): def wrapper(): print("Something is happening before the function is called.") func() print("Something is happening after the function is called.") return wrapper @my_decorator def say_whee(): print("Whee!")
So, @my_decorator is just an easier way of saying say_whee = my_decorator(say_whee). It’s how you apply a decorator to a function.
Reusing Decorators
Recall that a decorator is just a regular Python function. All the usual tools for easy reusability are available. Let’s move the decorator to its own module that can be used in many other functions.
Create a file called decorators.py with the following content:
def do_twice(func): def wrapper_do_twice(): func() func() return wrapper_do_twice
Note: You can name your inner function whatever you want, and a generic name like wrapper() is usually okay. You’ll see a lot of decorators in this article. To keep them apart, we’ll name the inner function with the same name as the decorator but with a wrapper_ prefix.
You can now use this new decorator in other files by doing a regular import:
from decorators import do_twice @do_twice def say_whee(): print("Whee!")
When you run this example, you should see that the original say_whee() is executed twice:
>>> say_whee() Whee! Whee!
Free Bonus: Click here to get access to a free "The Power of Python Decorators" guide that shows you 3 advanced decorator patterns and techniques you can use to write to cleaner and more Pythonic programs.
Decorating Functions With Arguments
Say that you have a function that accepts some arguments. Can you still decorate it? Let’s try:
from decorators import do_twice @do_twice def greet(name): print(f"Hello {name}")
Unfortunately, running this code raises an error:
>>>>>> greet("World") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: wrapper_do_twice() takes 0 positional arguments but 1 was given
The problem is that the inner function wrapper_do_twice() does not take any arguments, but name="World" was passed to it. You could fix this by letting wrapper_do_twice() accept one argument, but then it would not work for the say_whee() function you created earlier.
The solution is to use *args and **kwargs in the inner wrapper function. Then it will accept an arbitrary number of positional and keyword arguments. Rewrite decorators.py as follows:
def do_twice(func): def wrapper_do_twice(*args, **kwargs): func(*args, **kwargs) func(*args, **kwargs) return wrapper_do_twice
The wrapper_do_twice() inner function now accepts any number of arguments and passes them on to the function it decorates. Now both your say_whee() and greet() examples works:
>>> say_whee() Whee! Whee! >>> greet("World") Hello World Hello World
Returning Values From Decorated Functions
What happens to the return value of decorated functions? Well, that’s up to the decorator to decide. Let’s say you decorate a simple function as follows:
from decorators import do_twice @do_twice def return_greeting(name): print("Creating greeting") return f"Hi {name}"
Try to use it:
>>>>>> hi_adam = return_greeting("Adam") Creating greeting Creating greeting >>> print(hi_adam) None
Oops, your decorator ate the return value from the function.
Because the do_twice_wrapper() doesn’t explicitly return a value, the call return_greeting("Adam") ended up returning None.
To fix this, you need to make sure the wrapper function returns the return value of the decorated function. Change your decorators.py file:
def do_twice(func): def wrapper_do_twice(*args, **kwargs): func(*args, **kwargs) return func(*args, **kwargs) return wrapper_do_twice
The return value from the last execution of the function is returned:
>>>>>> return_greeting("Adam") Creating greeting Creating greeting 'Hi Adam'
Who Are You, Really?
A great convenience when working with Python, especially in the interactive shell, is its powerful introspection ability. Introspection is the ability of an object to know about its own attributes at runtime. For instance, a function knows its own name and documentation:
>>>>>> print <built-in function print> >>> print.__name__ 'print' >>> help(print) Help on built-in function print in module builtins: print(...) <full help message>
The introspection works for functions you define yourself as well:
>>>>>> say_whee <function do_twice.<locals>.wrapper_do_twice at 0x7f43700e52f0> >>> say_whee.__name__ 'wrapper_do_twice' >>> help(say_whee) Help on function wrapper_do_twice in module decorators: wrapper_do_twice()
However, after being decorated, say_whee() has gotten very confused about its identity. It now reports being the wrapper_do_twice() inner function inside the do_twice() decorator. Although technically true, this is not very useful information.
To fix this, decorators should use the @functools.wraps decorator, which will preserve information about the original function. Update decorators.py again:
import functools def do_twice(func): @functools.wraps(func) def wrapper_do_twice(*args, **kwargs): func(*args, **kwargs) return func(*args, **kwargs) return wrapper_do_twice
You do not need to change anything about the decorated say_whee() function:
>>> say_whee <function say_whee at 0x7ff79a60f2f0> >>> say_whee.__name__ 'say_whee' >>> help(say_whee) Help on function say_whee in module whee: say_whee()
Much better! Now say_whee() is still itself after decoration.
Technical Detail: The @functools.wraps decorator uses the function functools.update_wrapper() to update special attributes like __name__ and __doc__ that are used in the introspection.
A Few Real World Examples
Let’s look at a few more useful examples of decorators. You’ll notice that they’ll mainly follow the same pattern that you’ve learned so far:
import functools def decorator(func): @functools.wraps(func) def wrapper_decorator(*args, **kwargs): # Do something before value = func(*args, **kwargs) # Do something after return value return wrapper_decorator
This formula is a good boilerplate template for building more complex decorators.
Note: In later examples, we will assume that these decorators are saved in your decorators.py file as well. Recall that you can download all the examples in this tutorial.
Timing Functions
Let’s start by creating a @timer decorator. It will measure the time a function takes to execute and print the duration to the console. Here’s the code:
import functools import time def timer(func): """Print the runtime of the decorated function""" @functools.wraps(func) def wrapper_timer(*args, **kwargs): start_time = time.perf_counter() # 1 value = func(*args, **kwargs) end_time = time.perf_counter() # 2 run_time = end_time - start_time # 3 print(f"Finished {func.__name__!r} in {run_time:.4f} secs") return value return wrapper_timer @timer def waste_some_time(num_times): for _ in range(num_times): sum([i**2 for i in range(10000)])
This decorator works by storing the time just before the function starts running (at the line marked # 1) and just after the function finishes (at # 2). The time the function takes is then the difference between the two (at # 3). We use the time.perf_counter() function, which does a good job of measuring time intervals. Here are some examples of timings:
>>> waste_some_time(1) Finished 'waste_some_time' in 0.0010 secs >>> waste_some_time(999) Finished 'waste_some_time' in 0.3260 secs
Run it yourself. Work through the code line by line. Make sure you understand how it works. Don’t worry if you don’t get it, though. Decorators are advanced beings. Try to sleep on it or make a drawing of the program flow.
Note: The @timer decorator is great if you just want to get an idea about the runtime of your functions. If you want to do more precise measurements of code, you should instead consider the timeit module in the standard library. It temporarily disables garbage collection and runs multiple trials to strip out noise from quick function calls.
Debugging Code
The following @debug decorator will print the arguments a function is called with as well as its return value every time the function is called:
import functools def debug(func): """Print the function signature and return value""" @functools.wraps(func) def wrapper_debug(*args, **kwargs): args_repr = [repr(a) for a in args] # 1 kwargs_repr = [f"{k}={v!r}" for k, v in kwargs.items()] # 2 signature = ", ".join(args_repr + kwargs_repr) # 3 print(f"Calling {func.__name__}({signature})") value = func(*args, **kwargs) print(f"{func.__name__!r} returned {value!r}") # 4 return value return wrapper_debug
The signature is created by joining the string representations of all the arguments. The numbers in the following list correspond to the numbered comments in the code:
- Create a list of the positional arguments. Use
repr()to get a nice string representing each argument. - Create a list of the keyword arguments. The f-string formats each argument as
key=valuewhere the!rspecifier means thatrepr()is used to represent the value. - The lists of positional and keyword arguments is joined together to one signature string with each argument separated by a comma.
- The return value is printed after the function is executed.
Let’s see how the decorator works in practice by applying it to a simple function with one position and one keyword argument:
@debug def make_greeting(name, age=None): if age is None: return f"Howdy {name}!" else: return f"Whoa {name}! {age} already, you are growing up!"
Note how the @debug decorator prints the signature and return value of the make_greeting() function:
>>> make_greeting("Benjamin") Calling make_greeting('Benjamin') 'make_greeting' returned 'Howdy Benjamin!' 'Howdy Benjamin!' >>> make_greeting("Richard", age=112) Calling make_greeting('Richard', age=112) 'make_greeting' returned 'Whoa Richard! 112 already, you are growing up!' 'Whoa Richard! 112 already, you are growing up!' >>> make_greeting(name="Dorrisile", age=116) Calling make_greeting(name='Dorrisile', age=116) 'make_greeting' returned 'Whoa Dorrisile! 116 already, you are growing up!' 'Whoa Dorrisile! 116 already, you are growing up!'
This example might not seem immediately useful since the @debug decorator just repeats what you just wrote. It’s more powerful when applied to small convenience functions that you don’t call directly yourself.
The following example calculates an approximation to the mathematical constant e:
import math from decorators import debug # Apply a decorator to a standard library function math.factorial = debug(math.factorial) def approximate_e(terms=18): return sum(1 / math.factorial(n) for n in range(terms))
This example also shows how you can apply a decorator to a function that has already been defined. The approximation of e is based on the following series expansion:
When calling the approximate_e() function, you can see the @debug decorator at work:
>>> approximate_e(5) Calling factorial(0) 'factorial' returned 1 Calling factorial(1) 'factorial' returned 1 Calling factorial(2) 'factorial' returned 2 Calling factorial(3) 'factorial' returned 6 Calling factorial(4) 'factorial' returned 24 2.708333333333333
In this example, you get a decent approximation to the true value e = 2.718281828, adding only 5 terms.
Slowing Down Code
This next example might not seem very useful. Why would you want to slow down your Python code? Probably the most common use case is that you want to rate-limit a function that continuously checks whether a resource—like a web page—has changed. The @slow_down decorator will sleep one second before it calls the decorated function:
import functools import time def slow_down(func): """Sleep 1 second before calling the function""" @functools.wraps(func) def wrapper_slow_down(*args, **kwargs): time.sleep(1) return func(*args, **kwargs) return wrapper_slow_down @slow_down def countdown(from_number): if from_number < 1: print("Liftoff!") else: print(from_number) countdown(from_number - 1)
To see the effect of the @slow_down decorator, you really need to run the example yourself:
>>> countdown(3) 3 2 1 Liftoff!
Note: The countdown() function is a recursive function. In other words, it’s a function calling itself. To learn more about recursive functions in Python, see our guide on Thinking Recursively in Python.
The @slow_down decorator always sleeps for one second. Later, you’ll see how to control the rate by passing an argument to the decorator.
Registering Plugins
Decorators don’t have to wrap the function they’re decorating. They can also simply register that a function exists and return it unwrapped. This can be used, for instance, to create a light-weight plug-in architecture:
import random PLUGINS = dict() def register(func): """Register a function as a plug-in""" PLUGINS[func.__name__] = func return func @register def say_hello(name): return f"Hello {name}" @register def be_awesome(name): return f"Yo {name}, together we are the awesomest!" def randomly_greet(name): greeter, greeter_func = random.choice(list(PLUGINS.items())) print(f"Using {greeter!r}") return greeter_func(name)
The @register decorator simply stores a reference to the decorated function in the global PLUGINS dict. Note that you do not have to write an inner function or use @functools.wraps in this example because you are returning the original function unmodified.
The randomly_greet() function randomly chooses one of the registered functions to use. Note that the PLUGINS dictionary already contains references to each function object that is registered as a plugin:
>>> PLUGINS {'say_hello': <function say_hello at 0x7f768eae6730>, 'be_awesome': <function be_awesome at 0x7f768eae67b8>} >>> randomly_greet("Alice") Using 'say_hello' 'Hello Alice'
The main benefit of this simple plugin architecture is that you do not need to maintain a list of which plugins exist. That list is created when the plugins register themselves. This makes it trivial to add a new plugin: just define the function and decorate it with @register.
If you are familiar with globals() in Python, you might see some similarities to how the plugin architecture works. globals() gives access to all global variables in the current scope, including your plugins:
>>> globals() {..., # Lots of variables not shown here. 'say_hello': <function say_hello at 0x7f768eae6730>, 'be_awesome': <function be_awesome at 0x7f768eae67b8>, 'randomly_greet': <function randomly_greet at 0x7f768eae6840>}
Using the @register decorator, you can create your own curated list of interesting variables, effectively hand-picking some functions from globals().
Is the User Logged In?
The final example before moving on to some fancier decorators is commonly used when working with a web framework. In this example, we are using Flask to set up a /secret web page that should only be visible to users that are logged in or otherwise authenticated:
from flask import Flask, g, request, redirect, url_for import functools app = Flask(__name__) def login_required(func): """Make sure user is logged in before proceeding""" @functools.wraps(func) def wrapper_login_required(*args, **kwargs): if g.user is None: return redirect(url_for("login", next=request.url)) return func(*args, **kwargs) return wrapper_login_required @app.route("/secret") @login_required def secret(): ...
While this gives an idea about how to add authentication to your web framework, you should usually not write these types of decorators yourself. For Flask, you can use the Flask-Login extension instead, which adds more security and functionality.
Fancy Decorators
So far, you’ve seen how to create simple decorators. You already have a pretty good understanding of what decorators are and how they work. Feel free to take a break from this article to practice everything you’ve learned.
In the second part of this tutorial, we’ll explore more advanced features, including how to use the following:
Decorating Classes
There are two different ways you can use decorators on classes. The first one is very close to what you have already done with functions: you can decorate the methods of a class. This was one of the motivations for introducing decorators back in the day.
Some commonly used decorators that are even built-ins in Python are @classmethod, @staticmethod, and @property. The @classmethod and @staticmethod decorators are used to define methods inside a class namespace that are not connected to a particular instance of that class. The @property decorator is used to customize getters and setters for class attributes. Expand the box below for an example using these decorators.
The following definition of a Circle class uses the @classmethod, @staticmethod, and @property decorators:
class Circle: def __init__(self, radius): self._radius = radius @property def radius(self): """Get value of radius""" return self._radius @radius