A higher-order function is one that either takes a function as an argument or returns a function (or both!). These functions help us achieve more complex tasks in our business logic that would be much more tedious with first-class functions.
Python allows us to define our own custom higher-order functions. In fact, I demonstrate this feature in my previous article on lambda functions in Python. I won’t be covering that particular feature in this article. However, I will be covering 3 of the most useful higher-order functions that are built into Python:
- Map
- Reduce
- Filter
These functions make dealing with iterable objects much simpler than using first-class functions or regular loops. For each function, I will explain what the function does, demonstrate how it works with a code example, and suggest possible applications for the function.
Map
The map function in Python allows us to produce a new iterable object from an original iterable object. It takes 2 arguments: a function and an iterable object. The map function applies the function passed to every member of the iterable and then produces a map object. This object can be cast to an iterable type of our choosing. The function passed to the map object does not have to return a value (I will demonstrate this later).
Implementation
For our example, let’s generate a new list (L2) from an initial list (L1) such that each element of L2 is a double of its corresponding element in L1. Here is how we can achieve this with the map function:
def double(x):
return x * 2
L1 = [3, 1, 4, 22, 45, 1, 245, 6, 34, 4, 8, 14]
L2 = list(map(double, L1))
print("Doubled Numbers: {}".format(L2))
The code above passes the double function(notice we leave out the parenthesis) and the original list (L1). A map object is returned and then cast to a list. Note, we can cast this map object to any type of iterable we want (list, tuple, set, …).
The map function also works with lambda functions. To make the code above a little more compact, we can rewrite it as follows:
L1 = [3, 1, 4, 22, 45, 1, 245, 6, 34, 4, 8, 14]
L2 = list(map( (lambda x : x \* 2) , L1))
print(L2)
This would be a more desirable solution if your map function is rather simple and will not be needed elsewhere in the program. However, if you’re running a complex operation on an iterable and/or plan to reuse the function outside of this context, it’s best to stick to regular functions.
Remember I mentioned that the function does not actually have to return anything. If you want to map certain functionality to every member of an iterable but don’t care for the resulting map object, the map function will simply return None for every corresponding member of the original iterable.
In fact, we don’t even have to store the returned map object at all. If we have a list of people and we simply want to print out a greeting for each person on the list, we can use a map function:
def greet(person):
print("Hello, {}!".format(person['name']))
people = [
{ 'name' : 'John Doe', 'age' : 20},
{ 'name' : 'Mike Will', 'age' : 10},
{ 'name' : 'Jane Hill', 'age' : 18},
{ 'name' : 'Alan Smith', 'age' : 16},
{ 'name' : 'Chimuka Chinene', 'age' : 18},
]
list(map(greet, people ))
""" Prints :
Hello, John Doe!
...
Hello, Chimuka Chinene!
"""
We have to cast the map object returned in order to actually get the function to execute on all the members. The list produced is just a list of None type objects as the mapper function did not return anything. If you assign that list to a variable and print it, the output would be: [None, None, None, None, None]
Uses
The most obvious use of a map function is in creating a hash table. A hash table is a data structure that associates a key with a hash that’s generated using the key. The map function could be used to apply the hash function to a list of keys without the need for iteration.
The map function could also be used to replace loops. Instead of looping through every member of an iterable and executing some logic, we could map a function with that logic to the iterable. Remember, we don’t have to return anything from the mapper function nor do we have to care for the resulting map object.
Reduce
A reduce function works similar to a map function in the sense that it accepts an iterable and a function to apply to all the members of an iterable. However, it differs from map as it returns ONE value instead of an iterable mapping to the original. It reduces an iterable down to one value. The reduce function also takes a third argument, the initial value that it starts the iteration with. If this value is not provided, the first element of the iterable will be used as the initial value. With every iteration, the function takes its current value, and the next member in the iterable and returns a value to be used for the next element.
Implementation
Let’s calculate the sum of the doubles of all numbers in an array of integers L1. We could use a loop for this. However, a cleaner solution would be to use a reduce function as follows: Note in Python 3, reduce has been moved to functools so you will have to import it.
from functools import reduce
def addDoubles(a, b):
return a + (b * 2)
L1 = [3, 1, 4, 22, 45, 1, 245, 6, 34, 4, 8, 14]
doubleSum = reduce(addDoubles, L1, 0)
print(doubleSum)
In the code above, the add function takes 2 parameters: the current sum(a) and the current element in the iterable(b) which is doubled. The total of the double and current sum is returned and carried forward into the next iteration until the end of the iterable. The reduce function also accepts lambda functions. We can simplify the code above as follows:
from functools import reduce
L1 = [3, 1, 4, 22, 45, 1, 245, 6, 34, 4, 8, 14]
doubleSum = reduce( (lambda a,b : a + b * 2) , L1, 0)
print(doubleSum)
This is a simple example, we can define a much more complex reducer function. We can also pick another initial value. For example, if I wanted my starting point to be 1000, I would write the code differently:
from functools import reduce
L1 = [3, 1, 4, 22, 45, 1, 245, 6, 34, 4, 8, 14]
sum = reduce( (lambda a,b : a + b * 2) , L1, 1000)
print(sum)
Uses
The reduce function can be used anywhere that an iterable needs to be aggregated to one value. We can define the reducer function any way we like.
Filter
The filter function is also similar to the map function. It accepts an iterable, and a function to execute on all the members of the iterable. However, it does come with some different rules:
- A filter does not have to produce a 1 to 1 mapping between the new iterable and the original one like a map function does.
- A filter creates a new list based on the elements that return True when the filter function is executed on them
Implementation
Given a list of 5 people, let’s filter out all the people that are younger than 18.
def legal(person):
return person['age'] >= 18
people = [
{ 'name' : 'John Doe', 'age' : 20},
{ 'name' : 'Mike Will', 'age' : 10},
{ 'name' : 'Jane Hill', 'age' : 18},
{ 'name' : 'Alan Smith', 'age' : 16},
{ 'name' : 'Chimuka Chinene', 'age' : 18},
]
legal_people = list(filter(legal, people))
print(legal_people)
""" Prints [{'name': 'John Doe', 'age': 20}, {'name': 'Jane Hill', 'age': 18}, #{'name': 'Chimuka Chinene', 'age': 18}] """
In the code above, we run the legal function on each person. The function returns either True or False depending on the person’s age. Only the people whose age returns True from the legal function make it into the new list. A filter object is returned. We can cast this to any iterable type we desire.
The filter function can also take a lambda function as an argument. We can write the code above as follows:
people = [
{ 'name' : 'John Doe', 'age' : 20},
{ 'name' : 'Mike Will', 'age' : 10},
{ 'name' : 'Jane Hill', 'age' : 18},
{ 'name' : 'Alan Smith', 'age' : 16},
{ 'name' : 'Chimuka Chinene', 'age' : 18},
]
legal_people = list(filter( (lambda person : person['age'] >= 18) , people))
print(legal_people)
""" Prints [{'name': 'John Doe', 'age': 20}, {'name': 'Jane Hill', 'age': 18}, #{'name': 'Chimuka Chinene', 'age': 18}] """
Uses
This one is pretty straightforward. We use the filter function in all instances where we want to filter out undesirable members of the original list. We can run this on very complex elements in an iterable and define complex filter functions to achieve whatever desired outcome we want. Just remember, the filter function has to return True or False.
Conclusion
Higher-order functions are an invaluable tool when writing programs, especially ones working on iterable objects. They are not necessary as their functionality can be achieved by regular for/while loops. However, they help a great deal in keeping our code neat and concise compared to regular loops. I hope I’ve provided a sufficient explanation of the functions above that will allow you to implement them in your projects. Happy coding!
