Anyone with some experience of writing Python codes should be familiar with statements like import x or some variation of it. This is the Python's way of enabling its users use additional functionalities that come in the form of modules and functions. In this post, we will try to understand Python Modules.
Things that'll be discussed:
- What and Why?
- What is a module?
- Why using module is useful
- Understand a namespace
- How to Use a Module?
- Built in Modules
- What's Next?
What and Why?¶
Soon you start coding you will notice that your code tend to grow. An effective way of managing a growing code base is to decompose the code into smaller, manageable blocks. So that these blocks can not only run separately but also work together as part of a larger project. Module is a way how we can organize our independent code blocks.
"A module is a file containing Python definitions and statements." - Source: Python Tutorial
For example, let's assume that you have created re-usable functions that you regularly use in different projects. The non-modular way of using them would be either,
- Directly copying and pasting the functions in every project, or
- Saving them in a script and run the script during the project. For example, one could use magic command
%runto execute such a script.
The first alternative has the obvious limitation of repeated works. While the second alternative is a better idea but comes with these following limitations:
- By executing the script, you essentially create all the variables and functions available inside the script. Thus, you load your workspace with all the components of that script even if you needed only a select few.
- Also, the objects created will occupy the namespace and may lead to name conflict or unintended consequences. More on namespace later.
- Third, in extreme cases, it may expose you to a risk, especially if you are working on an open project or using someone else's script. If you execute such a script blindly, you may end up running some malicious code that can expose you to someone's ill intentions.
Using module gives us a way to overcome these limitations. We will demonstrate how using a module helps us overcome these limitations in the next section while we will be discussing about how to use a module.
💡 A Note about Namespace
We talked about the term - namespace. In simple terms, namespace is a memory space allocated in the computer where object names exist. Inside a namespace, names are unique inside a namespace. If we consider the namespaces as groups, then different groups may have members with the same names but inside a group there won't be any duplicate names. Using modules is a great way to keep the namespace separate and clean.
When a module is used properly, Python makes all the names from the module available but doesn't make them available to the code's namespace.
Demonstration in the following section should make it clear.
How to Use a Module?¶
Let's first create a Python script called messingAround.py and save it in the same directory where we will run our Python program. This will be the home for our functions and variable. To start off, let's define one variable - pi = 6.38, and one function - makeHalf(), inside messingAround.py. So the script will look like this:
# messingAround module
pi = 6.38
def makeHalf(val):
return val/2
Ways to Import¶
To make a module available, we must import it using the keyword import. There are different ways to import modules. Using messingAround as an example, we will see different ways of importing modules:
- We can import the entire module by simply calling
import messingAround.- This will make all the components of the module available to call. But won't load them as Python objects.
- We can also call multiple modules in one line like
import <module01> <module02>.
- We can import specific component from a module. In our example, we could load
pifrommessingAroundmodule by callingfrom messingAround import pi.- We can also import multiple components from the same module using the single line by adding the component names sepatated by comma:
from module01 import component01, component02.
- We can also import multiple components from the same module using the single line by adding the component names sepatated by comma:
- We can give aliases to either the module or components while we import them:
- Importing
messingAroundasma:import messingAround as ma. - Importing component01 as c01, component02 as c02:
from module01 import component01 as c01, component02 as c02. - 🚨 Remember, if you give aliases while importing you will have to call them by the alias. Calling them by their actual name wouldn't work anymore.
- Importing
- We can load everything from a module by calling
from messingAround import *.- 🚨 There is not likely any case where you want to do that. This is essentially the same thing as executing a script and comes with all the limitations that we discuss earlier.
Demo¶
We will use a build-in Python module called math along with our practice module messingAround for the demonstration purpose. These demonstrations will show how to use modules and how they help us avoid namespace complications.
Importing module and using its components¶
import messingAround as ma
import math as m
pi = m.pi * 3
print("My pi value = {} \nmessingAround's pi value = {} \nMath module's pi value = {}".format(pi, ma.pi, m.pi))
My pi value = 9.42477796076938 messingAround's pi value = 6.28 Math module's pi value = 3.141592653589793
Math module comes with a variable called pi that contains the value for π. We also have a variable with the same name but different value in our module. On top of that we have created a custom pi variable using the pi variable from math module.
Notice, how using a dot notation we have called both the pi variables from the modules along with our custom pi variable and didn't end up in a namespace conflict/confusion - pi variables from different sources retained their values.
Using magic commands and import *¶
On the contrary, consider the following example.
# import *
from messingAround import *
import math as m
print("pi value = {} \nMath module's pi value = {}".format(pi, m.pi))
print()
pi = m.pi * 3
print("pi value = {} \nMath module's pi value = {}".format(pi, m.pi))
pi value = 6.28 Math module's pi value = 3.141592653589793 pi value = 9.42477796076938 Math module's pi value = 3.141592653589793
Notice, how the value of pi keeps changing!
- As soon as we loaded everything from
messingAroundmodule, a variable calledpiwas created in our workspace - checkout the printed out value of 6.28. - But as soon as we created a new variable and gave it the same name -
pi, the original variable loaded frommessingAroundmodule was replaced by this new variable. Check how the value forpichanged later. - While notice how the value for
pivariable frommathmodule stays the same.
🛑 Replace this line from messingAround import * with %run -i messingAround.py. Are the results same? Think about 'Why'?
Peeping Inside a Module¶
When modules get larger, we may want to check what's inside the module right after importing it. dir() allows us to do exactly that. dir(<module name/alias>) prints out the components in alphabetical order.
dir(ma)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'makeHalf', 'pi']
Notice, how the output listed our two components makeHalf and pi. For now, ignore the components with all the underscores. We will come to those when we will discuss about building a Python package.
Built in Modules and Python Module Index¶
When we install Python we get a bunch of modules along with it. If you consider Python as a car, then solely the Python program itself is like the base model and all the modules are like the add-ins/upgrades. All these modules along with the built in functions are called Python Standard Library.
To find the extensive list of Python modules you can check official Python Module Index site. A word of caution though,
Don't get overwhelmed by the sheer number of modules. Pick the ones that are relevant to your goal i.e. data science vs app dev and focus on them only.
What's Next?¶
Understanding modules is the first step of understanding Python Packages. Package is practically a collection of modules. In this post, we learned about modules, benefits of using modules, and how to use them. In a follow up post we learn how to organize our modules in a package.