I am the creator of Dependency Injector . This is a dependency injection framework for Python.
This is the definitive guide to building applications with the Dependency Injector. Past tutorials cover how to build a web application with Flask , REST API with Aiohttp, and monitoring a daemon with Asyncio using dependency injection.
Today I want to show how you can build a console (CLI) application.
Additionally, I have prepared answers to frequently asked questions and will publish their postscript.
The manual consists of the following parts:
- What are we going to build?
- Preparing the environment
- Project structure
- Installing dependencies
- Fixtures
- Container
- Working with csv
- Working with sqlite
- Provider Selector
- Tests
- Conclusion
- PS: questions and answers
The completed project can be found on Github .
To start you must have:
- Python 3.5+
- Virtual environment
And it is desirable to have a general understanding of the principle of dependency injection.
What are we going to build?
We will be building a CLI (console) application that looks for movies. Let's call it Movie Lister.
How does Movie Lister work?
- We have a database of films
- The following information is known about each film:
- Name
- Year of issue
- Director's name
- The database is distributed in two formats:
- Csv file
- Sqlite database
- The application searches the database using the following criteria:
- Director's name
- Year of issue
- Other database formats may be added in the future
Movie Lister is a sample application used in Martin Fowler 's article on dependency injection and inversion of control.
This is how the class diagram of the Movie Lister application looks like:
The responsibilities between the classes are distributed as follows:
MovieLister- responsible for the searchMovieFinder- responsible for extracting data from the databaseMovie- entity class "movie"
Preparing the environment
Let's start by preparing the environment.
First of all, we need to create a project folder and a virtual environment:
mkdir movie-lister-tutorial
cd movie-lister-tutorial
python3 -m venv venv
Now let's activate the virtual environment:
. venv/bin/activate
The environment is ready. Now let's get into the structure of the project.
Project structure
In this section, we will organize the structure of the project.
Let's create the following structure in the current folder. Leave all files empty for now.
Initial structure:
./
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ └── containers.py
├── venv/
├── config.yml
└── requirements.txt
Installing dependencies
It's time to install the dependencies. We will use packages like this:
dependency-injector- dependency injection frameworkpyyaml- library for parsing YAML files, used to read the configpytest- testing frameworkpytest-cov- helper library for measuring code coverage by tests
Let's add the following lines to the file
requirements.txt:
dependency-injector
pyyaml
pytest
pytest-cov
And execute in the terminal:
pip install -r requirements.txt
Installation of dependencies is complete. Moving on to fixtures.
Fixtures
In this section, we will add fixtures. Test data are called fixtures.
We will create a script that will create test databases.
Add a directory
data/to the root of the project and add a file inside fixtures.py:
./
├── data/
│ └── fixtures.py
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ └── containers.py
├── venv/
├── config.yml
└── requirements.txt
Next, edit
fixtures.py:
"""Fixtures module."""
import csv
import sqlite3
import pathlib
SAMPLE_DATA = [
('The Hunger Games: Mockingjay - Part 2', 2015, 'Francis Lawrence'),
('Rogue One: A Star Wars Story', 2016, 'Gareth Edwards'),
('The Jungle Book', 2016, 'Jon Favreau'),
]
FILE = pathlib.Path(__file__)
DIR = FILE.parent
CSV_FILE = DIR / 'movies.csv'
SQLITE_FILE = DIR / 'movies.db'
def create_csv(movies_data, path):
with open(path, 'w') as opened_file:
writer = csv.writer(opened_file)
for row in movies_data:
writer.writerow(row)
def create_sqlite(movies_data, path):
with sqlite3.connect(path) as db:
db.execute(
'CREATE TABLE IF NOT EXISTS movies '
'(title text, year int, director text)'
)
db.execute('DELETE FROM movies')
db.executemany('INSERT INTO movies VALUES (?,?,?)', movies_data)
def main():
create_csv(SAMPLE_DATA, CSV_FILE)
create_sqlite(SAMPLE_DATA, SQLITE_FILE)
print('OK')
if __name__ == '__main__':
main()
Now let's execute in the terminal:
python data/fixtures.py
The script should output
OKon success.
We verify that the files
movies.csvand movies.dbappeared in the directory data/:
./
├── data/
│ ├── fixtures.py
│ ├── movies.csv
│ └── movies.db
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ └── containers.py
├── venv/
├── config.yml
└── requirements.txt
Fixtures are created. Let's continue.
Container
In this section, we will add the main part of our application - the container.
The container allows you to describe the structure of the application in a declarative style. It will contain all application components and their dependencies. All dependencies will be specified explicitly. Providers are used to add application components to the container. Providers control the lifetime of components. When creating a provider, no component is created. We tell the provider how to create the object, and it will create it as soon as necessary. If the dependency of one provider is another provider, then it will be called and so on along the chain of dependencies.
Let's edit
containers.py:
"""Containers module."""
from dependency_injector import containers
class ApplicationContainer(containers.DeclarativeContainer):
...
The container is still empty. We will add providers in the next sections.
Let's add another function
main(). Her responsibility is to run the application. For now, she will only create a container.
Let's edit
__main__.py:
"""Main module."""
from .containers import ApplicationContainer
def main():
container = ApplicationContainer()
if __name__ == '__main__':
main()
The container is the first object in the application. It is used to get all other objects.
Working with csv
Now let's add everything we need to work with csv files.
We need:
- The essence
Movie - Base class
MovieFinder - Its implementation
CsvMovieFinder - Class
MovieLister
After adding each component, we will add it to the container.
Create a file
entities.pyin a package movies:
./
├── data/
│ ├── fixtures.py
│ ├── movies.csv
│ └── movies.db
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ ├── containers.py
│ └── entities.py
├── venv/
├── config.yml
└── requirements.txt
and add the following lines inside:
"""Movie entities module."""
class Movie:
def __init__(self, title: str, year: int, director: str):
self.title = str(title)
self.year = int(year)
self.director = str(director)
def __repr__(self):
return '{0}(title={1}, year={2}, director={3})'.format(
self.__class__.__name__,
repr(self.title),
repr(self.year),
repr(self.director),
)
Now we need to add a factory
Movieto the container. For this we need a module providersfrom dependency_injector.
Let's edit
containers.py:
"""Containers module."""
from dependency_injector import containers, providers
from . import entities
class ApplicationContainer(containers.DeclarativeContainer):
movie = providers.Factory(entities.Movie)
Don't forget to remove the ellipsis ( ...). The container already contains providers and is no longer needed.
Let's move on to creating
finders.
Create a file
finders.pyin a package movies:
./
├── data/
│ ├── fixtures.py
│ ├── movies.csv
│ └── movies.db
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ ├── containers.py
│ ├── entities.py
│ └── finders.py
├── venv/
├── config.yml
└── requirements.txt
and add the following lines inside:
"""Movie finders module."""
import csv
from typing import Callable, List
from .entities import Movie
class MovieFinder:
def __init__(self, movie_factory: Callable[..., Movie]) -> None:
self._movie_factory = movie_factory
def find_all(self) -> List[Movie]:
raise NotImplementedError()
class CsvMovieFinder(MovieFinder):
def __init__(
self,
movie_factory: Callable[..., Movie],
path: str,
delimiter: str,
) -> None:
self._csv_file_path = path
self._delimiter = delimiter
super().__init__(movie_factory)
def find_all(self) -> List[Movie]:
with open(self._csv_file_path) as csv_file:
csv_reader = csv.reader(csv_file, delimiter=self._delimiter)
return [self._movie_factory(*row) for row in csv_reader]
Now let's add
CsvMovieFinderto the container.
Let's edit
containers.py:
"""Containers module."""
from dependency_injector import containers, providers
from . import finders, entities
class ApplicationContainer(containers.DeclarativeContainer):
config = providers.Configuration()
movie = providers.Factory(entities.Movie)
csv_finder = providers.Singleton(
finders.CsvMovieFinder,
movie_factory=movie.provider,
path=config.finder.csv.path,
delimiter=config.finder.csv.delimiter,
)
You
CsvMovieFinderhave a dependency on the factory Movie. CsvMovieFinderneeds a factory as it will create objects Movieas it reads data from a file. In order to pass the factory, we use the attribute .provider. This is called provider delegation. If we specify a factory movieas a dependency, it will be called when it csv_finderis created CsvMovieFinderand an object will be passed as an injection Movie. Using the attribute .provideras an injection will be passed by the provider itself.
It
csv_finderalso has a dependency on several configuration options. We've added a provider onfigurationto pass these dependencies.
We used the configuration parameters before setting their values. This is the principle by which the provider worksConfiguration.
First we use, then we set the values.
Now let's add the configuration values.
Let's edit
config.yml:
finder:
csv:
path: "data/movies.csv"
delimiter: ","
The values are set to the config file. Let's update the function
main()to indicate its location.
Let's edit
__main__.py:
"""Main module."""
from .containers import ApplicationContainer
def main():
container = ApplicationContainer()
container.config.from_yaml('config.yml')
if __name__ == '__main__':
main()
Let's go to
listers.
Create a file
listers.pyin a package movies:
./
├── data/
│ ├── fixtures.py
│ ├── movies.csv
│ └── movies.db
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ ├── containers.py
│ ├── entities.py
│ ├── finders.py
│ └── listers.py
├── venv/
├── config.yml
└── requirements.txt
and add the following lines inside:
"""Movie listers module."""
from .finders import MovieFinder
class MovieLister:
def __init__(self, movie_finder: MovieFinder):
self._movie_finder = movie_finder
def movies_directed_by(self, director):
return [
movie for movie in self._movie_finder.find_all()
if movie.director == director
]
def movies_released_in(self, year):
return [
movie for movie in self._movie_finder.find_all()
if movie.year == year
]
We update
containers.py:
"""Containers module."""
from dependency_injector import containers, providers
from . import finders, listers, entities
class ApplicationContainer(containers.DeclarativeContainer):
config = providers.Configuration()
movie = providers.Factory(entities.Movie)
csv_finder = providers.Singleton(
finders.CsvMovieFinder,
movie_factory=movie.provider,
path=config.finder.csv.path,
delimiter=config.finder.csv.delimiter,
)
lister = providers.Factory(
listers.MovieLister,
movie_finder=csv_finder,
)
All components are created and added to the container.
Finally, we update the function
main().
Let's edit
__main__.py:
"""Main module."""
from .containers import ApplicationContainer
def main():
container = ApplicationContainer()
container.config.from_yaml('config.yml')
lister = container.lister()
print(
'Francis Lawrence movies:',
lister.movies_directed_by('Francis Lawrence'),
)
print(
'2016 movies:',
lister.movies_released_in(2016),
)
if __name__ == '__main__':
main()
Everything is ready. Now let's launch the application.
Let's execute in the terminal:
python -m movies
You will see:
Francis Lawrence movies: [Movie(title='The Hunger Games: Mockingjay - Part 2', year=2015, director='Francis Lawrence')]
2016 movies: [Movie(title='Rogue One: A Star Wars Story', year=2016, director='Gareth Edwards'), Movie(title='The Jungle Book', year=2016, director='Jon Favreau')]
Our application works with a database of movies in
csv. We also need to add format support sqlite. We'll deal with this in the next section.
Working with sqlite
In this section, we will add another type
MovieFinder- SqliteMovieFinder.
Let's edit
finders.py:
"""Movie finders module."""
import csv
import sqlite3
from typing import Callable, List
from .entities import Movie
class MovieFinder:
def __init__(self, movie_factory: Callable[..., Movie]) -> None:
self._movie_factory = movie_factory
def find_all(self) -> List[Movie]:
raise NotImplementedError()
class CsvMovieFinder(MovieFinder):
def __init__(
self,
movie_factory: Callable[..., Movie],
path: str,
delimiter: str,
) -> None:
self._csv_file_path = path
self._delimiter = delimiter
super().__init__(movie_factory)
def find_all(self) -> List[Movie]:
with open(self._csv_file_path) as csv_file:
csv_reader = csv.reader(csv_file, delimiter=self._delimiter)
return [self._movie_factory(*row) for row in csv_reader]
class SqliteMovieFinder(MovieFinder):
def __init__(
self,
movie_factory: Callable[..., Movie],
path: str,
) -> None:
self._database = sqlite3.connect(path)
super().__init__(movie_factory)
def find_all(self) -> List[Movie]:
with self._database as db:
rows = db.execute('SELECT title, year, director FROM movies')
return [self._movie_factory(*row) for row in rows]
Add the provider
sqlite_finderto the container and specify it as a dependency for the provider lister.
Let's edit
containers.py:
"""Containers module."""
from dependency_injector import containers, providers
from . import finders, listers, entities
class ApplicationContainer(containers.DeclarativeContainer):
config = providers.Configuration()
movie = providers.Factory(entities.Movie)
csv_finder = providers.Singleton(
finders.CsvMovieFinder,
movie_factory=movie.provider,
path=config.finder.csv.path,
delimiter=config.finder.csv.delimiter,
)
sqlite_finder = providers.Singleton(
finders.SqliteMovieFinder,
movie_factory=movie.provider,
path=config.finder.sqlite.path,
)
lister = providers.Factory(
listers.MovieLister,
movie_finder=sqlite_finder,
)
The provider
sqlite_finderhas a dependency on configuration options that we haven't defined yet. Let's update the configuration file:
Edit
config.yml:
finder:
csv:
path: "data/movies.csv"
delimiter: ","
sqlite:
path: "data/movies.db"
Done. Let's check.
We execute in the terminal:
python -m movies
You will see:
Francis Lawrence movies: [Movie(title='The Hunger Games: Mockingjay - Part 2', year=2015, director='Francis Lawrence')]
2016 movies: [Movie(title='Rogue One: A Star Wars Story', year=2016, director='Gareth Edwards'), Movie(title='The Jungle Book', year=2016, director='Jon Favreau')]
Our application supports both database formats:
csvand sqlite. Every time we need to change the format, we have to change the code in the container. We will improve this in the next section.
Provider Selector
In this section, we will make our application more flexible.
You will no longer need to make changes in the code to switch between
csvand sqliteformats. We'll implement a switch based on an environment variable MOVIE_FINDER_TYPE:
- When an
MOVIE_FINDER_TYPE=csvapplication uses thecsv. - When an
MOVIE_FINDER_TYPE=sqliteapplication uses thesqlite.
The provider will help us with this
Selector. It chooses a provider based on the configuration option ( documentation ).
Let's edit
containers.py:
"""Containers module."""
from dependency_injector import containers, providers
from . import finders, listers, entities
class ApplicationContainer(containers.DeclarativeContainer):
config = providers.Configuration()
movie = providers.Factory(entities.Movie)
csv_finder = providers.Singleton(
finders.CsvMovieFinder,
movie_factory=movie.provider,
path=config.finder.csv.path,
delimiter=config.finder.csv.delimiter,
)
sqlite_finder = providers.Singleton(
finders.SqliteMovieFinder,
movie_factory=movie.provider,
path=config.finder.sqlite.path,
)
finder = providers.Selector(
config.finder.type,
csv=csv_finder,
sqlite=sqlite_finder,
)
lister = providers.Factory(
listers.MovieLister,
movie_finder=finder,
)
We created a provider
finderand specified it as a dependency for the provider lister. The provider finderchooses between providers csv_finderand sqlite_finderat runtime. The choice depends on the value of the switch.
The switch is the configuration option
config.finder.type. When its value is csvused by the provider from the key csv. Likewise for sqlite.
Now we need to read the value
config.finder.typefrom the environment variable MOVIE_FINDER_TYPE.
Let's edit
__main__.py:
"""Main module."""
from .containers import ApplicationContainer
def main():
container = ApplicationContainer()
container.config.from_yaml('config.yml')
container.config.finder.type.from_env('MOVIE_FINDER_TYPE')
lister = container.lister()
print(
'Francis Lawrence movies:',
lister.movies_directed_by('Francis Lawrence'),
)
print(
'2016 movies:',
lister.movies_released_in(2016),
)
if __name__ == '__main__':
main()
Done.
Run the following commands in the terminal:
MOVIE_FINDER_TYPE=csv python -m movies
MOVIE_FINDER_TYPE=sqlite python -m movies
The output for each command will look like this:
Francis Lawrence movies: [Movie(title='The Hunger Games: Mockingjay - Part 2', year=2015, director='Francis Lawrence')]
2016 movies: [Movie(title='Rogue One: A Star Wars Story', year=2016, director='Gareth Edwards'), Movie(title='The Jungle Book', year=2016, director='Jon Favreau')]
In this section, we got acquainted with the provider
Selector. With this provider, you can make your application more flexible. The switch value can be set from any source: configuration file, dictionary, other provider.
Hint:
Overriding a configuration value from another provider allows you to implement configuration overloading in your application without a hot restart.
To do this, you need to use provider delegation and the.override().
In the next section, we'll add some tests.
Tests
Finally, let's add some tests.
Create a file
tests.pyin a package movies:
./
├── data/
│ ├── fixtures.py
│ ├── movies.csv
│ └── movies.db
├── movies/
│ ├── __init__.py
│ ├── __main__.py
│ ├── containers.py
│ ├── entities.py
│ ├── finders.py
│ ├── listers.py
│ └── tests.py
├── venv/
├── config.yml
└── requirements.txt
and add the following lines to it:
"""Tests module."""
from unittest import mock
import pytest
from .containers import ApplicationContainer
@pytest.fixture
def container():
container = ApplicationContainer()
container.config.from_dict({
'finder': {
'type': 'csv',
'csv': {
'path': '/fake-movies.csv',
'delimiter': ',',
},
'sqlite': {
'path': '/fake-movies.db',
},
},
})
return container
def test_movies_directed_by(container):
finder_mock = mock.Mock()
finder_mock.find_all.return_value = [
container.movie('The 33', 2015, 'Patricia Riggen'),
container.movie('The Jungle Book', 2016, 'Jon Favreau'),
]
with container.finder.override(finder_mock):
lister = container.lister()
movies = lister.movies_directed_by('Jon Favreau')
assert len(movies) == 1
assert movies[0].title == 'The Jungle Book'
def test_movies_released_in(container):
finder_mock = mock.Mock()
finder_mock.find_all.return_value = [
container.movie('The 33', 2015, 'Patricia Riggen'),
container.movie('The Jungle Book', 2016, 'Jon Favreau'),
]
with container.finder.override(finder_mock):
lister = container.lister()
movies = lister.movies_released_in(2015)
assert len(movies) == 1
assert movies[0].title == 'The 33'
Now let's start testing and check the coverage:
pytest movies/tests.py --cov=movies
You will see:
platform darwin -- Python 3.8.3, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
plugins: cov-2.10.0
collected 2 items
movies/tests.py .. [100%]
---------- coverage: platform darwin, python 3.8.3-final-0 -----------
Name Stmts Miss Cover
------------------------------------------
movies/__init__.py 0 0 100%
movies/__main__.py 10 10 0%
movies/containers.py 9 0 100%
movies/entities.py 7 1 86%
movies/finders.py 26 13 50%
movies/listers.py 8 0 100%
movies/tests.py 24 0 100%
------------------------------------------
TOTAL 84 24 71%
We used the.override()provider methodfinder. The provider is overridden by mock. When contacting the provider, thefinderoverriding mock will now be returned.
The work is done. Now let's summarize.
Conclusion
We built a CLI application using the dependency injection principle. We used Dependency Injector as a dependency injection framework.
The advantage you get with Dependency Injector is container.
The container starts to pay off when you need to understand or change the structure of your application. With a container, this is easy because all the components of the application and their dependencies are explicitly defined in one place:
"""Containers module."""
from dependency_injector import containers, providers
from . import finders, listers, entities
class ApplicationContainer(containers.DeclarativeContainer):
config = providers.Configuration()
movie = providers.Factory(entities.Movie)
csv_finder = providers.Singleton(
finders.CsvMovieFinder,
movie_factory=movie.provider,
path=config.finder.csv.path,
delimiter=config.finder.csv.delimiter,
)
sqlite_finder = providers.Singleton(
finders.SqliteMovieFinder,
movie_factory=movie.provider,
path=config.finder.sqlite.path,
)
finder = providers.Selector(
config.finder.type,
csv=csv_finder,
sqlite=sqlite_finder,
)
lister = providers.Factory(
listers.MovieLister,
movie_finder=finder,
)
A container as a map of your application. You always know what depends on what.
PS: questions and answers
In the comments to the previous tutorial, cool questions were asked: "why is this necessary?", "Why do we need a framework?", "How does the framework help in implementation?"
I have prepared answers:
What is dependency injection?
- it is the principle that reduces coupling and increases cohesion
Why should I use dependency injection?
- your code becomes more flexible, understandable and better testable
- you have fewer problems when you need to understand how it works or change it
How do I start applying dependency injection?
- you start writing code following the dependency injection principle
- you register all components and their dependencies in the container
- when you need a component, you get it from the container
Why do I need a framework for this?
- you need a framework in order not to create your own. The object creation code will be duplicated and difficult to change. To avoid this, you need a container.
- the framework gives you a container and providers
- providers control the lifetime of objects. You will need factories, singletons and config objects
- the container serves as a collection of providers
What price am I paying?
- you need to explicitly specify dependencies in the container
- this is additional work
- it will start paying dividends when the project starts to grow
- or 2 weeks after its completion (when you forget what decisions you made and what is the structure of the project)
Dependency Injector concept
In addition, I will describe the concept of Dependency Injector as a framework.
Dependency Injector is based on two principles:
- Explicit is better than implicit (PEP20).
- Don't do any magic with your code.
How is Dependency Injector different from other frameworks?
- No automatic linking. The framework does not automatically link dependencies. Introspection, linking by argument names and / or types is not used. Because "explicit is better than implicit (PEP20)".
- Doesn't pollute your application code. Your application is unaware of and independent of the Dependency Injector. No
@injectdecorators, annotations, patching or other magic tricks.
Dependency Injector offers a simple contract:
- You show the framework how to collect objects
- The framework collects them
The strength of the Dependency Injector lies in its simplicity and straightforwardness. It is a simple tool to implement a powerful principle.
What's next?
If you are interested, but hesitate, my recommendation is this:
Try this approach for 2 months. He's not intuitive. It takes time to get used to and feel. The benefits become tangible when the project grows to 30+ components in a container. If you don’t like it, don’t lose much. If you like it, get a significant advantage.
- Learn more about Dependency Injector on GitHub
- Check out the documentation at Read the Docs
I would be glad to receive feedback and answer questions in the comments.