Django 2.0+ path converters

Hello!



Routing in Django from the second version of the framework received a wonderful tool - converters. With the addition of this tool, it became possible not only to flexibly configure the parameters in the routes, but also to separate the areas of responsibility of the components.



My name is Alexander Ivanov, I am a mentor at Yandex.Practicum at the back -end development faculty and a lead developer at the Computer Modeling Laboratory. In this article, I'll walk you through Django's route converters and show you the benefits of using them. The first thing to start with is the limits of applicability:

1. Django version 2.0+;
2. registration of routes should be done with django.urls.path
   
   
   
   .

So, when a request arrives at the Django server, it first goes through the middleware chain, and then the URLResolver ( algorithm ) is turned on . The task of the latter is to find a suitable one in the list of registered routes.



For a substantive analysis, I propose to consider the following situation: there are several endpoints that should generate different reports for a certain date. Let's assume the endpoints look like this:

users/21/reports/2021-01-31/
teams/4/reports/2021-01-31/
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

What would the routes in urls.py



? For example, like this:

path('users/<id>/reports/<date>/', user_report, name='user_report'),
path('teams/<id>/reports/<date>/', team_report, name='team_report'),
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Each item in < >



is a request parameter and will be passed to the handler.

Important: the name of the parameter when registering the route and the name of the parameter in the handler must match.

Then each handler would have something like this (pay attention to type annotations):

def user_report(request, id: str, date: str):
   try:
       id = int(id)
       date = datetime.strptime(date, '%Y-%m-%d')
   except ValueError:
       raise Http404()
  
   # ...
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

But this is not a royal business - to copy-paste such a block of code for each handler. It is reasonable to move this code into an auxiliary function:

def validate_params(id: str, date: str) -> (int, datetime):
   try:
       id = int(id)
       date = datetime.strptime(date, '%Y-%m-%d')
   except ValueError:
       raise Http404('Not found')
   return id, date
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

And in each handler then there will be a simple call to this helper function:

def user_report(request, id: str, date: str):
   id, date = validate_params(id, date)
  
   # ...
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

In general, this is already digestible. The helper function will either return the correct parameters of the required types, or abort the handler. Everything seems to be fine.



But in fact, here's what I did: I shifted some of the responsibility for deciding whether this handler should run for this route or not, from the URLResolver to the handler itself. It turns out that URLResolver did its job poorly, and my handlers not only have to do useful work, but also decide if they should do it at all. This is a clear violation of the SOLID principle of sole responsibility . This will not work. We need to improve.

Standard converters

Django provides standard route converters . It is a mechanism for determining whether a portion of the route is appropriate or not by the URLResolver itself. A nice bonus: the converter can change the type of the parameter, which means that the type we need can immediately come to the handler, and not the string.



Converters are specified before the parameter name in the route, separated by a colon. In fact, all parameters have a converter, if it is not specified explicitly, then the converter is used by default str



.

Beware: some converters look like types in Python, so it might seem like they are normal casts, but they are not - for example, there are no standard converters float



or bool



. Later I will show you what a converter is.

After looking at the standard converters, it becomes obvious what to id



use the converter for int



:

path('users/<int:id>/reports/<date>/', user_report, name='user_report'),
path('teams/<int:id>/reports/<date>/', team_report, name='team_report'),
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

But what about the date? There is no standard converter for it.



You can, of course, dodge and do this:

'users/<int:id>/reports/<int:year>-<int:month>-<int:day>/'

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Indeed, some of the problems were eliminated, because now it is guaranteed that the date will be displayed in three numbers separated by hyphens. However, you still have to handle problem cases in the handler if the client sends an incorrect date, for example 2021-02-29 or 100-100-100 in general. This means that this option is not suitable.

We create our own converter

Django, in addition to standard converters, provides the ability to create your own converter and describe the conversion rules as you like.



To do this, you need to take two steps:

1. Describe the class of the converter.
2. Register the converter.

A converter class is a class with a certain set of attributes and methods described in the documentation (in my opinion, it is somewhat strange that the developers did not make a base abstract class). The requirements themselves:

1. There must be an attribute regex
   
   
   
   describing the regular expression to quickly find the required subsequence. I'll show you how it is used later.
2. Implement a method def to_python(self, value: str)
   
   
   
   for converting from a string (after all, the transmitted route is always a string) into a python object, which will eventually be passed to the handler.
3. Implement a method def to_url(self, value) -> str
   
   
   
   to convert back from a python object to a string (used when invoking django.urls.reverse
   
   
   
   or tagging url
   
   
   
   ).

The class for converting the date will look like this:

class DateConverter:
   regex = r'[0-9]{4}-[0-9]{2}-[0-9]{2}'

   def to_python(self, value: str) -> datetime:
       return datetime.strptime(value, '%Y-%m-%d')

   def to_url(self, value: datetime) -> str:
       return value.strftime('%Y-%m-%d')
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

I am against duplication, so I will put the date format in an attribute - it’s easier to maintain the converter if I suddenly want (or need) to change the date format:

class DateConverter:
   regex = r'[0-9]{4}-[0-9]{2}-[0-9]{2}'
   format = '%Y-%m-%d'

   def to_python(self, value: str) -> datetime:
       return datetime.strptime(value, self.format)

   def to_url(self, value: datetime) -> str:
       return value.strftime(self.format)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

The class is described, so it's time to register it as a converter. This is done very simply: in the function register_converter



you need to specify the described class and the name of the converter in order to use it in routes:

from django.urls import register_converter
register_converter(DateConverter, 'date')
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Now you can describe the routes in urls.py



(I deliberately changed the name of the parameter to dt



so as not to confuse the entry date:date



):

path('users/<int:id>/reports/<date:dt>/', user_report, name='user_report'),
path('teams/<int:id>/reports/<date:dt>/', team_report, name='team_report'),
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Now it is guaranteed that the handlers will be called only if the converter works correctly, which means that the parameters of the required type will come to the handler:

def user_report(request, id: int, dt: datetime):
   #     
   #      
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Looks awesome! And this is so, you can check.

Under the hood

If you look closely, an interesting question arises: nowhere is there a check that the date is correct. Yes, there is a regular season, but an incorrect date is also suitable for it, for example 2021-01-77, which means there to_python



must be an error in it. Why does it work?



About this I say: "Play by the rules of the framework, and it will play for you." Frameworks take on a number of common tasks. If the framework cannot do something, then a good framework provides an opportunity to expand its functionality. Therefore, you should not engage in bicycle building, it is better to see how the framework offers to improve its own capabilities.



Django has a routing subsystem with the ability to add converters that takes care of the method call to_python



and catching errors ValueError



.



Here is the code from the Django routing subsystem without changes (version 3.1, file django/urls/resolvers.py



, class RoutePattern



, method match



):

match = self.regex.search(path)
if match:
   # RoutePattern doesn't allow non-named groups so args are ignored.
   kwargs = match.groupdict()
   for key, value in kwargs.items():
       converter = self.converters[key]
       try:
           kwargs[key] = converter.to_python(value)
       except ValueError:
           return None
   return path[match.end():], (), kwargs
return None

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

The first step is to search for matches in the route transmitted from the client using a regular expression. The one regex



that is defined in the converter class is involved in the formation self.regex



, namely, it is substituted instead of the expression in angle brackets <>



in the route.



For example,

users/<int:id>/reports/<date:dt>/
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

turn into

^users/(?P<id>[0-9]+)/reports/(?P<dt>[0-9]{4}-[0-9]{2}-[0-9]{2})/$
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

In the end, just the same regular from DateConverter



.



This is a quick search, superficial. If no match is found, then the route is definitely not suitable, but if found, then it is a potentially suitable route. This means that you need to start the next stage of verification.



Each parameter has its own converter, which is used to call the method to_python



. And here's the most interesting thing: the call is to_python



wrapped in try/except



, and type errors are caught ValueError



. That is why the converter works even in the case of an incorrect date: an error falls ValueError



, and this is considered so that the route does not fit.



So in the case of DateConverter



, we can say, lucky: in case of an incorrect date, an error of the required type falls. If there is an error of another type, then Django will return a 500 response.

Don't stop

It seems that everything is fine, the converters are working, the necessary types immediately come to the handlers ... Or not right away?

path('users/<int:id>/reports/<date:dt>/', user_report, name='user_report'),
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

In the handler for generating a report, you probably need it User



, and not it id



(although this may be the case). In my hypothetical situation, just an object is needed to create a report User



. What then turns out, again twenty-five?

def user_report(request, id: int, dt: datetime):
   user = get_object_or_404(User, id=id)
  
   # ...
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Shifting responsibilities to the handler again.



But now it is clear what to do with it: write your own converter! It will make sure the object exists User



and will pass it to the handler.

class UserConverter:
   regex = r'[0-9]+'

   def to_python(self, value: str) -> User:
       try:
           return User.objects.get(id=value)
       except User.DoesNotExist:
           raise ValueError('not exists') #  ValueError

   def to_url(self, value: User) -> str:
       return str(value.id)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

After describing the class, I register it:

register_converter(UserConverter, 'user')
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Finally, I describe the route:

path('users/<user:u>/reports/<date:dt>/', user_report, name='user_report'),
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

That's better:

def user_report(request, u: User, dt: datetime):  
   # ...
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Converters for models can be used often, so it is convenient to make the base class of such a converter (at the same time, I added a check for the existence of all attributes):

class ModelConverter:
   regex: str = None
   queryset: QuerySet = None
   model_field: str = None

   def __init__(self):
       if None in (self.regex, self.queryset, self.model_field):
           raise AttributeError('ModelConverter attributes are not set')

   def to_python(self, value: str) -> models.Model:
       try:
           return self.queryset.get(**{self.model_field: value})
       except ObjectDoesNotExist:
           raise ValueError('not exists')

   def to_url(self, value) -> str:
       return str(getattr(value, self.model_field))

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Then the description of the new converter to the model will be reduced to a declarative description:

class UserConverter(ModelConverter):
   regex = r'[0-9]+'
   queryset = User.objects.all()
   model_field = 'id'

      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
    

Outcome

Route converters are a powerful mechanism that helps you make your code cleaner. But this mechanism appeared only in the second version of Django - before that we had to do without it. This is where auxiliary functions of the type get_object_or_404



came from; without this mechanism, cool libraries like DRF are made.



But this does not mean that converters should not be used at all. This means that (yet) it will not be possible to use them everywhere. But where possible, I urge you not to neglect them.



I will leave one caveat: here it is important not to overdo it and not drag the blanket in the other direction - you do not need to take the business logic into the converter. It is necessary to answer the question: if such a route is in principle impossible, then this is the converter's area of ​​responsibility; if such a route is possible, but under certain circumstances it is not processed, then this is already the responsibility of the handler, serializer or someone else, but definitely not the converter.



PS In practice, I have made and used only a converter for dates, just the one shown in the article, since I almost always use DRF or GraphQL. Tell us if you use route converters and, if so, which ones?