Throughout the course of June, the dgrid StackOverflow tag saw a series of questions regarding usage of dgrid and dstore with the Django REST Framework. Based on the flurry of closely-related questions that popped up, I became quite curious as to the actual level of difficulty involved with this integration.

Over the holiday weekend here in the US, I decided to let that curiosity get the better of me. In this blog post, I share the results of that endeavor, stepping through the process of customizing a simple Django Rest Framework project to communicate with dgrid using dstore’s Rest store.

Before we Begin

If you’d like to follow along with the steps in this post, you’re going to need Python, Django, and the Django REST Framework. On OS X and Linux, this can be accomplished by installing pip, then using it to install django and djangorestframework. e.g.:

sudo easy_install pip
sudo pip install django
sudo pip install djangorestframework

(Some Linux distributions may also provide packages for pip and django.)

Getting Started

We will start with a zip archive containing a very simple dgrid project, with an example application which defines a Person model and comes complete with initial data.

The zip includes the following notable files:

  • dgrid/ – Contains project files
    • settings.py – Settings for this Django project, initially modified to add 'rest_framework' and 'example' to INSTALLED_APPS
    • urls.py – Sets up routes for the project; uses the Django REST Framework’s DefaultRouter to register the /people/ route
  • example/ – Contains the example application’s files
    • fixtures/initial_data.json – Initial data for populating the Person table
    • models.py – Defines the Person model
    • serializers.py – Defines a Serializer for the Person model, to allow CRUD operations
    • views.py – Defines a ViewSet for the Person model, allowing full read/write access for this example
    • static/ – Contains HTML, CSS, and JS for displaying data from the /people/ endpoint in dgrid instances
      • dojoConfig.js – Common dojoConfig to load dgrid, dstore, etc. from MaxCDN (c/o RawGit)
      • createGrid.js – Common code to create a dstore/Rest store and a grid, based on a specified dgrid constructor
      • index.html and pagination.html – Pages which display items from the store using OnDemandGrid and Pagination, respectively

Extract the zip into a directory of your choosing, then open a terminal in that directory and run ./manage.py syncdb to initialize tables and pre-populate the Person table with data. (If it prompts you to create a superuser, you can skip that step – we don’t need it for this example.)

Next, to make sure the example model and viewset is working, run ./manage.py runserver and navigate a browser to people/. It may take a moment, but you should see a page listing all of the entries in the database table.

You can also view the two dgrid pages mentioned above by browsing to static/index.html and static/pagination.html.

Now it’s time for some good news and some bad news.

The good news is, we can already see data rendered in those grids! Great!

The bad news is, the data isn’t really being requested correctly. Take a closer look and you may notice a few things:

  • Clicking the headers to sort has no effect
  • If you examine the Pagination example, you’ll notice it says “0 – 0 of 0 results”
  • If you look in the Network tab of your browser’s developer tools, every response from /people/ includes all of the entries

What’s going on here? Well, we haven’t fully made ends meet yet. The Django REST Framework has certain default behaviors, and includes some useful features that aren’t enabled by default. Meanwhile, dstore/Rest (or more precisely, dstore/Request, from which it inherits) has certain expectations of how it should interact with the service and how the service should respond.

Let’s take a look at each side of this puzzle.

Customizing the Django REST Framework

Based on the above observations, we know we need to address a couple of issues with the service — namely, the ability to request ranges of data (rather than receiving the full set), and the ability to sort the data.

Pagination

The Django REST Framework documentation has a full page detailing the various pagination styles it supports. It also includes information and examples on creating customized pagination classes.

The decision of which pagination class to use is ultimately dependent on what fits the needs of dstore/Rest. Specifically, it needs to be able to indicate a start and count, and expects the response to indicate the total number of items either via the Content-Range header, or via the total property if the response is an object. When the response is an object, it also expects the actual results to be present under the items property.

Looking at the Django REST Framework documentation, the LimitOffsetPagination class seems to fit the criteria of passing a start and count (or, in its terms, offset and limit). However, in looking at the example, it’s clear that it reports the total via a count property, and the items via a results property.

So close, and yet so far away. That’s okay, though — we just need to extend the class that almost meets our needs, and nudge it the rest of the way there.

The framework’s documentation on custom pagination styles provides a helpful example, but the example extends PageNumberPagination, not LimitOffsetPagination. However, by looking at that example and the source code of LimitOffsetPagination, we can figure out what we need pretty quickly.

Let’s add our own custom pagination class to example/pagination.py:

from rest_framework.compat import OrderedDict
from rest_framework.pagination import LimitOffsetPagination
from rest_framework.response import Response

class DstorePagination(LimitOffsetPagination):
    def get_paginated_response(self, data):
        return Response(OrderedDict([
            ('total', self.count),
            ('next', self.get_next_link()),
            ('previous', self.get_previous_link()),
            ('items', data)
        ]))

If you think this looks a whole lot like LimitOffsetPagination‘s get_paginated_response function, you’re not far off the mark — the only difference is we’ve replaced count with total and results with items.

Now we just need to tell the framework to use our custom pagination class. As indicated in the documentation, this can be accomplished centrally by specifying DEFAULT_PAGINATION_CLASS in our application’s settings. We simply add the following at the end of dgrid/settings.py:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'example.pagination.DstorePagination',
    'PAGE_SIZE': 25
}

(This also adds PAGE_SIZE so that if an unranged query is performed, it won’t bother consuming resources and bandwidth returning the full set.)

Ordering (Sorting)

While we’re editing dgrid/settings.py, let’s also tackle the issue of sorting. The Django REST Framework documentation’s page on Filtering mentions an OrderingFilter, which seems like exactly what we need, so let’s add that to DEFAULT_FILTER_BACKENDS as indicated in the documentation.

At this point the REST_FRAMEWORK dictionary should look like this:

REST_FRAMEWORK = {
    'DEFAULT_PAGINATION_CLASS': 'example.pagination.DstorePagination',
    'DEFAULT_FILTER_BACKENDS': (
        'rest_framework.filters.OrderingFilter',
    ),
    'PAGE_SIZE': 25
}

Are we There Yet?

At this point if you restart your server and browse again to pagination.html, you should notice some improvements. The footer now reports the proper number of items. The response also no longer includes all of the results. However, it still includes 25 when we’re asking for 10, and sort still isn’t working.

That’s okay for now, because we’ve still yet to configure the dstore/Rest store. It’s time to hop over to the client side.

Customizing the dstore/Rest store

As with the server-side, we’ve got two areas to resolve: pagination and sorting. Fortunately, dstore/Request (which dstore/Rest inherits all of its querying logic from) provides useful options that will allow us to send the parameters that the service needs.

Pagination

By default, dstore/Request expresses its ranged queries via a limit(count,start) syntax. (You may have already noticed this while looking at network requests earlier.) However, as seen in the documentation linked above, we can set rangeStartParam and rangeCountParam to override this.

Looking again at the Django REST Framework’s LimitOffsetPagination documentation, it expects offset and limit query parameters by default, so let’s set the store’s options accordingly.

Open example/static/createGrid.js and add the following properties to the object passed to the store’s constructor (around line 10):

rangeStartParam: 'offset',
rangeCountParam: 'limit'

Sorting

You may have already noticed sortParam in the dstore/Request documentation as well. This defaults to sort(+field) syntax, but when a value is specified, it will behave as a typical key/value pair.

Thinking back to the Django REST Framework’s OrderingFilter documentation, it expects an ordering query parameter by default, so let’s add that to the store’s constructor arguments as well.

sortParam: 'ordering'

However, there’s still a small issue — looking at the Django REST Framework examples, the sort field includes a - prefix to indicate descending sort, but does not include a prefix to indicate ascending sort. dstore/Request, on the other hand, defaults to specifying + or -. Fortunately, these are also overridable via ascendingPrefix and descendingPrefix properties, so all we need to do is set ascendingPrefix to an empty string:

ascendingPrefix: ''

Last but not Least

There’s one more very important thing we need to configure for this store. When you first looked at the results in the service response, you may have noticed that it provides a unique ID via the pk field. dstore stores expect unique IDs to reside in an id field by default, so we need to set idProperty to tell it otherwise.

With all this said and done, our store instantiation should now look like this:

var store = new TrackableRest({
    target: '/people/',
    idProperty: 'pk',
    rangeStartParam: 'offset',
    rangeCountParam: 'limit',
    sortParam: 'ordering',
    ascendingPrefix: ''
});

Going for a Test Drive

Now if you refresh index.html or pagination.html (you don’t need to restart the server since we only changed static assets since the last restart), scrolling/paging and sorting should function as expected!

Additionally, the grids in the example pages are set up with the dgrid/Editor mixin, configured so you can double-click any cell, edit the value, press Enter, and the new value will be sent to the server via a PUT request. This all Just Works by virtue of dstore/Rest‘s put implementation, which already operates the way the Django REST Framework expects.

In Conclusion

Let’s reflect back on what it took to get this working:

  • Thoroughly browsing the documentation for both the server-side framework and the client-side store, to see what each side needs and where we could meet in the middle
  • Piecing together some settings on both sides based on documentation and examples
  • Getting our hands a bit dirty to create a custom pagination class

All in all, this may have been a bit challenging, but the takeaway is that it was surely not an insurmountable task. Coming into this with no knowledge of Django or the Django REST Framework whatsoever (and very little practical Python experience), it took me roughly a day’s worth of work – 50% to get things to work, and the other 50% to reduce it down to the minimal example presented in this tutorial.

I’d wager there’s one more important takeaway from this — if you or your team is facing a tough JavaScript integration challenge, and the solution seems elusive even with help from the community, you don’t need to fight that battle alone. Drop us a line and let us provide a few hours of insight and support to prevent weeks of frustration and project delays.