Django Unchained… Literally…

The story of how we liberated our web application from the shackles of Django Framework into a new homemade of Create-React-App. Too dramatic??? I now have your attention. Read on…

Just as how for B2C users we have Goibibo, for B2B customers (hoteliers) we have In.Goibibo (Extranet), which is used to manage hotel property, inventory & pricing. The codebase for Extranet was a humongous Django setup which used to serve all the APIs, webapp & Admin views. So it was obvious that sooner or later we need to introduce some separation of concerns to breathe fresh air into each of the sub-entities.

• Having each of the sub-entities separate will not only help in planning, executing code changes faster but also make the process of debugging easier. Hence moving to a Service Oriented Architecture was the need of the hour.
• The website was also undergoing changes to be rewritten in React to make the code more maintainable and readable. But this was still being done in the Django setup which lags way far behind compared to the developer experience which newer JS frameworks provide.
• Also having the web app as a standalone project will let us easily introduce global level changes like a Router and help in performing perf optimizations easier.

Action Plan

This is what I do: - you decide what you want - you figure out what it would take to get there - you break that out into the smallest, most actionable bits you can, and schedule them - adjust along the way based on new information 🤷🏻‍♀️ that's really the model I work from

 — @sarah_edo

Since I was unaware of both the codebase & Django, and having never worked on Python; some initial time had to be spent in understanding how the code is setup and it is all connected. After spending about 2 days on understanding, I had figured out a rough idea of how the setup is and how the migration should proceed.

We chose to go with Create-React-App which I believe is the gold standard in terms of a dev environment for developing React apps. CRA in general is not customizable and for a good reason. But we were intending to maintain a fork with our customizations if needed. The new repo also comes with Prettier.

• The web app was a mix of Backbone + jQuery code forming the core and most of the UI, ReactJS being used in some of the tabs by directly mounting onto section elements.
• The ReactJS code will feel right at home in the new setup and would require very minimal changes.
• The old Backbone code was to be moved into the new setup and changed to adhere to modules standards so that it could be properly imported in the right order and bundled.
• The vendor JS library files were to be replaced with their node module equivalents so that we would gain from the Tree-Shaking ability of the new setup wherever possible.
• All the CSS along with assets like fonts, images etc were not to be disturbed.
• All the hierarchy of Django HTML templates was to be ported directly and hierarchy to be maintained.

Alright, I now have a step by step plan of what to do! Time to

I had a rough idea of how to execute everything in that plan, except for the last step. The web apps these days are just SPA, which means only one HTML and rest of the code in JS. But Backbone/jQuery code was reliant on a lot of HTML templates and this migration’s scope was to be limited to moving out of Django.

First Blocker

How to get all the HTML code which is a huge set of nested template files to get to work on Create-React-App which is primarily a setup for SPA??

• When I was digging into the CRA’s webpack config, the HTML loader included — html-webpack-plugin — ships a default loader which allows EJS like syntax to include dynamic stuff like webapp title.

• Upon digging further I noticed that it can support any JS since it is using lodash to process the templates. So I can throw in a require()call and load any HTML I want using html-loader.
• So adding html-loader as a dev dependency, I added a require statement for HTML resources which worked beautifully and thankfully CRA doesn’t block loader chains here.
• But there is one more problem here. Since we are using html-loader, I can’t nest another require inside the HTML, so deep level nesting of HTML was not possible with this setup.
• I could’ve gone for ejs-loader which would have allowed me to do all that, but I didn’t want to unnecessarily fork for this reason, specially since down the line, all the HTML files will vanish post complete migration to react.

So, we resolved our first blocker albeit with a little more work from our side to remove all the deep nesting of the HTML templates. Our index.html is now peppered with a lot of requires to HTML templates, in some cases I had to move the contents of a template file into the index.html so that I could require other set of HTML files.

Also I noticed an issue while doing this with the html-webpack-plugin. I filed an issue & PR for this. If you are interested you can read more here.

Next Gotchas

Once that is resolved, I migrated all the HTML files with minimal changes into the new repo. Now was time to move all the Backbone + jQuery code, along with all the CSS into the new repo. Without second thought, I moved all code into the src/ directory in the new repo, only to be bombarded with millions of lint errors/warnings. Then my sensible self reminded of something that I keep telling myself all the time

DRY KISS — Don’t Repeat Yourself & Keep It Simple Silly

I can’t go into another rabbit hole fixing all these issues now, the focus of this task was to just migrate. So I moved all the Backbone + jQuery code along with all the vendor JS files and CSS files under public/ folder, and included it directly in the HTML like before. Which also means that I have to setup production version of the same which combines multiple files into one and minifies and uglifies the code.

lodash templates in the index.html again to the rescue!!

Getting this to final shape took a little more work than I had imagined.

• First things first, I had to write a post_build.sh script which will combine, minify & uglify the files and remove the original ones from the build/ directory. This was already being done in the old Django setup, so copied the same script with some modifications.
• Then I noticed that the condition shown above was always evaluating to false, which upon spending a considerable amount of time, I found that it was happening because of the Create-React-App setup. The replacements for %SOME_VAR% stuff inside the index.html happens post generation of build/index.html as evident here.
• So I had to write a pre_build.sh script which will rewrite %NODE_ENV% as “production”. And set it back to %NODE_ENV% in the post build script. I didn’t think this was a generic use case for everyone, hence decided against filing an issue & PR for this on Create-React-App.
• Then I added a few more lines to the bash script to calculate the md5 hash of each of these combined files and rewrite the index.html in build/ directory to add the hash to the url. So /legacy_js/combined.min.js became /legacy_js/combined.min.<hash>.js
• Similar thing was done with our CSS as well.

Taking care of Django stuff

Through out the code, wherever the Django template syntax was used, it had to be migrated.

• Things commented out with {# #} were commented out using HTML comments <!-- -->
• Forms in which {% csrf_token %} was replaced with <input type=”hidden” name=”csrfmiddlewaretoken” />
• In some cases, the logic which was written in Django syntax, had to be written in lodash template syntax and made to work via _.template
• {%version <some file>%} which is used to version some file, was taken care by our new build process
• {% url … %} was replaced with proper URLs after going through the code.

Bringing it all together

With all this code logistics out of the way, it was now time to integrate all these individual migrated pieces and get it to work. So far into this migration, there was no output that could be checked. All these migrations was done under the hood and its results were yet to be seen.

But there was one more thing that had to be done, before we could start seeing the results. Django server-side renders (SSR) the initial HTML to inject some data before serving it to the client. So this data is something that is used throughout the code and is very critical to it’s functioning. So we had to tackle this.

We could have gone down the route of SSR for this too. But since the initial focus was limited to ripping the webapp out of Django, we decided to make an API for this and call it as the first thing, before letting any other code run. And once we get the response from this API, we put the data in all the right places on the client side where Django used to inject before.

Copy pasting the react code was the easiest part of the migration.

Some of the code had to be changed to wait until the data from the initial API call is available. And finally, all the marathon changes needed for the migration was complete!!

All is done! Time for celebrations!!!

For all the nerds, here are the file stats in the new repo

Production deployment setup was done and we went live with this!! The migration was a huge success. Now, where do we go from here??

Possibilities Ahead

• We already got a saving of ~200KB on minified file size in the new setup for the react code, this number will be even higher when we will move the Backbone + jQuery code and all the vendor JS libraries into the new setup due to tree shaking.
• Move all the CSS + assets into the new setup, so that it will also take the advantages of the optimizations setup for it.
• From the current tab by tab ReactJS code, we can introduce ReactJS at the top level, so that we can introduce Global state, Router etc. Also router will help us achieve deep linking to any screen we want.
• Migrate rest of the code to ReactJS, during which we can also setup set of reusable components.
• Typescript + Jest support is already present in the new repo. Start taking advantage of it!
• Think about Code Splitting/Lazy loading.
• Focus on improving our lighthouse score!
• PWA
• React Hooks? CSS-in-JS??

The possibilities are endless!!!

Edit 1: Renamed “Django Rest Framework” to “Django Framework” which is more correct

Django Unchained… Literally… was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.