How many times have you chosen not to login in an app because the process was tedious? Well, I have. Many times.

What a lot of apps out there do is that they ask you to fill in a lot of unnecessary information at the login step, thereby making the login process tedious and lengthy. Most of the time they wouldn’t be using all the information that they ask you to fill in.

Truth is — users are scared of lengthy forms and generally more than one Call To Action on any given screen. In fact, in his blog, Neil Patel (one of the well known faces in growth marketing) suggests that having just 3 fields in a form increases the conversion rate by 25%.

Another problem that I see is that these apps tell you to sign up but they don’t tell you why you need to sign up, and that is one of the biggest reasons for users to skip login — no value proposition. Bottom line — you need to show your users what are the benefits of logging in, otherwise there is a high possibility they won’t.

Why did we redesign the onboarding experience?
With our old onboarding process, we were kind of in the same bucket. Our onboarding process has basically 2 parts to it. The first part being the user signing up and the second part, the user linking their phonebook (syncing their contacts).

Initially, the value proposition wasn’t clear enough, CTAs were hidden behind a tap and too much incomprehensible information was there at the user’s disposal. As a result, the skip rate was, well, pretty high.

The reason why we want users to login is because, firstly, a logged in user would earn goCash (our virtual currency) on every booking they make. Secondly, a logged in user gets to see personalised prices, which means what they to see is what they need to pay (taxes, personalised offers and goCash pre-applied).

One of our features — the goCash+ gratification, requires our users to log in and link their phonebook. So, a non logged in user would not get gratified with goCash+ whenever their contact made a new booking. Every time a non logged in user would try to sync their contacts, we would take them to the login screen. That broke the experience.

It was then that we decided to take a step back and make things simpler and clearer for the user.

What did we exactly do?
We made mobile login mandatory using MobileConnect — which makes login possible with just one tap. Yeah, you read that right. What’s happening behind the scenes is that, if you are on mobile data (with any of these network operators — Vodafone, Aircel, Airtel, Idea, Tata Docomo, Telenor), and you are registered with us, you get logged in on that very step. No OTP required! Yes, its that simple (Oh and yes, this led to us getting featured by GSMA at MWC, Barcelona — the biggest conference for mobile tech in the world!). However, if you aren’t on mobile data or you aren’t on the mentioned network operators, you won’t be able to do a one-tap login. Instead, you will asked to login via OTP. Also, there is an option for social login via Facebook. The login flow thus became much simpler.

If you are on mobile data with Vodafone, Aircel, Airtel, Idea, Tata Docomo or Telenor and you are registered with us, you get logged in on that very step. No OTP required! Yes, its that simple.

And in case you’re wondering, we are working on getting more operators on board.

Now let’s talk about the design. The idea was to make the value proposition absolutely clear and easy to comprehend on a 5 inch screen and also to make the login experience super smooth, which meant the number of unnecessary taps had to be removed, and the CTAs had to be right there on the phone’s viewport. So, we revisited the visual design of the onboarding screen. We introduced a carousel (which occupied roughly 30% of the screen). This carousel had bold colours and fonts, and eye catching graphics.

The objective was to allow the user to consume all the information at one glance, so, the communication was also kept short and crisp. The remaining 70% of the screen had the text fields and CTAs.

The impact of this change was huge
- Login skips have been reduced by 35% (which was the primary objective)
- Phonebook syncs have increased by 5x

Haven’t had a taste of this amazing experience yet? Don’t worry, you can try it out right here.

Get Set Goibibo!

Designing the New Onboarding Experience was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Vaguely remembering when I first read binary system, I was in XI std (1989) during school days. We were used to decimal system and it was a nightmare for us to understand another number system.

That too understanding Binary Subtraction in terms of Addition (2’s compliment) or Borrow Method (http://www.wikihow.com/Subtract-Binary-Numbers)? Why simply computer doesn’t understand any other operator than ‘addition’ ? All these questions in my mind was disturbing not only to me but the whole class including teachers as Binary system was first introduced in the curriculum.

Being not very ‘studious’ kind of person, it was even more painful for me. This led me to figure out if I can use the same decimal system method to subtract binary numbers.

So here is what my method is (Not sure if it breaks, so please feel free to comment / correct) :

Sid Method :

Lets say we have to subtract 5–3 = 2 in Binary system.

5 = 101 , 3 = 11 and result is 10 = 2

So what we should do is, take the bigger number’s binary representation and just subtract the binary representation of smaller number using decimal system method only. i.e.,

101– 11 = 90

Now we know that ‘9’ is not a binary number. So just change ‘9’ to ‘1’ and we are good to go. We have ’10' as the answer which is what we want :)

Now question is : what all decimal numbers can come when we subtract two numbers in binary system ?

Answer is : 0 , 1 , 8 and 9.

What you have to do is : change ‘8’ with ‘0’ and ‘9’ with ‘1’

Why ? No reason :) But just think of it to be in binary form. ‘9’ is odd and therefore 9 mod 2 = 1 and 8 mod 2 = 0.

OR 8 more looks like two ‘0’s and 9 looks like 1 :) Whatever …

It saved my life just to verify my answer if I had to subtract two binary systems using ‘cpu’s method.

Please do comment / correct if you get to know any fault.

Subtraction in Binary — Human versus CPU was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

“We see our customers as invited guests to a party, and we are the hosts. It’s our job every day to make every important aspect of the customer experience a little bit better.”

- Jeff Bezos

Goibibo truly believes in this philosophy and leaves no stone unturned to make our customer’s travel experience the best possible.

In this endeavour, we are constantly looking for areas that create pain for our customers and try to bring in solutions and convert them into delight factors.

UNHAPPY when you Check-In to a Hotel??

We realised that Post Check-out “Review & Rating” of Hotel adds little value to the customer as there is little opportunity for the Hotel to work on the feedback or change the customer’s experience.

The best way to solve this was to share Customer’s feedback with the Hotel on a real-time basis to create an urgency for them to resolve and also delight customers

Presenting Goibibo In-Trip Review

We utilised our “Write a Review” feature to gather the customer’s voice and instantly share it with the Hotel for a resolution. During a Trip each time a customer rated the Hotel, 2 or below, the Goibibo system immediately notifies the Hotel about an unhappy guest and provides them an opportunity to resolve the customer’s issue in a stipulated time, else face the Goibibo ‘Stick’ (Impact Hotel Rating Negatively which directly influences their business)

Once the Hotel resolves the issue, customer is notified and if not resolved to their satisfaction, Review & Rating shared by the customer gets published on our website.

If the issue is resolved to the customer’s satisfaction they have an option to modify the review submitted for the Hotel (The Hotel has now earned a ‘Carrot’ for resolving the issue quickly)

Let’s see how it works!

While you’re in your hotel and are unsatisfied with the services, just dive into the goibibo app, open your booking voucher under My Trips and post a review. We’ll take care of the rest. Simple!

See your experience getting better in a couple of hours….

We are always striving to make each hotel stay, a memorable one.
Stay Tuned for more!

P.S. Shweta was a driving force in making this project come to life 👏🏻 🙌🏻

In-Trip Reviews was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

We have a good news for you! If you have ever:

• Booked a hotel room and paid for the complete overnight stay when you only needed the room for few hours or only for a day time? We guess the most of us have done so…..
• Wanted to use the hotel lounge, swimming pool or other luxurious services in a premium hotel and had to book the complete room night of the hotel for availing such services?
• Reached a hotel early in the morning, just after getting off from your flight , bus, or train had to wait at the hotel reception till the check-in time? We understand that is NOT AT ALL delightful…
• Waited at the airport for hours to catch the early morning flight? We’ve all been there.

To all of the long road trippers, have you ever wanted to take a pause while driving to take a nap and had to look for a nice room to stay?

We at Goibibo, spoke to 100s of our customers across India from the business cities like Delhi, Mumbai and Bangalore and from the pilgrimage towns like Katra, Tirupati, Shirdi, Varanasi, Amritsar, Rameshwaram and others. We have found a big challenge that needs to be solved.

“Why should we book a hotel room for a complete overnight price when we only need the room for few hours or only for a day-use”

To solve this conundrum, we reached out to many of our hotels partners and on boarded them with a new concept of providing the hotel rooms for day use or for few pack of hours i.e. 3 hours, 6 hours, 8 Hours, etc. in all of the major pilgrimage towns, transit locations near airport and bus stands, on major highways.

We talked to a lot of people and ran several pilot projects to understand what it will take for a customer to book a room by hours or by day and FINALLY we are rolling out our very first version of Hourly Bookings.

We have divided the room availability into time slots (pack of 3 hours, 6 hours, 9 hours etc.) and added an early morning check-in in every hotel.

This has been rolled out to our entire user base on:

1. Goibibo’s Mobile App (both iOS and Android)
2. Goibibo’s desktop website

So, next time whenever you need a hotel room for few hours, Try Hourly Booking Feature @ Goibibo and save a lot of money!

All you need to look for is an Hourly booking Logo on the hotels page or apply the hourly booking filter in you search & voila! you will be able to book rooms for just few hours

Believe us, this concept is no more alien in India and 1000s of people are adopting it day by day.

Be a smart traveler and Have a safe stay :)

Hourly Booking: Stay Short, Pay less! was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Goibibo, “We love Travel” and we are passionate about solving problems. We realised that people face problems while deciding on modes of transport to travel from one city to another, especially when there are a lot of available options. Paradox Of Choice!

Until now, you had to go to individual transportation and accommodation categories on an OTA to discover price, schedule and content. For example, if one needs to travel from Delhi to Jaipur : One can do so via Air, Cab, Bus, Shared Car or Rail. To discover prices of these one needs to go to individual verticals and never does one get a single view.

If one needs to travel from Delhi to Jaipur : One can do so via Air, Cab, Bus, Shared Car or Rail. To discover prices of these one needs to go to individual verticals and never does one get a single view.

With this problem statement, we decided to hit many birds with a single stone with the launch of Goibibo One View, Go Planner.

In a nutshell, we brought all modes of transport and accommodation in “One View” and integrated the GoContacts social graph and community with this context. We are proud to say that this is the first of its kind feature. This is just the start and we plan to solve many complex problems going forward including real time bundling of multiple modes, transit cities and towns.

How It Works

On the home screen Goibibo invites travellers with a question : “Where Do You Want to Go”.

a. Discover multiple modes of travel in a single view. Example : Delhi to Jaipur. Travellers can compare the price of Air, Cab, shared car and bus in a single view. Trains coming soon!

b. Get to discover air fare trends

c. Find accommodation in the same view

d. Discover which of their goContacts have travelled to that destination. Ask Questions from their contacts who have travelled to the destination. This is all backed by super privacy features on a user’s Go profile.

e. Ask questions to the community and discover UGC for the destination.

We’ve got lots more coming up. Stay Tuned!

God Speed Everyone!

Where Do You Want To Go? was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

It has been a Super Proud time, I wish to share with you all some of the highlights.

We have come a long way!!! I remember the days we started Goibibo, It was a dream and that dream is giving us all moments of pride, this is super exciting as well as emotional.

Super proud to see Goibibo showcased at Facebook for the awesome work done by mobile team with “places api”. The way facebook team have received us, shows how valuable are the products/tech that we are building and how closely they want to partner with us.

Our CEO Ashish (ashishkashyap), was a keynote speaker for Amazon AWS Summits in India, Created a mark on audience talking about how strong we were and how we will be on AWS, taking over the stage from Dr. Werner Vogels ( CTO — Amazon ). This tells how important we are for Amazon not only as a paying partner but as an evangelist to take their product to places and to get feedback. Gaurav Khurrana, took stage to tell how we migrated to AWS which we internal coined Blitz-Migration. Giving audience a sneak peak on our agility, For a lot of the new releases of Amazon we do work at times as the alpha partners.

@goibibo @ibiboryde @redBus_in @AWSonAir #AWSSummit #LiveBusTracking Zero procurement time 👍🏻

 — @jyoti1013

And then came Google I/O, Goibibo not only got mentioned in I/O multiple times, but became a part of a showcase in one of the presentations by google team on AMP.

This doesn’t stop here. Raghunandan Gupta (Raghunandan gupta) pushed for an indigenous implementation that team did for Goibibo, It got accepted as a feedback on GoogleChrome — Workbox to “pause pre-caching to avoid bandwidth contention” as an enhancement. This is a remarkable achievement.

Thats great gratification when #google accepta your concern and make efforts to mould their product. #io17 https://t.co/Ib0UWYAkUQ

 — @RGengage

Apple team is constantly working with the Mobile team to create experiences that are unique to iOS steering with our brand and reach to create richer experiences for Apple users.

We have really come a long way!!, There are many more stories from all the teams and it’s a super proud moment for me and for all to be part of this magical journey.

Magic can happen and is only possible if bright and high willed individuals come together, collaborate and build for future. I can see that happening with us, It’s rewarding just to be in a place that creates such magic. We have really come a long way!!

Many more milestones to achieve :)

We Have come a Long Way!! was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Recently one of our customers raised an issue about being denied Check-in by Hotel Deccan Erragadda due to her being a solo woman traveler.

We take such issues very seriously backed with action. We want to share with the entire community what actions we took and are taking

We gave Nupur an upgrade and higher rated hotel totally complementary. We refunded the full amount for the hotel Hotel Deccan Erragadda that was booked.

We have de-listed the Hotel Deccan Erragadda from our platform pending the investigation.

We are taking up with local authorities/ police and hotel management for the rationale behind such policies. So far we have the following response from the hotel, Hotel Deccan Erragadda

Policies, content and pricing of the hotels are generated by the hotels themselves. The policies are upfront. See screen shot below. As a platform, we have faced this issue for the first time, hence we are doing thorough investigation into the background behind this policy. We do not support such policies unless backed with a rationale. We will leave no stone unturned to have a scaled solution.

Goibibo has a Neutral Ratings and Reviews platform that enables travellers and hoteliers to share, engage and create most transparent experience. We would be posting feedback on our Reviews and Ratings and QnA platform as well. These tools ensures better decision making and trust.

Nupur has confirmed in her Facebook post what action we have taken

At Goibibo, we try to ensure trust and transparency for our customers be it a buyer ( traveller ) or a seller ( hotelier ). We strive to make Goibibo the most trustworthy and reliable platform and would work with authorities and all key constituents to create best of experience for all stakeholders in ecosystem.

Goibibo Cares for Women! was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Goibibo, we are obsessed with our customers and we strive hard to ensure we are omnipresent, easily reachable, and give them best of the best travel experience. With 200 million people using WhatsApp in India every month, we thought sending air tickets, hotel booking confirmation, etc. on email or SMS might not be the only way to deliver value to our customers. To strengthen delivery of our e-tickets and vouchers, and to create an experience that is easy and more convenient for our customers, Goibibo is testing an integration with WhatsApp enterprise solution to send all the above and more on WhatsApp. You will no longer have to bear the inconvenience of combing your email or SMS to get hold of air tickets while standing in a busy airport entrance queue — your Goibibo air ticket is just a WhatsApp away.

You might be wondering why WhatsApp. For starters, a lot of people who started their internet journey with mobile feel more at home with WhatsApp than with email. WhatsApp has become second nature to a majority of the country, and is easy to use and accessible on mobile. Goibibo being a mobile-first company wants to leverage the mobile ecosystem to its fullest. With mobile, we can communicate better with our customers and also get feedback from them, which will help us deliver a better experience. To summarise, the ubiquity of WhatsApp will help us make travel a much better experience for our users.

For this test, we are beginning to send international air tickets on WhatsApp. Now, whenever you book an international flight, you have the option to receive your e-ticket right in a WhatApp message as an attached PDF document. We will ramp this up soon to domestic flight bookings, hotels, trains, etc. With Goibibo, your travel is just a WhatsApp away.

Trying out WhatsApp for travel was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Goibibo has been listening to its users . As part of our NPS surveys we got a lot of feedback from our users that they were not happy with the high airline cancellation charges . At the same time few users complained that by mistake they filled a wrong spelling in their name and had to cancel the whole ticket . Goibibo thought of a noble plan to solve this user problem

Today we launched Cancellation Protection Program for our users who transact on our platform . As part of this program a user has to pay a nominal non refundable Cancellation Protection Premium . In return , when the user has to cancel his tickets due to any unforeseen circumstances , Goibibo refunds the entire booking amount , in short the cancellation charges are ZERO. If you pay an amount X while booking ( Excluding Convenience fee and Premium ) , we refund you X back . A sample of the user interface is below

What else do I need to know ?

a) Cancellation Protection is applicable on select one way domestic flights
b) User has to pay a 1 time Cancellation Protection premium to avail cancellation protection
c)Total Airline cancellation charges and goibibo cancellation charges would be waived off under cancellation protection scheme
d) Tickets should be cancelled 25 hours prior to scheduled departure of the flight to avail Cancellation Protection
e) Cancellation Should be done at Goibibo.com and not on Airline end

Currently Cancellation Protection Program is live for our web users , the same program would be live for Android and iOS in the coming weeks

High Cancellation Charges for flights ? No Worries now !! was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Being a de facto mobile messaging and media sharing app in India, WhatsApp is extensively used for peculiar use cases like wishing glittery Good Mornings to the whole family tree and friend circle,

Or sending your current location to your friends when you throw a party at a new place.

We, at Goibibo, have been rolling out WhatsApp based solutions for our customer requirements, be it seat selection in a flight , or receiving voucher and map directions for the booked hotel.

Now, we have also powered booking of hotels near you in a really simple way on WhatsApp. You got down at New Delhi Railway station, and are looking for some good hotels nearby ?

Just share your current location with Goibibo on WhatsApp !

We will do the rest for you and send you few best picked hotel options nearby from different budget categories for today’s check-in, which you can then easily book on mobile website or app.

You can of course modify your dates and selection on the website/app, or go ahead and confirm with the default same day check-in dates to do your Goibibo booking.

Remember next time when you need a hotel nearby, just share your location to Goibibo on WhatsApp. If you’re reading this on a mobile device, click here and share your location with our official WhatsApp account to start exploring.

Book hotels near you, with WhatsApp 📍 ! was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

You’ve spotted that last piece of commodity that you love and someone else is up for grabs for it. Will you let them have it, snatch it off them or resolve through fair means !?! Who gets the say? Well in our case, it’s…

The Story….

We are running our entire application on Django 1.7. Like any other legacy system( let’s call it OLD_PROJ) it was a labyrinth of complex code. To make life easier we decided to move on to a better approach with an API based system, micro services powered by Django Rest Framework, Django 1.10 (our test drive on it) and other such fancy solutions. We called it HULK !

But any oldie can vouch for it that you can’t get rid of them so easily. So we had to maintain a huge part of the older system and expand the newer one. But these two projects had something in common! The database. Now arises the actual problem. 2 projects, 2 django versions, one shared database! Its Django1.7 vs Django1.10…

This would be an easy deal if we were to use raw SQL queries to connect to multiple databases instead of leveraging Django ORM. But we love experimenting and whats the point, if we can’t utilise the framework to its fullest!!

So the problem in hand is that we need to reuse an existing database. How hard can it be? How about just add it to the Databases config in settings.py?

Oops…Trouble!

The Mystery…

Let’s call the database for HULK (NEW_PROJECT) as HULK_DB and the one for OLD_PROJ as OLD_DB. Lets say there is an app A in HULK and it needs to use models m1, m2, m3 from the OLD_PROJ. If we were to copy the model definitions from the OLD_PROJ to HULK into A/models.py , then, on running migrate django would just recreate these tables in the HULK_DB, which is not what we want as we want to reuse the existing data in OLD_DB.

Note: You can create the usual A/models.py and define all the native models for HULK and use them as you would normally. These tables would be registered in HULK_DB.

Coming back to reusing old data, the first step was to identify all the models that need to be reused and avoid django migrate from registering these models in HULK. For that we followed the steps below:

1. Created a django app in HULK called goibibo_models (to store all model definitions).
2. Created a models file, lets say common_models.py. (This can be named anything except models.py). All model definitions required from OLD_PROJ were copied into this.
3. By default, django generates <appname_modelname> as the table_name to be accessed for read/write operations, hence we have to modify the Meta of each model class to access the actual table names:
   * changing the ‘db_table’ property to actual name of the table in OLD_DB
   * setting a common app_label to all models copied from OLD_PROJ (explained later)

# Example:

class Notes(models.Model):
    notes = models.TextField()
    createdon = models.DateTimeField(auto_now_add=True)
    modifiedon = models.DateTimeField(auto_now=True)

    class Meta:
        app_label = 'goibibo_models'
        db_table = 'common_itemnotes'

Simple yet, but then when a project is created and you setup the ‘DATABASES’ property in settings.py and run migrations, Django internally creates all the basic tables for auth (eg: User), contentypes(for storing generic relations) and loads its registry based on the Django installed_apps for that project. In this case we wanted to reuse these inbuilt django models too from an already existing database. Oops again !!!

The Eureka Moment…

To solve this we had to override the built in types from Django source code! This was a pretty scary thing to do, but the efforts were fruitful. For all these models we named the app_label as ‘goibibo_inbuilt’ to differentiate them from other app models.

Below is the snippet of code overriding the django User model.

Similarly the code overriding other django inbuilt types are as follows.
1. custom_contenttypes.py
2. custom_genericcontenttypes.py

Now that we have overridden all inbuilt types, we can make use of them in our models definition to utilise, CustomUser and Generic Foreign Key types. This is illustrated below by adding few more attributes to the Notes model mentioned earlier.

1. The user field points to a user in the OLD_DB
2. The content_object uses content_type and object_id to build the custom GenericForeignKey field that can refer to multiple other models instead of a single one (like ForeignKey type).

The Finale

So now we have reached the final step. At this point we have 2 different sets of codes:
1. one native to hulk using the default inbuilt types, models.py
2. another, a set of files with custom code to access the old database.

The only thing left is to redirect the requests accordingly to the databases. For that we used the database routing feature of Django. We do this by ensuring that any model with app_label as ‘goibibo_models’ or ‘goibibo_inbuilt’ (as changed earlier) should access the old database.

# settings.py

DATABASE_ROUTERS = ['multiple_db_routers.HulkDataRouter']

DATABASES = {
    'hulk_db': {
        <configure HULK database>
    },
    'old_db': {
        <configure OLD_APP database>
    }
}

Below is the code for multiple_db_routers.py

And the winner is…

This completes our setup and changes and now we can effectively utilise the django ORM for different django projects with different versions and different databases.

# To access table specific to hulk from HULK_DB(Django1.10)
from models.py import HulkModel

# To access table from OLD_PROJ from OLD_DB(Django1.7)
from common_models import Notes

Time now to announce the winner. And the winner is…..
DJANGO!! Which one, you say..?

Duel of the Djangos was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Goibibo, we have always believed in velocity and ownership. Giving power to developers to Reliably & Continuously deploy code across all environments is a step towards bringing velocity & delegating ownership to individual developer. Today I would like to share how we achieve this at Goibibo.

We have upgrade ourselves from our earlier CI-CD ( EC2 + ELB based rolling release using Ansible ) pipeline and moved entirely on the containerized based CI -CD pipeline. We have used the docker to build & run the containers and ECS as orchestration service.

Why we preferred ECS over Kubernetes?

I know everybody has this question in mind. We also had this confusion initially but we don’t want to add the complexity of running our own container orchestration software and the integration between the AWS services is not out of the box when we run our own orchestration software.

Motivation:

The main motivation behind to go for containerized based CI-CD pipeline is to break out those traditional Dev versus Ops silos. Code was working on my machine or on staging machine, why it’s not working on production?

There are few more benefits we got along with the containerized way.

• More power to application owners to make the changes directly in configs & system without ops dependencies.
• Deployment velocity has increased and deployment time has been reduced.
• Visibility of production infra & services to all stake holders.
• Similar environment from Dev to Production.
• Immutable infra, build it from scratch.
• Moved towards infrastructure as a code and make every change accountable and traceable.
• faster ramp-up of new members of the teams.

We have placed the docker-compose file as a part of the code base for developers, through which they can reduce the build time on their local workstation. One of the best part of using docker-compose is we have mapped two volumes, one is for their codebase, through which they don’t need to do the build process for the code change, another for logs, for seeing logs no need to go inside container they can view on their local machine itself.

• Fast: last but not the least, this complete process should be fast. At Goibibo we learn fast, build fast, succeed or fail fast:)

ECS CI-CD Pipeline:

There are multiple parts of how we build the complete ECS CI-CD pipeline.

• Setting up the underlying infra.
• Setting up Services, Task definitions & Repository.
• Monitoring, Alerting, and log viewing.
• Building the automated pipeline to take the releases live.

Setting up the underlying infra:

It’s being assumed that AWS environment is already in place like VPC, subnets, Route tables security group etc hence that is not in the scope of this blogpost.

Just a few things to remember while setting up the environment.

• If you have selected the awsvpc as preferred networking mode in task definitions, please ensure that you have a good number of IPs available in the subnets because with awsvpc every container gets it’s own IP address.
• keep all the subnets in the backend and attach the NAT gateway. I’m not seeing any reason of running the services in the front end when you have load balancer configured.

To build the underlying infra we have created the empty ECS clusters and used the Spotinst to launch the SPOT instances and handle the autoscaling of infra. We have taken the Amazon ECS optimized AMI and made few changes on top of it using the user-data to supports monitoring and logging, will explain that during the monitoring and alerting part.

ECS Cluster : Cluster is logical segregation of EC2 infra. We are using only a few ECS cluster so that we can optimally use the infra.
Spotinst: Spotinst Elastigroup predicts EC2 Spot behavior, capacity trends, pricing, and interruptions rate. Whenever there’s a risk of interruption, Elastigroup acts accordingly to balance capacity up to 15 minutes ahead of time, ensuring 100% availability.

In our case, we are using ​Spotinst ECS integration with Spotinst ECS Autoscaler to autoscale the infra based on the reserved CPU & Memory utilization by the services & tasks in ECS.

Snapshot from our one of the cluster:

Benefits::

• Using the Spotinst & ECS help us to reduce down the EC2 cost drastically as the instances are running as SPOT and we don’t have to maintain the AMIs for backup.
• Pushing the changes on EC2 is easy and can happen using the rolling method. Where Spotinst launches the new instances with the new changes and removes the old instance one by one.

Setting up Services & Task definitions:

With the continuation of our philosophy of building the infrastructure as a code, we used the cloudformation way to build our services and tasks definitions.

This CloudFormation template builds the task definition, services, security groups, internal & external load balancer and task level autoscaling based on the CPU utilization. Every parameter is configurable from CPU units to the number of tasks any service will run.

Monitoring, Alerting, and logging:

Monitoring: We heavily rely on the NewRelic for our application and infrastructure monitoring. NewRelic also supports docker monitoring out of the box. Installing and setting up the latest agent is the part of the user-data script which we have configured in the Spotinst.

Alerting: We did an integration between AWS ECS & NewRelic so that all the data present at one place and we can setup the alerts on top of it. NewRelic sends all the alerts to a slack channel, email to our NOC team and a call notification via the VictorOPS.

Logging: Our Docker containers generate 2 type of logs one is docker stdout logs and another one is application logs. Docker stdout logs we are pushing into CloudWatch using ECS + CloudWatch integration and application logs are mounted on the host machine and all the logs are being pushed into the elasticsearch using filebeat in real-time. We also make a copy on S3 for our data processing pipelines.

Building the automated pipeline to take the releases live.

For CI-CD pipeline we used Jenkins. Jenkins is open source continuous integration & deployment server with thousands of plugins.

Initially, we thought of using the AWS code deploy but few of our services are not on AWS and we want a similar setup all across so we went ahead with the Jenkins. Also, Jenkins give us flexibility to mold everything the way we wanted.

The build pipeline as follows:

• Everytime a developer creates a tag starting djcb-* GitHub executes a webhook and pass the information to a small flask application which further triggers the build in Jenkins.
• Jenkins checkout the repo with given tag and looks for the Dockerfile. Post that it starts building the container as most of the layer is already present from the previous build the docker build setup take only 1–2 mins depending on the changes.
• In 3rd step, we run the docker image on the Pre-Production environment and start executing the test cases. If everything goes right, we push the image to AWS ECR and start updating the task definitions to update the application.
• ECS replace the old containers with the new one using rolling or blue-green way. You can change the way ECS deploy the containers by changing the minimum and maximum tasks in the service group.

• Once everything is done we send a notification to Newrelic, sentry and on Slack to track the changes.

What Next:

• Auto-Revert: Currently revert requires human intervention if anything goes for a toss post the production deployment.
• Evaluate using of EKS & Fargate, once these services are available in the Mumbai Region.

Containers @ Goibibo was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

At Goibibo, we always strive to make the experience of travel booking — simpler, smoother and faster for our users. Earlier last week, we have introduced a new way for our user authentication — ‘Login with Whatsapp’. This first-of-its-kind Whatsapp login authentication will help our mobile web users to login without needing to remember the password, keying in OTP or Mobile Number — rendering a seamless experience to the users.

The Problem Statement

Every day, lakhs of users go through the tedious login authentication process. Currently, users can login through Mobile number followed by Password or OTP. Alternatively, they can login using their Facebook or Email accounts. In all these processes, the following questions are always at the back of user’s mind

• ‘What is my password? I don’t remember the password I set’
• ‘I haven’t received my OTP yet. How do I login?’
• ‘Is my personal data shared if I login through Facebook?’
• ‘Is my data secure enough?’

We looked back and thought — Is there a simpler way to login while keeping it fast and also removes this kind of questions from user’s mind.

The Idea

Login on Goibibo is totally mobile number driven. So why not use the ubiquitous platform — Whatsapp? The idea is to make the workflow completely user-initiated while keeping it private, as the user has full control of writing to us over or not on Whatsapp.

The workflow

‘Login with Whatsapp’ mode of authentication is a seamless and frictionless experience for the user.

• No more using your keyboard to input the Mobile Number or OTP or Password
• It just takes 2-taps to login
• Super smooth and no data privacy issue (like Facebook or Google)

Impact

· Currently, ~1000 users daily are using login with Whatsapp on our mWeb platform

· 93% of the customers who sent a message to Goibibo have logged-in after receiving the link (The drop off for OTPs is ~17%)

The Learnings

While developing the workflow, one of the use case that struck our mind was that this simplifies the Mobile number login authentication for International numbers where the OTP costs are generally high. As a lot of expats book through Goibibo, this came as a really good solution for a mundane problem.

The Road Ahead

We are soon launching the ‘Login with Whatsapp’ feature of Android and iOS.

Kudos to the amazing team which made this possible in a short span — Tushar Kanti, Navneet Kumar, Alwin Babu, Bhoomit Vasani, Preetika & Shamik

Want to try it right now? Log-on to https://www.goibibo.com/ from your Mobile browser and Sign-in

Introducing ‘Login with Whatsapp’ on Goibibo was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Before the sleepless night after the interview and the wait for that one meaningful mail in my inbox could become a routine, I finally got a call for rendezvous with the best Tech Travel Aggregator brand — Goibibo. The toughest job I find here is to club up the flabbergasting journey of my internship of 2 months duration into few paragraphs. Nevertheless, here is the synopsis of my learning.

On the first day of my internship, I was filled with mixed emotions, mostly anxiety and excitement. After all the formalities were completed, I met my team amongst which I never felt like a newbie. On the second day itself, I was acknowledged of the whole architecture of the company. The main propaganda was to create a separate auth micro-server (Neo) from the initial server system where all the systems were included in a single server. Before creating the neo micro-server, there was a need of creating a testing environment in which we can run the tests before any deployment is made. I am grateful to my team members including Navneet Sir, Tushar Sir, Phani Sir and Abhirama Sir & the whole devops team who helped me and guided me throughout the project.

CORE PROJECT

AUTOMATING AND DOCKERIZING TESTS @ TESTING-MADE-EASIER

I will start with explaining about what our aim was and then the process of how we did it.

Our aim was to create a test infrastructure which neither depends on any other environment nor requires any other dependencies to be imported from any other project. We wanted to include this setup in the build pipeline so that any deployment should only succeed when all the tests are passed, otherwise the deployment should be stopped.

I started with unit testing so that I can understand about the Django REST Framework in more detail. During the unit testing itself, I got to know that in some API’s, the slave database is used to fetch the results. So, the problem with django is that it loses the cursor to the database after a specified time. Ergo, I explicitly imported connections inside the tests file and maintained a cursor pointing to the default database which was also pointed through the slave database. The other problem I encountered here was that the django-tests create their own test database every time one tries to run the tests. So, I was not able to connect the database we intended to connect. We tried mocking the database to run the tests and also used fixtures to solve the problem. After 2 days, we settled on using fixtures with py-test instead of nose-tests to run the tests. In fixtures, we have to use a py-test decorator with transaction set to true in parameters. Moreover, we explicitly created a Test section in settings.py file which will contain the database (test_goibibo) settings using the host as the container name (test_db) and the specified port (we can explicitly use a different settings.py file while running the py-test).

I created 3 test cases for every API.

1. The positive case in which I gave the correct input to the API which matched with the entry I created for testing it (True-positive case).

2. The negative case wherein the input to the API does not exist in database. So, if the API is giving the response which should be given for a non-existing input, I’ve intentionally passed it (False-negative case).

3. The last is the one in which I’ve given a wrong/invalid input to the API so that it gives an error in response. If it gives an error in response, I’ve passed it.

In this way, I basically covered all bases of testing any API. The next step was to dockerize the whole process. We tried to dockerize this whole process by running the tests in container only. We wanted to use Docker as a part of our process so that there remains no database dependency outside the test environment. So, what we have done is that we have created 2 containers through docker-compose: first is the test container in which we will run the tests (test_auth) and the other is the database container which will host the database for running the tests. The container hosting the database downloads the mysql/mongo image through which we have created our test database (test_goibibo). By running the database in container, we have ensured that we don’t need any mysql/mongo server installed on our local to run the tests. Moreover, we haven’t used the pp database so that the data created for running tests do not hamper the usage of the same.

We also dumped a schema file in the database which had the create queries of all the necessary tables related to Auth. The dumping part was done by mounting the volumes through the entry-point feature of docker-compose. We linked the testing container (test_auth) to the database container (test_db) through the ‘depends_on’ property of docker-compose.

For automating the whole thing on local, I created a shell script which will start the containers by building the docker containers and then running them. I intentionally gave a timeout of 2 minutes between running the containers and running the tests so that the tests will run after the schema file gets dumped in the database. After the containers starts running, I wrote a command which would run the tests inside the test container (test_auth) using the docker ‘exec’ command. We ensured that both the containers were turned down after the tests are completed. The tests are self-contained and the data generated for testing gets removed after every test. Hence, all tests were now running on local through the shell script file.

To automate this whole process and integrating it with the build pipeline, we used the ecs-jenkins to run the shell script. We created a jenkins file through which we first checkout the branch from which we want to run the tests. Then, we created a xml report within the test command only which we further rendered to create a test report using the J-unit plugin in jenkins-file. The test report separately shows the passed and the failed test cases with the console output. We can also customize the output which is displayed in each test result. The test report generated through jenkins will be similar to this :

The whole testing process can be summarized in the following flowchart :

How to add more tests?

Just add tests in the test.py file and create a tag of the repository which would be used in building the job in jenkins to run the tests. Build the job using that tag and the testing is done!!

Honestly, my time spent interning here proved to be one of the best summers of my life. Not only did I gain practical skills, but I also made really great friends and we had tons of fun together. I enjoyed every bit of it, so much that I wished I never had to return to college. The atmosphere here was always welcoming which made me feel right at home. Summer was well spent :)

RENDEZVOUS WITH Goibibo | Internship Experience was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

How Goibibo manages its Hotel Inventory Persuasions.

At Goibibo, we strive hard to give exciting experience to our customers and enable data-driven business. To achieve this we have a dedicated Data team, which consistently build highly reliable data products which in turn add value to business. One such product is “Inventory Cache”.

Introduction

With lakhs of hotels on-boarded on Goibibo and tens of room-types within each hotel, one can’t hit database to fetch inventory details every time. It creates a need to maintain a separate single cache for hotel-rooms which is extremely reliable and scalable. There are multiple use-cases for this cache and one such is Inventory Persuasion, where if a hotel room inventory is left with lesser than a defined threshold then show this inventory count to user, creating an urgency to book his/her room in that hotel.

Under the hood

It’s Kafka Streams! Yes, the main streaming pipeline behind the show is Kafka Streams. Every change in inventory is captured in Kafka through CDC.

Above flow diagram represents the high level architecture of Inventory Cache.

Change logs in hotels database are captured by Debezium and sent to Kafka topics. It consists of 3 streams — Hotel Details, Room Details and Inventory Details. Hotels and Rooms streams are persisted into GlobalKTable using KafkaStreams. GlobalKTable is a log compacted topic in Kafka, whose data is stored locally by streams application in rocksDB.

Now we join inventory details stream with hotels and rooms GlobalKTables to formulate the complete details and data is persisted in Redis in HashSet format with HotelID and CityID keysets. Also this data flows back to another Kafka topic, in case some other service want to use this.

Real Problem - Consistency of the Cache??

Above is the half part of the story. The biggest challenge here is to maintain consistency of the cache. If a single room inventory is not reflected correctly to customer then it can have severe impact on business and our reputation. There can be multiple reasons for such nuances like Debezium pipeline failure, Kafka topic deletion, etc.

To handle this critical piece we have introduced Continuous Integration Testing and Lambda Architecture.

Continuous Integration(CI) Testing:

In this two sources of truth are hit hourly for few random records —

* RedisCache

* HotelsAPI

If there is a significant mismatch in data, then metrics are sent to NewRelic where policies are set to alert maintainers.

Next step of this testing is a correction measure-:

Lambda Architecture:

In CI Testing, if mismatch happens then it is obvious that data is not consistent in cache. As a correctness measure we incorporated Lambda Architecture, where a batch pipeline runs beside realtime pipeline for past hour and feeds data from database to separate Kafka topic for streams to process and correct cache.

Once we have these things in place then we are certain about the consistency of our Inventory Cache. This architecture has helped us resolve the anomalies without any manual intervention and leading to win-win situation for both product and developers.

Impact

We have seen that when a hotel is sold out and suddenly a cancellation happens and we show “Almost gone! last 1 room left” persuasion message to customer for that inventory, then average time for it to go back to sold out is 20 minutes. This means that average selling time for last room left is ~20min. This data transparency helps customers to take fast decisions and hence serves business better.

We are solving plenty of interesting problems and would like to work with best talent out there. Join us, we’re hiring. Email at jointhecrew@go-mmt.com

Hurry! Book your room fast on Goibibo was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Top rated hotels share best deals with you in our new booking experience!!

Working for a travel company guarantees you a call from your friends and family every time they are looking to travel. A few months back, I got a call from a friend looking to travel for work to Bangalore. As with most conversations, it revolved around discounts and recommendations for hotels near Marathahalli.

At the end of the conversation, I realised that my friend didn’t really care much about the hotel name or whether it was a fancy brand. For him, booking a hotel was pretty similar to booking an Uber or Ola. All that was necessary for him to know was:

1. Hotel should be near Marathahalli
2. Hotel should have all basic amenities — clean room, linen, washroom, wifi, good reviews.
3. Should be in the price range of 1500–2000 per night
4. A promise that the price on goibibo will be better than any other platform.

Following this, We talked to a few more friends looking to travel to new, unfamiliar places. Last minute bookings after a party, unplanned overnight stays, hotels near a bus stand or railway station due to departure delay were some of the common use cases observed.

It was easy to observe a similar pattern for their booking requirements —

1. Proximity to location
2. Basic amenities
3. Fixed budget
4. Best price deal.

The best part of working with goibibo — if you have an idea that addresses a pain point, you can take the initiative and take it forward from 0 to 1. This week, we launched our newest offering to all our customers:

“Secret Hotels. Secret Prices”. You tell us the budget and amenities, we’ll pick the best hotel for you at a price much lower than any other website.

Our top rated hotels share bests discounts in our Secret Hotel funnel with users to ensure the Best price deal for them.

Want to experience the new way of booking hotels?

• Search for hotels on goibibo. On the search result page, find the entry point to secret hotels

• Tell us your location, Select your budget and amenities preference. The secret hotels are handpicked by our team to guarantee you an awesome stay!

• Review your preferences, Book in a single click. We’ll find you the best possible hotel, with some awesome reviews and rating.

Exceptional work in taking the project from idea to execution by Faris Mohammed & shankarlamba (our lead designers), Pooja M.D, Shruti Madan, Rohan Shah, Poojari Rohan, Vikrant Chaudhary, Anusheel Singh, Raghunandan Gupta, Ashwin Mathew (Engineering Team). Project mentoring by Anshuman Bapna, Deepak Tuli, Parikshit Choudhury & vikalp sahni

Download the app and try out our new feature now: Goibibo’s Android App

In a short span of time we are seeing some tremendous traction on this funnel with scope for a lot of improvement. Do share your feedback in comments below.

Secret Hotels, Secret Prices was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Our active Goibibo community always wants to help fellow travellers, through hotel reviews, or answers to the questions they ask. With millions of hotel reviews and hundreds of reviews per hotel, it becomes vital to provide the best information to our customers in an easy to consume way.

The implementation summarized millions of reviews into summary topics and their sentiments, reflecting the mood of the customers.

How was it built ? First, let’s mention all the Python goodness !

• spaCy : spaCy is the heart of all the NLP, supporting operations like lemmatizations, tokenizations, dependency parsing or noun phrase extraction.
• Scikit-learn : For topic modeling and building the primary sentiment analyzer to predict topic sentiment in hotel and travel context.
• difflib and jellyfish ( jaro_winkler ) : to detect highly similar strings

>>> #uses difflib and jellyfish
>>> find_string_similarity("SPACEOUS rooms!!","spacious rooms",normalized=True)
0.9461538461538461

• regex : the regex module is used as an enhancement alongside the standard re module, as it provides richer capabilities like finding a partially misspelled word within a sentence .

>>> import regex
>>> input = "Painted by Leonrdo da Vinci"
>>> regex.search(r'\b(leonardo){e<3}\s+(da)\s+(vinci){e<2}\b',input,flags=regex.IGNORECASE)
<regex.Match object; span=(10, 27), match=' Leonrdo da Vinci', fuzzy_counts=(0, 1, 1)>

*Tip* : bisect module is very handy in cases where we need to decide a value based on range of other variables. For example, error allowance of 3 for ‘leonardo’, and 2 for ‘vinci’ based on the string length.

• NLTK : for its inbuilt batteries like finding ranked collocations , and the emotional module vaderSentiment which acts as the fallback sentiment classifier.

NLP Concepts

The expressions of the people need to be read and understood well ! NLP concepts which extensively assisted in the content analysis are:

• Stemming and Lemmatization, to work with root forms of multiple variations.
• Fuzzy matching, for approximate phrase matches and paraphrase detection. Involves operations like string or word representation alterations through stemming, lemmatization, normalization, approximate string matching, and partial phrase matching . The number of operations or aggressiveness is chosen based on extent of coverage required in a step vs performance cost. Additional candidates like metaphone/soundex were not considered.

• Tokenization, retokenization, part of speech (POS) tagging to identify the concepts in the content
• Dependency Parsing to find relations between the concepts, and use that wisdom.

One more example.

Building the extractor

The raw data is first sanitized and standardized. Then, the key topics extractor uses multiple channels, both supervised and unsupervised, to find vital topics. These topics are then deduped and clustered based on similarity parameters like intent, sentiment, paraphrase match and string similarity. The topics are ranked and pruned to reach an approximate K*log(N) number of key topics from N user review documents. K is used to tune the extent of key topics intended to be shown to the end user.

As mentioned, the extractor pipeline consists of multiple data channels:

1. Known named entities : Important names which we might lie in the user reviews. Locations or point of interests near a hotel, for example.
2. Collocations
3. Noun Phrases
4. Topic Modeling and Dependency Parsing

The overview of whole pipeline looks like this :

Pipeline, under the hood

Let’s cover the individual components of the pipeline one by one, along with decisions made and methodologies used.

1. Known named entities : all geographical entities ( hotels or locations or airports) and non-geographical entities ( airlines, trains ) are modeled as a graph with nodes and their relations with each other in our application platform. This intelligence is then used to derive a set of possible mentions of relevant nodes in user reviews. For example, hotel reviews in Agra might be mentioning the Taj Mahal a lot. Usage of spelling variations or misspelled words is pretty common in this, hence the forementioned regex module comes in handy here to increase the extraction coverage, along with the frequencies
2. Noun phrases : Noun phrases form a vital component of language, and become even more relevant in reviews. “Front desk staff”, “rooftop restaurant food”, “candle light dinner” for example. We use spaCy to extract the noun phrases and their frequencies.

Both (1) and (2) are ranked by a tf-idf based scorer to judge relevance of the entity of the noun phrase in overall review documents.

3. Collocations : Wikipedia describes Collocations as :

In corpus linguistics, a collocation is a sequence of words or terms that co-occur more often than would be expected by chance

“More often than would be expected by chance” is an indication of their importance in the overall review. We extract bigram and trigram Collocations using inbuilt batteries provided by the evergreen NLTK .

The score of extracted collocations is a function of their gram score provided by NLTK scorer, frequency and gram token length.

4. Topic Modeling and Dependency Parsing : This is the most crucial channel of extraction.

• The first step is collect the subjects for which we want to learn the user utterances and sentiments.

— First input in this is a supervised list of hotel relevant subjects.

— This list is further enriched with Topic modeling, using both Non-negative Matrix Factorization (NMF) and Latent Dirichlet Allocation (LDA) powered by Scikit-learn.

• Then the marvelous spaCy comes into action to do Part Of Speech (POS) tagging and dependency parsing for the subjects to decipher what the users are saying about the subject. A tree parser obtains linguistic information in a standardized format from the free text writing by the users. Along with the subject, the formatted info contains vital parts like adjectives, adverbs, determiners, negations, conjunctions etc. which have been used in the context of the subject.
• This data of the phrases is passed through primary and secondary sentiment analyzers ( explained later) to assess its contextual polarity. This results in “the rooms were neat and clean”, “really large rooms”, or even misspelled “all room were specious” to be categorized as positive phrases.

• These utterances about the subjects get grouped together based on group of subjects ( staff, front desk, service), same sentiment polarity, or similar paraphrase meaning. ( unmatched service, courteous front desk, polite staff, fast room service ).
• But some customers of that hotel might be holding a reverse sentiment about the same subject, how do we define the democratic judgement for that subject ? The approach for this answer includes both: text reviews for which we have derived sentiment and keytopics data, and non-text reviews where the users provided only the rating. Combination of these data points gives stronger insights about customers being delighted or disgruntled with that specific aspect of their hotel experience.
• The importance of the topic subjects are calculated based on their importance per document ( tf-idf based ) and logarithm based boosting on number of occurrences in different reviews
• Once we have found clusters of similar subject topics and their mentions, combined based on similar sentiment polarity, paraphrase meaning, subject similarity, lemmatizers (and stemmers) , we chose a parent topic to define that cluster and display to our end users. The parent topic shall be representative of the clustered sentiments and expressive too.

There are multiple possible approaches to select the parent topic, shall it be the most frequent pattern, or most immaculate one ? As expected, frequent ones are “good”, “nice” or “great” , and immaculate ones are “exquisite”, “sumptous”, “stupendous” etc.

The caveat of choosing common ones was that the generated summary would be inundated with topics like “good food”,”good staff”,”good location”,”nice service”,”nice pool”,”nice coffee” . This is monotonous. On the other hand, if we carefully chose the immaculate and rare ones, the summary might not be easily comprehendible and not represent popular sentiments.

One approach evaluated to find frequent-immaculate balance was to include inputs like frequency distribution of the words in a linguistic corpus like brown corpus from NLTK , or our own corpus.

>>> from nltk import FreqDist
>>> from nltk.corpus import brown
>>> fdist = FreqDist(brown.words())
>>> fdist['good']
767
>>> fdist['delicious']
4

• We went ahead with a simpler empirical approach of selecting the parent topic out of secondary ranked items from the frequency ranking. And that was delicious ! That was courteous ! That was fantastic !

Indexes and Snippets

All the above channels also report indexes of relevant snippets from which the information was extracted, which come in handy in emphasizing the useful information.

Bad data filtering

All channels mentioned above might provide irrelevant or not so useful pieces of information like our brand name, or name of the hotel, or frequent verb phrases. The filtration layer specific to each channel tries to minimize such content through stopwords ( phrase matches), and grammatical structure rules.

Contextual Sentiment. Classification. Clustering.

Remarks like “Happy with the service” or “Extremely satisfactory vacation” are positive sentiments, while “Really disappointing experience” is negative.

However, certain mentions like “…very small pool..” or “..The rooms were really speccious..” ( spacious ) , have a neutral sentiment in general sense, but in travel or lodging domain, they become contextually positive/negative sentiments.

To measure and classify such contextual sentiment for the extracted key topics, we built a classifier in scikit-learn.

Building and using the sentiment classifier

• The training data consists of extreme polarity reviews from our users i.e. reviews where the users were extremely satisfied ( rating 5/5 ) or extremely dissatisfied ( rating 1/5).
• To have a broader coverage, hotel reviews are from different themes and locations, like leisure, business, beach, budget, luxury, or pilgrimage
• These reviews undergo similar information parsing and data standardization as mentioned in “Topic Modeling and Dependency Parsing” section above.
• The resulting data along with their known polarity are used to train scikit models.

Corrections and Cases:

• There could, of course, be disparities like sarcasm or mixed content in single review. Such outliers get diluted by good volume of training data, and additional checks done through correlation with numerical ratings and sub-ratings ( on food, location, facilities, value for money or staff ) provided by users. Conflicts are also looked up into our fallback sentiment analyzer Vader ( inbuilt in NLTK ), and opposite sentiment disparity between our training data sentiment and Vader provided sentiment is a signal to discard that row. Neutral-Positive ( or vice-versa ) is not a disparity, but positive-negative and that too with high polarity score is considered a case for rejecting that data chunk.
• Neutral stopwords are ignored irrespective of the mentioned numeric review rating. ‘Ok food’, ‘Average service’
• Named entity adjectives : ‘serene’ in ‘serene beach’ is relevant sentiment, but ‘calangute’ in ‘calangute beach’ is a named entity.

Analyzer usage:

Just like the training data, the unseen candidates are also converted to standardized forms for sentiment classification. Most utterances do get classified.

To have more coverage:

• The lookups are also done with the lemmatized forms and fuzzy matching if primary classification isn’t triggered.
• Negated lookups for already classified positive sentiments.
• Mix of all.

This assists in classifying “food not good” as negative, if “best food” is known as positive.

*Tip* : The models are persisted and loaded using cPickle and memoized with cachetools for faster lookup times.

Vader also comes in the picture as fallback sentiment analyzer, and high intensity scores reported by it are used for polarity determination.

Clustering:

To achieve the objective of information summarization and representation on the product in an optimal way, the extracted and classified topics are grouped on underlying intent, and a representative topic is chosen out of the group for conveying first level information.

Items are grouped on:

• Same intent / paraphrases : Similar subject and sentiment polarity. Only polarity is taken into account for grouping, not the intensity. This ensures that “bad service quality”, “worse service”, and “really slow and pathetic service quality” lie in one group.

• Fuzzy matching : as mentioned in the NLP concepts section above, similar phrases get grouped together
• Mix of both : “Speccious room”, “Spacious rooms”, “Comfortable rooms”

Below is a interactive cluster diagram for the reviews of a hotel on our platform. It depicts the extracted parent topics and sentiments, the grouped phrases, and the relevant highlights captured in the review snippets.

The product recipe has open source ingredients from:

1. displaCy
2. spaCy
3. Scikit-learn
4. Natural Language Tool Kit ( NLTK )
5. vaderSentiment ( also available within NLTK )

Any questions, feedbacks, suggestions for improvement are most welcome.

Key topics extraction and contextual sentiment of users’ reviews was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

The story of how we liberated our web application from the shackles of Django Framework into a new homemade of Create-React-App. Too dramatic??? I now have your attention. Read on…

Just as how for B2C users we have Goibibo, for B2B customers (hoteliers) we have In.Goibibo (Extranet), which is used to manage hotel property, inventory & pricing. The codebase for Extranet was a humongous Django setup which used to serve all the APIs, webapp & Admin views. So it was obvious that sooner or later we need to introduce some separation of concerns to breathe fresh air into each of the sub-entities.

• Having each of the sub-entities separate will not only help in planning, executing code changes faster but also make the process of debugging easier. Hence moving to a Service Oriented Architecture was the need of the hour.
• The website was also undergoing changes to be rewritten in React to make the code more maintainable and readable. But this was still being done in the Django setup which lags way far behind compared to the developer experience which newer JS frameworks provide.
• Also having the web app as a standalone project will let us easily introduce global level changes like a Router and help in performing perf optimizations easier.

Action Plan

This is what I do: - you decide what you want - you figure out what it would take to get there - you break that out into the smallest, most actionable bits you can, and schedule them - adjust along the way based on new information 🤷🏻‍♀️ that's really the model I work from

 — @sarah_edo

Since I was unaware of both the codebase & Django, and having never worked on Python; some initial time had to be spent in understanding how the code is setup and it is all connected. After spending about 2 days on understanding, I had figured out a rough idea of how the setup is and how the migration should proceed.

We chose to go with Create-React-App which I believe is the gold standard in terms of a dev environment for developing React apps. CRA in general is not customizable and for a good reason. But we were intending to maintain a fork with our customizations if needed. The new repo also comes with Prettier.

• The web app was a mix of Backbone + jQuery code forming the core and most of the UI, ReactJS being used in some of the tabs by directly mounting onto section elements.
• The ReactJS code will feel right at home in the new setup and would require very minimal changes.
• The old Backbone code was to be moved into the new setup and changed to adhere to modules standards so that it could be properly imported in the right order and bundled.
• The vendor JS library files were to be replaced with their node module equivalents so that we would gain from the Tree-Shaking ability of the new setup wherever possible.
• All the CSS along with assets like fonts, images etc were not to be disturbed.
• All the hierarchy of Django HTML templates was to be ported directly and hierarchy to be maintained.

Alright, I now have a step by step plan of what to do! Time to

I had a rough idea of how to execute everything in that plan, except for the last step. The web apps these days are just SPA, which means only one HTML and rest of the code in JS. But Backbone/jQuery code was reliant on a lot of HTML templates and this migration’s scope was to be limited to moving out of Django.

First Blocker

How to get all the HTML code which is a huge set of nested template files to get to work on Create-React-App which is primarily a setup for SPA??

• When I was digging into the CRA’s webpack config, the HTML loader included — html-webpack-plugin — ships a default loader which allows EJS like syntax to include dynamic stuff like webapp title.

• Upon digging further I noticed that it can support any JS since it is using lodash to process the templates. So I can throw in a require()call and load any HTML I want using html-loader.
• So adding html-loader as a dev dependency, I added a require statement for HTML resources which worked beautifully and thankfully CRA doesn’t block loader chains here.
• But there is one more problem here. Since we are using html-loader, I can’t nest another require inside the HTML, so deep level nesting of HTML was not possible with this setup.
• I could’ve gone for ejs-loader which would have allowed me to do all that, but I didn’t want to unnecessarily fork for this reason, specially since down the line, all the HTML files will vanish post complete migration to react.

So, we resolved our first blocker albeit with a little more work from our side to remove all the deep nesting of the HTML templates. Our index.html is now peppered with a lot of requires to HTML templates, in some cases I had to move the contents of a template file into the index.html so that I could require other set of HTML files.

Also I noticed an issue while doing this with the html-webpack-plugin. I filed an issue & PR for this. If you are interested you can read more here.

Next Gotchas

Once that is resolved, I migrated all the HTML files with minimal changes into the new repo. Now was time to move all the Backbone + jQuery code, along with all the CSS into the new repo. Without second thought, I moved all code into the src/ directory in the new repo, only to be bombarded with millions of lint errors/warnings. Then my sensible self reminded of something that I keep telling myself all the time

DRY KISS — Don’t Repeat Yourself & Keep It Simple Silly

I can’t go into another rabbit hole fixing all these issues now, the focus of this task was to just migrate. So I moved all the Backbone + jQuery code along with all the vendor JS files and CSS files under public/ folder, and included it directly in the HTML like before. Which also means that I have to setup production version of the same which combines multiple files into one and minifies and uglifies the code.

lodash templates in the index.html again to the rescue!!

Getting this to final shape took a little more work than I had imagined.

• First things first, I had to write a post_build.sh script which will combine, minify & uglify the files and remove the original ones from the build/ directory. This was already being done in the old Django setup, so copied the same script with some modifications.
• Then I noticed that the condition shown above was always evaluating to false, which upon spending a considerable amount of time, I found that it was happening because of the Create-React-App setup. The replacements for %SOME_VAR% stuff inside the index.html happens post generation of build/index.html as evident here.
• So I had to write a pre_build.sh script which will rewrite %NODE_ENV% as “production”. And set it back to %NODE_ENV% in the post build script. I didn’t think this was a generic use case for everyone, hence decided against filing an issue & PR for this on Create-React-App.
• Then I added a few more lines to the bash script to calculate the md5 hash of each of these combined files and rewrite the index.html in build/ directory to add the hash to the url. So /legacy_js/combined.min.js became /legacy_js/combined.min.<hash>.js
• Similar thing was done with our CSS as well.

Taking care of Django stuff

Through out the code, wherever the Django template syntax was used, it had to be migrated.

• Things commented out with {# #} were commented out using HTML comments <!-- -->
• Forms in which {% csrf_token %} was replaced with <input type=”hidden” name=”csrfmiddlewaretoken” />
• In some cases, the logic which was written in Django syntax, had to be written in lodash template syntax and made to work via _.template
• {%version <some file>%} which is used to version some file, was taken care by our new build process
• {% url … %} was replaced with proper URLs after going through the code.

Bringing it all together

With all this code logistics out of the way, it was now time to integrate all these individual migrated pieces and get it to work. So far into this migration, there was no output that could be checked. All these migrations was done under the hood and its results were yet to be seen.

But there was one more thing that had to be done, before we could start seeing the results. Django server-side renders (SSR) the initial HTML to inject some data before serving it to the client. So this data is something that is used throughout the code and is very critical to it’s functioning. So we had to tackle this.

We could have gone down the route of SSR for this too. But since the initial focus was limited to ripping the webapp out of Django, we decided to make an API for this and call it as the first thing, before letting any other code run. And once we get the response from this API, we put the data in all the right places on the client side where Django used to inject before.

Copy pasting the react code was the easiest part of the migration.

Some of the code had to be changed to wait until the data from the initial API call is available. And finally, all the marathon changes needed for the migration was complete!!

All is done! Time for celebrations!!!

For all the nerds, here are the file stats in the new repo

Production deployment setup was done and we went live with this!! The migration was a huge success. Now, where do we go from here??

Possibilities Ahead

• We already got a saving of ~200KB on minified file size in the new setup for the react code, this number will be even higher when we will move the Backbone + jQuery code and all the vendor JS libraries into the new setup due to tree shaking.
• Move all the CSS + assets into the new setup, so that it will also take the advantages of the optimizations setup for it.
• From the current tab by tab ReactJS code, we can introduce ReactJS at the top level, so that we can introduce Global state, Router etc. Also router will help us achieve deep linking to any screen we want.
• Migrate rest of the code to ReactJS, during which we can also setup set of reusable components.
• Typescript + Jest support is already present in the new repo. Start taking advantage of it!
• Think about Code Splitting/Lazy loading.
• Focus on improving our lighthouse score!
• PWA
• React Hooks? CSS-in-JS??

The possibilities are endless!!!

Edit 1: Renamed “Django Rest Framework” to “Django Framework” which is more correct

Django Unchained… Literally… was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Two States : The story of react-native to react-native-web

InGoMMT is a free hotel inventory management engine. Technically it’s a B2B platform where users are hoteliers unlike Goibibo which is a B2C platform. InGoMMT already has mobile apps (android + ios) developed in react-native as well as desktop app (Extranet) for hoteliers. We always had desktop app used for mobile web, which used to give bad user experience. This blog is about our journey of launching mweb, developed from our react-native codebase using react-native-web.

Problem Statement

In today’s world of apps, there is still a customer segment who prefer mobile web over mobile app. But how do we convert our app users into mobile web having same experience and behavior? The answer is react-native-web if your app is built in react-native. 💡

Lets understand react-native-web

As per official definition — react-native-web makes it possible to run React Native components and APIs on the web using React DOM. Using react-native you can develop native apps for android as well as ios and now using react-native-web, you can convert it into responsive web app. In short, write once, run anywhere.

First Things First

I created separate branch in our mobile codebase and added necessary packages for react-native-web-

yarn add react-scripts react-dom react-native-web react-art

1. I also created entry files public/index.html, src/index.js and added some node scripts- yarn start-web, yarn build-web.
2. I tried to run it but compilation failed.
3. I started getting errors like Support for the experimental syntax 'classProperties' isn't currently enabled, React Native - TypeError: Cannot read property 'bindings' of null, Plugin/Preset files are not allowed to export objects, only functions., Uncaught ReferenceError: regeneratorRuntime is not defined. [SyntaxError] "Unexpected token <" [render(<App/>)]
4. I concluded that we need more control over webpack configuration and settings, so we ejected react-scripts.
5. I renamed .babelrc to babel.config.js and made some changes to it-

module.exports = {
  'presets': [
    'module:metro-react-native-babel-preset',
    '@babel/preset-flow'
  ],
  'plugins': [
    [
      '@babel/plugin-proposal-decorators',
      {
        'legacy': true
      }
    ],
    ['@babel/plugin-proposal-class-properties', { 'loose': true }],
    [
      '@babel/plugin-transform-runtime',
      {
        'corejs': false,
        'helpers': true,
        'regenerator': true,
        'useESModules': false
      }
    ]
  ]
};

6. With these changes, compilation was successful. I did see our Login screen but it was not being rendered properly. Image was not getting loaded, input and checkbox components were misaligned. If you don’t believe me , check out yourself :)

7. Figured out that components were not getting height i.e. height 0px was getting assigned to all elements. I put below style in header to solve this issue.

<style>
  html, body {
    height: 100%;
  }
  #root {
    height: 100%;
  }
  #root > div:nth-child(1) {
    height: 100%;
  }
</style>

8. Bazinga!! 😄 😺 We saw our first screen in react-native-web just like app. Awesome.

Few hiccups on the way..

During the migration, we faced two major challenges-

• Certain components like Alert, Modal, DatePicker from react-native had no support in react-native-web (at the time of our development).
• Our codebase has multiple third party libraries and many of them are platform specific i.e. their equivalent support in web was not available.

Here goes the list of issues and how we solved them-

Few more extras never killed anybody :)

1. In app, we are using Firebase for analytics. Since Firebase Analytics has no support in web (as of today) so we added GTM (Google Tag Manager).
2. For better modularity and maintainability, we created separate repository for mweb and used our existing mobile codebase as git submodule

Setting up Server

1. For server we chose nodejs, an obvious choice for Javascript developers.
2. We decided to host mweb on ‘/mweb’ route i.e. in.goibibo.com/mweb. To serve all mweb traffic to nodejs server, we setup a proxy on /mweb route on our main Apache server.
3. Server routes-

app.use('/mweb', express.static(path.join(__dirname,'../build')));

// browser routes will be handled in Frontend
app.get('/mweb/*', function(req, res) {
  res.sendFile(path.join(__dirname, '../build', 'index.html'));
});

4. Dockerfile creation

..and finally the day arrived

1. We released mweb on 7th Feb, 2018 🍰
2. Total bundle size (gzip): 490 KB

Take Away for everyone

Sometimes it’s the journey that teaches you a lot about your destination.

I truly enjoyed this transformation journey which took me roughly 5 weeks. Beside introduction of a new channel, m-web has also made app debugging and development easy because now we can monitor network-tabs, add debugger etc in browser which are a bit tough in emulator. There are further challenges like build size optimization, support for chat feature etc which we'll conquer along the way.

If you like the journey like I did please give some love over social media and share it. Peace. 😃

Two States : The story of react-native to react-native-web was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.

Next Station — Goibibo!

Railway bookings on the rocks.

What comes to your mind first when you have to book a Train Ticket? IRCTC. That’s true for more than a million people, who are booking train tickets everyday on IRCTC’s website for 10+ year now.

Well YOU deserve better! To save users from being tested on maths & analytical skills on captcha, Goibibo took this #10YearChallenge to make your Train booking experience, just as soothing as the _kulhad wali chai_ in the journey.

The Journey

Team started with a Mobile-First philosophy! Idea was to handover the best train booking experience on finger-tips.

Before we could even start to innovate we had a huge obstacle. As per the directive of inventory supplier - IRCTC, our Look-To-Book Ratio (LBR) needs to be under 100. That means :

LBR : For every 100 api calls we make to irctc for getting data, we need to make at least 1 booking.

Oh boy! We were 4X out of LBR limit, and were being billed lakhs every day. It’s a chicken & egg problem, they said. You need more bookings to get your LBR down, and for more bookings you need more traffic, and traffic would increase the number of IRCTC calls and that would in-turn increase the LBR. So truly, we were in a fix.

We were left with only one option. Convert our traffic better.

With these facts jammed in head, team started on the mission. Mission of building Reasons for users to book from Goibibo.

The Magic Wand

How many times have you made a payment on IRCTC and ended up without a ticket? This may happen because of any technical glitch or may be your seat preferences. What you do next, pay again and give it another shot.

Goibibo believes, every payment that we have received, we need to deliver the ticket, just like a baby. None of those can be left un-delivered.

Once you make a payment for your train ticket, and if we fail to deliver a ticket to you, you get to retry the same transactions any number of times as you want. May be you forgot your password, may be your internet went down, may be IRCTC has some upstream issues, or God forbid, Goibibo’s server went down.

No Worries! Just a click and you can retry your transaction without paying again.

The functionality comes with host of features like getting your IRCTC password in just a click or resetting it, retry password/captcha or attempting the same booking with different IRCTC username etc…

Alternate Availability

Railways is a supply-short system. Users usually don’t get a confirmed tickets for their journey and end up not booking the tickets. Remember, each user dropping from our funnel means, increase in LBR.

This, Goibibo’s industry-first feature, searches through all possible combinations of stations & quotas to get you a confirmed seat in your desired train on your preferred date.

Got a waitlisted ticket? Worry not!

With just a click, we will find you a confirm ticket on the same train with altered source and destinations, but same boarding station. It might cost a few more bucks, but thats worth a confirmed ticket, just like Tatkal.

Another Example would be: Say user searched for Bangalore to Mysore and has got waitlist of #10. So we might suggest you an available seat on the same train by booking your ticket from Chennai to Mangalore & boarding you at Bangalore (which was your preferred source).

Availability Calendar

Better than a fortune teller, our Train Availability Calendar can actually see into the future.

To check any train’s availability, you probably need at least 3 step operation. Starting with search for your route with concerned dates, looking for your preferred train on search page, and then figuring the availability among classes. If that’s not enough you might have to hunt better availability on other dates. NO MORE!

The regular date selection calendar automatically turns into an availability calendar telling you which date is having availability, and on which dates train doesn’t run. In just one-click, user will now be able to make a booking straight from homepage.

Intelligent Results

Deriving from the fortune-telling calendar, we evolved our search results page to give you the best available class upfront.

ZERO clicks to check which train/class has better availability.

Just scroll through the train listing and select the best available train that suits you.

Noticed the funky icons on each train’s side? You can know upfront which train is Superfast, Serves Meal, Runs Overnight or has a Cleaning Service.

Where is my ticket?

Goibibo moved all of it’s user communications to Whatsapp-first almost a year back. So if you book a train ticket with us, we deliver you IRCTC ticket straight to your Whatsapp. Of course that’s not it. You can check your PNR status too on Whatsapp, just a message away.

Platform also hosts other special services like

CleanMyCoach service, PNR Status-change subscription, One-Click IRCTC Account creation, and ofcourse offers!

Those are just a few reasons to book with us.

अंकगणित (The Number Gain)

Goibibo’s Train vertical was running at hyperloop’s speed launching a number of industry first features in a span of just 4 months.

These features were accomplished by our in-house caching framework, we call DeepCache. All the API calls were cached in a single unit irrespective of their data and purpose. Hence we could pull out any data at any screen with just minimal information from the user. But that’s not the best part of any development cycle.

The best part of developing a feature is to see it fly.

Remember the Elephant in the room, LBR? Yeah! we killed it. LBR went straight down 4.5X. Booking grew more than 2.5X, around 50% conversion boost and customer satisfaction out of the roof.

In our Customer-Engineer Day, where the developers call the customers straight from their desks, we heard a joyous customer telling us

how their family could travel with a confirm ticket on Diwali, only with the help of Goibibo’s Train alternate availability feature.

No number is bigger than the customer’s satisfactory experience.

The team is on their toes, innovating and building better functionalities, to ensure our customers have a delightful trains booking experience. Looking forward to host you on the next-gen train’s booking platform — Goibibo.

Next Station — Goibibo! was originally published in Backstage on Medium, where people are continuing the conversation by highlighting and responding to this story.