Peeling the onion: The layers in a state-of-the-art recommendation system

Recommendation systems don't really need an introduction, they are used on nearly every digital platform and have become one of the staples of the data science toolkit. Up to 35% of Amazon's revenue is driven by their advanced recommendation systems [source], and even less advanced systems can easily claim double digit impact figures. Moreover, customers have grown accustomed to the comfort these systems provide during their on-line shopping experience, reducing the time they have to spend searching for the right products.

Whereas the basic principle of "finding people the stuff they are likely to enjoy" is very simple, it is good to think carefully about the overall structure of this system. This article gives an overview of this structure, and provides two key rules to keep in mind when creating recommendation engines.

A well-implemented recommender can be compared to an onion in that each layer builds upon its predecessor, adding functionality and improving performance as you go along. Starting at the 'core' of the onion, the key layers that can be observed are the following:

1. Core algorithm: The basic intelligence that scores various products or items for specific users. Collaborative filtering is the generally agreed upon golden standard for the core algorithm, although content-based or hybrid approaches are also in use. This is the place where the actual intelligence resides - i.e. where the magic happens - and is the place where a lot of data-geeks will love to spend time working on.
2. Business rules: Additional logic that favors or eliminates specific types of recommendations. Oftentimes the expertise of the people who have been running the company can be invaluable when tweaking the algorithm and judging whether or not recommendations are sensible. Business logic can take many shapes or forms, but in the most general form it comes down to adding or eliminating specific recommendations for specific types of clients. An example of this is recommending only clothing which is targeted at the gender of the client. Alternatively, business rules can also be used to favor specific products in order to overcome the cold-start problem of collaborative filtering; for example by recommending new movies to clients in spite of having a still insignificant amount of views to base recommendations on.
3. Displaying recommendations: The manner in which the recommendations are presented to the customer. Placing the recommendations in the right place where the customer will pay attention to them, and then selecting the right sequence in which the recommendations will be presented. This also means guaranteeing that the recommendations have a certain degree of diversity. For an application like Netflix this could mean presenting movies from diverse genres rather than only from a viewer's most favored genre. Such things are especially important when catering to families and you are unable to know with certainty which individual is navigating your product selection.
4. Integration with customer experience: Choosing at which point in the process the customer will be presented with recommendations. Recommendations can be used at many stages throughout the customer journey. For fashion retail customers can be presented with recommendations while they are browsing the website, in emails sent during direct marketing campaigns, in email campaigns directed at people who abandon their carts, or even during an in-store experience.

Building a recommendation system means starting out from the core and moving outwards. At each stage of this process the designer is presented with a plethora of options, and taking a wrong turn at any of the stages can have a serious detrimental impact on the performance of the algorithm.

Inversely, better performance at the core of the 'onion' translates in significantly better performance of the complete system. Because each layer of the onion builds upon the next, it is of paramount importance to make good design choices in the lower levels. Growing the core of the onion has a disproportionate effect on the overall volume of the finished product.

When constructing this system it is of the utmost importance to keep two guidelines into account:

Rule 1: Keep it as simple as possible (at least at the start)

It can be very tempting to try out the newest technique you have read about in some academic journal, or to try and collect an exhaustive list of the business rules your most experienced people use in their day-to-day interactions with clients. But however tempting this zealous approach can be for someone who truly loves working with algorithms, it must be avoided at all costs.

Beyond the simple economical logic which dictates that you should try to keep the development cost and time as low as possible, this also has a very good reason from a data-scientific perspective: simple algorithms will oftentimes outperform more complex approaches. Hence, it is not unimaginable that one would spend twice as much time on a more complex algorithm that actually performs worse than a more complex system - and then everybody loses.

Starting out simple gives you the ideal measuring stick, which is needed if you want to measure the impact of extra bells and whistles you would like to add to your system. This brings us to the second important guideline when constructing these systems:

Rule 2: Test to make sure you are not making things worse

One of the key risks when creating a recommendation system is making performance worse by adding unnecessary complexity.

This can be true when adjusting things within a layer of the onion, for example adding content-based logic on top of the collaborative filtering at the core of the algorithm. When doing something like this it is of the utmost importance to execute a well-constructed experiment to see if this added complexity adds value. If no statistically significant improvement can be measured don't be afraid to use Occam's razor, and go for the option with the lower degree in complexity.

Testing is also of paramount importance when moving towards the higher layers. Whereas the core algorithm is guaranteed to be using a rational strategy which is supported by the data and mathematical principles, the same is in no way guaranteed for the business logic which is added on top. For instance, the earlier example where recommendations are purposefully limited to within-gender recommendations could possibly be too restrictive when someone is also buying clothes for their spouse or even their children. As such, it is important to measure the impact of whatever logic you are adding to your model, departing from gut-based decision making.

Adhering to these principles and looking at the bigger picture when designing recommendation systems goes a long way in guaranteeing quality recommendations and a solid business impact. Make sure that your recommendation engine makes people cry tears of happiness rather than tears of desperation.