The Client is a large e-commerce retailer offering a wide range of products such as electronics, apparel, stationery, home decor, and groceries.
Today, every e-retailer is employing innovative methods to engage the end user. In an era where companies spend millions of dollars on acquiring new users, e-retailers are investing heavily on engaging and retaining their targeted users. This process requires websites to become increasingly personalized and customized towards the tastes of their users. An increasing number of players in this space are using recommendation engines to engage users. Simultaneously, the e-retailer who know their target audience well can engage them better.
Core Compete developed its recommendation engine using the SAS Real-Time Decision Manager (now branded as SAS Decision Manager (DM) and SAS Intelligent Decisioning Tool) and Hadoop technologies hosted on the Google Cloud Platform (GCP). The system processes a throughput of 3000 transactions per second (tps) in less than 80ms per transaction. Leveraging the cloud capabilities of GCP, the system can scale vertically and horizontally to process higher throughput in much lesser time.
Data management and orchestration play a central role in the development of a recommendation engine. The two types of data – real-time data and historical data are used by the system to create a user profile and make recommendation decisions. On the Google platform, clickstream data is collected from the website in the form of JSON packets and stored in the HBase database. These packets contain information such as product view action, cart view action, banner click action, etc. which help the system track the user activity on the website. This stream of information is consumed and redirected using Kafka and NiFi brokers. The customer profile data and summarized transaction data are stored in HBase using batch processes. This data is consumed by SAS DM to implement pre-defined business logic and score the data in real-time to generate decisions customized to each individual user.
On a user visit to a product page on the website, a request is triggered to the recommendation system. The request is redirected to one of the many Decision Manager engines by an application load balancer functioning on the round robin principle. The real-time session data, as well as historical data of the user stored in HBase, are consumed by DM to prepare a decision.
DM performs the following functions to generate a recommendation:
- Ingests user-specific input variables in the request
- Queries appropriate data from HBase DB using REST services
- Applies business logic (and potential real-time model scoring)
- Generates a decision response, suitable for the user
The generated response contains the list of recommended products to be shown to the user on the product page. In this project, the output list is dependent on the products viewed by the user in the previous sessions, the products purchased by the user, a generic products affinity model – quantifying relationship between products, the tenure of user’s relationship with the website and the list-placement location on the product page. The decisioning process can further be improved by including more user information and more complex product affinity models into the system.
This is just one of the examples of real-time personalisation in the retail industry. However, the necessity of real-time engagement and interactions is realised not only in the retail sector but also in other sectors such as Banking and Finance and Telecommunication. The systems can be used to validate credit eligibility for users in real-time. Institutions offering credit cards, loans, financial offers, etc. can perform eligibility checks in the real-time for its users, using the SAS DM and intelligent decisioning tool. The fast-paced decisioning along with real-time scoring methods can deliver high business value to the end users of the financial institutions.
In the telecom industry, the speed of delivering the next best offer to a subscriber plays a vital role in engaging them. With technologies such as geo-fencing, the end user can be offered attractive discounts on various brands or can be offered various personalized package plans. For example, SAS DM and the intelligent decisioning tool can be used to generate brand specific offers in real-time for a user who walks past a branded retail store. Such offers can be personalised based on user preferences and resource usage. Thus, SAS Decision Manager tool can play a pivotal role in increasing customer engagement for a wide range of applications across industries.