Abstract:
Purpose – Any business that opts to adopt a recommender engine (RE) for various potential benefits must
choose from the candidate solutions, by matching to the task of interest and domain. The purpose of this paper
is to choose RE that fits best from a set of candidate solutions using rule-based automated machine learning
(ML) approach. The objective is to draw trustworthy conclusion, which results in brand building, and
establishing a reliable relation with customers and undeniably to grow the business.
Design/methodology/approach – An experimental quantitative research method was conducted in which
the ML model was evaluated with diversified performance metrics and five RE algorithms by combining offline
evaluation on historical and simulated movie data set, and the online evaluation on business-alike near-realtime data set to uncover the best-fitting RE.
Findings – The rule-based automated evaluation of RE has changed the testing landscape, with the removal of
longer duration of manual testing and not being comprehensive. It leads to minimal manual effort with highquality results and can possibly bring a new revolution in the testing practice to start a service line “Machine
Learning Testing as a service”(MLTaaS) and the possibility of integrating with DevOps that can specifically help
agile team to ship a fail-safe RE evaluation product targeting SaaS (software as a service) or cloud deployment.
Research limitations/implications – A small data set was considered for A/B phase study and was
captured for ten movies from three theaters operating in a single location in India, and simulation phase study
was captured for two movies from three theaters operating from the same location in India. The research was
limited to Bollywood and Ollywood movies for A/B phase, and Ollywood movies for simulation phase.
Practical implications –The best-fitting RE facilitates the business to make personalized recommendations,
long-term customer loyalty forecasting, predicting the company’s future performance, introducing customers
to new products/services and shaping customer’s future preferences and behaviors.
Originality/value – The proposed rule-based ML approach named “2-stage locking evaluation” is selflearned, automated by design and largely produces time-bound conclusive result and improved decisionmaking process. It is the first of a kind to examine the business domain and task of interest. In each stage of the
evaluation, low-performer REs are excluded which leads to time-optimized and cost-optimized solution.
Additionally, the combination of offline and online evaluation methods offer benefits, such as improved quality
with self-learning algorithm, faster time to decision-making by significantly reducing manual efforts with endto-end test coverage, cognitive aiding for early feedback and unattended evaluation and traceability by
identifying the missing test metrics coverage.