SHARK: A Lightweight Model Compression Approach for Large-scale Recommender Systems
Beichuan ZhangChenggen SunJianchao TanXinjun CaiJun ZhaoMengqi MiaoKang YinChengru SongNa MouYang Song
4
Citation
16
Reference
10
Related Paper
Citation Trend
Abstract:
Increasing the size of embedding layers has shown to be effective in improving the performance of recommendation models, yet gradually causing their sizes to exceed terabytes in industrial recommender systems, and hence the increase of computing and storage costs. To save resources while maintaining model performances, we propose SHARK, the model compression practice we have summarized in the recommender system of industrial scenarios. SHARK consists of two main components. First, we use the novel first-order component of Taylor expansion as importance scores to prune the number of embedding tables (feature fields). Second, we introduce a new row-wise quantization method to apply different quantization strategies to each embedding. We conduct extensive experiments on both public and industrial datasets, demonstrating that each component of our proposed SHARK framework outperforms previous approaches. We conduct A/B tests in multiple models on Kuaishou, such as short video, e-commerce, and advertising recommendation models. The results of the online A/B test showed SHARK can effectively reduce the memory footprint of the embedded layer. For the short-video scenarios, the compressed model without any performance drop significantly saves 70% storage and thousands of machines, improves 30% queries per second (QPS), and has been deployed to serve hundreds of millions of users and process tens of billions of requests every day.Keywords:
Memory footprint
Component (thermodynamics)
It has become challenging to find relevant information needed for the user. In this evolving world of technology finding relevant data has become very crucial as most of the businesses revolve around data. To solve this problem, recommender systems are used. It has become very difficult to get relevant data without any proper recommender systems. Today, in every field recommender systems are used to provide relevant data to user on the basis of their choices, needs or interests. Content based recommender system and collaborative filtering recommender systems are two basics types of recommender systems are available. These two systems can be combined to make recommender system more efficient, these combined systems are called hybrid systems.
The purpose of the paper is to help new researchers to understand the working of basic recommender systems and identifies new research area for further improvement of recommender system.
Cite
Citations (0)
In this paper, we investigate novel quantization approaches to reduce memory and computational footprint of deep neural network (DNN) based keyword spotters (KWS). We propose a new method for KWS offline and online quantization, which we call dynamic quantization, where we quantize DNN weight matrices column-wise, using each column's exact individual min-max range, and the DNN layers' inputs and outputs are quantized for every input audio frame individually, using the exact min-max range of each input and output vector. We further apply a new quantization-aware training approach that allows us to incorporate quantization errors into KWS model during training. Together, these approaches allow us to significantly improve the performance of KWS in 4-bit and 8-bit quantized precision, achieving the end-to-end accuracy close to that of full precision models while reducing the models' on-device memory footprint by up to 80%.
Memory footprint
Keyword spotting
Learning vector quantization
Footprint
Spotting
Cite
Citations (21)
In the recent years, the Web has undergone a tremendous growth regarding both content and users. This has lead to an information overload problem in which people are finding it increasingly difficult to locate the right information at the right time. Recommender systems have been developed to address this problem, by guiding users through the big ocean of information. Until now, recommender systems have been extensively used within e-commerce and communities where items like movies, music and articles are recommended. More recently, recommender systems have been deployed in online music players, recommending music that the users probably will like. This thesis will present the design, implementation, testing and evaluation of a recommender system within the music domain, where three different approaches for producing recommendations are utilized. Testing each approach is done by first conducting live user experiments and then measure recommender precision using offline analysis. Our results show that the functionality of the recommender system is satisfactory, and that recommender precision differs for the three filtering approaches.
Information Overload
Cite
Citations (6)
The purpose of this study is to provide a comprehensive overview of the latest developments in the field of recommender systems. In order to provide an overview of the current state of affairs in this sector and highlight the latest developments in recommender systems, the research papers available in this area were analyzed. The place of recommender systems in the modern world was defined, their relevance and role in people's daily lives in the modern information environment were highlighted. The advantages of recommender systems and their main properties are considered. In order to formally define the concept of recommender systems, a general scheme of recommender systems was provided and a formal task was formulated. A review of different types of recommender systems is carried out. It has been determined that personalized recommender systems can be divided into content filtering-based systems, collaborative filtering-based systems, and hybrid recommender systems. For each type of system, the author defines them and reviews the latest relevant research papers on a particular type of recommender system. The challenges faced by modern recommender systems are separately considered. It is determined that such challenges include the issue of robustness of recommender systems (the ability of the system to withstand various attacks), the issue of data bias (a set of various data factors that lead to a decrease in the effectiveness of the recommender system), and the issue of fairness, which is related to discrimination against users of recommender systems. Overall, this study not only provides a comprehensive explanation of recommender systems, but also provides information to a large number of researchers interested in recommender systems. This goal was achieved by analyzing a wide range of technologies and trends in the service sector, which are areas where recommender systems are used.
Robustness
Relevance
Cite
Citations (3)
Recommender systems were introduced in the mid-1990s to help people select the most suitable product for them from the plethora of options available with them. The idea that led to their development was that we people often rely on the opinions of our peers before trying something new, say it be before buying a smart phone, a laptop, before going for a movie, before going to a new restaurant and even before visiting a doctor. Till date, we have numerous recommender systems developed for various areas, using different recommendation approaches. Yet, there are still a few limitations of recommender systems that need to be worked on. In this paper, we present an overview of recommender systems, the various approaches of recommender systems, the application areas for which various recommender systems have been developed and we also present the limitations of recommender systems. Keywords: Application Ares, Limitations, Metrics, Recommender Systems, Recommendation Appaches, Recommendation Algorithms
Laptop
Cite
Citations (37)
Researchers still believe that the information filtering system/ collaborating system is a recommender system or a recommendation system. It is used to predict the or of a user to an item. In other words, both predict rating or preference for an item or product on a specific platform. The aim of the paper is to extend the areas of the recommender system/recommendation systems. The basic task of the recommender system mainly is to predict or analyze items/product. If it is possible to include more products in the system, then obviously the system may be extended for other areas also. For example, Medicine is a product and doctors filter the particular medicine for the particular disease. In the medical diagnosis doctors prescribed a medicine and it a product. It depends on the disease of the user/patient so here doctor predicts a medicine or product just like an item is recommended in a recommender system. The main objective of the paper is to extend the Recommender System/Recommendation system in other fields so that the research works can be extended Social Science, Bio-medical Science and many other areas.
Cite
Citations (0)
A recommender system, which might assist in providing clients with new information and a better experience, is becoming increasingly popular in this era of modernization. Recommender systems are often used by various platforms to provide new products to consumers, which may also help in improving product sales. Additionally, the recommender system is essential in academic domains. It is common for users to take a while to find and access the materials they need. The recommender system is now available, which could reduce the time spent looking for materials and improve student achievement. Therefore, it is crucial to explore more on the theory and implementation of the recommender system. This paper aims to study a few types of recommender system techniques and implement it in the research article recommender system. Additionally, related research on each of the three recommender systems will be reviewed, along with a description of the related study, the dataset used, and the evaluation method.
Cite
Citations (1)
Recommender systems help humans in filtering and finding the right information from the enormous amount of data. Hostels are more famous than hotels for solo travelers, but no prior research related to recommender systems has been conducted in this domain. Hostels allow users to provide multi-criteria ratings and traditional recommender systems are not able to provide effective recommendations in case of multi-dimensionality i.e. contextual information and multi-criteriaratings. So, we have proposed a novel hybrid recommender system (SAFCHERS) that chooses the hostel's features for computation dynamically and provides explainable and better recommendations than the traditional recommender systems.
Cite
Citations (1)
Researchers still believe that the information filtering system/ collaborating system is a recommender system or a recommendation system. It is used to predict the "rating" or "preference" of a user to an item. In other words, both predict rating or preference for an item or product on a specific platform. The aim of the paper is to extend the areas of the recommender system/recommendation systems. The basic task of the recommender system mainly is to predict or analyze items/product. If it is possible to include more products in the system, then obviously the system may be extended for other areas also. For example, Medicine is a product and doctors filter the particular medicine for the particular disease. In the medical diagnosis doctors prescribed a medicine and it a product. It depends on the disease of the user/patient so here doctor predicts a medicine or product just like an item is recommended in a recommender system. The main objective of the paper is to extend the Recommender System/Recommendation system in other fields so that the research works can be extended Social Science, Bio-medical Science and many other areas.
Cite
Citations (1)
This chapter presents a brief and systematic overview of four major advanced recommender systems: group recommender systems, context-aware recommender systems, multi-criteria recommender systems, and cross-domain recommender systems. These advanced recommendations are characterized and compared in a unifying model as extensions of basic recommender systems. Future research topics and directions in the area of advanced personalized recommendations are discussed. Advanced recommender technologies will continue to advance.
Cite
Citations (1)