ICACT20220017 Slide.18
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Thank you for your attention. The floor is now open for any questions or comments.
|
|
ICACT20220017 Slide.17
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In conclusion, we have demonstrated that upsell return can be optimized using a binary classifier with misclassification costs. Training and testing the model produced very good performance results. The model outperformed the current traditional approach used. For future work we can look at maximizing long-term profitability of upselling. Finally, we can recommend products within a defined future time window (for example every quarter).
|
|
ICACT20220017 Slide.16
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The success rate S increases as η increases, as previously shown. Here, we see that S ranges from 65.4% to 86.1%. When the
semi-manual method of approaching customers was employed,
there was a total of 34 successful upsells out of a total of
80 customers approached, giving a 34/80 = 42.5% success
rate. Hence, the utilization of a predictive logistic regression
model to predict customer upsell for any given value of η
dramatically improves the success rate that was achieved using
the traditional semi-manual approach by at least 22.9%. The number of target customers selected for future approach for upsell is outlined in the last row of the table. We observe that this number decreases as increases, since the number of predicted outcomes is decreased.
|
|
ICACT20220017 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Here we see that the cost-optimal upsell cost is always less than or equal to the accuracy-optimal upsell cost for varying values of η, which is an expected result.
|
|
ICACT20220017 Slide.14
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
From this table, we can confirm that as η increases, the threshold t for optimizing cost also increases. Moreover, the accuracy of the model applied to both cross-validated training set and test set is also increased with increasing values of . It appears that the model performs better on the test data since the accuracy values are slightly higher.
|
|
ICACT20220017 Slide.13
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
We evaluated costs using different values of η.
We then compare the cost of the cost-based optimal result
C_cost with the cost of the accuracy-based optimal result
C_accuracy. This comparison is achieved by calculating the ratio
of the two costs R where the cost of the cost-based optimal
result is the numerator. In this figure we plot the value of R for various values of η. When η = 0 then false positive errors do not cost anything and therefore it is optimal to always choose a positive outcome in which case R = 0. If η = 1 then this corresponds to unit costs for both types of errors and hence the resulting optimal probabilities are the same that would be obtained for accuracy and hence in this case we get R = 1.
|
|
ICACT20220017 Slide.12
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
This figure shows the performance of Cross-Validated Training Data with Cost Optimal Threshold (blue) when η = 1 and Default Threshold (red) when η = 0.1. The trade-off between accuracy and cost when η = 0.1 and when η = 1 is clearly shown in this figure. For η = 0:1 the cost-optimal tuned threshold value was t =
0:0935 and the optimal expected cost was 0.2842. However,
there was a reduced accuracy of 76.3% when compared to the
accuracy if no costs are implemented (89.5%). In contrast, for
η = 1 costs are ignored and the threshold t selected optimizes
accuracy rather than cost where t = 0:5. The expected cost
for the accuracy-based optimal result was found to be much
higher with a value of 0.6737.
|
|
ICACT20220017 Slide.11
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The expected cost is given by equation 1.
A necessary condition for minimizing C bar can be obtained as shown in equation 2 where the threshold value t can be changed by varying η where η is given by this expression.
|
|
ICACT20220017 Slide.10
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Let cij denote the cost of classifying an instance of class i as class j .
Denote the pdf of the scores for class ’No’ instances by f0(s) and for class ’Yes’
scores by f1(s)
Denote the corresponding cumulative distribution functions by F0(s) and F1(s)
The prior probability of class j ϵ {’No’, ’Yes’} is denoted by πj.
|
|
ICACT20220017 Slide.09
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Initially when our model is trained without incorporating
costs, the resulting confusion matrix on the cross-validated
training set was computed as shown in the left table. This therefore gives an accuracy of 89.5%, with a success rate of 88.9%. However, there are many false positives (24) and false negatives (36) which would negatively impact the telecommunications provider financially. The cost of a false positive, that is, incorrectly predicting an upsell as successful, is much less than the cost of a false negative, that is, incorrectly predicting an upsell as unsuccessful. The cost of a false positive may only account for administrative costs. On the other hand, the cost of a false negative is the foregone benefit that would have been obtained if the customer was approached for upsell, and this benefit would be the revenue gained from the sale of the product. When applying cost weightings to the model, the number of false negatives is significantly reduced (3), as shown in the table on the right.
|
|
ICACT20220017 Slide.08
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The data was divided into two subsets as determined by the 'Approached for Upsell' variable. There were 270 customers who were approached for upsell during February 2018 – February 2019, while the remaining 2847 customers were not approached for upsell during this time period. We first split the data into two sets: one containing customers who have been approached for upsell in the past and one containing customers who have not yet been approached. Using the subset of customers who were already approached for upsell, the data was then split into a training and test set with a 70%/30% split, so the training set had 190 observations and the test set had 80 observations. The evaluation of performance metrics was done using repeated k-fold cross-validation with k = 10 using 3 repeats. The trained and tested model was then applied to customers who have not yet been
approached. This would result in the generation of new targets
which would then be ranked based on whether they have a high probability of choosing to upsell. There will of course still be a set of untargeted customers which the model will not recommend for approach.
|
|
ICACT20220017 Slide.07
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Before model construction, feature selection was performed so as to utilize the most important features of the customer data to maximize the model’s predictive power. We computed a rank of the variables by assigning an importance value to each feature with the use of information gain. The top five features were found to be the most important with a positive information gain while the information values for all other variables were computed to be zero. These top five features are PercentLegacyProducts, PercentLegacyRental, PercentLegacyMins, LastCISInstall and LegacyMins.
|
|
ICACT20220017 Slide.06
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Whether or not the customer has PBX, Single Lines, T1s and Trunk Lines. In addition, we have whether or not the customer was approached for upsell and if they were then whether or not the upsell was successful. These last two variables will be used as the target variables in our binary classifier model.
|
|
ICACT20220017 Slide.05
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Numbers of PBX, Single Lines, T1s, Trunk Lines, legacy product and rental information,
|
|
ICACT20220017 Slide.04
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The dataset used in this study was provided by a telecommunications service provider and applies to corporate customers. The dataset contains basic customer demographic information, transaction data and sales records concerning legacy products that the customer purchased. Here, we illustrate the data variables, data type and description of the attributes. We have demographic details such as Parent ID, Company ID, Sector, Company Age, Company Type, Industry, Sub-Industry, whether or not the customer signed a VIP contract, and the number of months until the VIP expires. Some transaction data include the number of days since the last CIS was installed, the number of CIS exchanges,
|
|
ICACT20220017 Slide.03
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
We consider upselling as the purchase of additional products and add-ons, and/or the switch from ’legacy’ products to upgraded ones. Traditional methods of upselling rely on a semi-manual approach to marketing such products to clients. One such provider assigns an Account Manager to corporate customers. Each manager will meet with their customers on a scheduled basis to discuss their needs, problems, and new product offerings which could result in upsell opportunities. However, some account managers may not completely understand each customer’s preferences, and this could lead to inappropriate recommendations. There is thus a greater chance for the semi-manual process to be time-consuming, subjective and costly.
|
|
ICACT20220017 Slide.02
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Today I will be outlining the details about upselling telecommunications products and why it poses a challenge. I will also discuss the proposed solution which is a recommender system that uses a logistic regression binary classifier associated with misclassification costs. I will then delve into the experimental model using data sourced from the telecommunications service provider. Finally, I will present some results obtained.
|
|
ICACT20220017 Slide.01
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Hi everybody, my name is Navin Dookeram. In this presentation for ICACT 2022, I have collaborated with Mrs. Zahira Hosein from the Telecommunications Service of Trinidad and Tobago, and Professor Patrick Hosein from the University of the West Indies, St. Augustine. Our presentation today is about a recommender system for the upselling of telecommunications products.
|
