ICACT20220012 Slide.17
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Thanks kindly for your attention! We now welcome any comments or questions. Hello everyone, my name is Mariella Rivas. For ICACT 2022, I have collaborated with Mr. Richard Roach and Prof. Patrick Hosein from The University of the West Indies.
|
|
ICACT20220012 Slide.16
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
References are listed
|
|
ICACT20220012 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In conclusion, both CART Decision Tree models achieved acceptable predictive accuracy and can serve as suitable foundations for further development. However, the cost-sensitive approach is highly recommended for future research due to its proven ability to provide the telecommunication company with tangible cost savings despite possible prediction errors in detecting fraud. A revision or resample of the training and testing datasets, possibly considering other call features not currently studied, may also assist in achieving a refined model performance.
|
|
ICACT20220012 Slide.14
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Conclusion and Future Work
|
|
ICACT20220012 Slide.13
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
While the accuracy-based model outperformed the cost-sensitive Decision Tree in terms of predictive accuracy by approximately 5%, we compare the cost of optimal results from both models using the cost ratio R, and examine this cost ratio for various values of η. We note that as error rates increase, it is expected that cost savings will also increase. This can potentially provide the telecommunication company with tangible cost savings when faced with the possibility of prediction errors, and thus presents a major advantage over the traditional approach to predicting fraudulent calls.
|
|
ICACT20220012 Slide.12
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
A false positive error result in poor service to the customer if a call is incorrectly terminated, while. a false negative error can bring about significant financial loss to the company. With the cost-sensitive approach to the CART DT model, we assign costs associated with each outcome of True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN) and then make predictions with the objective of minimizing total cost of outcomes, C̅, instead of minimizing error. Considering the cost of true predictions as zero, the cost of a FN error as 1, and for example, the cost of a FP error as one tenth the cost of a false negative (η = 0.1), a cost-sensitive CART Decision Tree was developed using the same data as the previous accuracy-based model. This approach yielded a predictive accuracy of 84.5%.
|
|
ICACT20220012 Slide.11
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
For our model, the dataset of the 18 selected features for 199,771 calls was split as 75% and 25% as training and testing sets respectively. The CART Decision Tree model was chosen due to its transparent, comprehensive nature and its ability to provide the telecommunication company with ease of use. Its predictive performance is evaluated by observing the resulting confusion matrix of correctly classified vs. mis-classified cases to calculate the model’s predictive accuracy (accuracy closer to 100% is preferred). The area under the Receiver Operating Characteristic Curve , commonly called AUC and ranging between 0 and 1, is also considered, with values closer to 1 being favoured. This model achieved a predictive accuracy of 89.7% and area under the Receiver Operating Characteristic Curve of 0.91, indicating acceptable performance, but with room for improvement.
|
|
ICACT20220012 Slide.10
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Using the 18 selected variables for a sample of 1000 calls (787 legitimate, 213 fraudulent), Ward’s Hierarchical Clustering algorithm with Euclidean distances under complete linkage was applied to further understand how the selected features may influence the classification of calls, thereby verifying that an appropriate combination of features are being considered in our model. This method builds a top-down tree-like hierarchy of clusters (data groups) by beginning with all the data points in one initial cluster and splitting clusters recursively until each data point has been assigned to a single cluster, and is particularly useful for large datasets. The algorithm’s effectiveness is evaluated by calculating its divisive coefficient, which measures the amount of clustering structure found. Coefficients close to 1 are preferable, indicating that the clusters formed describe the data well. In our case, a coefficient of 0.97 was obtained.
|
|
ICACT20220012 Slide.09
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
The dataset originally consisted of 23 premium rate call features. However, we examined each variable’s information gain and pairwise correlation between variables to eliminate 5 redundant features, leaving 18 as selected variables for further exploration. Information gain usually ranges between 0 and 1, with higher values preferred – indicating that the variable in question has higher influence on the independent variable. The correlation between each pair of variables takes values between -1 and 1, with absolute values closer to 1 indicating stronger relationship between the variables. Thus, features with information gain below 0.4 and/or correlation to another feature above 0.8 were considered negligible. To the right of this slide, we highlight the most influential call features in classifying premium rate calls as legitimate or fraudulent.
|
|
ICACT20220012 Slide.08
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Our study focused on a use case of a small developing island with a complex telecommunication infrastructure, but with constraints regarding fraud monitoring/management resources. Using historical premium rate calls data, we firstly identify the most influential parameters for detecting fraud by performing feature selection through considering the call variables’ information gain and pairwise correlation. Ward’s Hierarchical Divisive Clustering algorithm is then applied to the data using the selected features. Different from the related works, a traditional accuracy-based Classification and Regression Tree approach is adopted for classifying PRS calls, and compared to its cost-sensitive counterpart. The research aims to highlight the advantage of using cost-based analyses in the telecommunication industry, particularly as it relates to reducing revenue leakage associated with premium rate services fraud.
|
|
ICACT20220012 Slide.07
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Our Solution
|
|
ICACT20220012 Slide.06
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
In 2007 and 2020 respectively, Alberto Freitas et. al., and Shellyann Sooklal with Patrick Hosein, both studied medical data to explore cost-sensitive approaches to classification algorithms, with the outcomes promising remarkable advantage over traditional accuracy based classification.
|
|
ICACT20220012 Slide.05
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Several papers have presented machine learning as a popular and successful approach to premium rate services fraud detection. However, the situation varies by country. In 2015, Sharmila Subudhi and Suvasini Panigrahi explored a support vector machine approach to reducing the time in which fraudulent calls are detected accurately. In 2018, Hiuchen Li et. al. , also explored support vector machines, logistic regression, and random forest models using historical call data in China to develop models for accurately identifying telecom fraud – their best performing model was able to reduce malicious calls by 90%.
|
|
ICACT20220012 Slide.04
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Related work
|
|
ICACT20220012 Slide.03
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
PRS fraud occurs in either of two ways. In the first scenario, a premium rate number owner advertises his number to be called, for instance as a psychic hotline. Calls placed to the premium rate number are routed through the caller’s telecommunication company, but are then terminated on an Interactive Voice Response (IVR) system by a premium rate number provider. Extra “premium” charges are paid by the caller to their telecommunication company. These charges are paid to the premium rate number provider, who shares this revenue with the premium rate number owner. Alternatively, the fraudster may compromise a Private Branch Exchange (PBX) or IVR of a voicemail system, arranging for calls to be forwarded to premium rate destinations. The fraudster then places calls to the compromised number via a Public Switched Telephone Network (PSTN) or Voice over Internet Protocol (VOIP), which are routed through the hacked PBX to the high cost destination. The telecommunication company incurs costs to its carrier for these fraudulent calls, and the fraudster partners with the high cost destination to receive a portion of the revenue generated from the company. Thus, through either means, PRS fraud can imply significant loss of revenue for the telecommunication company.
|
|
ICACT20220012 Slide.02
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Premium Rate Services Fraud, commonly abbreviated as PRS fraud, refers to destination fraud, where illicit calls originate from a telecommunication network to a high-cost destination. This activity implies revenue leakage for telecommunication companies – a risk that is heightened by difficulties in accurately profiling fraudulent calls prior to their actual occurrences, especially in smaller developing countries. Therefore, early detection and termination are critical to reducing the high losses associated with such calls.
|
|
ICACT20220012 Slide.01
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Hi everyone, my name is Mariella Rivas. For ICACT 2022, I have collaborated with Mr. Richard Roach, and Prof. Patrick Hosein of The University of the West Indies. Today, our presentation will focus on a solution to detecting Premium Rate Services Fraud, with comparisons between accuracy-based and cost-sensitive approaches to the Classification Tree model.
|
