A Hybrid Machine Learning Approach for Improvised QoE in Video Services over 5G Wireless Networks

1  Introduction

In response to significant advancements in communication technologies and the widespread use of the internet, both content and service providers have begun providing video services to end users. Third-generation (3G) and fourth-generation (4G) mobile communication standards were developed due to the stable development of internet data services. Due to High internet traffic demands, technology is currently moving from its fourth-generation (4G) to its fifth-generation (5G) [1]. In particular, video streaming traffic has increased and has become a significant part of internet traffic. Mobile video traffic is expected to reach 73% in 2023 [2]. Several research aspects have been initiated as a result of this surge in video traffic. Particularly in recent years, the development of mobile data and smart devices is increasing. Both industry and academia have focused on 5G QoE research regarding video quality assessment [3]. Quality of Experience (QoE) revolves around how satisfied or dissatisfied users feel when using a multimedia service or application. Understanding the factors that affect QoE and allocating resources effectively to improve video quality to meet user expectations is crucial. There are approaches to evaluate Video QoE, including subjective tests [4], objective assessments [5], and data-driven analyses [6]. Among these options, data-driven research shows promise by utilizing datasets to measure user perception. It helps to overcome the limitations associated with subjective tests and objective assessments, such as costs and limited understanding of the human visual system.

To provide video services that satisfy users and their expectations, service providers and network operators must invest in QoE estimation [7,8], Due to this scenario, a model of economic interaction involves three types of participants: content providers (CPs), Internet service providers (ISPs), and end users. Their relationship is shown in Fig. 1. A bandwidth-based pricing model [9] and a flat-rate-based pricing model [10] are used to charge CPs and End Users by ISPs to enhance the QoE and QoS. Several factors may influence end users’ perceptions, including network, context, and content factors. These factors need to be considered when designing and optimizing video services. To guarantee the optimum experience for customers, a thorough examination of 5G video QoE should be conducted. To assess the QoE of a multimedia service, active users of video services will be surveyed using a subjective feedback method such as the MOS [11].

Figure 1: Relationships among Internet service providers (ISPs), content providers (CPs), and users

MOS is a well-known and accepted metric used to measure a multimedia service’s perceived quality. Additionally, it is a useful tool for evaluating the performance of different services and can help to identify areas of improvement. Table 1 shows scores from 1 to 5 denote bad, poor, fair, good, and excellent, respectively, according to the MOS standardized by the International Telecommunication Union (ITU) [12].

Most quality evaluation methods consider user behavior and contextual factors when considering evaluation approaches. Although the quality of experience is multidisciplinary, it is affected by factors such as economics, computer science, telecommunications, and social dynamics. A significant emphasis is placed on the effect of contextual factors in most of the existing research. It remains largely unexplored in academia how network factors and media-related parameters influence the process. Previous subjective studies suggest that the QoE for video services relies on various application QoS aspects. These include factors like the time taken for initial loading, the occurrence of rebuffering or stalling events, the quality of playback, and its fluctuations [13]. Due to encryption, network operators lack access to information about the video traces within their networks. The primary avenue for assessing the QoE in video services is to depend on network-level factors derived from encrypted video traces or utilize independent network measurement tools operating externally to the video application data plane. Previous studies on video QoE estimation have demonstrated that the performance of network-level factors, such as bandwidth, delay, and packet loss rate, directly influences QoE [14]. This prompts adopting supervised machine learning (ML) to establish a correlation between network-level measurements (network QoS) and QoE. The following are the primary contributions of this research work:

•   This research aims to identify how the quality of experience is affected by network and media-related parameters.

•   Analyse QoS metrics to determine which factors affect end-user QoE. Further, a two-level layering approach is used, in which various ML algorithms are used at the first level to predict the QoE of video services, and then the output of the first level is passed to the second-level Hybrid classifier for final prediction.

•   To validate the accuracy of the proposed work the performance analysis metrics were calculated.

The remainder of this article is organized as follows: Section 2 reviews the background works. Section 3 discusses the development of a QoE monitoring system. Section 4 presents a discussion of the Experimental evaluation. Section 5 includes the results and discussion. Finally, Section 6 concludes the work with future direction.

2  Related work

Internet traffic has increased in recent years, particularly IP video traffic, which accounts for most of all Internet traffic. In previous works, user behavior metrics were used to assess user engagement. In [15], the authors describe various methods for measuring engagement metrics, including loyalty, activity, and popularity. According to [16], the video view count is highly correlated with the popularity of its services on YouTube, and the content rating and favorites determine it. Video view counts are forecasted based on social sharing activities such as the number of times the videos are forwarded and the time of the session [17]. Video delivery over the internet and the user’s expectation for high-quality content are rising. This rise has strained internet infrastructure and existing video delivery technologies, resulting in increased latency in video data and reduced video quality. To satisfy the user’s needs, video service providers must know how QoS parameters affect user behavior [18].

As a result of examining the impact of degradation in quality of experience on different video types in [19], authors inferred two levels of user engagement. The first level measures the amount of time spent watching a video, and the second level measures the number of views per user and the total time spent watching videos. This analysis shows high buffering ratios reduce user engagement, while high bit rates deliver better video quality. Perceived quality of experience cannot be determined solely by social context factors and user engagement metrics. The correlation between QoS and users in a QoE service must be understood beyond social context factors. It has been noted in [20] and [21] that initial delays and stalling affect QoE in HTTP video streaming.

Moreover, the authors argue that QoE is also affected by video resolution, buffering time, and video playback. They also suggest that video QoE perception is affected by the amount of data transmitted during playback. Finally, the authors state that different QoE metrics can be used to evaluate the observed quality of video service.

To estimate the quality of video services, the authors in [22] have considered parameters such as bitrate, framerate, display size, and video content. The authors also point out that the streaming service provider can use these parameters to improve streaming video QoE. Furthermore, they suggest that machine learning methods can be beneficial to optimize QoE. The authors of [23] developed a mobile application to analyze the quality of experience and network parameters that can be installed on end users’ terminals. The application collects data from the user’s terminal, such as packet loss, jitter, and latency, and sends it to a cloud-based server. The data is then analyzed to provide real-time insights into the user’s network and service performance. The application can also be used to diagnose network problems and suggest solutions. According to [24], subjective studies and objective approaches are available to measure the QoE of adaptive video streaming. The researchers also compare machine learning-based and nonmachine learning-based models, concluding that the former exhibits superior results.

An SDN-based approach was proposed by [25] to prevent video freezes for HAS clients. Further, a framework based on machine learning was presented to assist clients in avoiding video freezes caused by congestion on the internet. An ML model was developed in another study [26,27] to investigate the relationship between QoS metrics and QoE to understand how end-users feel about QoE video services. Using supervised ML classification, the authors [28,29] demonstrated how QoS and quality of experience can be predicted and correlated for DASH video streaming in static and mobile scenarios. In this study, different ML algorithms are examined, but RFC is selected due to its highest accuracy of 76%.

This work observed that, although the QoE measures examined in the background work have been acquired in various forms, taking into account different influencing factors, there is a high priority given to social contests and user engagement. There is, however, a need for current research involving QoS, factors relating to video metrics, and the correlation between QoE. Further, Careful investigation of the results of the existing papers reveals that there is still a large scope for improving the accuracy metric.

To overcome the above issues, this paper proposes an innovative research agenda incorporating QoS and QoE metrics to understand user engagement and experience better. Then, a comparative analysis of the performance of the various ML classifiers with the 5G dataset is carried out to classify the quality of the video. Then the three best ML classifiers were selected based on training data at level 1 to enhance classification accuracy. Finally, a hybrid XGBStackQoE Classifier model is proposed; the model is created by concatenating the features of different machine learning classifiers at level 1, and its output is given to the proposed hybrid XGBStackQoE Classifier at level 2. Both academia and industry are exhibiting an increasing interest in this type of model. Table 2 illustrates a comprehensive comparison among machine learning-based Quality of Experience (QoE) prediction models designed for video services, including our novel proposed model.

3  Formulation of the QoE Monitoring System

This study examines the correlation between network parameters and end-user QoE [39]. This paper considers impact factors in the QoS metrics to be quality indicators for video streaming services. Further, it analyses the influence of various network parameters on the user experience and evaluates the performance of existing solutions. The results of the proposed system can be used to develop strategies for improving the quality of video streaming services. Using parameters related to QoS and content, this paper developed a framework that predicts user quality of experience based on the perceived quality of the end user. This framework helps us accurately assess user satisfaction and provides valuable data to improve user experience. QoS parameters and video metrics are highly correlated with the quality of experience. This framework can monitor user satisfaction over time, allowing us to spot trends in user behavior and make necessary adjustments. As a result of this outcome, the proposed work can determine that QoS metrics are the primary indicators of user satisfaction with a video.

3.1 QoE Measurement

The framework, as shown in Fig. 2, is divided into four sections. A parameter selection section evaluates the relationships between network parameters, selects the highly correlated parameters, and seeks to identify optimal parameters for a given task. Finally, the parameters are adjusted and tuned to improve performance. The parameters for this study were chosen based on literature and research. The parameters from the previous step are used to develop a metric used as input to the ML model in the QoE estimation section. The MOS prediction section computes the MOS using the ML model. The values from the previous sections are used in the QoE tracking section to keep track of the MOS.

Figure 2: QoE management system

This paper aims to examine the problem of categorizing video streaming quality into two categories: “bad” and “good”. A binary-dependent variable is used to denote the quality of experience. This variable can only take two values, either “0” or “1” as shown in Table 1. The number “1” represents “good” and the number “0” represents “bad”. The MOS score is classified as “bad” when it is within the (1–3) range and as “good” when it is within the (4–5) range.

3.2 Dataset Description

For this study, the 5G dataset available in [40] is used, which is accurate and adaptable to our purposes. In addition, this dataset covers the features this work needed to conduct our study. The 5G dataset contains network parameters and video metrics information for approximately 3756 samples.

To deal with the imbalance in the 5G dataset, methods such as imputation and under-sampling are used to standardize the data. After data processing, the final set of data is obtained. This data is essential to predict the quality of a video streaming session. A single sample provides network-level data and metrics that can be used to estimate user behavior at the network level. It allows us to analyze the correlation between the network parameters and video metrics to determine the optimal streaming conditions. Then, the performance evaluation metrics are used to evaluate and compare the model’s performance to other models. The proposed work found satisfactory results, and the model achieved the desired accuracy.

3.3 Hybrid XGBStackQoE Classifier

A novel hybrid XGBStackQoE analytical model using a two-level layering technique is shown in Fig. 3, in which multiple ML classifiers, like Decision Tree (DT), Bagging Classifier, Random Forest (RF), Adaboost Classifier, Gradient Boosting Classifier, XGBoost [41] is analyzed. The best three models are chosen based on their training and testing data accuracy for the first level. The layer 2 Hybrid XGBStackQoE-Model is fitted using the outputs (meta-features) of the layer 1 ML models. The Hybrid XGBStackQoE-Model is trained using the class labels predicted by the various ML models. The level 2 classifier is used to generate the final prediction.

Figure 3: Supervised ML-based QoE estimation workflow

The XGBStackQoE Classifier algorithm describes the steps used in the proposed work. The input training dataset C={xi,xi+1,…,xnandyi,yi+1,…,yn} has dimensionality with i = 1 to n. The training data position is indicated by the feature vector xi, which is a vector with i = 1 to m dimensions. yi is the class label associated with the ith training data. First-level classifiers in step 1 are trained with training and testing datasets. In steps 2 to 4, the for loop from t=1toT is used to fit first-level models. In step 5, a new dataset is created from the training set. From steps 6 to 8, the for loop runs from i=1ton and contains the modified feature vector xi′. It contains the first-level classifier’s predictions. The predicted class labels from level one are passed to step 9, where second-level classifiers learn from these predictions. Finally, the final prediction is generated using the level 2 classifier H(x)=c′(C1(x),C2(x),…,CT(x)).

3.4 Implementation

Google Colab Python 3.10.12 was used to run the classification models in a cloud environment. Intel Xeon CPU running at a top speed of 2.20 GHz with 12 GB of RAM. The proposed work utilizes a desktop PC with Windows 11 Operating System.

4  Experimental Evaluation

To create a model for QoE forecasting based on network-related parameters, this paper will examine the impact of different feasible parameters. The proposed work investigates the effects of these parameters on the user’s QoE. This work will analyze the model’s results to determine which parameters influence the user’s QoE most. Further, this work aims to present a method of evaluating MOS based on ML algorithms and other performance evaluation metrics. To assess the video QoE, ML models are trained with network and content parameters.

For training and testing the ML model, the data is obtained from a 5G dataset that contains subjective video information related to the network and content. Furthermore, 30% of the data is used for validation and 70% for training to determine the correlation between QoS and QoE metrics and estimate the MOS. To obtain better accuracy results for classification problems, this work applied three machine learning algorithms at level 1: Bagging Classifier, Random Forest, and Gradient Boost Classifier, followed by the XGBStackQoE Classifier at level 2.

As shown in Fig. 4a, if the Round-Trip Time (RTT) is high, the MOS of the video is lower, whereas if the RTT is low, the MOS of the video is higher. As we can see, for MOS to be high, the RTT value should be around 25 milliseconds; if the RTT value is high, the MOS of the video is reduced to 4, 3, 2, and 1. Therefore, RTT and MOS are inversely proportional. So, when the value of RTT is high, the value of MOS for the video will be low; this is the main information this work finds.

Figure 4: Analysis of network metrics with MOS

According to Fig. 4b, if the stall duration is high, the MOS of the video is low, and if the stall duration is low, the MOS of the video is high. For the MOS to be high, the stall duration value should be near 0, and if the value of the stall duration is high, the MOS will be reduced to 4, 3, 2, and 1. Therefore, stall duration and MOS are inversely proportional. As the stall duration of the video is high, the MOS of the video will be low. These are the main findings of this work.

Based on Figs. 5a and 5b, this work finds that since the throughput and byte size are high, the value of MOS is also high for video quality. A low throughput and byte size will result in a low MOS of video quality. Therefore, throughput and byte size are directly proportional to MOS. A similar relationship can be observed in Figs. 5c and 5d, where the width and height of a video frame are directly proportional to the MOS. As shown in Fig. 6. The proposed work uses a heatmap to determine the correlation between features.

Figure 5: Analysis of network and video metrics with mean opinion score

Figure 6: Heatmap matrix for correlation of features

Heatmaps allow for easy data interpretation by visually representing the data in a graphical format. It can be used to identify patterns and trends in data quickly. Analyzing the heat map makes it easy to understand the correlation matrix. The heatmap indicates which columns are negatively correlated and which are positively correlated. Heatmaps also provide an easy way to visualize statistical test results. It is useful to identify outliers and unexpected data points quickly.

4.1 Performance Measure

This work calculated prediction accuracy using Eq. (1) to evaluate the efficiency of the trained model.

Accuracy=Number of correct predictions Total number of data points×100%(1)

Using only an accuracy metric is unreliable for evaluating the trained model because the 5G dataset can have an uneven distribution of classes. It is vital to consider this when training the models to forecast the video quality based on MOS. There is a total sample size of 3591 in the 5G dataset. Hence, some uneven distribution of the samples can make the accuracy score misleading. So, the proposed work cannot rely only on the accuracy score. Therefore, metrics such as confusion matrix, precision, sensitivity, specificity, and F1 score are helpful. Table 3 includes a summary of these metrics. True Negative (TN), False Negative (FN), True Positive (TP), and False Positive (FP) values are computed to overcome the dependencies of accuracy.

4.2 Results for QoE Classification Using Hybrid XGBStackQoE-Model

The confusion matrix (CM) helps us understand the accuracy score’s limitations and how to utilize it effectively. This matrix is used to calculate the performance of classification models. In comparison to an accuracy score, CM provides more information. Tables 4 and 5 include a result of CM for training and testing data.

A true class represents the actual value, whereas a predicted class represents the value predicted by our model. Fig. 7 shows the comparison of CM for traditional ML models in Figs. 7a–7c with the proposed XGBStackQoE Classifier in Fig. 7d. In the XGBStackQoE CM Classifier, 301 samples in the first quadrant represent the TN. Therefore, the video has a poor MOS, and the model also predicted this correctly. The second quadrant 16 samples represent the true class is negative, which indicates that the video data has a poor MOS, but the model predicted this as a good MOS. This is referred to as an FP.

Figure 7: Confusion matrices for Level 1 and Level 2 classifiers: (a) random forest, (b) gradient boosting, (c) bagging classifier, (d) XGBStackQoE classifier

The third quadrant of 12 samples is FN, but its true class is positive. It indicates a good MOS in the video data, but the model predicted this as a bad MOS. Hence, it is known as FN. Lastly, 749 samples in the final quadrant are TP, indicating that the video data has Good MOS; the model also predicts it as positive. According to Eqs. (2) and (3), an XGBStackQoE model has 1050 correct and 28 wrong predictions.

TN+TP=Correct prediction(2)

FP+FN=Wrong prediction(3)

5  Results and Discussion

Performance metrics for the various traditional ML models, as well as the performance metrics for the XGBStackQoE model, are shown in Fig. 8. The XGBStackQoE has attained the highest precision of 97.90%.

Figure 8: Performance metrics for machine learning models

The enhanced performance of the proposed hybrid model is due to the features obtained from the Level 1 ML classifiers being concatenated and given as input to the Level 2 XGBStackQoE model. As a result of mapping the input features into a higher-dimensional space, the proposed XGBStackQoE classifier improves classification accuracy. Hence XGBStackQoE classifier is the most suitable model for MOS prediction.

5.1 Summary of Achievements

A novel hybrid XGBStackQoE analytical model using a two-level layering technique is proposed for QoE analysis. The idea behind the proposed method is to leverage the strengths of different classifiers and potentially improve the overall predictive performance. The proposed work dataset is split into 70% for training and 30% for testing sets. Base classifiers are chosen from a diverse traditional classifier based on their training and testing data accuracy for the first level. In this proposed work, the base classifiers used were Random Forest, Gradient Boosting, and Bagging Classifiers. Each base classifier can learn different aspects of the data or provide complementary predictions. Once the base classifiers are trained, the proposed work uses them to predict the testing set. By combining the forecasts from the base classifiers, a new feature matrix is generated. Each row of this matrix corresponds to an instance from the training set, and each column corresponds to the prediction made by a particular base classifier. The proposed XGBStackQoE-classifier uses this new feature matrix as input to train. Once the XGBStackQoE-classifier is trained, it uses the predictions of the base classifiers on the testing set as input to the XGBStackQoE-classifier. The XGBStackQoE-classifier will generate the final forecast for each instance in the testing set. The proposed work assesses the performance of the proposed XGBStackQoE model by comparing its predictions with the actual labels from the testing set. The proposed work uses appropriate evaluation metrics such as accuracy, precision, sensitivity, specificity, and F1 score to measure the performance.

6  Conclusion and Future Work

This paper investigated network and video quality parameters with user quality of experience. A fully automated data analytics framework for video quality prediction is developed. It has examined and measured the correlation between QoS and QoE parameters. Therefore, the causes of video quality degradation have been analyzed. The data framework is validated with a high level of accuracy in forecasting QoE. Here, various ML classifiers of level one, like Random Forest, Gradient Boosting, and Bagging Classifiers, are trained with the whole training data set. Additionally, the outputs (meta-features) of the layer one ML models are used to fit the level two Hybrid XGBStackQoE model. The hybrid XGBStackQoE analytical model outperformed traditional models, improving accuracy by 4% to 5%. The results of the XGBStackQoE model showed that it was more accurate and reliable than other models. This enhanced accuracy can improve the user experience of many applications, such as video streaming and gaming. This method will be further improved in future research to classify more classes. This could be achieved by increasing the number of training samples and refining the model architecture. Also, deep learning techniques could be employed to improve the model’s accuracy. The model could also be tested on different datasets to assess its generalizability. Lastly, transfer learning could be explored to reduce training time and improve model performance.

Acknowledgement: The authors would like to acknowledge the support and assistance they have received from VIT University, Chennai, India.

Funding Statement: The authors received no specific funding for this study.

Author Contributions: All authors contributed significantly to the conception, design, analysis, and interpretation of a novel hybrid XGBStackQoE analytical model. K. B. Ajeyprasaath played a key role in data collection, contributing to the manuscript’s writing and revision. P. Vetrivelan provided expertise in result analysis supervision and contributed valuable insights during the research process. All authors have reviewed and approved the final version of the manuscript and agree to be accountable for all aspects of the work.

Availability of Data and Materials: Publicly available datasets were analysed in this study. The data can be found here https://github.com/uccmisl/5Gdataset (accessed on 24/06/2023).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References

1. M. Agiwal, H. Kwon, S. Park and H. Jin, “A survey on 4G-5G dual connectivity: Road to 5G implementation,” IEEE Access, vol. 9, pp. 16193–16210, 2021. 10.1109/ACCESS.2021.3052462 [Google Scholar] [CrossRef]

2. F. Laiche, A. Ben Letaifa, I. Elloumi and T. Aguili, “When machine learning algorithms meet user engagement parameters to predict video QoE,” Wireless Personal Communications, vol. 116, no. 3, pp. 2723–2741, 2021. 10.1007/s11277-020-07818-w [Google Scholar] [CrossRef]

3. Y. J. Yu, A. C. Pang and M. Y. Yeh, “Video encoding adaptation for QoE maximization over 5G cellular networks,” Journal of Network and Computer Applications, vol. 114, no. 1, pp. 98–107, 2018. 10.1016/j.jnca.2018.04.008 [Google Scholar] [CrossRef]

4. A. K. Moorthy, K. Seshadrinathan, R. Soundararajan and A. C. Bovik, “Wireless video quality assessment: A study of subjective scores and objective algorithms,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 20, no. 4, pp. 587–599, 2010. 10.1109/TCSVT.2010.2041829 [Google Scholar] [CrossRef]

5. S. Chikkerur, V. Sundaram, M. Reisslein and L. J. Karam, “Objective video quality assessment methods: A classification, review, and performance comparison,” IEEE Transactions on Broadcasting, vol. 57, no. 2, pp. 165–182, 2011. 10.1109/TBC.2011.2104671 [Google Scholar] [CrossRef]

6. Y. Chen, K. Wu and Q. Zhang, “From QoS to QoE: A tutorial on video quality assessment,” IEEE Communications Surveys and Tutorials, vol. 17, no. 2, pp. 1126–1165, 2015. 10.1109/COMST.2014.2363139 [Google Scholar] [CrossRef]

7. G. Kougioumtzidis, V. Poulkov, Z. D. Zaharis and P. I. Lazaridis, “A survey on multimedia services QoE assessment and machine learning-based prediction,” IEEE Access, vol. 10, pp. 19507–19538, 2022. 10.1109/ACCESS.2022.3149592 [Google Scholar] [CrossRef]

8. R. Bhadada and K. L. Sharma, “Evaluation and analysis of buffer requirements for streamed video data in video on demand applications,” IETE Journal of Research, vol. 56, no. 5, pp. 242–248, 2010. 10.4103/0377-2063.72777 [Google Scholar] [CrossRef]

9. M. P. Dimitropoulos, X. Hurley, P. Kind and A. Stoecklin, “On the 95-percentile billing method,” in Passive and Active Network Measurement, pp. 207–216, Berlin, Heidelberg: Springer, 2009. 10.1007/978-3-642-00975-4_21 [Google Scholar] [CrossRef]

10. A. Dhamdhere and C. Dovrolis, “Can ISPs be profitable without violating “network neutrality”?” in Proc. of the 3rd Int. Workshop on Economics of Networked Systems (NetEcon ‘08), New York, NY, USA, Association for Computing Machinery, pp. 13–18, 2008. 10.1145/1403027.1403031 [Google Scholar] [CrossRef]

11. L. Krasula and P. Le Callet, “Emerging science of QoE in multimedia applications: Concepts, experimental guidelines, and validation of models,” in Academic Press Library in Signal Processing, vol. 6, pp. 163–209, 2018. 10.1016/B978-0-12-811889-4.00004-X [Google Scholar] [CrossRef]

12. ITU, “P.910: Subjective video quality assessment methods for multimedia applications,” 2022. [Online]. Available: https://www.itu.int/rec/T-REC-P.910 (accessed 24/06/2023). [Google Scholar]

13. R. K. P. Mok, E. W. W. Chan and R. K. C. Chang, “Measuring the quality of experience of HTTP video streaming,” in Proc. of the 12th IFIP/IEEE Int. Symp. on Integrated Network Management (IM 2011) and Workshops, Dublin, Ireland, pp. 485–492, 2011. 10.1109/INM.2011.5990550 [Google Scholar] [CrossRef]

14. G. Dimopoulos, I. Leontiadis, P. Barlet-Ros and K. Papagiannaki, “Measuring video QoE from encrypted traffic,” in Proc. of the 2016 Internet Measurement Conf., Santa Monica, California, USA, pp. 513–526, 2016. 10.1145/2987443.2987459 [Google Scholar] [CrossRef]

15. J. Lehmann, M. Lalmas, E. Yom-Tov and G. Dupret, “Models of user engagement,” in User Modeling, Adaptation, and Personalization, pp. 164–175, Berlin, Heidelberg: Springer, 2012. 10.1007/978-3-642-31454-4_14 [Google Scholar] [CrossRef]

16. M. U. N. Nisa, D. Mahmood, G. Ahmed, S. Khan, M. A. Mohammed et al., “Optimizing prediction of youtube video popularity using XGBoost,” Electronics, vol. 10, no. 23, pp. 2962, 2021. 10.3390/ELECTRONICS10232962 [Google Scholar] [CrossRef]

17. I. Bartolec, I. Orsolic and L. Skorin-Kapov, “Impact of user playback interactions on in-network estimation of video streaming performance,” IEEE Transactions on Network and Service Management, vol. 19, no. 3, pp. 3547–3561, 2022. 10.1109/TNSM.2022.3180114 [Google Scholar] [CrossRef]

18. A. A. Laghari, S. Shahid, R. Yadav, S. Karim, A. Khan et al., “The state of art and review on video streaming,” Journal of High Speed Networks, vol. 29, no. 3, pp. 211–236, 2023. 10.3233/JHS-222087 [Google Scholar] [CrossRef]

19. F. Dobrian, A. Awan, D. Joseph, A. Ganjam, J. Zhan et al., “Understanding the impact of video quality on user engagement,” Communications of the ACM, vol. 56, no. 3, pp. 91–99, 2013. 10.1145/2428556.2428577 [Google Scholar] [CrossRef]

20. Q. Wang, H. N. Dai, D. Wu and H. Xiao, “Data analysis on video streaming QoE over mobile networks,” EURASIP Journal on Wireless Communications and Networking, vol. 173, no. 1, pp. 1–10, 2018. 10.1186/S13638-018-1180-8/TABLES/8 [Google Scholar] [CrossRef]

21. S. Umrao, A. Roy and N. Saxena, “Multilevel hierarchical caching for efficient wireless video distribution,” IETE Journal of Research, vol. 63, no. 2, pp. 260–267, 2017. 10.1080/03772063.2016.1249965 [Google Scholar] [CrossRef]

22. R. Ul Mustafa, D. Moura and C. E. Rothenberg, “Machine learning approach to estimate video QoE of encrypted DASH traffic in 5G networks,” in Proc. of IEEE Statistical Signal Processing Workshop (SSP), Rio de Janeiro, Brazil, pp. 586–589, 2021. 10.1109/SSP49050.2021.9513804 [Google Scholar] [CrossRef]

23. D. Raca, D. Leahy, C. J. Sreenan and J. J. Quinlan, “Beyond throughput, the next generation: A 5G dataset with channel and context metrics,” in Proc. of the 11th ACM Multimedia Systems Conf., Istanbul, Turkey, pp. 303–308, 2020. 10.1145/3339825.3394938 [Google Scholar] [CrossRef]

24. W. Zhao, S. Wang, C. Wang and X. Wu, “Approximation algorithms for cell planning in heterogeneous networks,” IEEE Transactions on Vehicular Technology, vol. 66, no. 2, pp. 1561–1572, 2017. 10.1109/TVT.2016.2552487 [Google Scholar] [CrossRef]

25. S. Petrangeli, T. Wu, T. Wauters, R. Huysegems, T. Bostoen et al., “A machine learning-based framework for preventing video freezes in HTTP adaptive streaming,” Journal of Network and Computer Applications, vol. 94, no. 1, pp. 78–92, 2017. 10.1016/j.jnca.2017.07.009 [Google Scholar] [CrossRef]

26. J. P. Esper, A. Claudia Bastos Loureiro Moņcão, K. B. Chaves Rodrigues, C. Bonato Both, S. L. Corrêa et al., “QoE-DASH: DASH QoE performance evaluation tool for edge-cache and recommendation,” in Proc. of IEEE Int. Conf. on Communications, Seoul, Korea, pp. 757–762, 2022. 10.1109/ICC45855.2022.9839234 [Google Scholar] [CrossRef]

27. K. B. Ajeyprasaath and P. Vetrivelan, “Machine learning based classifiers for QoE prediction framework in video streaming over 5G wireless networks,” Computers, Materials & Continua, vol. 75, no. 1, pp. 1919–1939, 2023. 10.32604/cmc.2023.036013 [Google Scholar] [CrossRef]

28. K. B. Ajeyprasaath and P. Vetrivelan, “A QoE framework for video services in 5G networks with supervised machine learning approach,” in Machine Learning and Computational Intelligence Techniques for Data Engineering, pp. 661–668, Singapore: Springer, 2022. 10.1007/978-981-99-0047-3_56 [Google Scholar] [CrossRef]

29. A. Shahraki, T. Ohlenforst and F. Kreyß, “When machine learning meets network management and orchestration in edge-based networking paradigms,” Journal of Network and Computer Applications, vol. 212, no. 1, pp. 103558, 2023. 10.1016/j.jnca.2022.103558 [Google Scholar] [CrossRef]

30. R. Ul Mustafa, S. Ferlin, C. E. Rothenberg, D. Raca and J. J. Quinlan, “A supervised machine learning approach for DASH video QoE prediction in 5G networks,” in Proc. of the 16th ACM Symp. on QoS and Security for Wireless and Mobile Networks, Alicante, Spain, pp. 57–64, 2020. 10.1145/3416013.3426458 [Google Scholar] [CrossRef]

31. R. Shalala, R. Dubin, O. Hadar and A. Dvir, “Video QoE prediction based on user profile,” in Proc. Int. Conf. on Computing, Networking and Communications (ICNC), Maui, HI, USA, pp. 588–592, 2018. 10.1109/ICCNC.2018.8390347 [Google Scholar] [CrossRef]

32. L. Liu, H. Hu, Y. Luo and Y. Wen, “When wireless video streaming meets AI: A deep learning approach,” IEEE Wireless Communications, vol. 27, no. 2, pp. 127–133, 2020. 10.1109/MWC.001.1900220 [Google Scholar] [CrossRef]

33. L. Qian, H. Chen and L. Xie, “SVM-based QoE estimation model for video streaming service over wireless networks,” in Proc. of Int. Conf. on Wireless Communications & Signal Processing (WCSP), Nanjing, China, pp. 1–6, 2015. 10.1109/WCSP.2015.7341066 [Google Scholar] [CrossRef]

34. A. Hameed, R. Dai and B. Balas, “A decision-tree-based perceptual video quality prediction model and its application in FEC for wireless multimedia communications,” IEEE Transactions on Multimedia, vol. 18, no. 4, pp. 764–774, 2016. 10.1109/TMM.2016.2525862 [Google Scholar] [CrossRef]

35. Y. Ben Youssef, M. Afif, R. Ksantini and S. Tabbane, “A novel online QoE prediction model based on multiclass incremental support vector machine,” in Proc. of IEEE 32nd Int. Conf. on Advanced Information Networking and Applications (AINA), Krakow, Poland, pp. 334–341, 2018. 10.1109/AINA.2018.00058 [Google Scholar] [CrossRef]

36. D. Minovski, C. Ahlund, K. Mitra and P. Johansson, “Analysis and estimation of video QoE in wireless cellular networks using machine learning,” in Proc. of Eleventh Int. Conf. on Quality of Multimedia Experience (QoMEX), Berlin, Germany, pp. 1–6, 2019. 10.1109/QoMEX.2019.8743281 [Google Scholar] [CrossRef]

37. H. Zhang, L. Dong, G. Gao, H. Hu, Y. Wen et al., “DeepQoE: A multimodal learning framework for video quality of experience (QoE) PRediction,” IEEE Transactions on Multimedia, vol. 22, no. 12, pp. 3210–3223, 2020. 10.1109/TMM.2020.2973828 [Google Scholar] [CrossRef]

38. A. Ashiquzzaman, S. M. Oh, D. Lee, H. Jung and T. Won Um, “Deeplearning convolutional neural network based QoE assessment module for 4K UHD video streaming,” in Proc. of the 9th Int. Conf.on Simulation and Modeling Methodologies, Technologies and Applications, Prague, Czech Republic, pp. 1–6, 2019. 10.5220/0008117903920397 [Google Scholar] [CrossRef]

39. A. A. Laghari, X. Zhang, Z. A. Shaikh, A. Khan, V. V. Estrela et al., “A review on quality of experience (QoE) in cloud computing,” Journal of Reliable Intelligent Environments, vol. 9, no. 4, pp. 1–15, 2023. 10.1007/s40860-023-00210-y [Google Scholar] [CrossRef]

40. N. F. S. de Sousa, M. T. Islam, R. U. Mustafa, D. A. L. Perez, C. E. Rothenberg et al., “Machine learning-assisted closed-control loops for beyond 5G multi-domain zero-touch networks,” Journal of Network and Systems Management, vol. 30, no. 3, pp. 1–29, 2022. 10.1007/S10922-022-09651-X/METRICS [Google Scholar] [CrossRef]

41. T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in Proc. of the 22nd ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, San Francisco, California, USA, pp. 785–794, 2016. 10.1145/2939672.2939785 [Google Scholar] [CrossRef]

42. M. A. Haque, M. A. Rahman and M. S. Siddik, “Non-functional requirements classification with feature extraction and machine learning: An empirical study,” in Proc. 1st Int. Conf. on Advances in Science, Engineering and Robotics Technology (ICASERT), Dhaka, Bangladesh, pp. 1–5, 2019. 10.1109/ICASERT.2019.8934499 [Google Scholar] [CrossRef]