AWS PredSpot: Machine Learning for Predicting the Price of Spot Instances in AWS Cloud.

Alejandro Baldominos; Yago Saez; David Quintana; Pedro Isasi

doi:10.9781/ijimai.2022.02.003

Authors

Alejandro Baldominos Universidad Carlos III de Madrid
Yago Saez Universidad Carlos III de Madrid
David Quintana Universidad Carlos III de Madrid
Pedro Isasi Universidad Carlos III de Madrid

DOI:

https://doi.org/10.9781/ijimai.2022.02.003

Keywords:

Cloud Computing, Machine Learning, Prediction, Prices, Forecasting

Supporting Agencies

Authors would like to thank Data Science Awards committee for proposing the competition. This work has been supported by the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with UC3M in the line of Excellence of University Professors (EPUC3MXX), and in the context of the V PRICIT (Regional Programme of Research and Technological Innovation).

Abstract

Elastic Cloud Compute (EC2) is one of the most well-known services provided by Amazon for provisioning cloud computing resources, also known as instances. Besides the classical on-demand scheme, where users purchase compute capacity at a fixed cost, EC2 supports so-called spot instances, which are offered following a bidding scheme, where users can save up to 90% of the cost of the on-demand instance. EC2 spot instances can be a useful alternative for attaining an important reduction in infrastructure cost, but designing bidding policies can be a difficult task, since bidding under their cost will either prevent users from provisioning instances or losing those that they already own. Towards this extent, accurate forecasting of spot instance prices can be of an outstanding interest for designing working bidding policies. In this paper, we propose the use of different machine learning techniques to estimate the future price of EC2 spot instances. These include linear, ridge and lasso regressions, multilayer perceptrons, K-nearest neighbors, extra trees and random forests. The obtained performance varies significantly between instances types, and root mean squared errors ranges between values very close to zero up to values over 60 in some of the most expensive instances. Still, we can see that for most of the instances, forecasting performance is remarkably good, encouraging further research in this field of study.

Downloads

Download data is not yet available.

References

Amazon Web Services, “Amazon EC2 Spot Instances Pricing.” Accessed: Oct. 15, 2021, [Online]. Available: https://aws.amazon.com/ec2/spot/pricing

Amazon Web Services, “Amazon EC2 Instance Types.” Accessed: Oct. 15, 2021, [Online]. Available: https://aws.amazon.com/ec2/instance-types

O. A. Ben-Yehuda, M. Ben-Yehuda, A. Schuster, D. Tsafrir, “Deconstructing Amazon EC2 spot instance pricing,” ACM Transactions on Economics and Computation, vol. 1, no. 3, p. 16, 2013.

G. Portella, G. N. Rodrigues, E. Nakano, A. C. Melo, “Statistical analysis of Amazon EC2 cloud pricing models,” Concurrency and Computation. Practice and Experience, vol. 31, no. 18, p. e4451, 2018.

M. Lumpe, M. B. Chhetri, Q. B. Vo, R. Kowalcyk, “On estimating minimum bids for Amazon EC2 spot instances,” in 2017 17th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, Madrid, Spain, 2017, IEEE.

C. Tian, Y. Wang, F. Qi, B. Yin, “Decision model for provisioning virtual resources in Amazon EC2,” in 2012 8th Intl. Conf. Network and Service Management and 2012 Workshop on Systems Virtualization Management, Las Vegas, NV, USA, 2012, IEEE.

S. Tang, J. Yuan, X.-Y. Li, “Towards optimal bidding strategy for Amazon EC2 cloud spot instance,” in 2012 IEEE Fifth International Conference on Cloud Computing, Honolulu, HI, USA, 2012, IEEE.

S. Tang, J. Yuan, C. Wang, X.-Y. Li, “A framework for Amazon EC2 bidding strategy under SLA constraints,” CIEEE Transactions on Parallel and Distributed Systems, vol. 25, no. 1, pp. 2–11, 2014.

M. B. Chhetri, M. Lumpe, Q. B. Vo, R. Kowalczyk, “To bid or not to bid in streamlined EC2 spot markets,” in 2018 IEEE International Conference on Services Computing, San Francisco, CA, USA, 2018, IEEE.

M. B. Chhetri, M. Lumpe, Q. B. Vo, R. Kowalczyk, “On forecasting Amazon EC2 spot prices using time-series decomposition with hybrid look-backs,” in 2017 IEEE International Conference on Edge Computing, Honolulu, HI, USA, 2017, IEEE.

V. Khandelwal, A. Chaturvedi, C. P. Gupta, “Amazon EC2 spot price prediction using regression random forests,” IEEE Transactions on Cloud Computing, vol. 8, no. 1, pp. 59–72, 2020.

J. Lancon, Y. Kunwar, D. Stroud, M. McGee, R. Slater, “AWS EC2 instance spot price forecasting using LSTM networks,” SMU Data Science Review, vol. 2, no. 2, p. 8, 2019.

M. Malik, N. Bagmar, “Forecasting price of amazon spot instances using machine learning,” International Journal of Artificial Intelligence and Machine Learning, vol. 11, pp. 71–82, 07 2021.

V. Chittora, C. P. Gupta, “Dynamic spot price forecasting using stacked lstm networks,” in 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS), 2020, pp. 1080–1085.

W. Liu, P. Wang, Y. Meng, C. Zhao, Z. Zhang, “Cloud spot instance price prediction using knn regression,” Human-centric Computing and Information Sciences, vol. 10, no. 1, p. 34, 2020.

Kaggle, “AWS Spot Pricing Market Dataset.” Accessed: Oct. 15, 2021, [Online]. Available: https://www.kaggle.com/noqcks/aws-spot-pricingmarket

Western Sydney University, “Spot Price Archive.” [Online]. Available: http://spot.scem.uws.edu.au/ec2si

B. Javadi, R. Thulasiram, R. Buyya, “Statistical Modeling of Spot Instance Prices in Public Cloud Environments,” in 4th IEEE/ACM International Conference on Utility and Cloud Computing, Melbourne, Australia, 2011, 2011, IEEE.

Amazon Web Services, “EC2-Boto 3 Docs.” Accessed: Oct. 15, 2021, [Online]. Available: http://boto3.readthedocs.io/en/latest/reference/services/ec2.html#EC2.Client.describe_spot_price_history

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, E. Duchesnay, “Scikitlearn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011.