Feature selection is one of the most important challenges in machine learning and data science. This process is usually performed in the data preprocessing phase, where the data is transformed to a proper format for further operations by machine learning algorithm. Many real-world datasets are highly dimensional with many irrelevant, even redundant features. These kinds of features do not improve classification accuracy and can even shrink down performance of a classifier. The goal of feature selection is to find optimal (or sub-optimal) subset of features that contain relevant information about the dataset from which machine learning algorithms can derive useful conclusions. In this manuscript, a novel version of firefly algorithm (FA) is proposed and adapted for feature selection challenge. Proposed method significantly improves performance of the basic FA, and also outperforms other state-of-the-art metaheuristics for both, benchmark bound-constrained and practical feature selection tasks. Method was first validated on standard unconstrained benchmarks and later it was applied for feature selection by using 21 standard University of California, Irvine (UCL) datasets. Moreover, presented approach was also tested for relatively novel COVID-19 dataset for predicting patients health, and one microcontroller microarray dataset. Results obtained in all practical simulations attest robustness and efficiency of proposed algorithm in terms of convergence, solutions' quality and classification accuracy. More precisely, the proposed approach obtained the best classification accuracy on 13 out of 21 total datasets, significantly outperforming other competitor methods.

Department of Applied Cybernetics Faculty of Science University of Hradec Králové 50003 Hradec Králové Czech Republic

Department of Computational Mathematics Science and Engineering Michigan State University East Lansing MI 48824 USA

Department of Mathematics Faculty of Science Mansoura University Mansoura 35516 Egypt

Singidunum University Danijelova 32 11000 Belgrade Serbia

See more in PubMed

Luo S., Cheng L., Ren B. Practical swarm optimization based fault-tolerance algorithm for the Internet of Things. KSII Trans. Internet Inf. Syst. (TIIS) 2014;8(3):735–748.

Wu Q., Ding G., Xu Y., Feng S., Du Z., Wang J., Long K. Cognitive Internet of Things: A new paradigm beyond connection. IEEE Internet Things J. 2014;1(2):129–143. doi: 10.1109/JIOT.2014.2311513. DOI

Messaoud S., Bradai A., Bukhari S.H.R., Quang P.T.A., Ahmed O.B., Atri M. A survey on machine learning in Internet of Things: Algorithms, strategies, and applications. Internet Things. 2020;12 doi: 10.1016/j.iot.2020.100314. URL https://www.sciencedirect.com/science/article/pii/S2542660520301451. DOI

Zenggang X., Mingyang Z., Xuemin Z., Sanyuan Z., Fang X., Xiaochao Z., Yunyun W., Xiang L. Social similarity routing algorithm based on socially aware networks in the big data environment. J. Signal Process. Syst. 2022;94(11):1253–1267.

Chandrashekar G., Sahin F. A survey on feature selection methods. Comput. Electr. Eng. 2014;40(1):16–28. doi: 10.1016/j.compeleceng.2013.11.024. URL https://www.sciencedirect.com/science/article/pii/S0045790613003066, 40th-year commemorative issue. DOI

Li H., Peng R., Wang Z.-a. On a diffusive susceptible-infected-susceptible epidemic model with mass action mechanism and birth-death effect: analysis, simulations, and comparison with other mechanisms. SIAM J. Appl. Math. 2018;78(4):2129–2153.

Xue B., Zhang M., Browne W.N. Particle swarm optimisation for feature selection in classification: Novel initialisation and updating mechanisms. Appl. Soft Comput. 2014;18:261–276. doi: 10.1016/j.asoc.2013.09.018. URL https://www.sciencedirect.com/science/article/pii/S1568494613003128. DOI

Sun R., Wang J., Cheng Q., Mao Y., Ochieng W.Y. A new IMU-aided multiple GNSS fault detection and exclusion algorithm for integrated navigation in urban environments. GPS Solut. 2021;25:1–17.

Sharma M., Kaur P. A comprehensive analysis of nature-inspired meta-heuristic techniques for feature selection problem. Arch. Comput. Methods Eng. 2021;28(3)

Chuang L.-Y., Tsai S.-W., Yang C.-H. Improved binary particle swarm optimization using catfish effect for feature selection. Expert Syst. Appl. 2011;38(10):12699–12707.

Huang C., Han Z., Li M., Wang X., Zhao W. Sentiment evolution with interaction levels in blended learning environments: Using learning analytics and epistemic network analysis. Australas. J. Educ. Technol. 2021;37(2):81–95.

Yan A., Li Z., Cui J., Huang Z., Ni T., Girard P., Wen X. LDAVPM: a latch design and algorithm-based verification protected against multiple-node-upsets in harsh radiation environments. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. 2022

Yan A., Xu Z., Yang K., Cui J., Huang Z., Girard P., Wen X. A novel low-cost TMR-without-voter based HIS-insensitive and MNU-tolerant latch design for aerospace applications. IEEE Trans. Aerosp. Electron. Syst. 2019;56(4):2666–2676.

T. Bezdan, D. Cvetnic, L. Gajic, M. Zivkovic, I. Strumberger, N. Bacanin, Feature Selection by Firefly Algorithm with Improved Initialization Strategy, in: 7th Conference on the Engineering of Computer Based Systems, 2021, pp. 1–8.

Wang K., Zhang B., Alenezi F., Li S. Communication-efficient surrogate quantile regression for non-randomly distributed system. Inform. Sci. 2022;588:425–441.

Xie X., Xie B., Xiong D., Hou M., Zuo J., Wei G., Chevallier J. New theoretical ISM-K2 Bayesian network model for evaluating vaccination effectiveness. J. Ambient Intell. Humaniz. Comput. 2022:1–17. PubMed PMC

Yang D., Zhu T., Wang S., Wang S., Xiong Z. LFRSNet: a robust light field semantic segmentation network combining contextual and geometric features. Front. Environ. Sci. 2022:1443.

Zhu Z., Ong Y.-S., Dash M. Wrapper–filter feature selection algorithm using a memetic framework. IEEE Trans. Syst. Man Cybern. B. 2007;37(1):70–76. PubMed

Babatunde O.H., Armstrong L., Leng J., Diepeveen D. A genetic algorithm-based feature selection. IJECCE. 2014

Li X., Zhang X., Jia T. Humanization of nature: Testing the influences of urban park characteristics and psychological factors on collegers’ perceived restoration. Urban For. Urban Green. 2023;79

Zivkovic M., Bacanin N., Zivkovic T., Strumberger I., Tuba E., Tuba M. Enhanced grey wolf algorithm for energy efficient wireless sensor networks. 2020 Zooming Innovation in Consumer Technologies Conference; ZINC; IEEE; 2020. pp. 87–92.

Zivkovic M., Bacanin N., Venkatachalam K., Nayyar A., Djordjevic A., Strumberger I., Al-Turjman F. COVID-19 cases prediction by using hybrid machine learning and beetle antennae search approach. Sustainable Cities Soc. 2021;66 PubMed PMC

Zivkovic M., Bezdan T., Strumberger I., Bacanin N., Venkatachalam K. Computer Networks, Big Data and IoT. Springer; 2021. Improved Harris Hawks optimization algorithm for workflow scheduling challenge in cloud–edge environment; pp. 87–102.

Dorigo M., Birattari M. Springer; 2010. Ant Colony Optimization.

Kennedy J., Eberhart R. Proceedings of ICNN’95 - International Conference on Neural Networks, Vol. 4. 1995. Particle swarm optimization; pp. 1942–1948. DOI

Karaboga D., Basturk B. On the performance of artificial bee colony (ABC) algorithm. Appl. Soft Comput. 2008;8(1):687–697.

Yang X.-S., Hossein Gandomi A. Bat algorithm: a novel approach for global engineering optimization. Eng. Comput. 2012;29(5):464–483.

Yang X.-S. In: Stochastic Algorithms: Foundations and Applications. Watanabe O., Zeugmann T., editors. Springer Berlin Heidelberg; Berlin, Heidelberg: 2009. Firefly algorithms for multimodal optimization; pp. 169–178.

Khare N., Devan P., Chowdhary C.L., Bhattacharya S., Singh G., Singh S., Yoon B. Smo-dnn: Spider monkey optimization and deep neural network hybrid classifier model for intrusion detection. Electronics. 2020;9(4):692.

Hussien A.G., Amin M. A self-adaptive Harris Hawks optimization algorithm with opposition-based learning and chaotic local search strategy for global optimization and feature selection. Int. J. Mach. Learn. Cybern. 2021:1–28.

Baliarsingh S.K., Ding W., Vipsita S., Bakshi S. A memetic algorithm using emperor penguin and social engineering optimization for medical data classification. Appl. Soft Comput. 2019;85

Bezdan T., Zivkovic M., Bacanin N., Chhabra A., Suresh M. Feature selection by hybrid brain storm optimization algorithm for COVID-19 classification. J. Comput. Biol. 2022 PubMed

Zivkovic M., Stoean C., Chhabra A., Budimirovic N., Petrovic A., Bacanin N. Novel improved salp swarm algorithm: An application for feature selection. Sensors. 2022;22(5):1711. PubMed PMC

Strumberger I., Rakic A., Stanojlovic S., Arandjelovic J., Bezdan T., Zivkovic M., Bacanin N. Feature selection by hybrid Binary Ant Lion Optimizer with COVID-19 dataset. 2021 29th Telecommunications Forum; TELFOR; IEEE; 2021. pp. 1–4.

Bacanin N., Petrovic A., Zivkovic M., Bezdan T., Antonijevic M. International Conference on Advances in Computing and Data Sciences. Springer; 2021. Feature selection in machine learning by hybrid sine cosine metaheuristics; pp. 604–616.

Kareem S.S., Mostafa R.R., Hashim F.A., El-Bakry H.M. An effective feature selection model using hybrid metaheuristic algorithms for IoT intrusion detection. Sensors. 2022;22(4):1396. PubMed PMC

Bacanin N., Bezdan T., Tuba E., Strumberger I., Tuba M. Monarch butterfly optimization based convolutional neural network design. Mathematics. 2020;8(6):936.

Milosevic S., Bezdan T., Zivkovic M., Bacanin N., Strumberger I., Tuba M. Modelling and Development of Intelligent Systems: 7th International Conference, MDIS 2020, Sibiu, Romania, October 22–24, 2020, Revised Selected Papers 7. Springer International Publishing; 2021. Feed-forward neural network training by hybrid bat algorithm; pp. 52–66.

Yang J., Qu L., Shen Y., Shi Y., Cheng S., Zhao J., Shen X. International Conference on Swarm Intelligence. Springer; 2020. Swarm intelligence in data science: Applications, opportunities and challenges; pp. 3–14.

Yang X.-S., Xingshi H. Firefly algorithm: Recent advances and applications. Int. J. Swarm Intell. 2013;1(1):36–50.

I. Strumberger, E. Tuba, N. Bacanin, M. Zivkovic, M. Beko, M. Tuba, Designing convolutional neural network architecture by the firefly algorithm, in: Proceedings of the 2019 International Young Engineers Forum, YEF-ECE, Costa Da Caparica, Portugal, 2019, pp. 59–65.

Bacanin N., Tuba M. Firefly algorithm for cardinality constrained mean-variance portfolio optimization problem with entropy diversity constraint. The Scientific World Journal, Special Issue Computational Intelligence and Metaheuristic Algorithms with Applications. 2014;2014:16. doi: 10.1155/2014/721521. Article ID 721521. PubMed DOI PMC

I. Strumberger, N. Bacanin, M. Tuba, Enhanced Firefly Algorithm for Constrained Numerical Optimization, IEEE Congress on Evolutionary Computation, in: Proceedings of the IEEE International Congress on Evolutionary Computation, CEC 2017, 2017.

Wang H., Zhou X., Sun H., Yu X., Zhao J., Zhang H., Cui L. Firefly algorithm with adaptive control parameters. Soft Comput. 2017;3:5091–-5102.

Fan Q., Chen Z., Xia Z. A novel quasi-reflected Harris Hawks optimization algorithm for global optimization problems. Soft Comput. 2020:1–19.

Cazacu R. Comparative study between the improved implementation of 3 classic mutation operators for genetic algorithms. Procedia Eng. 2017;181:634–640. doi: 10.1016/j.proeng.2017.02.444. URL http://www.sciencedirect.com/science/article/pii/S1877705817310287, 10th International Conference Interdisciplinarity in Engineering, INTER-ENG 2016, 6-7 October 2016, Tirgu Mures, Romania. DOI

Bäck T., Schwefel H. An overview of evolutionary algorithms for parameter optimization. Evol. Comput. 1993;1(1):1–23.

S. Rahnamayan, H.R. Tizhoosh, M.M.A. Salama, Quasi-oppositional Differential Evolution, in: 2007 IEEE Congress on Evolutionary Computation, 2007, pp. 2229–2236.

Tuba M., Bacanin N. Improved seeker optimization algorithm hybridized with firefly algorithm for constrained optimization problems. Neurocomputing. 2014;143:197–207. doi: 10.1016/j.neucom.2014.06.006. DOI

Liu J., Mao Y., Liu X., Li Y. A dynamic adaptive firefly algorithm with globally orientation. Math. Comput. Simulation. 2020;174:76–101. doi: 10.1016/j.matcom.2020.02.020. URL http://www.sciencedirect.com/science/article/pii/S0378475420300598. DOI

Zhu Q., Xiao Y., Chen W., Ni C., Chen Y. 2016. Research on the improved mobile robot localization approach based on firefly algorithm; pp. 323–329. 37.

Kaveh A., Javadi S. Chaos-based firefly algorithms for optimization of cyclically large-size braced steel domes with multiple frequency constraints. Comput. Struct. 2019;214:28–39. doi: 10.1016/j.compstruc.2019.01.006. URL http://www.sciencedirect.com/science/article/pii/S0045794918314755. DOI

Yang X.-S. In: Research and Development in Intelligent Systems XXVI. Bramer M., Ellis R., Petridis M., editors. Springer London; London: 2010. Firefly algorithm, Lévy flights and global optimization; pp. 209–218.

Yu S., Zhu S., Ma Y., Mao D. A variable step size firefly algorithm for numerical optimization. Appl. Math. Comput. 2015;263:214–220. doi: 10.1016/j.amc.2015.04.065. URL http://www.sciencedirect.com/science/article/pii/S0096300315005275. DOI

Ghosh K.K., Guha R., Bera S.K., Kumar N., Sarkar R. S-shaped versus V-shaped transfer functions for binary Manta ray foraging optimization in feature selection problem. Neural Comput. Appl. 2021:1–15.

Dua D., Graff C. University of California, School of Information and Computer Science, Irvine, CA; 2019. UCI Machine Learning Repository.

Hussien A.G., Oliva D., Houssein E.H., Juan A.A., Yu X. Binary whale optimization algorithm for dimensionality reduction. Mathematics. 2020;8(10) doi: 10.3390/math8101821. URL https://www.mdpi.com/2227-7390/8/10/1821. DOI

Too J., Mirjalili S. A hyper learning binary dragonfly algorithm for feature selection: A COVID-19 case study. Knowl.-Based Syst. 2021;212

Sun G., Li J., Dai J., Song Z., Lang F. Feature selection for IoT based on maximal information coefficient. Future Gener. Comput. Syst. 2018;89:606–616.