The sine-cosine algorithm (SCA) is a new population-based meta-heuristic algorithm. In addition to exploiting sine and cosine functions to perform local and global searches (hence the name sine-cosine), the SCA introduces several random and adaptive parameters to facilitate the search process. Although it shows promising results, the search process of the SCA is vulnerable to local minima/maxima due to the adoption of a fixed switch probability and the bounded magnitude of the sine and cosine functions (from -1 to 1). In this paper, we propose a new hybrid Q-learning sine-cosine- based strategy, called the Q-learning sine-cosine algorithm (QLSCA). Within the QLSCA, we eliminate the switching probability. Instead, we rely on the Q-learning algorithm (based on the penalty and reward mechanism) to dynamically identify the best operation during runtime. Additionally, we integrate two new operations (Lévy flight motion and crossover) into the QLSCA to facilitate jumping out of local minima/maxima and enhance the solution diversity. To assess its performance, we adopt the QLSCA for the combinatorial test suite minimization problem. Experimental results reveal that the QLSCA is statistically superior with regard to test suite size reduction compared to recent state-of-the-art strategies, including the original SCA, the particle swarm test generator (PSTG), adaptive particle swarm optimization (APSO) and the cuckoo search strategy (CS) at the 95% confidence level. However, concerning the comparison with discrete particle swarm optimization (DPSO), there is no significant difference in performance at the 95% confidence level. On a positive note, the QLSCA statistically outperforms the DPSO in certain configurations at the 90% confidence level.

IBM Centre of Excellence Faculty of Computer Systems and Software Engineering Universiti Malaysia Pahang Lebuhraya Tun Razak 26300 Kuantan Pahang Darul Makmur Malaysia

Software Testing Intelligent Lab Department of Computer Science Faculty of Electrical Engineering Czech Technical University Karlovo nam 13 121 35 Praha 2 Czech Republic

Zobrazit více v PubMed

Holland JH Adaptation in Natural and Artificial Systems. University of Michigan Press, 1975.

Eberhart R, Kennedy J A new optimizer using particle swarm theory. In: Proc. Sixth International Symposium on Micro Machine and Human Science. IEEE; 1995. p. 39–43.

Kirkpatrick S, Gelatt CD, Vecchi MP Optimization by Simulated Annealing. Science. 1983; 220(4598): 671–680. doi: 10.1126/science.220.4598.671 PubMed DOI

Yang XS A New Metaheuristic Bat-Inspired Algorithm In: Proc. Nature Inspired Cooperative Strategies for Optimization (NICSO 2010). Springer; 2010. p.65–74.

Mirjalili S SCA: A Sine Cosine Algorithm for Solving Optimization Problems Knowledge Based Systems. 2016, 96: 120–133. doi: 10.1016/j.knosys.2015.12.022 DOI

Watkins CJCH and Dayan P Technical Note Q-Learning Machine Learning. 1992, 8: 279–292. doi: 10.1007/BF00992698 DOI

Yang XS and Deb S Cuckoo Search via Levy Flight In: Proc. World Congress on Nature and Biologically Inspired Computing. IEEE; 2009, p. 210–214.

Ahmed BS, Zamli KZ, Lim CP. Application of particle swarm optimization to uniform and variable strength covering array construction. Applied Soft Computing. 2012;12(4):1330–1347. doi: 10.1016/j.asoc.2011.11.029 DOI

Wu H, Nie C, Kuo FC, Leung H, Colbourn CJ A Discrete Particle Swarm Optimization for Covering Array Construction IEEE Transactions on Evolutionary Computation. 2015; 19(4): 575–591. doi: 10.1109/TEVC.2014.2362532 DOI

Mahmoud T, Ahmed BS. An efficient strategy for covering array construction with fuzzy logic-based adaptive swarm optimization for software testing use. Expert Systems with Applications. 2015;42(22):8753–8765. doi: 10.1016/j.eswa.2015.07.029 DOI

Ahmed BS, Abdulsamad TS, Potrus MY. Achievement of minimized combinatorial test suite for configuration-aware software functional testing using the cuckoo search algorithm. Information and Software Technology. 2015;66(C):13–29. doi: 10.1016/j.infsof.2015.05.005 DOI

Cohen MB, Colbourn CJ, Ling ACH. Constructing strength three covering arrays with augmented annealing. Discrete Mathematics. 2008;308(13):2709–2722. doi: 10.1016/j.disc.2006.06.036 DOI

Younis MI, Zamli KZ, Isa NAM. Algebraic strategy to generate pairwise test set for prime number parameters and variables. In: Proc. International Symposium on Information Technology. IEEE; 2008, p. 1–4.

Colbourn CJ, Keri G, Soriano PPR, Schlage-Puchta JC. Covering and radius-covering arrays: Constructions and classification. Discrete Applied Mathematics. 2010;158(11):1158–1180. doi: 10.1016/j.dam.2010.03.008 DOI

Othman RR and Zamli KZ. ITTDG: Integrated t-way test data generation strategy for interaction testing Scientific Research and Essays. 2011;6(17):3638–3648. doi: 10.5897/SRE10.1196 DOI

Cohen MB. Designing test suites for software interaction testing. PhD Thesis, Department of Computer Science, University of Auckland, New Zealand, 2005.

Din F, Alsewari ARA, Zamli KZ. A parameter free choice function based hyper-heuristic strategy for pairwise test generation. In: Proc. IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C). IEEE; 2017, p. 85–91.

Harman M, and Jones BF. Search-based software engineering. Information and Software Technology. 2001;43(14): 833–839. doi: 10.1016/S0950-5849(01)00189-6 DOI

Alsewari ARA and Zamli KZ. Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support. Information and Software Technology. 2012;54(6): 553–568. doi: 10.1016/j.infsof.2012.01.002 DOI

Zamli KZ, Alkazemi BY, Kendall G. A tabu search hyper-heuristic strategy for t-way test suite generation. Applied Soft Computing. 2016;44(C): 53–574.

Zamli KZ, Din F, Kendall G, Ahmed BS. An experimental study of hyper-heuristic selection and acceptance mechanism for combinatorial t-way test suite generation. Information Sciences. 2017;399(C): 121–153. doi: 10.1016/j.ins.2017.03.007 DOI

Alsewari ARA and Zamli KZ. A harmony search based pairwise sampling strategy for combinatorial testing. International Journal of the Physical Science. 2012;7(7): 1062–1072.

Zamli KZ, Din F, Baharom S, Ahmed BS. Fuzzy adaptive teaching learning-based optimization strategy for the problem of generating mixed strength t-way test suites. Engineering Applications of Artificial Intelligence. 2017;59(C): 35–50. doi: 10.1016/j.engappai.2016.12.014 DOI

Cohen MB, Gibbons PB, Mudgridge WB, Colbourn CJ, Collofello JS. Variable strength interaction testing of components In: Proc. Twenty Seventh Annual International Computer Software and Applications Conference. IEEE Computer Society; 2003, p. 413–418.

Chen X, Gu Q, Li A, Chen D. Variable strength interaction testing with an ant colony system approach. In: Proc. Sixteenth Asia-Pacific Software Engineering Conference. IEEE Computer Society; 2009, p. 160–167.

Shiba T, Tsuchiya T, Kikuno T. Using artificial life techniques to generate test cases for combinatorial testing. In: Proc. Twenty Eighth Asia-Pacific Software Engineering Conference. IEEE Computer Society; 2004, p. 72–77.

Ahmed BS and Zamli KZ. A variable-strength interaction test suites generation strategy using particle swarm optimization. Journal of Systems and Software. 2011;84(12): 2171–2185. doi: 10.1016/j.jss.2011.06.004 DOI

Camastra F, Ciaramella A, Giovannelli V, Lener M, Rastelli V, Staiano A, et al. A fuzzy decision system for genetically modified plant environmental risk assessment using Mamdani inference. Expert Systems with Applications. 2015;42(3): 1710–1716. doi: 10.1016/j.eswa.2014.09.041 DOI

Cordón O. A historical review of evolutionary learning methods for Mamdani-type fuzzy rule-based systems: Designing interpretable genetic fuzzy systems. International Journal of Approximate Reasoning. 2011;52(6): 894–913. doi: 10.1016/j.ijar.2011.03.004 DOI

Wolpert DH and Macready WG. No Free Lunch Theorems for Optimization. IEEE Transactions on Evolutionary Computation. 1997;1(1): 67–82. doi: 10.1109/4235.585893 DOI

Sutton RS and Barto AG. Reinforcement learning: An introduction. Cambridge: MIT Press, 1998.

Zamli KZ, Din F, Baharom S, Ahmed BS Fuzzy adaptive teaching learning-based optimization strategy for the problem of generating mixed strength t-way test suites. Engineering Applications of Artificial Intelligence, 2017; 59: 35–50. doi: 10.1016/j.engappai.2016.12.014 DOI

Zamli KZ, Din F, Kendall G, Ahmed BS An experimental study of hyper-heuristic selection and acceptance mechanism for combinatorial t-way test suite generation Information Sciences, 2017;399,:121–153. doi: 10.1016/j.ins.2017.03.007 DOI

Gambardella LM and Dorigo M. Ant-Q: A reinforcement learning approach to the traveling salesman problem. In: Proc. Twelfth International Conference on Machine Learning. Morgan Kaufmann Publishers Inc.; 1995, p. 252–260.

Khamassi I, Hammami M, Ghédira K. Ant-Q hyper-heuristic approach for solving 2-dimensional cutting stock problem. In: Proc. IEEE Symposium on Swarm Intelligence. IEEE; 2011, p. 1–7.

Machado L and Schirru R. The Ant-Q algorithm applied to the nuclear reload problem. Annals of Nuclear Energy. 2002;29(12): 1455–1470. doi: 10.1016/S0306-4549(01)00118-9 DOI

Samma H, Lim CP, Junita MS. A new reinforcement learning-based memetic particle swarm optimizer. Applied Soft Computing. 2016;43(C): 276–297. doi: 10.1016/j.asoc.2016.01.006 DOI

Yang XS. Nature-inspired metaheursitic algorithm. Luniver Press, 2010.

Center KAMS Bio-Complexity: Computational Model Tutorial. available from https://www3.nd.edu/tutorial/walksReport.html, last accessed on November.

Press WH, Teukolsky SA, Vetterling WT, Flannery BP Numerical recipes in C- The art of scientific computing. Cambridge University Press, 1992.

Guo M, Liu Y, Malec J A new Q-learning algorithm based on the metropolis criterion. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics. 2004; 34(5): 2140–2143. doi: 10.1109/TSMCB.2004.832154 PubMed DOI

Robinson J, Rahmat-Samii Y Particle swarm optimization in electromagnetics. IEEE Transactions on Antennas and Propagation. 2004;52(2): 397–407. doi: 10.1109/TAP.2004.823969 DOI

Holm S A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics. 1979;6(C): 65–70.