Population and Gradient Based Optimization Techniques, a Theoretical Overview

  • G. Friedl Széchenyi István University
  • M. Kuczmann Széchenyi István University
Keywords: optimization, soft computing, evolutionary algorithms, memetic algorithms


Many practical problems can be modeled only as a nonlinear continuous global optimization problem. It is usually impossible and impractical to solve them exactly. Evolutionary and hybrid algorithms are modern techniques to find optima for complex search spaces. This paper is a short summary about the optimization techniques, especially about the evolutionary based global searching algorithms, and the gradient based local searching algorithms. After the introduction, the second chapter gives a brief summary about population based techniques, especially about genetic and bacterial evolutionary algorithm, and particle swarm optimization. The next chapter discusses two gradient based searching technique, the steepest descent and the Levenberg-Marquardt method. This paper is a theoretical overview of the basics of my future Ph.D dissertation.

Author Biographies

G. Friedl, Széchenyi István University
Department of Automation, assistant lecturer
M. Kuczmann, Széchenyi István University
Department of Automation, Head of Department


Holland, J. H.: Adaption in Natural and Artificial Systems, The MIT Press, Massachusetts, 1992

Nawa, N. E., Hashiyama, T., Furuhashi, T., Uchikawa, Y.: Fuzzy Logic Controllers Generated by Pseudo-Bacterial Genetic Algorithm, In Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 2408–2413, Houston, 1997

DOI: 10.1109/ICNN.1997.614446

Nawa, N. E., Furuhashi T.: Fuzzy system parameters discovery by bacterial evolutionary algorithm, IEEE Transactions on Fuzzy Systems, vol. 7, no. 5, pp. 608–616, 1999

DOI: 10.1109/91.797983

Kennedy, J., Eberhart R.: Particle swarm optimization In Proceedings of the IEEE International Conference on Neural Networks, vol. 4, pp. 1942–1948, Perth, 1995

DOI: 10.1109/ICNN.1995.488968

Balázs, K.: Advanced Approaches in the Application Methodologies of Evolutionary Algorithms. Ph.D. dissertation, Budapest, Hungary, 2013

Sumathi, S., Hamsapriya, T., Surekha P.: Evolutionary Intelligence - An Introduction to Theory and Applications with Matlab, Springer, India, 2008

Gál, L.: Optimalization of Fuzzy Models by Bacterial Type Algorithms (in Hungarian), Ph.D. dissertation, Győr, Hungary, 2012

del Valle, Y., Venayagamoorthy, G. K., Mohagheghi, S., Hernandez, J.-C., Harley, R. G.: Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems, IEEE Transactions on Evolutionary Computation, vol. 12, no. 2, pp. 171-195, 2008

DOI: 10.1109/TEVC.2007.896686

Levenberg, K.: A method for the solution of certain non-linear problems in least squares, Quarterly Journal of Applied Mathematics, vol. 2, no. 2, pp. 164–168, 1944

Marquardt, D.: An Algorithm for Least-Squares Estimation of Nonlinear Parameters, SIAM J. Appl. Math., vol. 11, no. 2, pp. 431-441, 1963

Yamada, I.: The Hybrid Steepest Descent Method for the Variational Inequality Problem over the Intersection of Fixed Points Sets of Nonexpansive Mapping, Studies in Computational Mathemathics, vol. 8, pp. 473–504, 2001

DOI: 10.1016/S1570-579X(01)80028-8

Snyman, J.: Practical Mathematical Optimization, Springer, New York, 2005

How to Cite
Friedl, G. and Kuczmann, M. (2014) “Population and Gradient Based Optimization Techniques, a Theoretical Overview”, Acta Technica Jaurinensis, 7(4), pp. pp. 378-387. doi: 10.14513/actatechjaur.v7.n4.342.
Information Technology and Electrical Engineering