Abstract
The main purpose of this paper is to suggest a method for finding the minimum of a functionf(x) subject to the constraintg(x)=0. The method consists of replacingf byF=f+λg+1/2cg 2, wherec is a suitably large constant, and computing the appropriate value of the Lagrange multiplier. Only the simplest algorithm is presented. The remaining part of the paper is devoted to a survey of known methods for finding unconstrained minima, with special emphasis on the various gradient techniques that are available. This includes Newton's method and the method of conjugate gradients.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hestenes, M. R.,A General Problem in the Calculus of Variations with Applications to Paths of Least Time, The RAND Corporation, Research Memorandum No. RM-100, 1950.
Mengel, A. S.,Optimum Trajectories, The RAND Corporation, Report No. P-199, 1951.
Hestenes, M. R.,Numerical Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, The RAND Corporation, Research Memorandum No. RM-102, 1949.
Stein, M. L.,On Methods for Obtaining Solutions of Fixed-Endpoint Problems in the Calculus of Variations, Journal of Research of the National Bureau of Standards, Vol. 50, No. 5, 1953.
Hestenes, M. R.,Iterative Computational Methods, Communications on Pure and Applied Mathematics, Vol. 8, No. 1, 1955.
Balakrishnan, A. V.,On a New Computing Technique in Optimal Control and Its Application to Minimal-Time Flight Profile Optimization, Journal of Optimization Theory and Applications, Vol. 4, No. 1, 1969.
Hestenes, M. R.,An Indirect Sufficiency Proof for the Problem of Bolza in Nonparametric Form, Transactions of the American Mathematical Society, Vol. 62, No. 3, 1947.
Hestenes, M. R., andStiefel, E.,Methods of Conjugate Gradients for Solving Linear Systems, Journal of Research of the National Bureau of Standards, Vol. 49, No. 6, 1952.
Hestenes, M. R.,The Conjugate Gradient Method for Solving Linear Systems, Proceedings of the Sixth Symposium in Applied Mathematics, Edited by J. H. Curtiss, American Mathematical Society, Providence, Rhode Island, 1956.
Hayes, R. M.,Iterative Methods for Solving Linear Problems in Hilbert Space, Contributions to the Solutions of Systems of Linear Equations and the Determinations of Eigenvalues, Edited by O. Tausky, National Bureau of Standards, Applied Mathematics Series, US Government Printing Office, Washington, D.C., 1954.
Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, No. 2, 1964.
Myers, G. E.,Properties of the Conjugate-Gradient and Davidon Methods, Journal of Optimization Theory and Applications, Vol. 2, No. 4, 1968.
Horwitz, L. B., andSarachick, P. E.,Davidon's Method in Hilbert Space, SIAM Journal on Applied Mathematics, Vol. 16, No. 4, 1968.
Author information
Authors and Affiliations
Additional information
The preparation of this paper was sponsored by the U.S. Army Research Office, Grant No. DA-31-124-ARO(D)-355. This paper was presented at the Second International Conference on Computing Methods in Optimization Problems, San Remo, Italy, 1968.
Rights and permissions
About this article
Cite this article
Hestenes, M.R. Multiplier and gradient methods. J Optim Theory Appl 4, 303–320 (1969). https://doi.org/10.1007/BF00927673
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00927673