Powell's method

From DDWiki

Powell's method, also called conjugate direction method, is a zero-order optimization method improved from univariate search method.

The basic concept of Power's conjugate direction approach is to utilize the sequential search directions in one-dimension search and generate a new direction toward next iterative point. As unidirectional search vector S_i, i = 1...n (for n variables) is defined, the conjugate direction is determined by sum of all unidirectional vectors:

where S_n+1 is conjugate direction and α_i is scaling factor.

The solution search path of Power's method for a simple two-dimensional case is shown in the following figure:

While Powell's method is one the most efficient and reliable zero-order optimization method, there are some situations that the method may not perform well.

• When a search direction cannot provide further improvement, the subsequent search direction will not be conjugate.
• Sometimes the search directions may become parallel after few iterations.

Reference

• Steven C. Chapra and Raymond Canale, 2002, Numerical Methods for Engineers: With Software and Programming Applications, 4th Ed., McGraw-Hill.
• Garret N. Vanderplaats, 1993, Numerical Optimization Techniques for Engineering Design: With Applications, McGraw-Hill.