0.13 Least squares design of iir filters

[link] introduced the IIR least squares design problem, as illustrated in [link] . Such problem cannot be solved in the same manner as in the FIR case; therefore more sophisticated methods must be employed. As will be discussed later in [link] , some tradeoffs are desirable for $l_p$ optimization. As in the case of FIR design, when designing $l_p$ IIR filters one must use $l_2$ methods as internal steps over which one iterates while moving between diferent values of $p$ . Clearly this internal iteration must not be too demanding computationally since an outer $l_p$ loop will invoke it repeatedly (this process will be further illustrated in [link] ). With this issue in mind, one needs to select an $l_2$ algorithm that remains accurate within reasonable error bounds while remaining computationally efficient.

This section begins by summarizing some of the traditional approaches that have been employed for $l_2$ rational approximation, both within and outside filter design applications. Amongst the several existing traditional nonlinear optimization approaches, the Davidon-Fletcher-Powell (DFP) and the Gauss-Newton methods have been often used and remain relatively well understood in the filter design community. A brief introduction to both methods is presented in [link] , and their caveats briefly explored.

An alternative to attacking a complex nonlinear problem like [link] with general nonlinear optimization tools consists in linearization , an attempt to "linearize" a nonlinear problem and to solve it by using linear optimization tools. Multiple efforts have been applied to similar problems in different areas of statistics and systems analysis and design. [link] introduces the notion of an Equation Error , a linear expression related to the actual Solution Error that one is interested in minimizing in $l_2$ design. The equation error formulation is nonetheles important for a number of filter design methods (including the ones presented in this work) such as Levy's method, one of the earliest and most relevant frequency domain linearization approaches. [link] presents a frequency domain equation error algorithm based on the methods by Prony and Padé. This algorithm illustrates the usefulness of the equation error formulation as it is fundamental to the implementation of the methods proposed later in this work (in [link] ).

An important class of linearization methods fall under the name of iterative prefiltering algorithms, presented in [link] . The Sanathanan-Koerner ( SK ) algorithm and the Steiglitz-McBride ( SMB ) methods are well known and commonly used examples in this category, and their strengths and weaknesses are explored. Another recent development in this area is the method by Jackson, also presented in this section. Finally, Soewito's quasilinearization (the method of choice for least squares IIR approximation in this work) is presented in [link] .

Traditional optimization methods

One way to adress [link] is to attempt to solve it with general nonlinear optimization tools. One of the most typical approach in nonlinear optimization is to apply either Newton's method or a Newton-based algorithm. One assumption of Newton's method is that the optimization function resembles a quadratic function near the solution (refer to Appendix [link] for more information). In order to update a current estimate, Newton's method requires first and second order information through the use of gradient and Hessian matrices. A quasi-Newton method is one that estimates in a certain way the second order information based on gradients (by generalizing the secant method to multiple dimensions).