21.1 Ordinary Least Squares Methods (.NET, C#, CSharp, VB, Visual Basic, F#)
NMath includes least squares classes that compute solutions using various methods: Cholesky factorization, QR decomposition, and singular value decomposition. The interface is virtually identical for all least squares classes.
Least Squares Using Cholesky Factorization
The Cholesky least squares classes
solve least square problems by using the Cholesky factorization to solve
the normal equations. The normal equations for the least squares problem
are:
where 
denotes the transpose of a real matrix A
or the conjugate transpose of a complex matrix A.
If A has full rank, then 
is symmetric
positive definite—the converse is also true—and
the Cholesky factorization may be used to solve the normal equations.
This method will fail if the matrix A is rank deficient.
Finding least squares solutions using the normal equations is often the best method when speed is the only consideration.
Least Squares Using QR Decomposition
The QR decomposition least squares
classes solve least squares problems by using a QR decomposition to find
the minimal norm solution to the linear system 
. That is, they
find the vector x that minimizes the 2-norm
of the residual vector 
. Matrix A must have more rows than
columns, and be of full rank.
Finding least squares solutions via QR decomposition is the "standard" method for least squares problems, and is recommended for general use.
If the matrix A is close to rank-deficient, the QR decomposition method described above has less than ideal stability properties. In such cases, a method based on singular value decomposition is a better choice.