Levmar:Levenberg-Marquardt非線性最小二乘演算法

演算法 · 發表 2018-09-26 20:53:00

摘要： Levmar:Levenberg-Marquardt非線性最小二乘演算法 [email protected] Abstract.Levmar is GPL native ANSI C implementations of the Levenberg-Marquardt optimizat...

Levmar:Levenberg-Marquardt非線性最小二乘演算法

ofollow,noindex" target="_blank">[email protected]

Abstract.Levmar is GPL native ANSI C implementations of the Levenberg-Marquardt optimization algorithm.The blog focus on the compilation of levmar on Windows with Visual Studio.

Key Words.Levmar, C, LM least squares

1. levmar簡介

Gauss-Newton演算法是一個古老的處理非線性最小二乘問題的方法。該方法在迭代過程中要求矩陣J(x)滿秩。為了克服這個困難，Levenberg(1944)提出了一種新的方法，但未受到重視。後來Marquardt(1963)又重新提出，並在理論上進行了控討，得到Levenberg-Marquardt方法，簡稱LM方法。在此基礎上，Fletcher(1971)對其實現策略進行了改進，得到了Levenberg-Marquardt-Fletcher方法（LMF）。再後來，More(1978)將LM方法與信賴域方法結合，建立了帶信賴域的LM方法。

LM演算法的產生主要是解決曲線最小二乘擬合問題，現在很多軟體使用LM演算法來解決通用的曲線擬合問題。

本文主要介紹GPL開源庫levmar2.6使用Visual Studio在Windows上進行編譯。這個開源庫的官方網站是： http://users.ics.forth.gr/~lourakis/levmar/

2. 編譯levmar

下載原始碼levmar-2.6解壓，在其README.txt中對levmar的授權GPL、編譯等進行了說明。在Windows作業系統中，可以使用nmake /f Makefile.vc來編譯levmar和一個示例程式。

從官網介紹可知，levmar有些演算法依賴LAPACK庫，一個線性代數計算開源庫。所以如果要使用那些演算法，編譯的時候必須包含這個庫。從示例程式的原始檔lmdemo.c中可以看出，有些問題的求解是需要LAPACK庫的，相關原始碼列出如下：

/* uncomment the appropriate line below to select a minimization problem */
problem=
//0; // Rosenbrock function
//1; // modified Rosenbrock problem
//2; // Powell's function
//3; // Wood's function
4; // Meyer's (reformulated) problem
//5; // Osborne's problem
//6; // helical valley function
#ifdef HAVE_LAPACK
//7; // Boggs & Tolle's problem 3
//8; // Hock - Schittkowski problem 28
//9; // Hock - Schittkowski problem 48
//10; // Hock - Schittkowski problem 51
#else // no LAPACK
#ifdef _MSC_VER
#pragma message("LAPACK not available, some test problems cannot be used")
#else
#warning LAPACK not available, some test problems cannot be used
#endif // _MSC_VER
#endif /* HAVE_LAPACK */
//11; // Hock - Schittkowski problem 01
//12; // Hock - Schittkowski modified problem 21
//13; // hatfldb problem
//14; // hatfldc problem
//15; // equilibrium combustion problem
#ifdef HAVE_LAPACK
//16; // Hock - Schittkowski modified #1 problem 52
//17; // Schittkowski modified problem 235
//18; // Boggs & Tolle modified problem #7
//19; // Hock - Schittkowski modified #2 problem 52
//20; // Hock - Schittkowski modified problem #76"
#endif /* HAVE_LAPACK */
switch(problem){
default: fprintf(stderr, "unknown problem specified (#%d)! Note that some minimization problems require LAPACK.\n", problem);
exit(1);
break;

從上述原始碼可知，如果LAPACK庫不可用的時候，示例程式中的問題

l 7 Boggs & Tolle’s problem 3

l 8 Hock - Schittkowski problem 28

l 9 Hock - Schittkowski problem 48

l 10 Hock - Schittkowski problem 51

l 16 Hock - Schittkowskit modified #1 problem 52

l 17 Schittkowski modified problem 235

l 18 Boggs & Tolle modified problem #7

l 19 Hock - Schittkowski modified #2 problem 52

l 20 Hock - Schittkowski modified probem #76

這些問題的求解功能是不能使用的。從標頭檔案levmar.h中要以看出，

#ifdef LM_DBL_PREC
/* double precision LM, with & without Jacobian */
/* unconstrained minimization */
extern int dlevmar_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, int itmax, double *opts,
double *info, double *work, double *covar, void *adata);
extern int dlevmar_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, int itmax, double *opts,
double *info, double *work, double *covar, void *adata);
/* box-constrained minimization */
extern int dlevmar_bc_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, double *dscl,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
extern int dlevmar_bc_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, double *dscl,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
#ifdef HAVE_LAPACK
/* linear equation constrained minimization */
extern int dlevmar_lec_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *A, double *b, int k,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
extern int dlevmar_lec_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *A, double *b, int k,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
/* box & linear equation constrained minimization */
extern int dlevmar_blec_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, double *A, double *b, int k, double *wghts,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
extern int dlevmar_blec_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, double *A, double *b, int k, double *wghts,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
/* box, linear equations & inequalities constrained minimization */
extern int dlevmar_bleic_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub,
double *A, double *b, int k1, double *C, double *d, int k2,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
extern int dlevmar_bleic_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, 
double *A, double *b, int k1, double *C, double *d, int k2,
int itmax, double *opts, double *info, double *work, double *covar, void *adata);
/* box & linear inequality constraints */
extern int dlevmar_blic_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, double *C, double *d, int k2,
int itmax, double opts[4], double info[LM_INFO_SZ], double *work, double *covar, void *adata);
extern int dlevmar_blic_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *lb, double *ub, double *C, double *d, int k2,
int itmax, double opts[5], double info[LM_INFO_SZ], double *work, double *covar, void *adata);
/* linear equation & inequality constraints */
extern int dlevmar_leic_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *A, double *b, int k1, double *C, double *d, int k2,
int itmax, double opts[4], double info[LM_INFO_SZ], double *work, double *covar, void *adata);
extern int dlevmar_leic_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *A, double *b, int k1, double *C, double *d, int k2,
int itmax, double opts[5], double info[LM_INFO_SZ], double *work, double *covar, void *adata);
/* linear inequality constraints */
extern int dlevmar_lic_der(
void (*func)(double *p, double *hx, int m, int n, void *adata),
void (*jacf)(double *p, double *j, int m, int n, void *adata),
double *p, double *x, int m, int n, double *C, double *d, int k2,
int itmax, double opts[4], double info[LM_INFO_SZ], double *work, double *covar, void *adata);
extern int dlevmar_lic_dif(
void (*func)(double *p, double *hx, int m, int n, void *adata),
double *p, double *x, int m, int n, double *C, double *d, int k2,
int itmax, double opts[5], double info[LM_INFO_SZ], double *work, double *covar, void *adata);
#endif /* HAVE_LAPACK */
#endif /* LM_DBL_PREC */

從標頭檔案levmar.h中的程式碼可以看出，在#ifdef HAVE_LAPACK和#endif /* HAVE_LAPACK */之間的函式都是不可用的。除此之外的函式是可用的，如基本的dlevmar_der和dlevmar_dif等函式是不依賴LAPACK庫的。如果只使用這幾個函式，則可以不用配置LAPACK庫，編譯levmar就很簡單了。

如果不使用LAPACK庫，可以先在標頭檔案levmar.h中把#define HAVE_LAPACK 這一行註釋掉：