Checkgradient without solving optimization problem in MATLAB

user_6366901 0
Viewed 80

I have a relatively complicated function and I have calculated the analytical form of the Jacobian of this function. However, sometimes, I mess up this Jacobian.

MATLAB has a nice way to check for the accuracy of the Jacobian when using some optimization technique as described here.

The problem though is that it looks like MATLAB solves the optimization problem and then returns if the Jacobian was correct or not. This is extremely time consuming, especially considering that some of my optimization problems take hours or even days to compute.

Python has a somewhat similar function in scipy as described here which just compares the analytical gradient with a finite difference approximation of the gradient for some user provided input.

Is there anything I can do to check the accuracy of the Jacobian in MATLAB without having to solve the entire optimization problem?

mathematical-optimization scipy matlab python
2 Answers

A laborious but useful method I've used for this sort of thing is to check that the (numerical) integral of the purported derivative is the difference of the function at the end points. I have found this more convenient than comparing fractions like (f(x+h)-f(x))/h with f'(x) because of the difficulty of choosing h so that on the one hand h is not so small that the fraction is not dominated by rounding error and on the other h is small enough that the fraction should be close to f'(x)

In the case of a function F of a single variable, the assumption is that you have code f to evaluate F and fd say to evaluate F'. Then the test is, for various intervals [a,b] to look at the differences, which the fundamental theorem of calculus says should be 0,

Integral{ 0<=x<=b | fd(x)} - (f(b)-f(a))

with the integral being computed numerically. There is no need for the intervals to be small. Part of the error will, of course, be due to the error in the numerical approximation to the integral. For this reason I tend to use, for example, and order 40 Gausss Legendre integrator.

For functions of several variables, you can test one variable at a time. For several functions, these can be tested one at a time.

I've found that these tests, which are of course by no means exhaustive, show up the kinds of mistakes that occur in computing derivatives quire readily.

user_25646960

Have you considered the usage of Complex step differentiation to check your gradient? See this description

user_135605980
Related