Figure 1: Graph of function with first and second derivatives
The analytical solution is found by taking the first and second derivatives of the provided equation. Then plug in values at various points in x to find the analytical solutions at those points. Compare to the forward, backward, and central finite difference approximations of the first and second order near those points as well. The analytical equations are written below and example calculations are provided for each. See Figure 1 above for first and second analytical derivatives plotted alongside the original function.
%Declare variables
h; x; y;
%Declare analytical derivatives
%Declare first-order approximations
y_1f; y_1c; y_1b;
%Declare second-order approximations
y_2f; y_2c; y_2b;
%Calculate percent error for 1st and 2nd order
y_1f%; y_1c%; y_1b%; y_2f%; y_2c%; y_2b%;
%Calculate true error for 1st and 2nd order approximations
y_1fABS; y_1cABS; y_1bABS; y_2fABS; y_2cABS; y_2bABS;
%plots
We are going to look at all three approaches and compare the results to the same-order analytical derivative to see which is superior. Note that deciding which difference approximation is most accurate depends on the function and where you are evaluating along that function. Although we will see the central finite approximation is best for both the first and second order derivatives, this is not always the case.
Figure 2: Illustration of results for the forward, backward, and centered difference appriximations alongside theoretical derivative
Looking at Figure 2, we see the first order results from the difference approximations plotted alongside the analytical first derivative. What is interesting to note here is that the forward approximation seems to underestimate the derivative at x values below zero, and overestimate it at x values above zero. The reverse is true for the backward approximation, which overestimates the derivative at x values below zero and underestimates it at x values above zero. This makes sense, as the forward approximation takes the average slope between a point at the current x value and a point one "step" ahead of the current x value. So when the derivative is decreasing (as it is between x = -2 and x = 0), the forward approximation should return underestimations as it is factoring in a smaller f(x) value one step ahead. When the function is increasing, as it does after x = 0, the forward approximation will overestimate because it is factoring in larger increases at greater values of x.
As we can see visually, the centered finite approximation appears to be extremely close to the first derivative. As we get into error calculations, we will observe how the centered-finite approach is comparatively more accurate at predicting the analytical solutions relative to the forward and backward approaches.
Figure 3 shows the absolute value of error for the forward, backward, and centered finite approximations. Notice that below x = 0, the forward approach has a smaller error compared to the backward approach and that this switches at x = 0. The centered approximation appears to be near zero regardless of the value for x.
This is also observed in the percentage error. The backward error exceeds that of the forward error below x = 0, and the opposite is true at values above x = 0. Notice how the percent error rapidly increases at x = 0.25 and x = -0.25. These are the locations one "step size" away from x = 0, which is no coincidence. If you look at the equations for the first-order difference approximations notice how when x = 0 the step size accounts for the only difference in the output of the function. As x increases or decreases, the step size 'h' becomes a smaller portion of what is input into the function, and so its influence is decreased.
Figure 5: Illustration of results for 2nd-order forward, backward, and centered difference approximations.
Looking at Figure 5, we see how the center finite approximation most accurately predicts the behavior of the analytical second derivative. As we will observe in our error graphs, the center approximation is actually identical to the second derivative at all x values on this interval.
The backward approximation is always an underestimation of the second derivative, which makes sense given the second derivative has a constant slope upward (around 6). Because the backward approximation is always using the current and the previous value of x to estimate change, it will always underestimate the function in the case where the function is increasing at a constant rate. The same logic applies for the forward approximation. If the forward approximation uses the average rate of change between the current value of x and a future value of x, then at any point x the forward approximation will be too high as it factors in larger outputs of the function.
As we observed in Figure 5, the second derivative increases at a constant rate along the interval -2 to 2. This will cause our forward and backward approximations to have a constant value for numerical error, as we see in Figure 6. Notice how the forward and backward error are identical. This is because the forward and backward approximations are the same distance away from the analytical second derivative at all points in the interval. The second order center approximation always has an error of zero because it is identical to the analytical second derivative.
Looking at Figure 7, we see the error for the forward and backward approximations rapidly increase near x = 0. This makes sense, as the absolute value of the numerical error is always the same yet the second derivative approaches zero at x = 0. Because the denominator of percentage error is the "true value" (i.e. the analytical second derivative), as this approaches zero the total percent error rapidly increases until it becomes undefined at x = 0.