Fitting Data

Suppose you charge a capacitor through a resistor and read its voltage with a voltmeter every half second. You end up with a list of numbers, but what you want is a single number: the time constant $\tau = RC$, which says how quickly the capacitor charges. A fit is how you get from one to the other. You choose a model (an equation with adjustable parameters), then find the parameter values that bring the curve closest to the data.

“Closest” is made precise with residuals: the residual of a data point is the measured value minus the model’s value at that same time,

Whatever the model, plot the residuals and check for two things:

• Random scatter about zero, with a size comparable to the error bars
• A systematic pattern (an arch, a trend, a run of points all on one side): the model is wrong, or the parameters are not at their best values.

The first model most of us reach for is a straight line, $V = a + bt$. Fitting a line has real advantages: the best-fit $a$ and $b$ come from a closed-form formula, and the uncertainties in $a$ and $b$ do too. For data that really are linear, such as distance against time at constant speed, nothing more is needed.

Try it on the data below. Each voltmeter reading has an uncertainty of $\pm 0.1\ \text{V}$ (shown with error bars).

The line follows the overall trend in rising voltage, and the software reports confident-looking values of $a$ and $b$. However, many residuals are much larger than the reported uncertainties, and they do not appear randomly scattered about zero. This indicates that we have the wrong model. Charging slows down as the capacitor fills, so we need a different fitting function.

A functional fit

In the ideal theoretical case, a capacitor charging toward the battery voltage follows

which has two parameters with physical meaning: $V_b$, the battery voltage the capacitor is heading toward, and $\tau$, the time it takes to cover about $63\%$ of the remaining “distance”. If you switch the fit model above, and the residuals scatter about zero and lie mostly inside the $\pm 0.1\ \text{V}$ band.

There is no formula that hands you the best $V_b$ and $\tau$. Instead the computer (or you, in Desmos) adjusts them iteratively until the curve can’t be improved.

It is tempting to dodge this by straightening the data. Since $V_b - V = V_b e^{-t/\tau}$, a graph of $\ln(V_b - V)$ against $t$ is a straight line with slope $-1/\tau$. This works, but poorly. You need $V_b$ before you can plot anything, even though it is one of the things you are trying to fit. Late in the charge, $V_b - V$ is tiny, so the same voltmeter noise becomes a huge relative error (and can even make the logarithm undefined). Taking the log also distorts the error bars. Fitting the original curve avoids all of this.

Uncertainty by hand

Your best-fit $\tau$ is not a perfect number. Slightly different values of $\tau$ would have described the data almost as well. The uncertainty in $\tau$ should quantify the range of values that still describe the data acceptably, and you can find it directly: leave the best fit, increase $\tau$ until the residuals clearly develop a pattern, and note where that happened. Then do the same in the other direction. Half the distance between these two extreme values is a reasonable estimate of uncertainty.

Real data rarely start at the instant the switch closes. In the run below, recording began early, so the first few readings sit at zero until the capacitor starts charging at some time $t_0$. The model gains a third parameter:

Note that the three fit parameters are linked. A larger $\tau$ makes the curve rise more slowly, which can be partly compensated by a larger $V_b$. Delaying the start time also has a similar effect on the curve as a larger $\tau$. When adjusting $\tau$ to estimate the uncertainty, you should also re-adjust the other free parameters before judging the residuals.

If you find this process tedious, you are not alone! Luckily, there are ways of automating this with a computer. A common method uses the so-called chi-square statistic, which is discussed below.

A first look at $\chi^2$

Manually adjusting fit parameters by hand is subjective, and as the number of free parameters increases beyond two, this process can get rather unwieldy. The chi-square statistic allows us to make this fitting process more objective. For each point, measure the residual in units of that point’s own uncertainty $\sigma_i$, square it so that positive and negative residuals do not cancel, and add them up:

This reframes the fitting process into an optimization problem: the best-fit parameters (contained in the model function) are the ones that make $\chi^2$ as small as possible. Furthermore, we can make some quantitative statements about uncertainty. A data point that “misses” by about one $\sigma$ adds about $1$ to this sum, so we expect a good fit to $N$ points with $p$ parameters has $(\chi^2)_\text{min} \approx N - p$ (the minus p requires a bit of thought).

The uncertainty in each best-fit parameter has a clean rule: the uncertainty in a parameter is how far you can adjust it, re-optimizing the others, before $\chi^2$ rises by $1$ above its minimum.