The Theory of Curve Fitting through the Least Squares Method
Total word count: 1061
Time spent researching and writing: ~10 hrs
There are millions of statistics out in the world today, from numerous shopping demographics to scientific research. The main objectives when using statistical data can be to find trends, see what averages are, and predict what may happen in the future. This is done through curve fitting, a process in which a line is fit on a scatter plot of data to connect different points of data to the same line, creating a trend or curve. Doing so in the math world will create an algebraic polynomial in which any point in the domain can then be found using this function according to the graph. So what is the theory of curve fitting, and how does it work?
The first thing to do when curve fitting is find your data and graph it onto a scatter plot, making sure that there is at least some pattern or congruency between the points so that a general line can at least somewhat accurately follow the data points. Once you have this onto a data plot, you can use the least squares method in order to find the polynomial equation that best fits the scatter plots, whether it be a quadratic, cubic, quartic, exponential, power, logarithmic, etc.
Looking at an introductory statistics book will help to understand how a computer or mathematician who knows calculus goes about fitting a line to a scatter plot of data. According to book Statistics Demystified by Stan Gibilsco, the law of least squares is used to find the “best overall straight-line averages of the positions of the points” (192).
In a nutshell, this least squares method allows a computer to use calculus and in essence look through different possible equations of a type of function (quadratic, exponential, etc.) that best fits the line. The distance between each point and the attempted line is then measured. These numbers are then squared and added together, getting a final sum D of which the least squares line is where the lowest value of D is reached (Gibilisco 192). There can only be one least squares line, and this line is found with through functions of calculus. After the line is found, an equation of the line is then generated that best fits the data points, allowing for values other than the already obtained data points to be found algebraically.
Furthermore, in the book First (and Second) Steps in Statistics by Wright and London, the least squares method is explained as “fitting a line that minimizes the sum of squares of all the distances from the line to the points. This is called the…regression line” (139). This section of the book goes on to explain that the line found is the one that minimizes åei2, “where ei are called residuals or the distance between the line and the observed value for the y variable” (Wright and London 139).
To illustrate this, let’s look at a specific example from the “Focus on Modeling” section of the algebra textbook Precalculus for Boise State University. In chapter 3, a calculator is used to place points on a graph and create the least squares line for you. In this paper, I will use the computer program pro Fit to better illustrate the effects. Our example problem is one that is used to calculate the height of a baseball thrown upward. This problem states that “A baseball is thrown upward and its height measured at 0.5-second intervals using a strobe light” (Stewart, Redlin, and Watson 325). Using the data that we were given, we would enter these points in a data chart in the computer program:
After this is done, the computer can then use these values to produce a scatter plot of the function, using the x-values (time) in correlation with the y-values (height), and we find this graph:
From what we see here, we can conclude that we will probably want to apply a quadratic function to best represent this data, given the shape of the graph. To do so, the program will then have to calculate the function using the method of least squares as described above. For each point on the graph, the computer program will measure the distance between such points, square them, and add then together for the least square value until this value is as low as it will go. In this case, our Chi squared value is 2.1429e-2, which correlates to 0.021429, which is makes this a very accurate line as shown below:
The quadratic function that models this graph is T(x) = 51.8429x2 – 16.0000x + 4.2071.
So how was the quadratic equation reached from the data points? The computer program used calculus to find a line that had the smallest chi-squared, or least squares value from the data points. The way we can check this to see is by taking the original values of the actual points given and subtract these from the points given in the same spot from that line, where d is a data point on the graph, like so:
d1 = (0, 4.2); Actual = (0, 4.2071429)
d2 = (.5, 26.1); Actual = (.5, 26.128571)
d3 = (1.0, 40.1); Actual = (1, 40.05)
d4 = (1.5, 46.0); Actual = (1.5, 45.971429)
d5 = (2.0, 43.9); Actual = (2.0, 43.892857)
d6 = (2.5, 33.7); Actual = (2.5, 33.814286)
d7 = (3.0, 15.8); Actual = (3.0, 15.735714)
Actual distance points are away from line (vertically):
d1 = (.0071429)2 = .00005102102041
d2 = (.028571)2 = .000815302041
d3 = (.05)2 = .0025
d4 = (.028571)2 = .00081630241
d5 = (.007143)2 = .000051022449
d6 = (.114286)2 = .013061289796
d7 = (.064286)2 = .004132689796
Then add all of these together to get the Sum of least squares: 0.02142762751241. We can see that this is almost exactly the same result that the computer gave us (the chi-squared value), which came out to be 0.021429. This is how the least-squares example works. The line of least squares is the line that has the smallest of these numbers.
Overall, the least-squares line can found through calculus and ultimately be used to allow a mathematician to predict changes in the graph or examine changes between the original data points given. In the real world, this would allow a statistician to predict real changes happening, as well as find average rates of change and patterns in the statistics. This least-squares line is still one of the most used statistical methods today, and helps us to better understand the demographics people as well as the world we live in.
Gibilisco, Stan. Statistics Demystified. New York: McGraw-Hill, 2004. Print.
Stewart, James et al. Precalculus for Boise State University. Fifth edition. S.l.: Brooks Cole Pub Co, 2007. Print.
Wright, Daniel B, Kamala London, and Daniel B. Wright. First (and Second) Steps in Statistics. Los Angeles: SAGE, 2009. Print.