Simple linear regression

In statistics, simple linear regression is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the x and y coordinates in a Cartesian coordinate system) and finds a linear function (a non-vertical straight line) that, as accurately as possible, predicts the dependent variable values as a function of the independent variables.The adjective simple refers to the fact that the outcome variable is related to a single predictor. In statistics, simple linear regression is a linear regression model with a single explanatory variable. That is, it concerns two-dimensional sample points with one independent variable and one dependent variable (conventionally, the x and y coordinates in a Cartesian coordinate system) and finds a linear function (a non-vertical straight line) that, as accurately as possible, predicts the dependent variable values as a function of the independent variables.The adjective simple refers to the fact that the outcome variable is related to a single predictor. It is common to make the additional stipulation that the ordinary least squares (OLS) method should be used: the accuracy of each predicted value is measured by its squared residual (vertical distance between the point of the data set and the fitted line), and the goal is to make the sum of these squared deviations as small as possible. Other regression methods that can be used in place of ordinary least squares include least absolute deviations (minimizing the sum of absolute values of residuals) and the Theil–Sen estimator (which chooses a line whose slope is the median of the slopes determined by pairs of sample points). Deming regression (total least squares) also finds a line that fits a set of two-dimensional sample points, but (unlike ordinary least squares, least absolute deviations, and median slope regression) it is not really an instance of simple linear regression, because it does not separate the coordinates into one dependent and one independent variable and could potentially return a vertical line as its fit. The remainder of the article assumes an ordinary least squares regression.In this case, the slope of the fitted line is equal to the correlation between y and x corrected by the ratio of standard deviations of these variables. The intercept of the fitted line is such that the line passes through the center of mass (x, y) of the data points. Consider the model function which describes a line with slope β and y-intercept α. In general such a relationship may not hold exactly for the largely unobserved population of values of the independent and dependent variables; we call the unobserved deviations from the above equation the errors. Suppose we observe n data pairs and call them {(xi, yi), i = 1, ..., n}. We can describe the underlying relationship between yi and xi involving this error term εi by This relationship between the true (but unobserved) underlying parameters α and β and the data points is called a linear regression model. The goal is to find estimated values α ^ {displaystyle {widehat {alpha }}} and β ^ {displaystyle {widehat {eta }}} for the parameters α and β which would provide the 'best' fit in some sense for the data points. As mentioned in the introduction, in this article the 'best' fit will be understood as in the least-squares approach: a line that minimizes the sum of squared residuals ε ^ i {displaystyle {widehat {varepsilon }}_{i}} (differences between actual and predicted values of the dependent variable y), each of which is given by, for any candidate parameter values α {displaystyle alpha } and β {displaystyle eta } , In other words, α ^ {displaystyle {widehat {alpha }}} and β ^ {displaystyle {widehat {eta }}} solve the following minimization problem: By expanding to get a quadratic expression in α {displaystyle alpha } and β , {displaystyle eta ,} we can derive values of α {displaystyle alpha } and β {displaystyle eta } that minimize the objective function Q (these minimizing values are denoted α ^ {displaystyle {widehat {alpha }}} and β ^ {displaystyle {widehat {eta }}} ):