Trend Analysis and Polynomial Regression

 

I.                    Experiments

 

A.                 Problem: 

1.                  Often, one will conduct an experiment in which an independent variable is theoretically continuous but it is sampled at various levels for convenience; e.g., dosage level of a drug, numbers of exposures, hours of deprivation.  One might then want to enter this factor into analyses as a single continuous variable.  Normally, multilevel variables are represented as k-1 dummy variables.  To reduce the k-1 variables to a single variable implies that the factor they represent is linearly related to the dependent variable.  Therefore, before using a single variable to represent a multilevel factor, one should test for linearity.

 

B.                 Tests for linearity

1.                  ANOVA strategy

a)                  Perform ANOVA

b)                  Using contrast weights test linear trend and residual deviations from linear trend.

c)                  If the residual deviations from linear are not significant and small (e.g., F < 2), then one may decide to accept the null hypothesis that there are no higher order effects.  One can then use a single variable to represent this factor in a multiple regression.

2.                  Regression strategy

a)                  Regress Y on the factor coded as k-1 dummy variables (full model).

b)                  Regress Y on the factor coded as a single continuous variable (reduced model).

c)                  Test the change in R2.

d)                  If the change in R2 is not significant, then one may decide to accept the null hypothesis that there are no higher order effects.  One can then use a single variable to represent this factor in a multiple regression.

e)                  If the change in R2 is significant, then one can use dummy coding or polynomial regression.

 

II.                 Polynomial Regression

A.                 Polynomial regression is regression of the criterion on a predictor raised to powers, i.e.:  Y=a + b1X1 + b2X12 + b3X13 + ...

B.                 This regression must be performed hierarchically, testing the change in R2 as each higher order term is added to the model.  The predictors in a polynomial are highly correlated so it is not wise to interpret the b's out of context.  The best interpretation of a polynomial regression is given by a graph of the predicted values.

 

III.               Regression on orthogonal polynomials

A.                 An alternative to polynomial regression is to use“dummy” predictors that are coded with orthogonal polynomial weights (like those used in orthogonal coding or contrast analyses). 

B.                 With equally spaced levels and equal cell sizes, these dummy variables are uncorrelated and hence the hierarchical strategy is unnecessary.

C.                 The “dummy” predictors can be coded to reflect unequal cell sizes or unequally spaced levels.

 

IV.              Non-experimental Research

A.                 Hypothesis Testing:  In non-experimental research, one may have a predictor that takes on so many different values that it is effectively continuous.  One can still test the hypothesis that the effect of the predictor on the criterion is non-linear (e.g. a power function) by using the hierarchical strategy.

B.                 Exploratory:  To explore whether a polynomial function of an independent variable is a better predictor of the dependent variable than a linear or lower order polynomial, one can proceed hierarchically and test whether the proportion of the variance explained by each added term is significant.  After finding a non-significant change, one should add another higher order term and test the change in R2 caused by adding both terms to be sure that the highest useful order polynomial has been reached.  Once the highest useful order has been determined, one may want to recompute the F ratios for the effects of the lower order terms using the MS error from the final regression because this is a better estimate of the "true error".

C.                 With multiple independent variables, one may be inclined to overfit using multiple power terms and interactions.  That is, one may fit a model to the random error present in the data.

1.                  When explanation is the goal, higher order terms are often problematic because they are difficult to interpret.  To help interpret interactions, multiplicative terms, and power terms use a hierarchical analysis and compare the simple effects of the independent variables in a model without interactions.  These coefficients will usually reflect the general trends.  The higher order terms can then be thought of as modifying this trend.

2.                  When prediction is the sole aim, fitting error will be problematic if it reflects only peculiarities in the sample studied.