### Introduction

We use regression analysis to understand the relationships, patterns, and causalities in data. Often we are interested in understanding the impacts that changes in the dependent variables have on our outcome of interest.

Some models provide coefficients that can be directly interpreted as these *marginal effects*. The coefficients directly represent the predicted change in *y* caused by a unit change in *x*.

However, not all models provide such straightforward interpretations. Coefficients in more complex models may not always provide direct insights into the relationships we are interested in.

In this blog, we look more closely at the interpretation of marginal effects in three types of models:

- Purely linear models
- Models with transformations in independent variables
- Models with transformations of dependent variables

## Purely linear models

Marginal effects measure the impact that an instantaneous change in one variable has on the outcome variable while all other variables are held constant.

In the simple OLS model with linear effects, estimated coefficients are always equal to marginal effects. To understand why let's consider the model:

$$ price = \beta_0 + \beta_1*weight $$

with estimated coefficients $\beta_0 = -6.707$ and $\beta_1 = 2.04$.

Consider what happens to our predicted outcome, `price`

, as we incrementally increase the vehicle `weight`

.

Weight (lbs) | Predicted Price | Change in Predicted Price | Marginal Effects |
---|---|---|---|

1000 | $4081.42 | $2044.06 | $2044.06/1000 = $2.04 |

2000 | $6125.48 | $2044.06 | $2044.06/1000 = $2.04 |

3000 | $8169.54 | $2044.06 | $2044.06/1000 = $2.04 |

4000 | $10213.61 | $2044.06 | $2044.06/1000 = $2.04 |

5000 | $12257.67 | -- | -- |

With each one unit increase in `weight`

there is a $2.04 increase in `price`

. There are two key things to note about the marginal effect of `weight`

on `price`

in this case:

- It is equal to the estimated coefficient, $\beta_1$.
- It is constant across all values of
`weight`

.

## Transformed data

In the case of the purely linear model, the estimated coefficient is, conveniently, equal to the marginal effects. This result doesn't hold if we consider more complicated models with non-linearities, such as interactions terms, logarithmic terms, or power terms.

### Interaction terms

For example, let's add an interaction term into the model

$$ price = \beta_0 + \beta_1*weight + \beta_2*weight*mpg $$

with estimated coefficients:

$$ \beta_0 = 2787.94\\ \beta_1 = 2.227\\ \beta_2 = -0.055 $$

Let's look at what happens to our predicted outcome, `price`

, as we incrementally increase the vehicle `weight`

while holding `mpg`

constant at 16.

Weight (lbs) | Predicted Price | Change in Predicted Price | Marginal Effects |
---|---|---|---|

1000 | $5478.57 | $1345.31 | $1345.31/1000 = $1.345 |

2000 | $6823.89 | $1345.31 | $1345.31/1000 = $1.345 |

3000 | $8169.21 | $1345.31 | $1345.31/1000 = $1.345 |

4000 | $9514.53 | $1345.31 | $1345.31/1000 = $1.345 |

5000 | $10859.84 | -- | -- |

With each one unit increase in `weight`

there is a $1.35 increase in `price`

. In this case, the marginal effect of `weight`

on `price`

is no longer equal to the estimated coefficient, $\beta_1$. This is because `weight`

also influences `price`

through the interaction term, $\beta_2 * weight * mpg$.

$$ \begin{aligned} \beta_1 & + \beta_2 * mpg &= 1.345\\ 2.227 & - 0.0552 * 16 &= 1.345 \end{aligned} $$

Therefore, while the marginal effect of a change in the `weight`

variable is still independent of the value of `weight`

, it does depend on the value of `mpg`

.

As we can see on the above graph, increasing `mpg`

dampens the impact that increasing `weight`

has on price. For example, at a `mpg`

of 16, increasing `weight`

increases `price`

by $1.345 for every pound. However, if we increase `mpg`

to 25 the marginal effect of increasing `weight`

decreases to $0.849 for every pound.

In our example linear model with interaction terms, we have seen that the marginal effects:

- Are no longer equal to the estimated coefficient, $\beta_1$.
- Are independent of the value of
`weight`

but do depend on the value of`mpg`

. - Must account for the impact of
`weight`

on`price`

which occurs through the interaction term.

### Power terms

For our next example, we will look at a model that includes a quadratic term. For this example, we will examine the impact that `weight`

has on `mpg`

:

$$ mpg = \beta_0 + \beta_1*weight + \beta_2*weight^2 $$

with estimated coefficients:

$$ \beta_0 = 51.183\\ \beta_1 = -14.158\\ \beta_2 = 1.3244 $$

Now let's incrementally increase the vehicle `weight`

and see what happens to predicted `mpg`

.

Weight (lbs) | Predicted Price | Change in Predicted Price | Marginal Effects |
---|---|---|---|

1000 | 38.349 | -$10.186 | -$10.186/1 = -10.186 |

2000 | 28.160 | -$7.538 | -$7.538/1 = -7.538 |

3000 | 20.622 | -$4.890 | -$4.890/1 = -4.890 |

4000 | 15.732 | -$2.242 | -$2.242/1 = -2.242 |

5000 | 13.490 | -- | -- |

In this case, the marginal effect of a change in the `weight`

on `mpg`

changes as we change the value of `weight`

.

**Analytical marginal effects**

The table gives us a numerical estimate of the marginal effects. However, in this case we can easily compute the precise analytical marginal effect:

$$ \begin{aligned} &\beta_1 + 2*\beta_2*weight \:=\\ -&14.16 + 2.6488*weight \end{aligned} $$

Using this, let's find the analytical marginal effect when $weight = 2$:

$$ \begin{aligned} \beta_1 & + 2* \beta_2 * weight &= &\\ -14.16 & + 2 * 1.3244 * 2 &= &-8.8624 \end{aligned} $$

Right away we can see that our numeric estimate of the marginal effect when $weight = 2$ does not equal the analytical marginal effect.

**Numerically approximated marginal effects**

Why is this? Let's look more carefully at how we compute the marginal effects in our table.

We start by predicting the outcome `mpg`

at each value of `weight`

using our regression results:

$$ \widehat{mpg} = 51.183 - 14.16*weight + 1.3244*weight^2 $$

Once we have found these predicted prices we can determine the change as we increase `weight`

by one:

$$ \text{Marginal effects} = \frac{\widehat{mpg}_{weight=3} - \widehat{mpg}_{weight=2}}{3-2}\\ \ \\ \text{Marginal effects} = \frac{20.622 - 28.160}{3-2} = -7.538 $$

In more general terms, we can numerically estimate the marginal effects using the numerical derivative

$$\lim_{\Delta x\to0} \frac{ f(x + \Delta x) - f(x)}{\Delta x} $$

This approximation is valid only as $\Delta x$ gets close to zero.

In our table we use $\Delta x = 1$ to approximate our marginal effects. Let's see what happens if we use $\Delta x = 0.001$ to approximate the marginal effects when $weight = 2$.

First, we find the predicted price when $weight = 2$

$$ \widehat{mpg}_{weight=2} = 51.18 - 14.158*2 + 1.324*2^2 = \$28.160$$

Next, we find the predicted price when $weight = 2 + 0.001$

$$ \widehat{price}_{weight=2.001} = 51.18 - 14.158*2.001 + 1.324*2.001^2 = \$28.151$$

Finally, we subtract the two and divide by the change in `weight`

$$\text{Marginal effects} = \frac{28.151 - 28.160}{0.001} = -8.861$$

By using a smaller change in `weight`

to compute our numerical approximation we obtain a closer estimate of the analytical solution.

## Transformed dependent variables

Sometimes we have models with transformed dependent variables such as log, logit, or probit regressions. In these models we relate a function of `Y`

to our dependent variables:

$$ F(Y) = Y' = X\beta + \epsilon$$

The coefficients from these models do not always intuitively reflect the direct relationships between our observed outcomes and regressors. For this purpose, we can use the marginal effects to better interpret the relationships between our regressors and outcome.

### Probit example

Let's consider a simple example of a probit model. The probit model transforms binary outcomes to continuous Y data using the cumulative normal distribution:

$$Y = \Phi (X\beta + \epsilon)$$ $$\Phi^{-1} Y = X\beta + \epsilon$$ $$ Y' = X\beta + \epsilon$$

In this model, we treat $X\beta$ as a z-score. Therefore, as $X\beta$ increases the event measured by `Y`

is more likely to occur.

The estimated coefficient, $\beta$, reflects the increase in the z-score that occurs with an incremental increase in `X`

. However, we may be more interested in knowing how a variable impacts the probability that the event will occur.

**For this we use the marginal probability effect**, which reflects two things :

- As
`X`

increases, the z-score increases through the term $X \beta$. As we noted earlier, the marginal impact of an increase of`X`

on the z-score is equal to $\beta$. - As the z-score increases, the probability of the event changes. This probability change is measured by the value of the standard normal p.d.f at $X \beta$, $\phi(X\beta)$.

The total marginal probability effect is equal to the combined effect of $\beta$ and $\phi(X\beta)$:

$$\beta * \phi(X\beta).$$

Note that the marginal probability effect is dependent on `X`

. There are a number of ways to choose the most useful `X`

values :

- We can use theoretically relevant
`X`

values. - We can use the mean
`X`

values. This will yield the marginal effects at the mean (MEM). - We can compute the marginal effects at all
`X`

values and take the average. This will yield the average marginal effects (AME).

## Conclusion

In today's blog we discuss marginal effects. Here are a few key points to take away from this discussion:

- Marginal effects allow us to interpret the direct effects that changes in regressors have on our outcome variable.
- Marginal effects are equal to the estimated coefficients in only a few select cases.
- To understand the direct relationship between regressors and outcomes we need to properly compute the marginal effects based on the functional form of our regression.

Eric has been working to build, distribute, and strengthen the GAUSS universe since 2012. He is an economist skilled in data analysis and software development. He has earned a B.A. and MSc in economics and engineering and has over 18 years of combined industry and academic experience in data analysis and research.