Logistic Models and the margins command

When looking at the results of a logistic model, there are several different measures of the relationship between the predictors and the probability of a positive outcome that can be used to interpret the model:

If you are unclear which you are looking at, confusion can abound. This is doubly-confounded in Stata (in my opinion) where certain margins commands will produce a different measure than perhaps expected.

Intercept only model

Let’s start simple and consider a model with only an intercept. We’ll use the "auto" data set, and fit a model predicting the probability of a car being foreign made.

. sysuse auto
(1978 automobile data)

. logit foreign, nolog

Logistic regression                                     Number of obs =     74
                                                        LR chi2(0)    =   0.00
                                                        Prob > chi2   =      .
Log likelihood = -45.03321                              Pseudo R2     = 0.0000

------------------------------------------------------------------------------
     foreign | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       _cons |  -.8602013   .2543331    -3.38   0.001    -1.358685   -.3617176
------------------------------------------------------------------------------

. logit, or

Logistic regression                                     Number of obs =     74
                                                        LR chi2(0)    =   0.00
                                                        Prob > chi2   =      .
Log likelihood = -45.03321                              Pseudo R2     = 0.0000

------------------------------------------------------------------------------
     foreign |       Odds   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       _cons |   .4230769   .1076025    -3.38   0.001     .2569985     .696479
------------------------------------------------------------------------------

The default output from the logit command are the log odds, \(-.86\), and passing the or option (or calling logistic instead of logit) gives the odds ratio of \(.423\). The conversion between these values is straightforward:

\[ \textrm{logodds} = \log(\textrm{odds}) \] \[ \textrm{odds} = \exp^{\textrm{logodds}} \]

Let’s backtrack to see how we arrive at those values.

First, let’s look at the breakdown of foreign and domestic cars.

. tabulate foreign

 Car origin |      Freq.     Percent        Cum.
------------+-----------------------------------
   Domestic |         52       70.27       70.27
    Foreign |         22       29.73      100.00
------------+-----------------------------------
      Total |         74      100.00

We see that \(P(\textrm{foreign}) = .297\) and \(P(\textrm{domestic}) = .703\). We can convert these probabilities into odds, using the formula

\[ \textrm{Odds}(\textrm{foreign}) = \frac{P(\textrm{foreign})}{1 - P(\textrm{foreign})}. \]

Therefore we can easily see that

\[ \textrm{Odds}(\textrm{foreign}) = \frac{.297}{1 - .297} = .423 \] \[ \textrm{Odds}(\textrm{domestic}) = \frac{.703}{1 - .703} = 2.36 \]

For completeness, to convert from odds to probability you can use

\[ P(\textrm{foreign}) = \frac{\textrm{Odds}(\textrm{foreign})}{1 + \textrm{Odds}(\textrm{foreign})}. \]

Note that the odds of a car being foreign is exactly the result we saw above from logit, or. So in a logistic model with only an intercept, the coefficient on the intercept is the odds of a positive outcome.

Rather than calculate these manually, Stata can produce these automatically.

\(P(\textrm{foreign})\):

. margins
warning: prediction constant over observations.

Predictive margins                                          Number of obs = 74
Model VCE: OIM

Expression: Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       _cons |   .2972973   .0531331     5.60   0.000     .1931583    .4014363
------------------------------------------------------------------------------

\(\textrm{Odds}(\textrm{foreign})\):

. margins, expression(exp(xb()))
warning: prediction constant over observations.

Predictive margins                                          Number of obs = 74
Model VCE: OIM

Expression: exp(xb())

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
       _cons |   .4230769   .1076025     3.93   0.000       .21218    .6339739
------------------------------------------------------------------------------

The expression(exp(xb())) is a bit odd, but the easiest way to obtain what we need. Think of it as just saying “give me the odds”.

A single binary predictor

In the intercept only example, we had no concept of an odds ratio. Let’s add a fixed effect, in this case a binary predictor, which will require interpreting an odds ratio.

. generate highmileage = mpg > 25

. label define highmileage 0 "Low Mileage" 1 "High Mileage"

. label value highmileage highmileage

. tabulate foreign highmileage

           |      highmileage
Car origin | Low Milea  High Mile |     Total
-----------+----------------------+----------
  Domestic |        45          7 |        52
   Foreign |        15          7 |        22
-----------+----------------------+----------
     Total |        60         14 |        74

. logit foreign i.highmileage, nolog

Logistic regression                                     Number of obs =     74
                                                        LR chi2(1)    =   3.18
                                                        Prob > chi2   = 0.0746
Log likelihood = -43.444169                             Pseudo R2     = 0.0353

------------------------------------------------------------------------------
     foreign | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
 highmileage |
High Mile..  |   1.098612   .6120483     1.79   0.073    -.1009804    2.298205
       _cons |  -1.098612   .2981424    -3.68   0.000    -1.682961   -.5142639
------------------------------------------------------------------------------

. logit, or

Logistic regression                                     Number of obs =     74
                                                        LR chi2(1)    =   3.18
                                                        Prob > chi2   = 0.0746
Log likelihood = -43.444169                             Pseudo R2     = 0.0353

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
 highmileage |
High Mile..  |          3   1.836145     1.79   0.073     .9039507    9.956295
       _cons |   .3333333   .0993808    -3.68   0.000      .185823    .5979406
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

Probabilities

Let’s look at the probabilities. Here we have conditional probabilities since we have a predictor. So we are interested in \(P(\textrm{foreign} | \textrm{high mileage})\) and \(P(\textrm{foreign} | \textrm{low mileage})\).

From the table above, we can easily compute this:

. tabulate foreign highmileage, col

+-------------------+
| Key               |
|-------------------|
|     frequency     |
| column percentage |
+-------------------+

           |      highmileage
Car origin | Low Milea  High Mile |     Total
-----------+----------------------+----------
  Domestic |        45          7 |        52
           |     75.00      50.00 |     70.27
-----------+----------------------+----------
   Foreign |        15          7 |        22
           |     25.00      50.00 |     29.73
-----------+----------------------+----------
     Total |        60         14 |        74
           |    100.00     100.00 |    100.00 

We see \(P(\textrm{foreign} | \textrm{high mileage}) = .25\) and \(P(\textrm{foreign} | \textrm{low mileage}) = .5\).

We can also obtain these via margins:

. margins highmileage

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
 highmileage |
Low Mileage  |        .25   .0559017     4.47   0.000     .1404347    .3595653
High Mile..  |         .5   .1336306     3.74   0.000     .2380888    .7619112
------------------------------------------------------------------------------

If you do not flag highmileage as a categorical with i., you can use instead margins, over(highmileage). If you pass a continuous variable, it will compute the probability at each discrete value of the continuous variable.

Testing for equality between these percentages:

. margins highmileage, pwcompare(pv)

Pairwise comparisons of adjusted predictions                Number of obs = 74
Model VCE: OIM

Expression: Pr(foreign), predict()

---------------------------------------------------------------------
                             |            Delta-method    Unadjusted
                             |   Contrast   std. err.      z    P>|z|
-----------------------------+---------------------------------------
                 highmileage |
High Mileage vs Low Mileage  |        .25   .1448521     1.73   0.084
---------------------------------------------------------------------

Odds

We can compute the odds using the formulas above, giving us

\[ \textrm{Odds}(\textrm{foreign} | \textrm{high mileage}) = 1 \]

and

\[ \textrm{Odds}(\textrm{foreign} | \textrm{low mileage}) = .333. \]

To obtain with margins, we again pass the expression option:

. margins highmileage, expression(exp(xb()))

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: exp(xb())

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
 highmileage |
Low Mileage  |   .3333333   .0993808     3.35   0.001     .1385505    .5281161
High Mile..  |          1   .5345225     1.87   0.061    -.0476448    2.047645
------------------------------------------------------------------------------

Note that we do not want to test if the odds are different using pwcompare as that’s what the odds ratio is for!

Odds ratio

The odds ratio is often very confusing to interpret, but is straightforward: An odds ratio predicts the number of positive outcomes we expect to see for every negative outcome. So an odds ratio of \(2\) would mean for every domestic car, we’d expect to see \(2\) foreign cars. An odds ratio of \(.25\) would mean for every domestic car, we’d expect \(.25\) foreign cars - or, for every \(4\) domestic cars, we’d expect \(1\) foreign car (since \(.25 = 1/4\)).

The odds ratio is literally the ratio of the odds.

\[ \textrm{OR}(\textrm{foreign} | \textrm{high mileage}) = \frac{\textrm{odds}(\textrm{foreign} | \textrm{high mileage})}{\textrm{odds}(\textrm{foreign} | \textrm{low mileage})} = 1/.333 = 3 \]

Looking at the regression results again:

. logit, or

Logistic regression                                     Number of obs =     74
                                                        LR chi2(1)    =   3.18
                                                        Prob > chi2   = 0.0746
Log likelihood = -43.444169                             Pseudo R2     = 0.0353

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
 highmileage |
High Mile..  |          3   1.836145     1.79   0.073     .9039507    9.956295
       _cons |   .3333333   .0993808    -3.68   0.000      .185823    .5979406
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

The intercept is \(\textrm{odds}(\textrm{foreign} | \textrm{low mileage})\), the odds of a positive outcome in the baseline group, and the coefficient on highmileage is the odds ratio!

Note that we cannot use the margins command to obtain the odds ratio. (If I'm wrong about that, please let me know!) Instead, we use lincom:

. lincom _b[1.highmileage], or

 ( 1)  [foreign]1.highmileage = 0

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         (1) |          3   1.836145     1.79   0.073     .9039507    9.956295
------------------------------------------------------------------------------

(I obtained the _b[1.highmileage] name by running logit, coeflegend to obtain the legend.) Note the or option, without it we obtain the log odds.

A categorical predictor

Moving from a binary predictor to a categorical predictor is fairly straightforward; instead of a single odds ratio, we have one for each level of the categorical predictor, excluding the reference category.

. generate pricecat = price < 7500

. replace pricecat = 2 if price >= 7500 & price < 10000
(5 real changes made)

. replace pricecat = 3 if price >= 10000 & price < .
(10 real changes made)

. logit foreign i.pricecat, or nolog

Logistic regression                                     Number of obs =     74
                                                        LR chi2(2)    =   2.47
                                                        Prob > chi2   = 0.2905
Log likelihood = -43.797041                             Pseudo R2     = 0.0275

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
    pricecat |
          2  |   3.705882   3.546757     1.37   0.171     .5678577    24.18487
          3  |   .6176471   .5195704    -0.57   0.567     .1187686    3.212026
             |
       _cons |   .4047619   .1163527    -3.15   0.002     .2304165    .7110264
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

Probabilities

. tabulate foreign pricecat, col

+-------------------+
| Key               |
|-------------------|
|     frequency     |
| column percentage |
+-------------------+

           |             pricecat
Car origin |         1          2          3 |     Total
-----------+---------------------------------+----------
  Domestic |        42          2          8 |        52
           |     71.19      40.00      80.00 |     70.27
-----------+---------------------------------+----------
   Foreign |        17          3          2 |        22
           |     28.81      60.00      20.00 |     29.73
-----------+---------------------------------+----------
     Total |        59          5         10 |        74
           |    100.00     100.00     100.00 |    100.00 
. margins pricecat

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
    pricecat |
          1  |   .2881356   .0589618     4.89   0.000     .1725725    .4036987
          2  |         .6    .219089     2.74   0.006     .1705934    1.029407
          3  |         .2   .1264911     1.58   0.114     -.047918     .447918
------------------------------------------------------------------------------

Odds

The intercept is the odds of a foreign car in pricecat 1, or \(.288/.712 = .405\). We can obtain the odds of each pricecat in the typical way.

. margins pricecat, expression(exp(xb()))

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: exp(xb())

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
    pricecat |
          1  |   .4047619   .1163527     3.48   0.001     .1767148     .632809
          2  |        1.5   1.369306     1.10   0.273    -1.183791    4.183791
          3  |        .25   .1976424     1.26   0.206    -.1373719    .6373719
------------------------------------------------------------------------------

Odds ratios

Finally, the two coefficients in the model are the odds ratios of being in pricecat 2 or 3 versus 1. Again we can use lincom to obtain these.

. lincom _b[2.pricecat], or

 ( 1)  [foreign]2.pricecat = 0

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         (1) |   3.705882   3.546757     1.37   0.171     .5678577    24.18487
------------------------------------------------------------------------------

. lincom _b[3.pricecat], or

 ( 1)  [foreign]3.pricecat = 0

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         (1) |   .6176471   .5195704    -0.57   0.567     .1187686    3.212026
------------------------------------------------------------------------------

Note here that multiplying the odds ratios by the odds in pricecat 1 (the intercept) gives the odds in the other group. Such as \(3.71\times .408 \approx 1.5\).

A continuous predictor

Now let’s replace the categorical predictor with a continuous one. Again, most interpretations stay the same.

. logit foreign headroom, or nolog

Logistic regression                                     Number of obs =     74
                                                        LR chi2(1)    =   6.72
                                                        Prob > chi2   = 0.0095
Log likelihood = -41.671379                             Pseudo R2     = 0.0747

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
    headroom |   .4304066   .1490474    -2.43   0.015     .2183295    .8484873
       _cons |   4.780366   4.755595     1.57   0.116      .680253    33.59324
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

Instead of talking about probability or odds in a level of a categorical predictor, it is instead at a specific level of headroom. The intercept is the odds of having a positive outcome when the headroom is identically \(0\), which in this case, as is often the case, is not interesting.

Probabilities

We cannot look at crosstabs as we did before the compute probabilities, but the margins command still works.

. margins, at(headroom = (2.5 5))

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: Pr(foreign), predict()
1._at: headroom = 2.5
2._at: headroom =   5

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         _at |
          1  |   .3674795   .0655664     5.60   0.000     .2389717    .4959873
          2  |   .0659516    .050366     1.31   0.190    -.0327639     .164667
------------------------------------------------------------------------------

These are the predicted probabilties of a positive outcome at the referenced levels of headroom, i.e. \(P(\textrm{foreign} | \textrm{headroom} = 2.5)\) and \(P(\textrm{foreign} | \textrm{headroom} = 5)\).

Odds

We can directly compute the odds given the probabilities above, but it’s easier to continue using margins.

. margins, at(headroom = (2.5 5)) expression(exp(xb()))

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: exp(xb())
1._at: headroom = 2.5
2._at: headroom =   5

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         _at |
          1  |   .5809764   .1638824     3.55   0.000     .2597729    .9021799
          2  |   .0706083   .0577296     1.22   0.221    -.0425396    .1837562
------------------------------------------------------------------------------

Odds ratio

The coefficient in the logistic regression is interpreted as the odds ratio when increasing headroom by \(1\). In other words, if we had a collection of cars with headroom of \(x\) and magically change their headroom to \(x + 1\), we would expect for every one additional domestic car, we’d see \(.43\) additional foreign cars.

We can obtain this odds ratio by again using lincom.

. lincom _b[headroom], or

 ( 1)  [foreign]headroom = 0

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         (1) |   .4304066   .1490474    -2.43   0.015     .2183295    .8484873
------------------------------------------------------------------------------

Interactions

Let’s consider interactions now. We’ll interact two binary variables.

. generate highprice = price > 5000

. label define highprice 0 "Low Price" 1 "High Price"

. label value highprice highprice

. logit foreign i.highprice##i.highmileage, or nolog

Logistic regression                                     Number of obs =     74
                                                        LR chi2(3)    =  10.54
                                                        Prob > chi2   = 0.0145
Log likelihood = -39.763201                             Pseudo R2     = 0.1170

------------------------------------------------------------------------------
     foreign | Odds ratio   Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
   highprice |
 High Price  |   6.795455    5.54509     2.35   0.019     1.372897    33.63558
             |
 highmileage |
High Mile..  |       11.5    10.7684     2.61   0.009      1.83505    72.06888
             |
   highprice#|
 highmileage |
 High Price #|
High Mile..  |   .1471572   .2548493    -1.11   0.269     .0049392    4.384364
             |
       _cons |   .0869565   .0641052    -3.31   0.001     .0205016    .3688215
------------------------------------------------------------------------------
Note: _cons estimates baseline odds.

Probabilities

Because we have two categorical predictors, we can return to looking at crosstabs as a way of obtaining probabilities. The margins call will also return them.

. table foreign highmileage highprice

----------------------------------------------------
                   |             highprice
                   |  Low Price   High Price   Total
-------------------+--------------------------------
Car origin         |
  Domestic         |
    highmileage    |
      Low Mileage  |         23           22      45
      High Mileage |          6            1       7
      Total        |         29           23      52
  Foreign          |
    highmileage    |
      Low Mileage  |          2           13      15
      High Mileage |          6            1       7
      Total        |          8           14      22
  Total            |
    highmileage    |
      Low Mileage  |         25           35      60
      High Mileage |         12            2      14
      Total        |         37           37      74
----------------------------------------------------

. margins highprice#highmileage

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: Pr(foreign), predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
   highprice#|
 highmileage |
  Low Price #|
Low Mileage  |        .08   .0542586     1.47   0.140     -.026345     .186345
  Low Price #|
High Mile..  |         .5   .1443376     3.46   0.001     .2171036    .7828964
 High Price #|
Low Mileage  |   .3714286   .0816735     4.55   0.000     .2113515    .5315056
 High Price #|
High Mile..  |         .5   .3535534     1.41   0.157    -.1929519    1.192952
------------------------------------------------------------------------------

This is similar to the categorical predictor, where there are four groups. For example, low price and low mileage, \(2\) out of \(25\) cars are foreign, so the probability is \(2/25 = .08\).

Odds

. margins highprice#highmileage, expression(exp(xb()))

Adjusted predictions                                        Number of obs = 74
Model VCE: OIM

Expression: exp(xb())

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
   highprice#|
 highmileage |
  Low Price #|
Low Mileage  |   .0869565   .0641052     1.36   0.175    -.0386874    .2126004
  Low Price #|xo
High Mile..  |          1   .5773503     1.73   0.083    -.1315857    2.131586
 High Price #|
Low Mileage  |   .5909091   .2067148     2.86   0.004     .1857554    .9960628
 High Price #|
High Mile..  |          1   1.414214     0.71   0.480    -1.771808    3.771808
------------------------------------------------------------------------------

If you look at the logistic results above, the baseline categories are low mileage and low price. So, as before, the intercept is the odds of a foreign car in that subcategory, which we see here.

We do not obtain the odds for the other categories in the regression output.

Odds ratios

The odds ratios reported in the regression output only present part of the story. Let’s take a look at them again.

The coefficient on highprice is the odds ratio of being foreign between high price and low price cars, in the low mileage category.

\[ \frac{\textrm{OR}(\textrm{foreign}|\textrm{high price, low mileage})}{\textrm{OR}(\textrm{foreign}|\textrm{low price, low mileage})} \]

The coefficient on highmileage is the odds ratio of being foreign between high mileage and low mileage cars, in the low price category.

\[ \frac{\textrm{OR}(\textrm{foreign}|\textrm{low price, high mileage})}{\textrm{OR}(\textrm{foreign}|\textrm{low price, low mileage})} \]

This work is licensed under CC BY-NC 4.0 Creative Commons BY-NC image