Project: The Implications of Income, Taste and Substitution Effects for the Assessment of User Benefits Using Discrete Choice Models C77

Reference: STP 14/5/29

Last update: 31/12/2009 10:42:11

Objectives

The specific objectives are to develop recommendations regarding how welfare assessment should be carried out using different mixed multinomial logit (MMNL) and generalised extreme value (GEV) model forms and utility specifications.

Description

The overall aim of the research is to develop a better understanding of the likely effect on policy appraisal of the increasing use of advanced discrete choice models of mixed multinomial logit (MMNL) and extended generalised extreme value (GEV) form. An investigation will also be carried out into the extent to which heterogeneity of tastes, complex substituition patterns and income effects would, if present and detected by such models, affect the appraisal of transport policy measures (relative to current MNL/NL approaches).

Contractor(s)

Imperial College London
Exhibition Road, London , SW7 2AZ
020 7589 5111

Contract details

Cost to the Department: £49,975.00

Actual start date: 07 July 2003

Actual completion date: 26 August 2004

Publication(s)

The Impact of Income, Taste, and Substitution Effects on the Assessment of User Benefits Using Discrete Choice Models
Author: Elisabetta Cherchi, John Polak, Geoff Hyman
Publication date: 15/10/2004
Source: ers@dft.gsi.gov.uk

Summary of results

  1. This research investigated systematically the consequences of using different choice model structures on the accuracy of alternative methods of computing benefit measures. The results indicate that the benefit measures most typically used in practice (especially the rule-of-a-half) often give seriously biased results when compared to the "correct" value (i.e. that obtained from a simulation of the disaggregate compensating variation). The bias is often large and appears either in the case of fix parameters, due mainly to probabilities and attributes aggregations. However, bias increases enormously in presence of heterogeneity in tastes with both the magnitude of the heterogeneity and the spread of the distribution of income (which gives rise to systematic taste heterogeneity due to the income effect). The effects of heterogeneity in the parameters related to travel cost is particularly marked, mainly due to the extra error in the aggregation of the SVT. In particular, for non-marginal change, it was found that the error of disregarding heterogeneity in the cost parameter is often much bigger than the error of disregarding taste ran-domness in other attributes (e.g. travel time). This result adds to the discussion on the appropriateness of fixing the cost parameter when estimated models show different evidence.

    The results also indicate that, in certain circumstances, aggregate benefits measures give apparently good results, but unfortunately for the wrong reason - i.e., through the accidental cancelling out of significant errors acting in opposite directions. Moreover, results are also strongly influenced by the type of attribute and the market share of the alternatives involved in the policy. All these different effects make any generalization and recommendation quite risky. However, from the analysis, it seems that same cases could be high-lighted where particular care must be adopted. For example, it was found that policies that imply changes in an alternative with an actual market share very low are more likely to produce quite big wrong results. Other cases where the application of aggregate benefit measure is particular doubtful is when the policies imply a change in the same parameter that shows randomness in taste, especially if the heterogeneity is specific among alternatives. Specific taste heterogeneity, in fact, increases the differences among alternatives and thus increases the effect (i.e. the error) of a change in the associated attribute. On the other hand, generic randomness induces correlation among alterna-tives, and consistent benefit measures should be used in this case. Finally, it was found that the error due to disregarding income effect is not negligible when the cost of travelling is more than 25% of the income. The numerical results need, obviously, more evidence, but highlight that it is possible to derive a guidance to check in which case the RoH can be applied safely and in which must be doubted.

    Finally, it is worth noting that the results raise again the question of the importance of the demand modelling into the appraisal of transport policy measures. In fact, benefit measures are highly sensitive to the value of the taste parameters and unfortunately, as demonstrated by Cherchi and Polak (2005), in presence of taste heterogeneity it is very likely to obtain biased estimates of the parameters.

    As an overall conclusion from these results, it seems that once one departs from the relatively simple structures associated with conventional models, aggregate approaches to benefit assessment become highly unreliable and should be questioned as a basis for decision making.