PRP: Statement of the Peer Review Panel

1. This report summarises the review process applied to the report ("the Report") of three technical panels set up by the Department for Transport ("DfT") to investigate issues relating to the future operation and development of Heathrow Airport. The review was carried out by an independent peer review panel ("PRP"), chaired by Professor Bernard Silverman FRS. Based on examination of the Report and observation of the processes leading to its completion, the overall unanimous views of the PRP are as follows:

• The PRP believes that the Report takes appropriate account of the current state of scientific knowledge and its application to the subject of the review, whilst acknowledging that certain elements of the most recent work referred to in the Report have so far only appeared in pre-publication format, pending submission to the standard review processes of scientific publication.
• The PRP considers the conclusions of the Report to be clearly and fairly presented.
• The PRP believes that the conclusions set out in the Report are justified in light of the current state of knowledge.
• The PRP considers the Report to be appropriately comprehensive and fit for purpose.
• The PRP believes that the Panels have fairly identified areas where there is uncertainty and/or the potential for specific future research. The PRP is grateful that the Technical Panel members have been able to address a number of issues raised by the PRP.
• The PRP considers that the summaries in the Report are appropriate and adequate, and that the first chapter of the Report contains a clear and fair summary of the key issues and findings of the Report as a whole.
• The PRP commends the overview section to readers who wish to gain an overall appreciation of the Report without necessarily covering the material in detail.
• The PRP considers the Report to contain clear and correct bibliographic references. Although many of the references are company reports or consultants' reports and are, therefore, not necessarily readily available in the public domain, the PRP has been assured that the DfT holds copies of all the cited reports and documents.
• The discussions at Technical Panel meetings were robust and open; debate was not dictated, or constrained, by either the respective Chairs of those meetings, by the presence of representatives of DfT or other Government Departments, or by the presence of the PRP members. Decisions were made after proper debate and consideration and were not pre-determined or imposed on the Technical Panel members.

Background

2. The White Paper "The Future of Air Transport", published by the Secretary of State for Transport in December 2003, identified the need for a national strategic framework for the future development of airport capacity in the United Kingdom, looking forward 30 years. One reason given in the White Paper for this strategic framework was the requirement to address the environmental impacts that air travel generates. When considering the possibility that a third runway might be constructed at Heathrow Airport, the White Paper made reference to the air quality standards that will become mandatory from 2010 and stated that one of the key conditions for such a development would be the need to ensure compliance with these air quality limits.
3. A programme of action was therefore instituted, to consider how these conditions could be met. To deal with the issue of air quality, three Technical Panels were established to revisit the air quality assessments made in the run up to the White Paper. The Technical Panels were to focus, respectively, on: modelling approaches (Panel 1); data measurement (Panel 2); and sources of emissions (Panel 3) and to produce the Report. The membership of the Technical Panels comprised independent experts from a variety of scientific, technical and operational backgrounds, supported by officials from relevant Government Departments. The need for further research and data collection was identified and it was envisaged that this work would be completed by the end of 2005; in the event the Report was finalised in March 2006.

Constitution of the Peer Review Panel

4. The PRP was established by the DfT in September 2005, as a result of the Government's intention that the work of the Technical Panels would be subject to rigorous peer review. The PRP was to play a role in assessing and analysing the suitability and reasonableness of the processes and technical actions of the Technical Panels in response to their respective remits and to report its conclusions on their work.

PRP Remit

5. The stated brief of the Peer Review Panel was as follows: "The Peer Review Panel's main objective will be to review the work of the Technical Panels and publish its conclusions on whether their work has been unbiased and fair in terms of establishing a technical basis for future assessments of air quality impacts. This is to be used in Ministers' consideration of the future development of Heathrow."
6. It was expected that the work of the PRP would reach and publish conclusions on the work of the Technical Panels by

• learning about the basis for and conduct of the ongoing work of the Technical Panels;
• scrutinising their work for reasonableness in terms of both their evidence base and conclusions;
• questions to and further information from the Technical Panel Chairs and, if necessary, the Technical Panel members in response to presentation of interim information on the work of the Technical Panels;
• consideration and assessment of the full technical report of the Technical Panels.

Membership of the PRP

7. The Peer Review Panel was constituted through a formal process in line with guidance from the Office of the Commissioner for Public Appointments (OCPA). Applications were sought through advertising in the national press and through the use of an executive search agency. Following a sift and interview process, monitored by an OCPA assessor, a chair and three additional panel members were appointed. The selection panel comprised the DfT Chief Scientist, the manager of the air quality work associated with the DfT Project for the Sustainable Development of Heathrow (PSDH) and an independent OCPA assessor. The resulting membership of the PRP comprised:
8. Professor Bernard Silverman (Chair), Master of St Peter's College, Oxford and Professor of Statistics at Oxford University. He is a Fellow of the Royal Society and a Fellow, Council Member and Past President of the Institute of Mathematical Statistics. His research interests range from general statistical theory and methodology to the application of statistics in a wide range of subject areas across the physical and biological sciences, engineering and medicine.
9. Stephen Boughton, retired Solicitor, formerly a partner in Linklaters, specialising in business and company law, corporate finance and mergers and acquisitions. He currently undertakes a range of charitable, voluntary and public service roles.
10. Dr Roy Colvile, Senior Lecturer in Air Quality Management at Imperial College London. He originally trained as an experimental physicist and now has wide experience in the field of atmospheric dispersion modelling. He is a member of the UK Air Quality Expert Group.
11. Professor Ian Poll OBE, Professor of Aerospace Engineering at Cranfield University and Technical Director of Cranfield Aerospace Ltd. He is a Fellow, Council member and a Past President of the Royal Aeronautical Society, a Fellow and Council member of the Royal Academy of Engineering and a Fellow of the American Institute of Aeronautics and Astronautics. His principal areas of expertise are aerodynamics and aircraft performance.
12. The expertise of the PRP therefore covered a range of disciplines, from mathematics and statistics, through aerodynamics/aeronautics and atmospheric dispersion, to legal. All members of the PRP were independent from Government and from the institutions providing representatives to the Technical Panels. Potential conflicts of interest were disclosed at an early stage, discussed and judged not to be of concern to the work of the PRP.

PRP Process and Procedures

13. In order for the peer review process to be effective, it was necessary for the PRP to work independently of the Technical Panels and at arms' length from them, and also to be independent of the DfT. The PRP did, however, allow representatives of the DfT to attend their meetings, and also to provide technical and secretarial support for their work.
14. Roger Gardner of the Air Quality and Environmental Technology Branch, Aviation Environmental Division attended throughout (except where it was felt that a confidential discussion was needed). Once the Report was at an advanced stage, he was joined by Paul Taylor of Atkins (formerly with Halcrows), who was retained by DfT as technical support throughout the Air Quality technical panel process, and was a member of all three Technical Panels.
15. The presence of Roger Gardner and Paul Taylor enabled rapid feedback to be given from the PRP to the members of the Technical Panels and assisted in clarifying certain issues raised by the members of the PRP. The PRP is satisfied that the presence at meetings of representatives of the DfT and of Paul Taylor did not compromise the independence of the review process, not least because of the mutual understanding and acceptance of the need for independence.
16. The PRP decided at an early stage to limit its work to the terms of the remit described above and to focus their work as narrowly and precisely as was reasonable. Particularly bearing in mind that the PRP's work should not itself require further external independent review, the PRP sought to review, not to repeat, the work of the Technical Panels, and also to concentrate on the Report itself. The PRP have not reviewed or commented on the various Annexes.
17. The process adopted emulated a publication peer review, providing interim feedback on advanced drafts of the Report in the hope that any issues raised could be accounted for in the final version. Since only near final drafts were considered, the PRP was able to maintain an appropriate distance and independence from the work of the Technical Panels and to avoid becoming enmeshed in their processes and deliberations.
18. At the same time, however, this interim review process allowed the PRP to gain a deeper appreciation of the likely content of the final Report and to raise points which the Technical Panels were able to address.
19. In addition to reviewing the written Report itself, the PRP monitored the later part of the process by which the Report had been constructed, to provide further quality control of the Report and additional confirmation that it was based on sufficiently sound and rigorous work by the Technical Panels. Accordingly, the PRP attended certain meetings of the Technical Panels, in order to observe their operations, as follows:

• Meeting of Panel 1 held on 28 September 2005;
• Plenary Session of all three Panels held on 12 and 13 October 2005; and
• Final meeting of Panel 1 held on 31 October 2005.

20. All PRP members attended at least one of these meetings, but none attended all. The objective of planning for this variation in attendance was to ensure, firstly, that all PRP members would be able to comment on "process issues" and, secondly, that no PRP member would become too closely identified with the work of any particular Technical Panel, such that the independence of that PRP member would be compromised.
21. The role played by PRP members at the Technical Panel meetings they attended was primarily to act as observers of the discussion and decision-making processes, asking questions of Technical Panel members only where the PRP members felt that clarification of issues was necessary or where amplification was required. The PRP did not seek to influence the conduct of proceedings at those meetings or the conclusions and recommendations reached. An "arm's length" relationship with the Technical Panels was maintained by the PRP.
22. The PRP wishes to add an additional comment on the process followed at the final meeting of Panel 1 held on 31 October 2005, at which agreement was reached as to which models were to be recommended. As the Panel 1 report states, these decisions were taken after the making of full presentations on each of the five models considered, and after a full inter-comparison and assessment of model performance. The decisions were taken in open forum, with the whole of Panel 1, including representatives of the five modelling groups, present. The PRP considered this process to be demonstrably open, fair and reasonable.
23. Following on from attendance at the Technical Panel meetings and review by the individual PRP members of draft sections of the Report, the PRP discussed and agreed its responses to, and comments on, the drafts. Further discussion took place by e-mail, before comments and questions were passed (by e-mail) to the Chairs and members of the Technical Panels. The Technical Panels were invited to respond in writing to PRP comments, where necessary, but no direct communications about the Report took place between PRP members and Panel members. The PRP suggested that members of each Technical Panel should review the draft reports produced by the other Technical Panels, to ensure consistency of approach and so that conclusions and recommendations contained in one Panel report which might affect the contents of another Panel report were properly dealt with.
24. At every stage, the PRP's decisions were made by consensus, and the process described led to the PRP's unanimous conclusions about the final Report.