Methodology

This page gives links to some useful short reports on methodological topics. The page does not attempt to be comprehensive but gives a selection of relevant publications. Further useful publications can be found in the reference sections of the documents cited below. Please note that following successive reorganisations in England, the regional public health observatories (PHOs) and the Association of Public Health Observatories (APHO) became part of Public Health England. Public Health England was then replaced by the UK Health Security Agency and the Office for Health Improvement and Disparities.

We welcome comments and ideas about what topics to include here.

Terminology

See our Guide to terminology (49Kb) for guidance on some of the terms used on the ScotPHO website.

Standardisation

Public Health Scotland (PHS) has produced a guide to standardisation (196Kb) and an Excel tool (65Kb), which allows you to standardise your own data.

The English Eastern Region Public Health Observatory report INphoRM 6: Standardisation (196Kb) describes and compares different methods of standardisation.

Information on calculating confidence intervals for standardised rates can be found in the Association of Public Health Observatories (APHO) Technical briefing 3: Commonly used public health statistics and their confidence intervals. The accompanying excel spreadsheet tool allows you to calculate standardised rates and confidence intervals on your own data.

Please note: For the calculation of European age-sex standardised rates (EASRs), the original European Standard Population (ESP) introduced in 1976 has been replaced by the ESP 2013, which has an older population structure. Further details are given on the ONS website. Statistics providers across the UK, including ScotPHO, started to use ESP 2013 from January 2014. The effect of this is that for events that occur primarily amongst older people (e.g. all-cause deaths), the calculated EASR increases by changing to use the ESP 2013. Conversely, for events that happen mainly amongst younger people (e.g. hospital admissions for drugs misuse), the calculated EASR is likely to decrease by changing to use the ESP 2013. All EASRs should be clearly annotated to explain which version of the ESP has been used (but some older analyses of EASR based on ESP 1976 may not yet be annotated). When looking at time trends in EASRs, it is important to ensure that these are based on only one release of the ESP, as it is meaningless to compare an EASR calculated using ESP 1976 with one calculated using ESP 2013. Note that the numbers of events and crude rates are unaffected by the choice of standard population used.

Confidence intervals

The APHO Technical briefing 3: Commonly used public health statistics and their confidence intervals includes how to calculate confidence intervals for rates, proportions, means, and age-standardised rates and ratios. It also describes the use of confidence intervals, i.e. what they are and when they are used, and presents their preferred methods for calculating confidence intervals for the types of statistics described. An excel spreadsheet tool allows you to calculate confidence intervals for your own data.

Process control charts

The APHO Technical briefing 2: Statistical process control methods in public health intelligence (1.3Mb) gives a good overview of this topic. Excel spreadsheet tools are available to download, which allow you to produce funnel plots using your own data (using counts, crude rates, proportions and percentages, or directly and indirectly standardised rates).

Health inequalities

The ScotPHO webpage Measuring health inequalities covers different methods for measuring inequalities in health, and has an accompanying downloadable excel spreadsheet tool that allows you to perform the different tests on your own data.

PHS has produced a comprehensive document Deprivation guidance for analysts (82Kb). This offers background information, outlines the deprivation indices used by PHS, and makes recommendations on the choice of deprivation index to use in health analyses.

The Public Health Observatory Handbook of Health Inequalities Measurement focuses on the measurement and interpretation of health inequalities.

Inequalities in Health - Report of the Measuring Inequalities in Health Working Group set out recommendations made by the Working Group on Measuring Inequalities in Health in Scotland.

The Scottish Government's annual Long-term monitoring of health inequalities report includes the most up-to-date government recommendations for measuring and reporting inequalities in Scotland.

Life expectancy and healthy life expectancy

See our Health and life expectancies website section.

Developing indicators

For a good overview see The Good Indicators Guide: Understanding how to use and choose indicators (535Kb) produced by APHO.

Target setting

The APHO Technical briefing 4: Target setting in a multi-agency environment looks at key issues to consider when setting targets in a multi-agency environment, including the choice of appropriate methodologies, indicators and statistics, and consideration of the wider political and ethical context.

Health Inequalities Impact Assessment

Public Health Scotland have published information and resources relating to health inequalities impact assessments. 

Small area modelled (or synthetic) estimates

See the APHO Technical briefing 8: Prevalence modelling for details around the need for prevalence modelling and for an overview of various methods of generating estimates of the prevalence of diseases or risk factors in local populations.

Local prevalence of health behaviours can be modelled mathematically from national surveys. This has been done in England for a range of behaviours in their local health profiles, and in Scotland for tobacco smoking. Modelling is a low-cost way of producing local estimates for small areas across the whole country since it makes use of existing survey data and requires no new data collection. However, there are substantial limitations which need to be considered when interpreting the estimates, and which mean that modelled / synthetic estimates can't be used for monitoring the effectiveness of interventions.

Other methodological resources that may be of interest:

Public Health Scotland have web pages and resources on: 

• Tools for monitoring and evaluation.
• Health Inequalities Impact Assessments (HIIA).
• The Place Standard tool.
• WEMWB scale and mental health indicators.

Further technical briefings from Public Health England on:

• Measuring sustainable development.
• Geodemographic segmentation.
• Using small area data in public health intelligence.
• Measuring smoking prevalence in local populations.

The British Medical Journal (BMJ) have published on:

• How to read a paper.
• Epidemiology for the uninitiated.
• Academic medicine.
• Statistics at square one.

The Evidence-Based Medicine "What is....?" series provides short bulletins explaining the key concepts behind evidence-based medicine.