- Home
- Administrative data
- Our research
- About us
- Contact us
- News and events
- BLOG - Access to secure data during the Covid-19 pandemic - a model for the future?
- Covid-19 and Care Homes: Advances in Administrative Data Research during the pandemic
- DATA INSIGHTS -Deprivation and informal care at the end of life
- NEWS - Innovative new residential linkage tool launched
- BLOG SERIES - Dramatic increase in deaths at home- No.4
- DATA INSIGHTS - Youth Movements, Social Mobility and Health Inequalities
- NEWS - New report warns of deepening poverty crisis for Scottish families
- New report on Infants Born into Care in Scotland
- Spotlight on Dr Elizabeth Lemmon
- Spotlight on Jan Savinc
- BLOG - The value of social science and administrative data research in Scotland: how we are helping respond to COVID-19
- DATA INSIGHTS - Exploring illegal drug consignments in Scotland
- DATA INSIGHTS -Linking two administrative data sets about looked after children
- NEWS – ADR Scotland data ambassadors launched
- The importance of administrative data
- Virtual Conference - Data Linkage: Information to Impact
- BLOG SERIES - Dramatic increase in deaths at home - No.7
- BLOG SERIES - Dramatic increase in deaths at home
- BLOG SERIES - Dramatic increase in deaths at home- No.3
- DATA INSIGHTS - Investigating the effects of class composition and class size on pupils’ attainment in Scottish primary schools
- NEWS - New opportunity to join ADR Scotland’s Public Panel
- BLOG - Exploring the potential of synthetic data
- DATA INSIGHTS - Automatic Coding of Occupations: Methods to create the Scottish Historic Population Database (SHPD)
- DATA INSIGHTS - Selective schools: do they improve health?
- EVENT - Active Travel: New Data, New Insights
- EVENT - Holyrood Evidence Week: Doing Data Better for Policy and Public Good
- BLOG - Geospatial Ambitions
- BLOG - Taking historical death records and developing a database for future analysis
- DATA INSIGHTS - Community mortality due to Covid-19
- Future-proofing investment into administrative data research announced in Scotland
- NEWS - Understanding the dynamics of the nursing workforce: the potential of routinely collected data
- Spotlight on Katherine Falconer
- ADR Scotland publishes its strategy for 2022-2026
- BLOG: Developing a cross-national research agenda on crime and convictions
- BLOG: Working together to make a difference with data
- DATA INSIGHTS - Homelessness duration in Scotland: how long does rehousing take?
- DATA INSIGHTS - Occupation and COVID-19 deaths: Scotland in a comparative perspective
- DATA INSIGHTS -The health and economic benefits of active commuting in Scotland
- IPDLN Conference - Data linkage research: informing policy and practice
- Spotlight on Dr Evan Williams
- Spotlight on Fernando Pantoja
- Spotlight on Laurie Berrie
- ADR Scotland Winter Partnership Session - **internal event**
- BLOG - AGEING AND HOMELESSNESS IN SCOTLAND
- BLOG - Can we use linked administrative data to identify social disadvantage?
- BLOG - Commuting and its impact on health
- BLOG - The Dynamics of the Nursing Workforce in the UK: Using data to support our nurses
- BLOG: Growing up in kinship care
- DATA INSIGHTS - Analysing a season of death and excess mortality in Scotland’s past
- EVENT - ADR UK Virtual Half Day event
- Event - Public data for public good: towards better understanding children's lives
- NEWS - Data research initiative secures £90 m funding extension
- NEWS: Our role supporting the new Covid-19 research data service in Scotland
- Spotlight on Michelle K Jamieson
- Webinar - An Introduction to Looked-After Children Dataset
- BLOG - An Inside Job: Using Criminology, Police Data and a Lot of Nouse
- BLOG - Improving justice data to promote data justice in Scotland
- BLOG - Location of death in 2020: a changing trend from hospitals to homes
- BLOG - Seeking feedback on Research Data Scotland’s core principles via our public panel
- BLOG - What skills, training and support are required by those wishing a career as an administrative data researcher?
- BLOG No. 9 - Final blog in this 'deaths at home' series
- BLOG SERIES - Dramatic increase in deaths at home - No. 6
- BLOG SERIES - Dramatic increase in deaths at home - No.8
- BLOG SERIES - Dramatic increase in deaths at home- No.5
- BLOG: 5 things I've learnt about working with policymakers...
- BLOG: Automating Coding for Large Historical Datasets
- BLOG: COVID-19- How increased deaths at home impact the carer community
- DATA INSIGHTS -Postal deliveries of drugs in Scotland
- EVENT - 'Getting things done with data in government'
- EVENT - Linking public sector data for research: an ADR UK showcase event
- EVENT Seminar - Administrative data for social policy research: potential and pitfalls
- NEWS - Additional funding for Understanding Children’s Lives and Outcomes
- NEWS - Engaging children and young people
- NEWS: Police use of Fixed Penalty Notices under the Covid-19 regulations in Scotland: A new data report highlights links with deprivation and inequality
- NEWS: Police use of the new Covid-19 powers: Using administrative data to analyse and evaluate practice
- Research Data Scotland - New user forum
- Spotlight on Dr Patricio Troncoso
- Summary of ADR Scotland Winter Partnership session
- Directorship of the International Population Data Linkage Network (IPDLN) for 2021-22.
- BLOG: In the light of experience: InterRAI and the final thousand days of life
- Multiple health conditions and social care
- NEWS - Susan McVie elected as Fellow of the Academy of Social Sciences
- SCADR relocates to the Bayes Centre
- EVENT: Four day introduction to using administrative data for social and health research
- BLOG: The value of administrative data: DALYs and the Scottish Burden of Disease study
- BLOG: Where to start with parliamentary and policy engagement
- EVENT - International Conference on Administrative Data Research, Cardiff
- EVENT - Using data to realise the potential of the 'Last 1000 days'
- EVENT: TalkingData: ADR Scotland mini-summit
- EVENT: “Let’s use data to save time, money and lives”: ADR Scotland partners gather for mini-summit
- EVENTS: ADR Scotland researchers present at international conference in Cardiff
- SafePod Network
BLOG - Automating Coding for Large Historical Datasets
Colleagues from our eCohorts strand led an online workshop this month which showcased the innovative coding methods that we are developing.
On 5th May 2020, SCADR hosted an International Workshop on Automated Coding for research teams from the UK, Denmark, Norway and Sweden, held virtually due to the COVID-19 pandemic. Bringing together computer scientists, social scientists, historians and historical demographers, each brought their experience and expertise to discuss the complexities of applying coding schemes to variables such as occupation or cause of death.
Increased computing power in recent years has meant that historians are now moving from studying single communities or sample populations to exploring patterns and processes amongst the inhabitants of whole regions or countries. This often means making very large sets of data machine readable. The transcription, cleaning and coding of original and often handwritten sources such as census records and church registers are time-consuming, labour-intensive and expensive stages in the research process and researchers are increasingly seeking ways of automating them.
The aim of the workshop was to discuss the possibility of applying coding schemes which would allow international comparison between the sources available in the participating countries. It was recognised that not only would there be language barriers to be overcome, but that even within countries the terms used might vary or change their meaning over time. New terms would emerge and older expressions die out. The way sources recorded information could also vary and this would have to be taken into account when coding, interpreting and analysing the data.
Presentations and experiences were shared from the different nations. The UK team members come from the Universities of Edinburgh, St Andrews and Cambridge and have all been involved in the wider Digitising Scotland project, led by Professor Chris Dibben. This work now involves researchers from SCADR who are focusing on making the data research ready. Richard Tobin introduced his work using natural language processing to assign codes to occupations and causes of death in the data collected through Digitising Scotland.
The Danish team was led by Barbara Revuelta Eugercios and Anne Løkke from the BioHistory Research Group and the Centre for Health Research in the Humanities, University of Copenhagen. Both are closely involved in the Link- Lives project, working to create an historical reconstruction of the Danish population. Hilde Sommerseth, Director of the Norwegian Historical Data Centre, The Arctic University of Norway, brought members of the team who work on the Historical Population Register for Norway. The Swedish team was headed by Elisabeth Engberg and Maria Hiltunen Maltesdotter, from the Centre for Demographic and Aging Research at Umeå University, which hosts the Demographic Data Base, DDB.
From these range of projects, the workshop explored the different programmes and platforms being utilised and the challenges of working across disciplines and languages. Future workshops are planned to unite the teams’ computer scientists and bring twenty-first century technology to bear on eighteenth, nineteenth and twentieth century population data in greater detail, with a proposed future Hackathon to bring ideas into practice.
If historic information on individuals can be made machine readable, it is then possible to create very powerful research resources at scale. By automatically linking these records through time, it is then possible to reconstruct someone’s life, such as: what jobs they did, where they were living when their first child was born or who they married. Using data science approaches these life studies or ‘eCohorts’ can be produced for tens of thousands of individuals. This allows us to explore important questions about how health or wellbeing is impacted through life events and these can be studied at a whole population level.
This article was published on 29 May 2020