- Home
- Administrative data
- Our research
- About us
- Contact us
- News and events
- BLOG - Access to secure data during the Covid-19 pandemic - a model for the future?
- Covid-19 and Care Homes: Advances in Administrative Data Research during the pandemic
- DATA INSIGHTS -Deprivation and informal care at the end of life
- NEWS - Innovative new residential linkage tool launched
- BLOG SERIES - Dramatic increase in deaths at home- No.4
- DATA INSIGHTS - Youth Movements, Social Mobility and Health Inequalities
- NEWS - New report warns of deepening poverty crisis for Scottish families
- New report on Infants Born into Care in Scotland
- Spotlight on Dr Elizabeth Lemmon
- Spotlight on Jan Savinc
- BLOG - The value of social science and administrative data research in Scotland: how we are helping respond to COVID-19
- DATA INSIGHTS - Exploring illegal drug consignments in Scotland
- DATA INSIGHTS -Linking two administrative data sets about looked after children
- NEWS – ADR Scotland data ambassadors launched
- The importance of administrative data
- Virtual Conference - Data Linkage: Information to Impact
- BLOG SERIES - Dramatic increase in deaths at home - No.7
- BLOG SERIES - Dramatic increase in deaths at home
- BLOG SERIES - Dramatic increase in deaths at home- No.3
- DATA INSIGHTS - Investigating the effects of class composition and class size on pupils’ attainment in Scottish primary schools
- NEWS - New opportunity to join ADR Scotland’s Public Panel
- BLOG - Exploring the potential of synthetic data
- DATA INSIGHTS - Automatic Coding of Occupations: Methods to create the Scottish Historic Population Database (SHPD)
- DATA INSIGHTS - Selective schools: do they improve health?
- EVENT - Active Travel: New Data, New Insights
- EVENT - Holyrood Evidence Week: Doing Data Better for Policy and Public Good
- BLOG - Geospatial Ambitions
- BLOG - Taking historical death records and developing a database for future analysis
- DATA INSIGHTS - Community mortality due to Covid-19
- Future-proofing investment into administrative data research announced in Scotland
- NEWS - Understanding the dynamics of the nursing workforce: the potential of routinely collected data
- Spotlight on Katherine Falconer
- ADR Scotland publishes its strategy for 2022-2026
- BLOG: Developing a cross-national research agenda on crime and convictions
- BLOG: Working together to make a difference with data
- DATA INSIGHTS - Homelessness duration in Scotland: how long does rehousing take?
- DATA INSIGHTS - Occupation and COVID-19 deaths: Scotland in a comparative perspective
- DATA INSIGHTS -The health and economic benefits of active commuting in Scotland
- IPDLN Conference - Data linkage research: informing policy and practice
- Spotlight on Dr Evan Williams
- Spotlight on Fernando Pantoja
- Spotlight on Laurie Berrie
- ADR Scotland Winter Partnership Session - **internal event**
- BLOG - AGEING AND HOMELESSNESS IN SCOTLAND
- BLOG - Can we use linked administrative data to identify social disadvantage?
- BLOG - Commuting and its impact on health
- BLOG - The Dynamics of the Nursing Workforce in the UK: Using data to support our nurses
- BLOG: Growing up in kinship care
- DATA INSIGHTS - Analysing a season of death and excess mortality in Scotland’s past
- EVENT - ADR UK Virtual Half Day event
- Event - Public data for public good: towards better understanding children's lives
- NEWS - Data research initiative secures £90 m funding extension
- NEWS: Our role supporting the new Covid-19 research data service in Scotland
- Spotlight on Michelle K Jamieson
- Webinar - An Introduction to Looked-After Children Dataset
- BLOG - An Inside Job: Using Criminology, Police Data and a Lot of Nouse
- BLOG - Improving justice data to promote data justice in Scotland
- BLOG - Location of death in 2020: a changing trend from hospitals to homes
- BLOG - Seeking feedback on Research Data Scotland’s core principles via our public panel
- BLOG - What skills, training and support are required by those wishing a career as an administrative data researcher?
- BLOG No. 9 - Final blog in this 'deaths at home' series
- BLOG SERIES - Dramatic increase in deaths at home - No. 6
- BLOG SERIES - Dramatic increase in deaths at home - No.8
- BLOG SERIES - Dramatic increase in deaths at home- No.5
- BLOG: 5 things I've learnt about working with policymakers...
- BLOG: Automating Coding for Large Historical Datasets
- BLOG: COVID-19- How increased deaths at home impact the carer community
- DATA INSIGHTS -Postal deliveries of drugs in Scotland
- EVENT - 'Getting things done with data in government'
- EVENT - Linking public sector data for research: an ADR UK showcase event
- EVENT Seminar - Administrative data for social policy research: potential and pitfalls
- NEWS - Additional funding for Understanding Children’s Lives and Outcomes
- NEWS - Engaging children and young people
- NEWS: Police use of Fixed Penalty Notices under the Covid-19 regulations in Scotland: A new data report highlights links with deprivation and inequality
- NEWS: Police use of the new Covid-19 powers: Using administrative data to analyse and evaluate practice
- Research Data Scotland - New user forum
- Spotlight on Dr Patricio Troncoso
- Summary of ADR Scotland Winter Partnership session
- Directorship of the International Population Data Linkage Network (IPDLN) for 2021-22.
- BLOG: In the light of experience: InterRAI and the final thousand days of life
- Multiple health conditions and social care
- NEWS - Susan McVie elected as Fellow of the Academy of Social Sciences
- SCADR relocates to the Bayes Centre
- EVENT: Four day introduction to using administrative data for social and health research
- BLOG: The value of administrative data: DALYs and the Scottish Burden of Disease study
- BLOG: Where to start with parliamentary and policy engagement
- EVENT - International Conference on Administrative Data Research, Cardiff
- EVENT - Using data to realise the potential of the 'Last 1000 days'
- EVENT: TalkingData: ADR Scotland mini-summit
- EVENT: “Let’s use data to save time, money and lives”: ADR Scotland partners gather for mini-summit
- EVENTS: ADR Scotland researchers present at international conference in Cardiff
- SafePod Network
BLOG - What do Administrative Data Researchers need to know?
The Scottish Centre for Administrative Data Research (SCADR) have been exploring what people using administrative data think they need to know, and what’s already available regarding training, support and access in this area.
What do people want?
We ran a Twitter poll for researchers and also asked data providers, the electronic Data Research and Innovation Service (eDRIS) team and Early Career Researchers Using Scottish Administrative Data (eCRUSADers) network what they wished researchers knew or had known more about when accessing and analysing administrative data.
In our Twitter poll, nearly twenty responses revealed the areas of key interest:
These results and our wider discussions highlighted 7 themes:
- The data access process – including how to write documents, timescales, and how things are different in Scotland
- Data protection – legal and ethical issues specific to admin data use – including the risks of using admin. data, the legislation that underpins data linkage, the differences between anonymisation and pseudonymisation, and when a Data Protection Impact Assessment (DPIA) is required.
- How to work within a safe haven – including processes (e.g. how to request software, disclosure control rules), and data management within a secure environment (versioning, following published standards such as RECORD).
- Understanding data/admin data in general – including understanding the provenance and uses of data, data management techniques for large messy datasets and understanding linkage.
- Specific analysis/data science techniques – including causal inference methods, survival analysis and longitudinal modelling.
- Information on specific datasets – including information on variable codes, variable quality (including how variables are derived), and overall what datasets can and can’t be used for.
- Other – including data visualisation, creating innovative policy relevant projects and how to fit this process into wider career development.
Interestingly, data protection was not mentioned by any researchers specifically, but was seen as an area lacking by data controllers and teams supporting researchers in Scotland and elsewhere. Perhaps researchers feel this area is covered by the training required for data access or conversations with the Data Protection Officer (DPO) and therefore they believe they know all that is required? Or perhaps specific data protection information on linking datasets needs more of a focus?
A few groups asked for training for all parties involved, i.e. sometimes support teams and data controllers could benefit from some training/development too. This could be on the legality and safety of data sharing and the system, or overviews of statistical methods to make output checking easier. A potential opportunity might be to deliver training to data controllers, support teams and researchers together to share knowledge, points of view and discuss how best to co-create research projects that are both feasible and impactful.
What’s already out there?
Looking at the themes, you might be thinking “there must be guidance published on a lot of this already?” You’d be right, however, this exercise indicates that researchers may not be aware of the huge amounts of information available on the process and datasets from sources such as the ISD National Data Catalogue, eDRIS website, eCRUSADers website or Scottish Government website.
It’s also possible sources such as these are not covering the information that researchers need, or that the information isn't easily discoverable.
A look through some major training providers found the majority of live courses and events are focused on data management, statistical methods or information on datasets. Gaps were noticed around specific information on Scottish datasets and the use of a safe haven. Health datasets for example are one of the biggest data assets in Scotland, and often are required to be used within a secure environment or safe haven - but there are no courses that train researchers in how to understand features such as hospital episodes and practically working with the data to recognise irregularities, or how to deal with working with the data within a secure environment.
Timing of events also could be an issue. Data management and statistical methods courses tend to be run annually. This may mean that people starting just after a course have nearly a full year to wait. Similar problems may arise for researchers that aren’t aware what methods they need training in until they have access to their data.
Through this exercise we identified four comprehensive courses that covered most of the above themes. More details on course content can be found on our training page, but include:
- SCADR's Introduction to Administrative Data (IDS-ADR) course
- Administrative Data Training - Centric
- Data Awareness Training – The Health Foundation & Cancer Research UK
- Analysis of Linked Health Data (introductory and advanced) – Swansea University / University of Sydney
These and other events are available for support staff as well as researchers, and there are courses in development from ONS / University of West of England (UWE) on output checking specifically for support staff.
Another important aspect we found is that the most useful courses use admin. data as the training dataset and provide interactive sessions for hands on practice. The materials are also available for a long time afterwards to allow researchers to come back to them when they get to different stages of their data access/analysis journey.
What can we do about it?
With this in mind, SCADR is now looking into how they we can best support filling the gaps identified. The development of Research Data Scotland is likely to be instrumental at signposting to all the available resources out there, to help first time researchers navigate the joy of Administrative Data Research.
We are very interested to hear your thoughts
- What are your views on the issues we’ve raised? Do you agree with the gaps identified?
- Do you have suggestions on what other training and support can be given to new researchers in this area?
Any answers or feedback from this blog would be gratefully received - please email to scadr@ed.ac.uk.
This article was published on 21 Jan 2021