Overview
The project’s main aim is to create a curated, three-tier open database of research publications and replication data on ‘social trust’. The work underpinning the database combines: (1) a large-scale, AI-assisted bibliometric and methodological mapping of the field, drawing on tens of thousands of records from open citation databases such as OpenAlex; (2) AI-assisted extraction of detailed methodological features from a substantively selected subset of full-text publications, validated through systematic human review; and (3) the manual reconstruction of analytical datasets and annotated reproducible code for a smaller curated set of studies whose data and methods are well-suited to use as teaching resources. The primary pedagogical purpose of this last tier is to underpin a textbook on ‘Sociological Methodology Applied to Research on Trust’ with an innovative focus that introduces specific quantitative research methods as part of a journey through inter-disciplinary academic research on the concept. Beyond its pedagogical use, the database is designed to serve as a public empirical resource for studying methodological and reproducibility practices in quantitative trust research, freely available to educators and researchers worldwide.
Statement of purpose:
The main aim of the proposed project is to create a curated database of secondary quantitative replication data on the topic of (social, institutional and interpersonal) ‘trust’. This database will serve several purposes, of which the following will be actively pursued within the timeframe of the BA/Leverhulme Small Research Grant:
- It will be the data foundation for a planned sociological research methodology textbook;
- It will allow educators teaching quantitative methods in the social sciences to easily develop demonstrative examples of statistical analyses beyond those to be included in the textbook;
- It will allow students of quantitative research methods to reproduce results from published research beyond the examples included in the textbook, but related to the additional exercises and methods presented there;
- It will provide a publicly available online resource for researchers starting out in the area of empirical social trust research;
- It will contribute and advance current initiatives to promote open research practices through data and code sharing;
- It will provide a public empirical base for studying methodological and reproducibility practices in three decades of quantitative trust research, complementing recent large-scale meta-scientific initiatives in the social and behavioural sciences.
The motivation behind the project has two scholarly foundations. One is educational and relates to the need to develop a more applied and engaging quantitative pedagogy for social science disciplines that are methodologically and epistemologically eclectic (such as sociology). The other motivation is meta-scientific and relates to ongoing struggles across the social sciences to establish more open and reproducible research practices. Recent advances in open-weight large language models and locally-deployable AI infrastructure have made it feasible to pursue both motivations at a substantially larger scale than was originally envisaged: by automating the systematic methodological review of large bodies of literature while keeping all sensitive processing in-house and under human supervision. The sections that follow describe how the two motivations interrelate and how the project’s programme of activities addresses both.
Context and background:
Quantitative sociology lags behind other social science disciplines – such as political science or economics – in its adoption of open research practices, which aim to ensure that published results are reproducible and the analytical procedures transparent (Ferguson et al. 2023; Freese 2007; Freese and Peterson 2020; Moody et al. 2022; Weeden 2023). The reasons for this are multifaceted. Weeden (2023) highlights the internal fragmentation of sociology along a plurality of epistemological and methodological orientations. Another reason might be that quantitative sociologists rarely use experimental methods (Gërxhani and Miller 2022) , which have been at the forefront of the recent shift towards open science practices in psychology (Ferguson et al. 2023). On the other hand, pursuing causal explanations, which is at the heart of econometrics, is somewhat marginal in sociological science (Gërxhani and Miller 2022; Goldthorpe 2016); sociologists tend instead to rely on large-scale survey data to target broad estimands that generally result in low p-values (and low effect sizes). As many have argued, that’s not ideal for the purposes of scientific advancement (Lundberg et al. 2021; Smaldino and McElreath 2016; McElreath and Smaldino 2015).
Regardless of the reasons for their limited adoption, based on large surveys of academic practitioners, Ferguson et al. (2023), p. 3 have found that “there are fairly high levels of stated support for open science, even among scholars in a discipline like sociology, where there is less institutionalization of these practices”. Institutionalisation and research culture are key elements in achieving advancements in respect to open science practices. One major factor affecting both, which sociology shares with its disciplinary peers, is the lack of methodological socialisation in reproducibility practices. Potential solutions have long been promoted in some trailblazing centres of excellence – like Harvard University’s Institute for Quantitative Social Science (King 2006) – but have only recently started gaining wider adoption through teaching reproducibility practices as part of applied research methods training at both undergraduate (Ball 2023; Vilhuber et al. 2022) and postgraduate levels (Stojmenovska et al. 2019). It takes small steps to achieve giant leaps, and these developments will undoubtedly change the teaching of applied quantitative research methods in the social sciences for the better, which will in turn shape the scientific practices of future generations of researchers.
My own teaching practice has sought to contribute to these positive trends. I have had the opportunity to design quantitative research modules at both undergraduate and postgraduate levels at two institutions, and a few years ago I implemented a simple but unorthodox approach: instead of introducing commonly used methods in turn and separately from one another, take the students on a journey across one coherent research theme, deconstructing the data and methods underpinning selected key articles and incrementally reproducing (parts of) the original analyses. It not only introduced students to replication packages and data repositories, but they could learn how basic methods relate to the more complex analyses reported in published research. As for the research theme, I settled on “social trust”, which is not only the focus of a large body of academic scholarship but is facilitated by the inclusion of useful measurement variables in all the major cross-national large-scale social survey programmes (such as the European Social Survey or the World/European Values Survey/Study) that provide publicly available data.
Designed around this basic idea, my recent methodological teaching has incorporated a number of open research practices, including the public sharing of computer code usable to prepare raw data from popular secondary survey datasets into datasets ready for analysis on public course websites that I wrote and maintain. This teaching portfolio has now matured into a concrete need to develop my course notes and programming code into a more structured textbook that would make a broader contribution to the advancement of quantitative pedagogy in sociology and other social sciences. It will fit well among other recent approaches that place the idea of story-telling and replication at the centre of quantitative teaching and research dissemination (Alexander 2023; Gelman et al. 2020; Gelman and Basbøll 2014). This project aims to create the essential data infrastructure that will underpin this textbook.
A second strand of recent change has come from outside teaching, in the meta-scientific examination of how research is actually conducted. Large multi-analyst initiatives have shown that even seemingly straightforward empirical claims in the social and behavioural sciences are frequently contingent on analysts’ choices (Aczel et al. 2026), and a wider Nature collection of companion contributions has used ‘replication games’ and crowd-sourced re-analyses to interrogate the robustness of published findings at scale (Sánchez-Tójar et al. 2026; Jones 2026; Nature Editorial 2026). The most direct methodological impetus for the present project’s enhanced design, however, came from economics: Garg and Fetzer (2025) demonstrated the power of language models to extract methodological structure – in their case, causal-claim graphs – from over forty thousand working papers, providing both a working template and a proof of feasibility for systematic, AI-assisted methodological review of large literatures.
These meta-scientific developments have been mirrored on the technical and ethical side. The maturing literature on AI-assisted systematic review (Dijk et al. 2023; Fütterer et al. 2026) and the parallel debate on the privacy, governance, and ethical dimensions of using large language models in research (Deng et al. 2025; Barberá 2025; Dennstädt et al. 2025) together establish a defensible pathway: locally-deployed, open-weight models running on institutional high-performance computing infrastructure make substantial extraction and classification work at literature scale feasible while keeping all source materials and intermediate outputs under institutional control. Both developments – the meta-scientific case for systematic methodological scrutiny, and the technical and ethical case for in-house AI infrastructure – have shaped the present iteration of the project, which is broader in scope and more methodologically ambitious than was originally envisaged.
Methodology and activities:
The project follows a three-tier methodological design that produces three nested datasets – THEME, META, and METHOD – moving from a broad bibliometric mapping of “social trust” research to a curated collection of fully replicable studies suitable for use as teaching resources. Each tier is constructed through a combination of bibliometric retrieval, AI-assisted classification using locally-deployed open-weight large language models, and human methodological judgement, with the latter remaining central at every stage.
The first tier produces the THEME dataset: a comprehensive metadata table of English-language social science publications on social trust over 1990–2025. Bibliographic records are drawn primarily from OpenAlex, complemented by Web of Science and Scopus where institutional access permits, then deduplicated by DOI and fuzzy title matching, standardised across journal and author conventions, and filtered by language and publication window. A subsequent processing step enriches this metadata with broad methodological and disciplinary classifications (theoretical; empirical-qualitative; empirical-quantitative; mixed methods; review/meta-analysis) generated by open-weight LLMs (e.g. DeepSeek, Qwen, GLM families) running on the university’s GPU-enabled high-performance computing nodes. Batch outputs are validated against a stratified manual sample, and misclassification patterns feed back into iterative prompt refinement. The completed THEME dataset supports the project’s bibliometric questions about time trends, geographies, disciplinary distributions, and open-access patterns in trust research over the past three decades.
The second tier produces the META dataset: a reduced subset of a few hundred primary empirical-quantitative studies in which social trust is a central variable (dependent, independent, or in another substantively load-bearing role). Candidate studies are screened against criteria covering substantive centrality, publication quality, and recency, with automated scoring (keyword density, citation metrics, journal rank) supporting but not replacing human judgement. Full texts are acquired through institutional library access and Unpaywall queries. The acquired texts are parsed using open-source tools such as GROBID and segmented into structured sections, after which a second locally-deployed LLM pipeline extracts detailed features from each publication: the role played by the trust variable in the analysis, the data source (primary collection or named secondary dataset), the statistical models used, the presence and substance of data- and code-availability statements, and the direction and significance of key trust-related findings. All AI-extracted records are reviewed against the original publications for logical consistency and source alignment, with systematic disagreements informing iterative prompt refinement. No automated scraping of paywalled content is used at this stage, and no full-text sources are processed through commercial Large Language Models hosted on external (cloud-based) servers. The META dataset supports the project’s meta-analytical questions about reproducibility practices, data sourcing, methodological orientation, and the substantive claims made in quantitative trust research.
The third tier produces the METHOD dataset: a smaller curated collection of approximately fifty studies selected on the basis of public availability of the underlying data, sufficient documentation of the analytical approach, and pedagogical value across a range of methods and social science disciplines. For each selected study, the analytical dataset is rebuilt from the original raw source using reproducible R code, the data manipulation steps are fully documented, and annotated R scripts are developed that replicate key parts of the originally published analyses. This is the most labour-intensive stage per publication and produces the project’s primary teaching resource: a curated set of analytical datasets and code that students and instructors can use to introduce or illustrate specific quantitative methods – linear and logistic regression, multilevel modelling, structural equation modelling, causal inference, time series, and others – through engagement with real-world social-trust research.
The three dataset tiers are integrated into a single harmonised database with controlled vocabularies for methods, models, and variable roles, and with explicit indicators recording the provenance of each annotation (human-coded, AI-extracted, or both). The database will be released as a versioned open-access repository on GitHub – the Trust Research Methodology Database (TReMeDa) – accompanied by a data dictionary, user documentation, and stable archived releases with DOI assignment via Zenodo. Pipeline documentation covering LLM model versions, prompt templates, and validation decisions ensures that the workflow can be re-run and that future batches of publications can be appended and reprocessed as the literature on social trust continues to grow.