Interobserver variability in synovial tissue biopsy markers: results from OMERACT Special Interest Group on Synovial Tissue Biopsy

Background:
Analysis of synovial tissue biopsy (STB) has emerged as a promising tool in Rheumatoid Arthritis (RA) research, considering that synovium is the main target of inflammatory arthritis. In particular, STB can play a pivotal role in the quest to predict treatment responses, setting a new standard for personalised medicine in managing RA. However, STB is not considered as part of routine care, due to intrinsic difficulties in its performance and evaluation. Whilst histological semi-quantitative analysis (SQA) is relatively easy to perform, heterogeneity between centres still burdens its application in clinical trials. Recognising this, the OMERACT STB Special Interest Group (SIG) aimed to refine and standardise STB analysis as a tool for guiding treatment choices in the future.

Objectives:
Establishing interobserver variability in synovial tissue staining and analysis across expert centres.

Methods:
STBs were collected by 7 centres, from 38 RA patients with clinically active disease about to commence a new disease modifying anti-rheumatic drug (DMARD), with STB performed prior to the start of any treatment. STB slides were stained with CD68, CD3, CD19/20, Factor VIII (FVIII) and Hematoxylin/Eosin (HE), and uploaded on a centralised online platform to facilitate independent, blinded, scoring. Before scoring, a quality control process was carried out to assess slides' staining quality, tissue preservation, and overall readability. SQA scorings were then performed on a 5-point ordinal scale (from 0 to 4) for each staining, using whole tissue scans or selected images. 11 observers, from 7 centres, took part in the first round of scoring (from March to July 2023), that was followed by an online workshop (December 2023). An online rescore was carried out between February and May 2024 followed by a second online workshop (October 2024). For interobserver variability, Kendall's correlation coefficients were calculated for SQA scores. Kendall's coefficients scores were categorised as weak (0.1 - 0.39), moderate (0.4 - 0.69), strong (0.7 - 0.89) and very strong (0.9 - 1).

Results:
Patients had a mean age of 58.7 (±13.7) years, 23 (63%) were females, with a mean disease duration of 9 (±12.7) months. Results from initial SQA scoring showed low agreement between observers (Figure 1), with an overall unsatisfactory level of interobserver agreement. This was particularly true for some scores, like FVIII and HE vascularity that showed a range of different answers. In the online workshop that followed, observers collaboratively discussed slides with lowest agreement and rescored them. Interestingly, during the discussion that followed the scoring, it was noted that a higher agreement was reached for those slides that were considered qualitatively optimal. Following discussion, participants independently voted to score SQA parameters using the highest score observed in the slide. Based on this agreement, the same slides were later re-scored (using the same online platform used for the initial score). 7 observers participated in the re-score. Results showed an increase in the agreement between participants (Figure 2A and 2B). Notably, when comparing results from the two scorings among observers who took part in both, there was a shift in correlations' coefficients towards higher values. This mostly encompassed moving from “weak” to “moderate” correlations, and from “strong” to “very strong” ones. As observers utilised a maximum score approach, there was a particular rise in the agreement for HE - where correlation improved in 70% of cases - and in FVIII (improvement in 57% of cases). That ultimately led to more homogeneous results among observers.

In a second online workshop, the STB SIG, to maximise agreement, further proposed:
- To evaluate, for each patient, different slides with different staining on adjacent slides, as well as to have sets of slides that are non-consecutive;
- To discuss between scorers whenever disagreement occurs.

Conclusion:
As the landscape of precision medicine evolves, the identification of specific biomarkers within synovial tissue is becoming increasingly critical. Here we report an initial attempt to standardise STB scoring among different observers with the future goal of using them as predictors of therapy response. Overall, this study shows that it is possible to have a common procedure to score STB among different experts. Building on our findings, continued efforts are needed to standardise STB evaluation as a potential tool for predicting response to therapy in RA patients.