New thyroid scintigraphy datasets: building a benchmark for detecting residual thyroid tissue

This post summarises and explains the paper "New thyroid scintigraphy datasets: Construction and benchmark assessment in diagnosis of residual thyroid tissue", published in the Journal of Military Science and Technology, No. 88 (2023), pp. 131–138. DOI: 10.54939/1859-1043.j.mst.88.2023.131-138. Beyond the paper itself, I've added background and some notes of my own so it reads well for people outside nuclear medicine.

TL;DR

• The problem: after a total thyroidectomy for cancer, doctors need to know whether any thyroid tissue remains, to decide on further radioiodine (I-131) treatment. The answer is on a thyroid scintigraphy scan.
• The gap: almost no thyroid scintigraphy dataset is publicly shared. The only sizeable study (446 patients, Guo et al., 2021) used private data.
• Contribution: we built two standardised datasets from 559 cases at the Department of Nuclear Medicine, 108 Central Military Hospital (Hanoi): one of whole-body images and one of neck-region images, with labels confirmed by a panel of physicians.
• Benchmark: we fine-tuned 8 popular CNNs (VGG16, InceptionV3, ResNet152V2, Xception, MobileNetV2, NASNetMobile, EfficientNetB0/B7). Best results:
  • neck images: Accuracy 0.955, Recall 0.984, F1 0.961 (Xception, VGG16)
  • whole-body images: Accuracy 0.893, F1 0.908 (Xception)
• Main lesson: cropping to the right region (the neck) matters more than the choice of architecture.

1. Background: what is thyroid scintigraphy, and why look for residual tissue?

Treating differentiated thyroid cancer

For differentiated thyroid cancer (mostly papillary), standard treatment usually has three steps:

1. Surgery to remove the whole thyroid gland.
2. I-131 therapy to ablate any remaining thyroid tissue (and metastases, if present).
3. Long-term TSH-suppression therapy with thyroid hormone.

However careful the surgery, it rarely removes 100% of the thyroid tissue. What remains, however small, still takes up iodine, and that is exactly what nuclear medicine exploits.

How scintigraphy works

Thyroid cells (including most differentiated cancer cells) actively pump iodine into themselves. Give the patient a small oral dose of radioactive iodine and it concentrates wherever thyroid tissue remains. A gamma camera outside the body captures the emitted gamma rays and forms an image: bright/dark spots are areas of radioiodine uptake.

Unlike CT or MRI, which produce anatomical images (what an organ looks like), scintigraphy produces functional/metabolic images (what an organ is doing). The images are low resolution and very noisy (Poisson counting noise), but extremely sensitive to active thyroid tissue.

In this study, patients took a diagnostic dose of 5 mCi and were scanned 36–48 hours later. The clinical question is simple:

Is there radioiodine uptake in the thyroid bed on this scan?

That makes it a binary classification problem: Uptake or Non-uptake.

Sample from the Whole Body (WB) dataset, anterior and posterior views. The dark area at the top is the neck; the dark spots in the abdomen are physiological uptake (stomach, bowel), not thyroid tissue.

2. The problem: no standard data

CADx (Computer-Aided Diagnosis) systems have shown promising results, but very few are approved for clinical use. A major reason is the lack of standard datasets to evaluate and compare them. A model that scores well on a small private dataset is not enough evidence that it is safe in practice.

Our literature survey found:

Anatomical imaging data (DDSM, MIAS, IRMA…) is relatively plentiful. For thyroid SPECT, though, exactly one dataset appears in the literature, and it isn't shared. So any group entering the field has to collect data from scratch or find a hospital partner, which takes a long time.

Nuclear medicine studies also show that each population has its own pathological characteristics on SPECT images. That makes a dataset of Vietnamese patients necessary for research here.

3. Building the datasets

Collection

• Source: Department of Nuclear Medicine, 108 Central Military Hospital.
• Period: January 2020 to December 2021, 1,777 scans in total. The paper uses 559 images for its experiments.
• Patients: aged 21 to 59 (mean 39), all receiving their first treatment to ablate thyroid tissue after surgery. Most (~97%) had papillary thyroid carcinoma.
• The study was approved by the Department of Nuclear Medicine, 108 Central Military Hospital.

Labelling

Each image gets one of two labels: radioiodine uptake or non-uptake. Labels come from the physician's diagnostic conclusion for each case and are re-checked in panel consultation case by case. With medical data, this is the most labour-intensive step and the one that decides the dataset's quality.

Two datasets

From the same set of cases we built two datasets:

1. Full Body (WB): whole-body scans combining anterior and posterior views.
2. Head & Neck (HN): the neck region cropped from the anterior view.

Sample from the Head & Neck (HN) dataset, cropped from the anterior view. Uptake in the thyroid bed is much easier to see than on the whole-body scan.

flowchart LR
  A[1,777 scans<br/>108 Central Military Hospital<br/>2020–2021] --> B[559 cases<br/>selected]
  B --> C[Labelling<br/>Uptake / Non-uptake<br/>+ panel review]
  C --> D[WB: anterior + posterior]
  C --> E[HN: neck crop<br/>from anterior view]
  D --> F[Train / Val / Test]
  E --> F
  F --> G[Fine-tune 8 CNNs<br/>ImageNet-pretrained]

Both datasets use the same split:

The class ratio is about 1.24 : 1, slightly skewed towards uptake but not severely imbalanced. The results show that even this small skew has an effect.

4. Benchmark: 8 CNNs with transfer learning

Why transfer learning?

559 images is very small by deep learning standards. Training a CNN with tens of millions of parameters from scratch on that would almost certainly overfit.

Transfer learning takes a network already trained on ImageNet (over a million natural images), keeps its feature extractor, and fine-tunes it on medical data. The early layers of a CNN learn generic features (edges, blobs, textures) that remain useful even though a scintigram looks nothing like a cat photo. The technique has worked well on bone X-rays, chest X-rays, myocardial perfusion SPECT and brain MRI. It gives better initial accuracy, faster convergence and better final results than training from scratch.

Models tested

Metrics

Treating uptake (residual tissue present) as the positive class:

• Accuracy = (TP + TN) / total: overall share of correct predictions.
• Recall (Sensitivity) = TP / (TP + FN): of the cases that really have residual tissue, how many the model catches. High recall = few misses.
• TNR (Specificity) = TN / (TN + FP): of the cases with no residual tissue, how many the model gets right. High TNR = few false alarms.
• F1-score = 2TP / (2TP + FP + FN): balances misses against false alarms.

Clinically, missing residual tissue (a false negative) is usually worse than a false alarm, because the patient may not get the follow-up treatment they need. That makes recall the metric to watch.

5. Results

Whole-body dataset (WB)

Neck-region dataset (HN)

Reading the results

1. Neck images beat whole-body images. Most models improve on every metric going from WB to HN. On a whole-body scan, several other organs take up iodine physiologically: salivary glands, stomach, bowel, bladder. On the image they look like uptake in the neck and confuse the model. Cropping to the neck removes the misleading information and focuses the model on the right place.

This is a familiar lesson in medical imaging: putting domain knowledge into preprocessing (choosing the ROI) often pays off more than a more complex model. With little data, it matters even more.

2. Xception is the most consistent. It comes out on top on both datasets. On neck images, a recall of 0.984 means that of the 62 uptake cases in the test set, the model missed only one.

3. Small models hold their own. NASNetMobile (about 5M parameters) reaches 0.984 recall on neck images, matching Xception and VGG16, while VGG16 has about 138M parameters. EfficientNetB7, the largest model, comes last on HN. With a small dataset, bigger is not better.

4. Label skew leaves a trace. On whole-body images most models have recall > TNR: they lean towards predicting "uptake". That matches the training set having about 1.24× more uptake labels. EfficientNetB7 is the exception, with TNR (0.92) above recall (0.871), so it seems less affected by the skew.

6. Extra perspective: caveats when reading the results

This section is my own commentary, not part of the paper.

A small test set means wide confidence intervals. With 112 test images, each mistake costs about 0.9 percentage points of accuracy. The 95% confidence interval for an accuracy of 0.955 is roughly 0.90–0.98. So 1–3 point gaps between models (say 0.911 vs 0.920) aren't enough to call one model better. A stricter comparison could use k-fold cross-validation, bootstrapping, or McNemar's test on the same test set.

Single-centre data. Every image comes from one hospital and one acquisition protocol. A model may pick up scanner- or protocol-specific quirks, so it needs external validation on data from other hospitals before any talk of clinical use.

Scintigrams aren't natural images. They are greyscale, Poisson-noisy and low contrast, very different from ImageNet. Things that may help: denoising, count-based intensity normalisation, physics-aware augmentation (left-right flips, mild intensity changes), or self-supervised pretraining on unlabelled nuclear medicine data.

Doctors trust what they can inspect. Techniques like Grad-CAM show whether the model is "looking" at the thyroid bed or relying on something else. On whole-body images, this is also a way to test the hypothesis that abdominal physiological uptake is what confuses the models.

From classification to localisation and quantification. Uptake vs non-uptake is only the first step. Clinicians also need to know where the tissue is, how much there is, and whether there are distant metastases. Those are detection and segmentation problems, which need region-level labels, not just image-level ones.

7. Conclusion and next steps

The paper makes two contributions:

1. Two standardised thyroid scintigraphy datasets (whole-body and neck) from Vietnamese patients, with labels confirmed by a panel of physicians: a reliable data source for CADx research groups in Vietnam and abroad.
2. Benchmark results for 8 modern CNNs, as a baseline for future work. On neck images, the best model reaches 95.5% accuracy and 98.4% recall.

Next steps for the group:

• Collect more data to grow the sample (1,777 scans exist; the paper uses 559).
• Address label imbalance and improve the models (Xception and EfficientNetB7 for whole-body; Xception, VGG16 and NASNetMobile for neck images).
• Extend to locating radioactive regions and detecting metastases elsewhere in the body.
• Add clinical indicators (such as thyroglobulin and TSH levels) to support treatment decisions.

This is the first post on the blog. If you work in nuclear medicine or medical imaging, or would like to collaborate on this dataset, get in touch.

---

Citation

L. P. Minh, N. C. Thanh, P. N. Hai, D. N. Thang, N. T. Trung, C. M. Duc, N. T. Ha, N. D. Thuan, "New thyroid scintigraphy datasets: Construction and benchmark assessment in diagnosis of residual thyroid tissue," Journal of Military Science and Technology, No. 88, pp. 131–138, 2023. DOI: 10.54939/1859-1043.j.mst.88.2023.131-138