How to read stem cell trial results — phase I versus phase II through autism cord blood research

Summary — Information about stem cells often says that 'an effect was reported in a study'. But the same word 'reported' means very different things depending on the study design. Using cord blood research in autistic children by a team at Duke University in the United States, this article sets out five criteria to check when reading clinical trial results.

The case: phase I and phase II from the same team

In 2017 the Duke team published a phase I study in which 25 young autistic children received a single intravenous infusion of their own cord blood, banked at birth. The infusions were carried out safely, and improvements in social and communication measures were reported on parent ratings and clinician observation. Read on its own, this could look like evidence of an effect.

To test this, the team ran a randomised, placebo-controlled phase II study of 180 children aged 2–7, published in 2020. On the primary outcome for the whole group, a socialisation score, there was no statistically significant difference between cord blood and placebo. When children without intellectual disability were analysed separately, differences were reported in some communication and EEG measures. Same team, same condition, same cells — the reason the two studies read so differently lies in study design.

Criterion 1: study phase — what is the study trying to find out?

Each phase has a different aim. Phase 1 mainly tests safety and feasibility, phase 2 explores possible efficacy and dose, and phase 3 confirms efficacy and safety in more patients. Changes seen in phase 1 are grounds for 'a further study is worthwhile', not proof of efficacy. The Duke phase I study, too, was mainly about safety.

Criterion 2: control group — compared with what?

In open-label, single-arm studies, everyone receives treatment and before-and-after changes are compared. But children develop naturally over time and receive behavioural and speech therapy in the same period, and if parents and assessors know treatment was given, expectations can influence ratings. That is why the phase I study could not tell whether changes were due to cord blood or to these factors. With a randomised placebo group, such factors act similarly on both groups, so the remaining difference can be attributed to treatment.

Criterion 3: blinding — who knew what?

In a double-blind study, neither participants nor assessors know who received treatment. Blinding is especially important for assessments involving observers' judgement, such as behaviour and development. Because the phase II study had this design, it is regarded as much more reliable evidence than phase I.

Criterion 4: primary outcome — what did the study decide to look at first?

A well-designed trial specifies and registers in advance the primary outcome that will count as evidence of effect. The phase II primary outcome was the socialisation score, and there was no difference on it. Because looking at many measures after a study can turn up chance differences, the pre-specified outcome is given the most weight.

Criterion 5: subgroup analysis — hypothesis or conclusion?

The phase II results in children without intellectual disability are a subgroup analysis. Subgroup analyses are useful for generating new hypotheses — 'it may be different in these children' — but cannot prove an effect by themselves; the investigators also described them as exploratory. Confirming them requires a new study planned for that group from the start.

Other studies worth reading alongside

The same team confirmed safety and dose in a phase I study of cord tissue-derived mesenchymal stromal cells in 12 children (2020), and in a randomised, placebo-controlled study of autologous cord blood in 63 children with cerebral palsy (2017) reported greater improvement in gross motor function in children who received at least a certain dose. The cerebral palsy result is significant because it came from a design with placebo control and dose comparison.

In short

When reading stem cell research results, check five things together: study phase, control group, blinding, primary outcome and whether results come from subgroup analyses. Detailed condition information is on the 'Autism Spectrum Disorder' page.

Frequently asked questions

Q. If children improved in phase 1, doesn't that mean it works?
Phase 1 mainly checks safety and often has no control group. Children develop naturally over time and receive other therapies in the same period, so changes seen in phase 1 are grounds for a further study, not evidence of efficacy.

Q. If phase 2 showed no difference, was it a complete failure?
No. The phase 2 study re-confirmed safety in a large group of children and left a new hypothesis that results may differ depending on intellectual disability. A finding of 'no difference' is also important information for designing the next study.

Q. How can I check whether a study was registered in advance?
Searching public registries such as ClinicalTrials.gov in the US or jRCT in Japan by study name or condition shows the primary outcome and eligibility criteria registered before the study began. Comparing the outcomes in the results paper with the registration is also a good check.

The above summarises results reported in published papers. University and institute names are given to identify research sources, not to introduce or recommend any medical institution. The suitability of any treatment is decided by the medical team.

Autism, Developmental Disorders & Cerebral Palsy →