Marina Sarris

A version of this article first appeared on iancommunity.org.

Consumers and patients are bombarded with information about medical research and autism treatments almost daily. Often these articles and social media posts include terms that can mean many different things. J. Kiely Law, M.D., MPH, research director and co-founder of the Interactive Autism Network (IAN), answers some common questions about what research really is – and isn’t. Dr. Law is a physician, a researcher, and the mother of a young adult with autism spectrum disorder (ASD).

The terms “evidence-based medicine and treatment” or “science-based medicine” are used a lot. What do they mean?

It means that an idea or treatment has been scientifically tested and, based on those test results, the idea is valid or, in the case of a treatment, the treatment works. The more times the test is repeated with the same result, the more evidence there is that the idea or treatment is valid. For example, cars go through a series of tests to make sure they are safe, and medical science does the same thing. Science uses a specific way of testing an idea, called a hypothesis. This way of testing is called the scientific method, and its purpose is to remove anything from the test that could bias or invalidate the results.

Sometimes providers or businesses publish patient testimonials about how a therapy or product works to treat autism. Is that similar to research?

No. Individual testimonials, especially when presented in the context of marketing a product, should be viewed with significant skepticism. Generally, these products have not been tested using the rigors of the scientific method.

Can testimonials – the personal experiences of patients – ever become part of research? How would that happen?

Yes, definitely! Patients have the best information about the day-to-day experience of living with a disease or disorder. When many patients report similar experiences or problems, then it sends a powerful message to the research community. This is why projects like IAN and SPARK collect data about autism directly from families and adults with ASD. With this information, researchers can create valid studies of the issues reported to them by patients.

Research is supposed to be objective and unbiased, but what does that mean? How do I know if something really is objective?

In the context of science, it means that when an idea or treatment is tested, the researcher tries to remove all outside influences, so that the results are based solely on the factors being tested. There are many types of bias that can impact the results of a study. Although we are not always aware of it, our own experiences and feelings can influence how we judge outcomes. This is one of the reasons why drug treatment studies are often set up such that neither the researcher nor the patient knows who is receiving the active treatment versus a placebo, or sugar pill. That is called a double-blind experiment because both the researchers and the participants don’t know who is getting the treatment being studied.

In studies that rely on people answering surveys, researchers have to design the questions so they don’t lead the person to answer in a certain way. For example, if a question asks a person how much they “liked” a treatment, that’s a leading question that assumes they liked it. It’s better to ask participants if their impression of the treatment was negative, neutral, or positive. That removes the bias that the treatment should be liked.

There are lots of different types of bias. Let’s say you’re doing a study on the stress levels of parents raising a child with autism. All the parents in your study are well-resourced – they have access to a lot of therapy, treatment, and support for their child. The results show that parents have relatively little stress. Those results are biased by that fact that the parents studied happen to have many resources that reduce family stress. That’s called selection bias because the people selected for the study do not represent all types of parents of a child with autism.

It’s probably impossible to remove all forms of bias from a study so it’s important to know what the biases are and to report in your findings how these biases may have impacted the results. For example, in studies involving the Simons Simplex Collection, in which families have only one child with autism, researchers often report that the results may not apply to families who have several children with autism. That’s because it’s possible that ASD may be different in families that have many members with autism.

What does peer review mean, and why is it important?

Peer review is when other researchers and experts evaluate and critique the scientific integrity or merit of a new research idea being proposed, or study results being published. If a researcher has a new idea and wants to present it for funding, other experts look at it to see if has merit, to start with.

Also, after a study has been conducted, other researchers review it before it is published. They evaluate the techniques used in the study and the interpretation of the results. They see whether the researcher has read what has already been published on the topic. They decide whether this study gets published in that journal.

Peer review is a way to make sure that the potential biases of an individual researcher are removed as much as possible from their interpretation of the results. It is supposed to help address outside influences that could affect the researcher, such as who will benefit from the outcome of a study. What if a researcher owns stock in a company that is producing a drug that he or she is studying? Peer review is trying to ensure that those conflicts of interest are addressed.

Can you name an example of how research has changed how we think about autism in recent years?

Research has really changed the way we approach autism. Here’s one example that began with information from families. Parents have known that wandering and running away were a huge problem in autism, although a lot of them didn’t receive any guidance from their pediatricians about it. If they met other parents of kids with ASD, they would ask, ‘Is your kid a runner?’ However, there wasn’t any evidence to know what the burden was, or to know how many kids were affected. IAN researched wandering several years ago, with results published in 2012.¹ That research found that nearly half of children with ASD wandered or ran away, and more than half of those children went missing. Now that we know how common wandering is, we can come up with solutions to help families keep their children safe.

REFERENCES: 

1. 1. Anderson, C., Law, J.K., Daniels, A., Rice, C., Mandell, D.S., Hagopian, L., & Law, P. (2012). Occurrence and family impact of elopement in children with autism spectrum disorders. Pediatrics. 130(5), 870-877. Abstract. View abstract.