The COVID-19 pandemic necessitated a dramatic shift from the traditional teaching model to online learning. Although e-learning is not a new concept, many schools are beginning to realize its importance as many state governments impose lock-down policies. Tomaino et al.’s (2021) study on “assessing the feasibility and effectiveness of distance learning for students with DDBN” is framed around this theme. The paper critically and systematically analyzes the research processes undertaken in the Tomaino and associates’ study and provides feedback on its strengths and weaknesses. It assesses whether the research design and methodology are consistent with the study’s purpose to establish if Tomaino et al.’s survey is scientifically sound and if the results are valid based on the study’s design and methodology.
Article’s Information
The “Behavior Analysis in Practice” journal is a transnational scientific journal that provides scientific information relevant to service delivery in behavior analysis. Its content targets front-line social workers, educators, scientist-practitioners, and supervisors in the behavioral field. The journal covers empirical reports that describe or evaluate behavior-analytic programs and procedures. Tomaino et al.’s study is one of the journal’s peer-reviewed articles and was published by SpringerLink. Peer-reviewed journal articles are credible and legitimate sources of information.
Analysis of the Research Article
Introduction
The introduction is concise and captures the readers’ interest in the research topic. It is logically written and adequately introduces the research problem. The authors claim that distance learning has been widely adopted following the COVID-19 outbreak, yet its effectiveness is not fully established. Moreover, research findings on the subject have been inconsistent, making it difficult to determine distance learning’s true impact on learners. Without knowing the actual impact or effectiveness of the learning modality, decision-makers can be misled into assuming that the program is effective when in reality, it isn’t, which can affect learning outcomes.
Therefore, the authors sought to establish two truths in the study. First, they aimed to determine distance learning’s feasibility and effectiveness when working with students with developmental disabilities and high behavioral needs (DDBN). Second, they purposed to ascertain whether the transition to distance learning affected the target population’s Individualized Education Plan (IEP). IEPs help educators to provide specialized instruction to students with disabilities and attend to their specific needs.
Literature Review
The literature review, embedded in the study’s introduction section, is relevant to the survey. Although the researchers do not demonstrate any potential biases in the literature review, all the included studies are centered on the researchers’ central theme. The literature review is organized and synthesized around the research topic and allows the reader to understand how it fits into past studies. The evidence used to support the researcher’s claims is solid and reliable because most are empirical studies conducted within the last ten years. Out of the 36 studies used in the literature review, 26 were performed between 2010 and 2020. The literature review justifies the need for the study; it allows the reader to understand why the research needs to be conducted.
The authors demonstrate that research on the topic is scarce, especially for the target population. Additionally, little is known about the factors that mediate or moderate distance learning and whether outcomes would be different for different socioeconomic groups. Third, even though research on the topic is limited, the findings are inconsistent. One view holds that distance learning is superior to face-to-face learning, while the other claims that it has negative consequences. This perspective is noteworthy, especially given that the studies supporting it were conducted within the last two years. Using the most recent data reveals where the industry currently stands on distance learning’s effectiveness and feasibility.
Finally, the research findings that focused on the authors’ target population cannot be generalized to the group. The authors describe some previous studies’ methods to help readers understand the need for the research. The authors note the survey specifically focused on their target population and involved participants aged between 18 and 50 years with IQ scores between 95 and 120; hence, it cannot be generalized. The study’s sample included younger students aged 5 to 22 years with intellectual disabilities.
The literature review primarily focused on distance learning effectiveness; however, little attention was given to the authors’ second objective: determining distance learning’s impact on students’ IEP. Out of the 36 studies in the literature review, only two studies focused on the effect of distance learning on IEP. The author’s second objective is equally as important as the first, and the authors should have given it just as much attention.
The authors note that most studies on IEP focused on students taking mainstream classes and very few on distance learning. The evidence to support this finding is supported by a systematic review that analyzed six other empirical studies. Additionally, another study with a large sample size (2600 students with learning disabilities) affirmed the authors’ notion. Thus, they demonstrated why they needed to determine the impact of distance learning on students’ IEP. Based on these factors, the scholars highlighted the need for more research needs to ascertain distance learning effectiveness and feasibility on the target population.
Concept Definition
The central concept, distance learning, has been clearly defined and described. Distance learning (DL) is defined as the teaching method where the teacher and student are separated physically. DL involves using audio, video, the internet, computers to deliver an education program, with at least 80% of the curriculum’s content is conveyed online. The other concepts, developmental disabilities and behavioral challenges, have not been adequately defined. The authors state that developmental disabilities are any conditions that are caused by cognitive or physical impairment. They give examples of developmental disabilities and challenging behaviors to let readers know what kind of disabilities their study participants had. The study participants had been diagnosed with Autism spectrum disorder (ASD), intellectual disability (ID), and “other developmental, behavioral, and cognitive disabilities” (Tomaino et al., 2021, p.4) It is not clear what conditions constitute “other developmental, behavioral, and cognitive disabilities” (Tomiano et al., 2021, p.4). The challenging behaviors included physical aggressiveness and self-injurious behavior.
Study Significance
This study hopes to make families, educators, and students understand distance learning’s impact on the target population. It contributes to the current literature base by providing parents’ and educators’ perspectives on distance learning’s feasibility and effectiveness.
The Comprehensiveness and Thoroughness of the Literature Review
The introduction and literature review are well-written and indicate the need for the research. All articles were relevant, and most were primary empirical research, validating the research problem and need for the study. The literature review’s main weakness is that it does not clearly define the research questions. According to Fandino (2019), a poorly-formulated research question may compromise the study’s quality and its results. Research questions help guide the choice of study design, target population, and what data needs to be collected and analyzed. Without it, one risks choosing a poor research design, which could affect the study’s chance of having any scientific relevance.
Purpose Statement Evaluation
The article clearly states the purpose statement: the purpose of study 1 was to determine distance learning’s social validity for students DDBN. The purpose statements are based on the arguments outlined in the literature review. The authors’ central view is that there is limited research on the topic, and even then, the findings have been inconsistent. The study hopes to help educators, families, and parents understand the true impact and effectiveness of distance learning on the target population. From this perspective, the authors purposed to determine the DL’s feasibility and appropriateness and the effect of transitioning to DL on students’ IEP.
Methods Section Evaluation
Research Design
The study conducted surveys that were appropriate for the purpose presented in the introduction. The authors used surveys to gauge teachers’ and parents’ opinions of DL’s impact and effectiveness on students with DDB. Although surveys are typically suited for descriptive studies, they can also be applied in studies that attempt to explore specific aspects of a situation. The survey was appropriate because the authors wanted to explore DL’s effectiveness and feasibility based on teachers’ and parents’ perspectives. Therefore, the used research design was suitable for the study. According to the hierarchy of evidence, this study’s findings are ranked as level VI evidence (Evidence-Based Practice Toolkit: Levels of Evidence, 2021). A qualitative study’s findings are level VI evidence, meaning that the confidence in its estimated effects is moderate to low.
Sampling Method
The sample was appropriate for the target population as members were selected from a program that offers specialized education to that population. The authors purposively selected the NPS program because it fits the study purpose. It runs two campuses in two different California counties and offers education, including distance learning to DDBN children. The authors issued questionnaires to all parents and educators in the NPS program. An inclusion and exclusion criteria were used to determine which participants to include in the study.
The study identifies the sample population demographics, i.e., age, educational level, enrolment duration, etc. The sample size was large but not sufficient; it included 110 members- 40 parents, 11 teachers, and 59 paraeducators. The authors note that the sample size is not fully representative of the target population. Only one parent, teacher, and paraeducators from each NPS class was selected. The authors acknowledge the sampling selection method’s biases, especially in the inclusion and exclusion criteria. Their exclusion criteria (parents whose children had low attendance and educators not older than three months) may have excluded eligible participants from contributing to the total DL effectiveness ratings. The authors used questionnaires with open and closed-ended questions to collect data. The study adequately described the protocol used in administering the instruments. The participants’ data was only included in the final analysis if they had responded to 90% of the questions. The authors ensured that only questionnaires with high response rates were included in the final analysis to reduce biases from low survey rates.
While quantitative researchers try to establish validity and reliability, qualitative researchers aim for trustworthiness, credibility (internal validity), dependability, and confirmability (authors’ objectivity). Noble and Smith (2015) assert that qualitative researchers can achieve these concepts by accounting for personal biases and acknowledging those made in the sampling method. The authors accounting for the bias in the sampling method improved their study’s trustworthiness.
Study’s Credibility
Although this study was well designed, it has a few limitations. The authors did not account for the questionnaires’ content validity as an online surveyor, Survey Monkey, developed them. Content validity in qualitative studies is usually done by experts who independently review the questionnaire items. According to Tenny and Sharts-Hopko (2019), ensuring content validity can improve a qualitative study’s credibility. The sampling technique, purposive sampling, is also problematic. Etikan et al. (2014) state that a purposive sampling study’s finding cannot be generalized to the larger population. Relying on the researcher’s judgment can negatively affect the study’s credibility.
Analytical Methods
The authors transcribed each participant’s responses and then coded them to identify common themes in the responses. The analytical test (coding) was appropriate because the study was qualitative. The authors verified the identified themes by interobserver agreement, which increases the study’s dependability (reliability). An interobserver agreement test can help the researcher determine to what extent two or more observers report the same values after measuring the same condition. A study conducted by Walter et al. (2019) showed that interobserver agreement (IOA) improves a study’s reliability and general quality. Although subject to methodological limitations, IOA validates and justifies the authors’ findings.
Results Evaluation
The authors clearly describe the results, and data has been presented in tables. The parent questionnaires show that parents had neutral perceptions of the DL feasibility and effectiveness as neutral. The three top themes about what the parents loved about distance learning include its structure, increased engagement in a child’s education, and improved socialization. The leading common themes about what the parents disliked were that DL increased the child’s challenging behaviors, negatively affected the child’s progress, and missed their friends and instructors.
The educators agreed that the DL platform was easy to use, their students used DL, and improved students’ educational benefits. The educators also strongly disagreed that their responsibilities made it difficult to instruct students on the platform. The top themes of what the educators liked about the program include increased staff collaboration and social contact. The common themes about what they disliked include reduced social interaction and nonattendance. This information has been well presented in tables, with text links to help the reader refer back to the original data. In the results section, the authors only report the findings. The description and illustration of the data are done in the discussion section.
Discussion
The authors began the discussion section by restating the study’s purpose and how they approached the problem. They concluded that both parents had neutral perspectives of DL effectiveness while educators felt the program improved the students’ learning outcomes. Furthermore, the educators revealed that the students maintain their learning goals, and some made progress on newly-developed goals. The authors use two past studies to highlight the importance of this finding. According to the authors’ studies, students with DDBN always show deterioration in skills when their routine is interrupted. However, in this case, the students showed improved learning outcomes based on the educators’ perception.
Another key finding of this study is that both parents and educators believed DL improved social skills. According to the authors, social skills are an essential consideration and a justification for implementing a DL program for this specific population. The authors use at least four studies to demonstrate how critical social skills are in a child’s development. The studies cited showed that social skills could positively influence academic achievement, social acceptance, and specific development areas. Because DL can instil in these students some social skills, the authors recommend that it be considered in the educational realm.
The authors postulate that parents’ neutral opinions on the programs’ feasibility and effectiveness were because DL required some parental support level to work. Both parents and educators agreed that the students needed more parental assistance during the program. They suggest that parents who can afford to sit with their children throughout the day rated the program favorably, while those who couldn’t afford such involvement rated it negatively. Interestingly, the study demonstrated that educators did not consider their personal lives an issue even though the program required the teachers to be accessible for at least six hours a day.
The researchers integrate the literature review into the discussion section to explain how these findings support existing literature. The literature review is used to guide the disagreements and agreements made between their conclusions and other authors’ previous findings. However, from a personal perspective, the integration of the literature view in the discussion section is inadequate. The authors only use a few studies to support their claims, and most of them are older than 20 years.
Limitations and Their Implications of the Study
The study adequately identifies its limitations, especially in its sample selection method. They recognize that the sample selection is not representative of students with developmental disabilities and behavioral needs. Therefore, its findings cannot be generalized to children with DDBN. The authors also note that because the data was collected during the COVID-19 pandemic, it is highly likely that it influenced the participants’ responses. The pandemic might have altered the respondents’ perceptions of the program. The authors recommend further research in determining what variables can improve parents’ response towards distance learning. The authors also recommend further research into the impact of distance learning, especially its effects on challenging behaviors. An experimental study that directly compares face-to-face education and distance learning is also needed.
Other Questions
The qualitative study method’s aspect that I find challenging is the methods used in analyzing the data. While the quantitative study has a standard analysis method, it seems that each research warrants its analytical approach in qualitative research. For example, some authors use a cross-case analysis and data triangulation method for a multi-case study design. In this case, the authors used the coding and interobserver agreement to analyze data and determine credibility. I found and addressed all the information on the provided list.
Conclusion
This study was well conducted, designed, and is also logically organized. The authors set out to determine DL’s feasibility and effectiveness on students with DDBN. The results revealed that DL improved the target population’s social skills and should be seriously considered given the wide-ranging benefits of social skills. However, these results cannot be generalized to the larger population because the sample selection method is biased. Additionally, the authors do not account for the instruments’ content validity, further invalidating its credibility. As for the results, the authors ensured its reliability by conducting the interobserver agreement. Therefore, the study is scientifically sound, but its evidence quality is moderate to low based on the evidence hierarchy.
References
Etikan, I., Musa, S. A., & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1–4. Web.
Evidence based practice toolkit: Levels of evidence. (2021). Winona State University. Web.
Fandino, W. (2019). Formulating a good research question: Pearls and pitfalls. Indian Journal of Anaesthesia, 63(8), 611–616. Web.
Noble, H., & Smith, J. (2015). Issues of validity and reliability in qualitative research. BMJ Evidence Based Nursing, 18(2), 34–35. Web.
Tenny, S., & Sharts-Hopko, N. C. (2019). Qualitative study. StatPearls Publishing.
Tomaino, M. A. E., Greenberg, A. L., Kagawa-Purohit, S. A., Doering, S. A., & Miguel, E. S. (2021). An assessment of the feasibility and effectiveness of distance learning for students with severe developmental disabilities and high behavioral needs. Behavior Analysis in Practice, 2021, 1–17. Web.
Walter, S. R., Dunsmuir, W. T. M., & Westbrook, J. I. (2019). Inter-observer agreement and reliability assessment for observational studies of clinical work. Journal of Biomedical Informatics, 100, 1–8. Web.