In their study, Jones, Hampshire, and McDonnell (2020) examined the issue of how teacher education programs are preparing pre-service ECSE teachers to collaborate with families and develop self-efficacy to do so. The researchers used quantitative methods by creating and administering a national survey administered to pre-service ECSE teachers near the finale of their teacher education programs. Besides aims to identify the levels of efficacy among the target group to build relationships with students and their parents as well as the capacity of teacher education programs to prepare them for collaboration with families. In addition, the research investigated the types of experiences that teachers of ECSE students had in their training programs related to partnering with families.
The sample population consisted of university students aged 18 and older, including those involved in an ECSE teacher education program at an IHE within the United States. To distribute the survey among the participants, a link to an online survey was emailed to program coordinators or department chairs at 195 IHEs nationwide. Among the 163 surveys that were opened, 103 were completed and returned to the scholars. This shows that the response rate was 63%, and it could have been higher if the researchers were more persistent in sampling participants. Because the researchers chose to use the quantitative method of data collection, several issues must be addressed.
Specifically, survey research represents the most fundamental tool for all quantitative outcome methodologies and studies. To help answer the established research questions, Jones et al. (2020) used a survey to ask questions to a sample of participants using an online method. Throughout the recent development and advancement of research methodologies, survey studies have evolved into a rigorous approach to research, with scientifically-tested strategies that detail who to include, what and how to distribute, and when to implement a survey and follow-up procedures with non-responders, to ensure a high-quality research process and outcome.
In their survey research, Jones et al. (2020) carried out an online survey with 24 items, which is a standard methodology for quantitative studies. However, the sampling procedures implemented in the survey are questionable, especially because there was no random selection of participants. For example, the primary investigator sent out the surveys to department chairs and program coordinators, who were later responsible for disseminating them among student teachers in their departments. Such a data collection method is referred to as convenience sampling, which is implemented when researchers cannot afford to survey every member of the population.
Convenience sampling is a subset of non-probability sampling, which are methods that do not utilize some form of random selection (Jager et al., 2017). In convenience sampling, participants are chosen in an ad hoc manner based on their accessibility or proximity to the researcher. Such a sampling technique is advantageous because it is inexpensive, efficient, and simple to implement. The main disadvantage of convenience sampling is that it lacks generalizability.
While probability samples in research yield results with clearer generalizability, they are more expensive and less efficient, which is why Jones et al. (2020) chose to focus on a non-probability technique. Besides, as suggested by Jager et al. (2017), convenience samples have become the norm within developmental and social sciences, with such a method being over 16 times more likely to be used than probability samples.
However, convenience sampling is flawed because the samples it offers do not produce representative results, which creates issues in reliability and validity. Also, there is a natural tendency of researchers to extrapolate from convenience samples and treat results as representative despite the fact that they are not. Besides, the results of convenience samples are complicated to replicate in future research. If scholars analyze the results of a convenience-based survey by list source, they will often find significant differences in the answers from different lists, in ways that confound easy explanation.
Therefore, Jones et al. (2020) should have considered the fact that non-probability sampling, and the convenience method especially, would not provide representative results. Even though an online survey is highly valuable for collecting quantitative data, the fact that the participants were chosen non-randomly presents a challenge to validity and reliability.
The poor generalizability of convenience sampling can also lead to estimate bias and issues of internal validation. Because the generalizability of convenience samples is not clear, the estimates that researchers derive from them can often be biased, which means that they do not reflect the actual effects among the target population because the sample inadequately represents its characteristics (Jager et al., 2017). Such bias extends to the estimates of population effects along with the estimates related to subpopulation differences. These effects can be illustrated with the help of outlining the known parameters of the population for a specific association between variables and comparing the known parameters to estimates obtained from convenience samples.
When it comes to the use of chi-square statistics, it is commonly used for testing connections between categorical variables. The chi-square test’s null hypothesis is that there is no relationship between categorical variables in the population, suggesting that they are independent of one another. Jones et al.’s (2020) research questions can be answered with the help of the chi-square test because they aim to find the connection between the outcomes of ESCE teacher education and their capacity to collaborate with students and parents for reaching the desired results in the education of students and the effectiveness of teachers.
The statistic is often used for evaluating Tests of Independence when using a crosstabulation (a bivariate table). Crosstabulation presents the distribution of two categorical variables at the same time, with the intersections of the variable categories appearing in the cells of the table (von Eye & Wiedermann, 2017). The Test of Independence is necessary for assessing whether there is an association between the two variables through comparing the observed response pattern in the cells to the pattern that would be expected if the variables had true levels of independence from one another.
Thus, making a calculation of the chi-square statistic and comparing it against a critical value from the distribution allows for an assessment of whether the observed cell counts are significantly different from the expected results. However, the chi-square test was not addressed by Jones et al. (2020), which is another limitation of the study.
The researchers should have carried out the chi-squared test because the sample size was only 103 completed surveys, and chi-square is sensitive to sample size, not allowing for reliable results if they are higher than 500 respondents. Besides, chi-square statistics can be sensitive to the distribution within the cells, with the SPSS usually warning users if cells have fewer than five cases. This is possible for addressing by always using categorical variables with a lower number of categories. For example, researchers can combine categories if necessary for producing a smaller table.
In their study, Jones et al. (2020) found that the respondents had high levels of self-efficacy when collaborating with families and that they had adequate preparation with the help of teacher education programs. In addition, student teachers stated that they had lower levels of self-efficacy when working with families whose language was different from their own. The research presented several limitations; for example, it remains unclear whether ESCE student teacher education programs focused on collaborating with families as a whole or on students separately. Also, the researchers did not determine if the survey respondents participated in a teacher education program, which placed more emphasis on the work with families.
Therefore, in the future, the study could be modified in several ways, with new research questions added. For instance, Jones et al. (2020) proposed to measure the competence of student teachers when building partnerships with families using more objective measures than self-reports, such as rating scales completed by student teachers’ supervisors or cooperating teachers. Such results can inform teacher educators about how well their programs are preparing ECSE student teachers to collaborate with their families.
However, a new study must consider using a non-probability sampling technique to ensure the representativeness of the sample, which means that convenience sampling cannot be used again and risk the reliability and validity of results. Besides, it is possible to conduct a mixed-methods study to reap the benefits of both qualitative and quantitative methodologies.
References
Jager, J., Putnick, D. L., & Bornstein, M. H. (2017). More than just convenient: The scientific merits of homogeneous convenience samples. Monographs of the Society for Research in Child Development, 82(2), 13-30. Web.
Jones, J., Hampshire, P., & McDonnell, A. (2020). Authentically preparing early childhood special education teachers to partner with families. Early Childhood Education Journal, 48, 767-779. Web.
von Eye, A., & Wiedermann, W. (2017). Direction of effects in categorical variables: Looking inside the table. Journal for Person-Oriented Research, 3(1), 11-27. Web.