Addressing The Issues of Low Degrees of Freedom and ‎Associated Poor Validation Estimates of Science Education Models: An Advocacy for The Application of Confirmatory ‎Network Analysis

  • Authors

    • Syamala Pilla School of Education, Lovely Professional University, Phagwara, Punjab, India https://orcid.org/0009-0007-4779-0969
    • Rajib Chakraborty School of Education, Lovely Professional University, Phagwara, Punjab, India
    https://doi.org/10.14419/k92yd937

    Received date: August 30, 2025

    Accepted date: September 27, 2025

    Published date: October 8, 2025

  • Bland-Altman Plots; Confirmatory Network Analysis; Confirmatory Factor Analysis; Degree of Freedom; Science Education Models

  • Abstract

    Generally, science education models have a low degree of freedom df, and this psychometric aspect negatively impacts the overall ‎goodness of fit estimates obtained on conducting CB-SEM-based Confirmatory Factor Analysis (CFA) on the hypothesized latent ‎models using the Maximum Likelihood (ML) estimator. To address this issue, the state-of-the-art Confirmatory Network Analysis ‎‎(CNA) approach is advocated, which uses the adjacency matrix in estimation and treats the data type of the data collected using ‎survey questionnaires as ordinal using the appropriate Diagonally Weighted Least Squares (DWLS) estimator to provide the proper ‎scaled goodness of fit estimates of the network. In the present study, the goodness of fit estimates of a science education model ‎involving the perception of 918 secondary school students of India on their science constructivist learning environment, ‎epistemological beliefs in science, science self-efficacy, science academic flow, and science academic achievement were found from ‎the traditional latent variable modeling and the latest network perspectives. Data analysis was conducted using SPSS AMOS Ver ‎‎23.0, several packages of R and through generated using ChatGPT Ver. 5. The scaled goodness of fit estimates of confirmatory ‎network analysis produced a better picture of the reality over confirmatory factor analysis, and provided deeper insights on the ‎interactions among the studied variables as were graphically established by the differences in the Bland-Altman plots of the CFI, ‎TLI, RMSEA and SRMR estimates of the two approaches. Educational and psychometric implications of the study are discussed‎.

  • References

    1. Altman, D. G., & Bland, J. M. (1983). Measurement in Medicine: The Analysis of Method Comparison Studies. Journal of the Royal Statistical So-ciety. Series D (The Statistician), 32(3), 307–317. https://doi.org/10.2307/2987937.
    2. Arslantas, H.A. (2016). Epistemological Beliefs and Academic Achievement. Journal of Education and Training Studies, 4(1). 215-220. https://doi.org/10.11114/jets.v4i1.1107.
    3. Arbuckle, J. L. (2006). Amos (Version 7.0) [Computer Program]. Chicago: SPSS.
    4. Arbuckle, J. L. (2014). Amos (Version 23.0) [Computer Program]. Chicago, IL: IBM SPSS, Inc
    5. Borsboom, D. (2008). Psychometric perspectives on diagnostic systems. Journal of Clinical Psychology, 64, 1089–1108. https://doi.org/10.1002/jclp.20503.
    6. Bybee, R. W., (2015). The BSCS 5E instructional model: creating teachable moments. Arlington, Virginia :NSTA Press.
    7. Burger, J., Andikkhash, V., Jäger, N., Anderbro, T., Blanken, T. F., & Klintwall, L. (2024). A novel approach for constructing personalized net-works from longitudinal perceived causal relations. Behaviour Research and Therapy, 173, 104456. https://doi.org/10.1016/j.brat.2023.104456.
    8. Cortes, S.T., Lorca, A.S., Pineda, H.A., Tubog, R., & Vilbar, A. (2024). Strengthening science education in basic education through a professional development program on participatory action research for science teachers, Social Sciences and Humanities Open, 10, https://doi.org/10.1016/j.ssaho.2024.101194.
    9. Christensen, A. P., Golino, H., & Silvia, P.J. (2020). A psychometric network perspective on the validity and validation of personality trait ques-tionnaires. European Journal of Personality. https://doi.org/10.1002/per.2265.
    10. Christensen, A.P., Golino, H. (2021). On the equivalency of factor and network loadings. Behavior Research Methods. 53, 1563–1580. https://doi.org/10.3758/s13428-020-01500-6.
    11. Cho, G., M. Sarstedt, & H. Hwang. (2022). A Comparative Evaluation of Factor- and Component-based Structural Equation Modelling Approach-es Under (In) Correct Construct Representations. British Journal of Mathematical and Statistical Psychology, 75 (2):220–251. https://doi.org/10.1111/bmsp.12255.
    12. Cramer, A. O. J. (2012). Why the item “23+ 1” is not in a depression questionnaire: Validity from a network perspective. Measurement: Interdisci-plinary Research & Perspective, 10, 50–54. https://doi.org/10.1080/15366367.2012.681973.
    13. Conley, A. M., Pintrich, P. R., Vekiri, I., & Harrison, D. (2004). 26-item Epistemological Beliefs Measure [Database record]. APA PsycTests. https://doi.org/10.1037/t15211-000.
    14. Du, X., Skjerdingstad, N., Freichel, R., Ebrahimi, O. V., Hoekstra, R. H. A., & Epskamp, S. (2025). Moving from exploratory to confirmatory net-work analysis: An evaluation of structural equation modeling fit indices and cutoff values in network psychometrics. Psychological Methods. Ad-vance online publication. https://doi.org/10.1037/met0000760.
    15. DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). Thousand Oaks, CA: SAGE Publications.
    16. Epskamp, S., Maris, G., Waldorp, L. J., & Borsboom, D. (2018). Network psychometrics. In P. Irwing, T. Booth, & D. J. Hughes (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 953–986). Wiley Blackwell. https://doi.org/10.1002/9781118489772.ch30.
    17. Epskamp S (2025). psychonetrics: Structural Equation Modeling and Confirmatory Network Analysis, R Package Version 0.7.1. Available online: https://cran.r-project.org/web/packages/psychonetrics/index.html.
    18. Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23, 617–634. https://doi.org/10.1037/met0000167.
    19. Epskamp S, Borsboom D, Fried EI (2018). “Estimating Psychological Networks and their Accuracy: A Tutorial Paper.” Behavior Research Meth-ods, 50, 195–212. https://doi.org/10.3758/s13428-017-0862-1.
    20. Epskamp, S., Cramer, A. O. J., Waldorp, L. J., Schmittmann, V. D., & Borsboom, D. (2012). qgraph: Network Visualizations of Relationships in Psychometric Data. Journal of Statistical Software, 48(4), 1–18. https://doi.org/10.18637/jss.v048.i04.
    21. Fan, X., & Sivo, S. A. (2007). Sensitivity of fit indices to model misspecification and model types. Multivariate Behavioral Research, 42(3), 509–529. https://doi.org/10.1080/00273170701382864.
    22. Golino, H., & Epskamp, S. (2017). Exploratory Graph analysis: A new approach for estimating the number of dimensions in psychological research. PloS One, 12, e0174035. https://doi.org/10.1371/journal.pone.0174035.
    23. Glynn, S.M., & Koballa, T. R., Jr. (2006). Motivation to learn college science. In J. J. Mintzes & W. H. Leonard (Eds.), Handbook of college sci-ence teaching (pp. 25-32). Arlington, VA: National Science Teachers Association Press.
    24. Giavarina, D. (2015). Understanding Bland Altman analysis. Biochemia Medica (Zagreb). 25(2):141-51. https://doi.org/10.11613/BM.2015.015.
    25. Heutte, J., Fenouillet, F., Kaplan, J., Martin-Krumm, C. and Bachelet, R. (2016). The EduFlow model – A contribution toward the study of optimal learning environments. In L. Harmat, F. Ø. Andersen, F. Ullén, J. Wright and G. Sadlo (dir.). Flow Experience: Empirical Research and Applica-tions (p. 124-143). Cham, Suisse: Springer. https://doi.org/10.1007/978-3-319-28634-1_9.
    26. Honicke, T., Broadbent, J., & Fuller-Tyszkiewicz, M. (2023). The self-efficacy and academic performance reciprocal relationship: the influence of task difficulty and baseline achievement on learner trajectory. Higher Education Research & Development, 42(8), 1936–1953. https://doi.org/10.1080/07294360.2023.2197194.
    27. Haslbeck, J.M.B., & Waldorp, L J. (2017). How well do network models predict observations? On the importance of predictability in network models. Behavior Research Methods. 50(2):853-861. https://doi.org/10.3758/s13428-017-0910-x.
    28. Hair, J.F., Babin, B.J., Ringle, C.M., Sarstedt, M., & Becker, J. (2025). Covariance based structural equation modeling (CB SEM): a SmartPLS 4 software tutorial, Journal of Marketing Analytics, https://doi.org/10.1057/s41270-025-00414-6.
    29. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struc-tural Equation Modeling, 6(1), 1–55. https://doi.org/10.1080/10705519909540118.
    30. Hallquist, M., Wright, A. C. G., & Molenaar, P. C. M. (2019). Problems with centrality measures in psychopathology symptom networks: Why network psychometrics cannot escape psychometric theory. Multivariate Behavioral Research. https://doi.org/10.1080/00273171.2019.1640103.
    31. Jöreskog, K. G. (1978). Structural Analysis of Covariance and Correlation Matrices. Psychometrika, 43 (4): 443–477. https://doi.org/10.1007/BF02293808.
    32. Jöreskog, K. G., & Wold, H. (1982). The ML and PLS techniques for modeling with latent variables: Historical and comparative aspects. In H. Wold & K. G. Jöreskog (Eds.), Systems under indirect observation, part I (pp. 263–270). Amsterdam: North-Holland.
    33. Jöreskog, K. G. (1993). Testing Structural Equation Models. In Testing Structural Equation Models, ed. K. A. Bollen and J. S. Long, 294–316. Thousand Oaks: Sage.
    34. Jia,F., Meng, J., Ma, Y., & Mao,Y. (2024). Flow experience and self-efficacy in undergraduates’ English learning: A weekly diary investigation with cross-lagged panel modeling approach, System, 123, 103312, ISSN 0346-251X, https://doi.org/10.1016/j.system.2024.103312.
    35. Kline, M.S. (2013). Application of Structural Equation Modeling in Educational Research and Practice, Springer. https://doi.org/10.1007/978-94-6209-332-4.
    36. Kline, R. B. (2015). Principles and Practice of Structural Equation Modeling. New York: Guilford Publications.
    37. Kan, K.J., de Jonge, H., van der Maas, H.L.J., Levine, S.Z., & Epskamp, S. (2020). How to Compare Psychometric Factor and Network Models. Journal of Intelligence, 8(4):35. https://doi.org/10.3390/jintelligence8040035.
    38. Kazu, I.Y. & Yalcin, C.K. (2021). The Effect of Stem Education on Academic Performance: A Meta-Analysis Study, TOJET: The Turkish Online Journal of Educational Technology, 20(4), 101-116.
    39. Kim, J. (2005). The Effects of a Constructivist Teaching Approach on Student Academic Achievement, Self-concept, and Learning Strategies, Asia Pacific Education Review, 6(1)., 7-19. https://doi.org/10.1007/BF03024963.
    40. Klintwall, L., Bellander, M., & Cervin, M. (2023). Perceived causal problem networks: Reliability, central problems, and clinical utility for depres-sion. Assessment, 30(1), 73–83. https://doi.org/10.1177/10731911211039281.
    41. Kline, R. B., & Little, T. D. (2023). Principles and practice of structural equation modeling (Fifth edition). The Guilford Press.
    42. Li, CH. (2021). Statistical estimation of structural equation models with a mixture of continuous and categorical observed variables. Behavior Re-search Methods. 53, 2191–2213. https://doi.org/10.3758/s13428-021-01547-z.
    43. Lehnert, B. (2015). BlandAltmanLeh: Plots (Slightly Extended) Bland-Altman Plots. R package version 0.3.1, https://CRAN.R-project.org/package=BlandAltmanLeh.
    44. McLure, F., Won, M., & Treagust, D. F. (2024). Science teachers’ understanding of creative thinking and how to foster it as mandated by the Aus-tralian Curriculum. Journal of Science Teacher Education. https://doi.org/10.1080/1046560X.2024.2313882.
    45. Marsman, M., Borsboom, D., Kruis, J., Epskamp, S., van Bork, R., Waldorp, L. J., & Maris, G. (2018). An introduction to network psychometrics: Relating Ising network models to item response theory models. Multivariate Behavioral Research, 53, 15–35. https://doi.org/10.1080/00273171.2017.1379379.
    46. Mindrila, M. (2010). Maximum Likelihood (ML) and Diagonally Weighted Least Squares (DWLS) Estimation Procedures: A Comparison of Esti-mation Bias with Ordinal and Multivariate Non-Normal Data, International Journal of Digital Society (IJDS), 1(1), 60-66. https://doi.org/10.20533/ijds.2040.2570.2010.0010.
    47. Nix, R.K., Fraser, B.J. & Ledbetter, C.E. (2005). Evaluating an Integrated Science Learning Environment Using the Constructivist Learning Envi-ronment Survey. Learning Environments Research. 8, 109–133. https://doi.org/10.1007/s10984-005-7251-x.
    48. Pitt, M. A., Myung, I. J., & Zhang, S. (2002). Toward a method of selecting among computational models of cognition. Psychological Review, 109(3), 472–491. https://doi.org/10.1037/0033-295X.109.3.472.
    49. Pedersen, T. (2025). patchwork: The Composer of Plots. R package version 1.3.2.9000, https://patchwork.data-imaginist.com.
    50. Popper, K. R. (1959). Logic of scientific discovery. London : Hutchinson. https://doi.org/10.1063/1.3060577.
    51. R Core Team (2020). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/.
    52. Rosseel, Y. (2012). Lavaan: An R Package for Structural Equation Modeling. Journal of Statistical Software 48: 1–36. https://doi.org/10.18637/jss.v048.i02.
    53. Shi, D., DiStefano, C., Maydeu-Olivares, A., & Lee, T. (2021). Evaluating SEM Model Fit with Small Degrees of Freedom. Multivariate Behavior-al Research, 57(2–3), 179–207. https://doi.org/10.1080/00273171.2020.1868965.
    54. Shi D, Lee T, Maydeu-Olivares A. (2018). Understanding the Model Size Effect on SEM Fit Indices. Educational and Psychological Measurement. 79(2):310-334. https://doi.org/10.1177/0013164418783530.
    55. Schmittmann, V. D., Cramer, A. O. J., Waldorp, L. J., Epskamp, S., Kievit, R. A., & Borsboom, D. (2013). Deconstructing the construct: A net-work perspective on psychological phenomena. New Ideas in Psychology, 31(1), 43–53. https://doi.org/10.1016/j.newideapsych.2011.02.007.
    56. Sadi, O. & Dagyar, M. (2015). High School Students’ Epistemological Beliefs, Conceptions of Learning, and Self-efficacy for Learning Biology: A Study of Their Structural Models, Eurasia Journal of Mathematics, Science & Technology Education, 11(5), 1061-1079. https://doi.org/10.12973/eurasia.2015.1375a.
    57. Taasoobshirazi, G., & Wang, S. (2016). The performance of the SRMR, RMSEA, CFI, and TLI: An examination of sample size, path size, and de-grees of freedom. Journal of Applied Quantitative Methods, 11(3), 31- 39.
    58. Ullman, J.B. (2006) Structural Equation Modeling: Reviewing the Basics and Moving Forward, Journal of Personality Assessment, 87:1, 35-50, https://doi.org/10.1207/s15327752jpa8701_03.
    59. Van Der Maas, H. L. J., Dolan, C. V., Grasman, R. P. P. P., Wicherts, J. M., Huizenga, H. M., & Raijmakers, M. E. J. (2006). A dynamical model of general intelligence: The positive manifold of intelligence by mutualism. Psychological Review, 113(4), 842–861. https://doi.org/10.1037/0033-295X.113.4.842.
    60. Vandekerckhove, J., Matzke, D., & Wagenmakers, E.-J. (2015). Model comparison and the principle of parsimony. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels (Eds.), Oxford library of psychology. The Oxford handbook of computational and mathematical psychology (p. 300–319). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780199957996.013.14.
    61. van Bork, R., Rhemtulla, M., Waldorp, L. J., Kruis, J., Rezvanifar, S., & Borsboom, D. (2019). Latent variable models and networks: Statistical equivalence and testability. Multivariate Behavioral Research, pp. 1–24. https://doi.org/10.1080/00273171.2019.1672515.
    62. Wickham H (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. ISBN 978-3-319-24277-4, https://ggplot2.tidyverse.org.
    63. Xia, Y., & Yang, Y. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. https://doi.org/10.3758/s13428-018-1055-2.

  • Downloads

  • How to Cite

    Pilla, S., & Chakraborty, R. (2025). Addressing The Issues of Low Degrees of Freedom and ‎Associated Poor Validation Estimates of Science Education Models: An Advocacy for The Application of Confirmatory ‎Network Analysis. International Journal of Basic and Applied Sciences, 14(6), 152-159. https://doi.org/10.14419/k92yd937