Our research examines how cognitive tools, computational algorithms, and interactive media can improve learning outcomes and learner engagement. We aim to invent successful technologies, processes, and know-how that can be replicated on a wider scale. Our close partnership with the Institute of Cognitive Science (ICS) at the University of Colorado brings expertise and leading research in fields critical to educational innovation, including computer science, cognitive psychology, educational theory, and computational linguistics. In collaboration with ICS and other University partners, we engage in both use-inspired basic research and applied research. Use-inspired basic research extends the frontiers of knowledge in ways that are motivated by considerations of use and human or societal need.

Curriculum Customization Service (CCS)

The CCS is one of the tools developed by DLS and the University of Colorado Boulder through a joint partnership. The CCS is an online planning tool that transforms traditional print-based materials into concept-driven, interactive teachers’ guides. These online guides support curriculum planning and implementation, are customizable and extensible to meet local needs, and provide targeted online professional development. Through our research-practice partnerships with K12 school districts, the CCS projects work closely with teachers in the ongoing development of the CCS as well as with increasing teacher capacity for understanding and implementing standard aligned (e.g. Common Core State Standards for Mathematics and Next Generation Science Standards) curricular materials. Additionally, the current project, titled Inquiry Hub, is working to develop teachers understanding of the standards and how to implement the aligned materials into their classrooms.

Selected Publications about CCS:

Butcher, K., Leary, H., Foster, J., Devaul, H. (accepted). Adopting an Online Curriculum Planning Tool: Facilitation for teachers’ thinking about constructive pedagogy and technology integration. Journal of Technology and Teacher Education.

Leary, H., Lee, V.R., & Recker, M. (2014). More Than Just Plain Old Technology Adoption: Understanding Variations in Teachers’ Use of an Online Planning Tool. In Proceedings of the 11th International Conference of the Learning Sciences. International Society of the Learning Sciences.

Lee, V. R., Leary, H. M., Sellers, L., & Recker, M. (2014). The Role of School District Science Coordinators in the District-wide Appropriation of an Online Resource Discovery and Sharing Tool for Teachers. Journal of Science Education and Technology, 23(3), 309-323. doi: 10.1007/s10956-013-9465-5

Dibie, O., Maull, K., & Sumner, T. (2012). Teacher Sociality and Information Diffusion in Educational Digital Libraries. In Proceedings of the 12th ACM/IEEE-CS Joint Conference on Digital Libraries(JCDL ‘12), Washington, DC, 27-30, June 10-14. Maull, K. E., Saldivar, M. G., & Sumner, T. R. (2012). Automated Approaches to Characterizing Educational Digital Library Usage: Linking Computational Methodologies with Qualitative Analyses. International Journal of Digital Libraries, 13(1), 51-64.

Saldivar, M. G., Maull, K., Kirshner, B., & Sumner, T. (2012). A Two-dimensional Framework for Evaluating Teachers’ Technology Adoption. In M. Orey, S. A. Jones & R. M. Branch (Eds.), Educational Media and Technology Yearbook (Vol. 36, Part 1, pp. 49-65). New York, NY: Springer

Maull, K. E., Saldivar, M. G., & Sumner, T. (2011). Understanding Digital Library Adoption: A Use Diffusion Approach. In Proceedings of the 11th annual international ACM/IEEE joint conference on Digital Libraries, Ottawa, CA, June 13-17.

Sumner, T. (2010). Customizing Science Instruction with Educational Digital Libraries. Paper presented the In Proceedings of the 10th ACM/IEEE-CS Joint Conference on Digital Libraries(JCDL 2010), Surfer’s Paradise, Brisbane, Australia.(June 22-24)

Customized Learning Service for Concept Knowledge (CLICK)

A key educational finding from learning research is that every student brings preconceptions about how the world works to every learning situation, and that these initial understandings need to be explicitly targeted as part of an effective instructional process. This research, funded by the Advanced Learning Technologies program of the National Science Foundation, is designing and evaluating an end-to-end prototype of a “customized learning service for concept knowledge”. This software service will support learning environments to perform customizations based on a real-time analysis of what students understand about a particular topic. Learner-centered customizations are performed by algorithmically comparing a learner’s current conceptual understanding, depicted as a concept map, with a domain competency model generated automatically from selected digital library resources. These comparisons will enable learning environments to provide customized retrieval, delivery, and presentation of educational resources drawn from digital libraries. In computer science, this research is contributing to the development of multi-document summarization techniques capable of analyzing and synthesizing educational resources. In learning science, this research is contributing to our understanding of how common student misconceptions can be reliably identified by computer algorithms capable of analyzing student work.

Selected Publications about CLICK:

Ahmad, F., de la Chica, S., Butcher, K., Sumner, T. and Martin, J.H. Towards automatic conceptual personalization tools. In Proceedings of the 7th ACM/IEEE-CS Joint Conference on Digital Libraries, (Vancouver, British Columbia, Canada, 2007), accepted for publication. Download pdf

de la Chica, S. SciNews Online: Scaffolding the construction of scientific explanations. In Proceedings of the 2007 SIGCHI Conference on Human Factors in Computing Systems, Extended Abstracts - ACM Student Research Competition, (San Jose, California, 2007), 2183-2188. Download pdf

de la Chica, S., Ahmad, F., Martin, J.H. and Sumner, T.R. Supporting science understanding through a customized learning service for concept knowledge. In Proceedings of the Combined Workshop on Language-Enabled Educational Technology and Development and Evaluation of Robust Spoken Dialogue Systems, 17th European Conference on Artificial Intelligence, (Riva del Garda, Italy, 2006), 9-16. Download pdf

Developing a Computational Model of Quality for Digital Repositories (QUALITY)

Digital resource quality has emerged as a dominant yet poorly understood concern within digital library efforts and other content repositories, particularly those supporting community and user contributions. Evaluating resource quality involves making complex, time-consuming, and variable human judgments. Developing computational models of quality that approximate expert human judgments is a foundational requirement for developing interfaces and tools that can optimize and scaffold human judgments on quality. This research, funded by the Information and Intelligent Systems program at the National Science Foundation, is investigating the characteristics of digital resources that serve as key markers of quality for experts engaged in resource selection and collection curation, and determining how to model these markers computationally. Additionally, this research is assessing how machine learning and natural language processing techniques can be applied to recognize these markers with sufficient discrimination to approximate and scaffold effective human-decision making.

Selected Publications about QUALITY:

Custard, Myra and Tamara Sumner. (2005). Using machine learning to support quality judgements. D-Lib Magazine, 11(10). Download article

Educational Standards

Digital Learning Sciences is working with the Center for Natural Language Processing (CNLP) at Syracuse University to develop and evaluate tools and algorithms to support the automatic identification of relevant K-12 science and math education standards for a given digital learning resource. CNLP is developing algorithms that use natural language processing and machine learning techniques to analyze the content of a digital resource and suggest the most likely standards that are correlated with the content. Digital Learning Sciences is evaluating the performance of these algorithms and developing tools and interfaces to integrate them into cataloging and collection management workflows. This project is also creating algorithms for automatically aligning state educational standards to selected national standards to support state to state correlations. Outcomes from this project will improve both the ability of teachers to locate resources in the National Science Digital Library that support standards–based instruction and the ability of library builders to efficiently develop collections that support standards-based education.

Selected Publications about Educational Standards Alignment:

Devaul, Holly, Anne Diekema, and Jonathan Ostwald. (2007). Computer-Assisted Assignment of Educational Standards Using Natural Language Processing. Unpublished technical report. Boulder, CO.

Diekema, Anne and Holly Devaul. (2006). Computer Assisted Standard Assignment and Alignment. On NSDL ToolTime series [ReadyTalk demonstration]. Boulder, CO: National Science Digital Library (NSDL) retrieved April 14, 2006.

Devaul, Holly and Karon Kelly. (2003). Searching by educational standards in Digital Library for Earth System Education (DLESE): What does it mean and what do users want? (Poster presentation). 2003 National Science Digital Library Annual Meeting; National Science Digital Library (NSDL). Retrieved April 14, 2006.