Beyond Threaded Discussions: Learning with Knowledge Building Analytics
December 10, 2020
Keywords: Education, Feedback, Writing, Writing Analytics
Target readers: Educators; Learning Technologist; Learning Scientists; Instructional Designers; Researchers
Author: Alwyn Vwen Yen Lee
Position: Research Fellow, National Institute of Education, Nanyang Technological University, Singapore
Author: Teo Chew Lee
Position: Senior Research Scientist, Programme Director (Learning Sciences & Innovation), National Institute of Education, Nanyang Technological University, Singapore
Beyond Threaded Discussions: Learning with Knowledge Building Analytics
We need better analytics for student discourse
Online discussion systems are a standard feature in learning platforms. They’re used for a range of purposes, including deepening student’s engagement with curriculum concepts, team communication in project investigations, and community building. However, such tools provide no support to help students understand how to make better contributions for knowledge building, and they provide very limited forms of analytics for educators to understand the quality of the conversation. Number of forum posts, replies, and wordcount may be useful for site administrators, and provide a very rough indicator of “student engagement” for teachers — but could we provide more insights into the quality of contributions, that would be beneficial for both teachers and students?
We’ve studied analytics in forums and are convinced we need analytics that are more meaningful and impactful in improving the quality of the discourse. Here are 3 reasons:
- Teaching and assessment remain heavily dependent on visual cues during face-to-face interactions but with online instruction necessitated by emergent situations (e.g., the pandemic), we can no longer rely on such cues. Learning analytics can help fill this gap and provide greater insights into student discourse.
- Students’ development of important 21st century competencies such as collaboration and communication skills may still be captured if teachers choose to conduct online discussions with students. With learning analytics, teachers can also remotely assess these attributes and students’ development.
- Apart from using online discussions to achieve learning objectives, students also engage in problem-sharing and idea-building. Discourse-centric learning analytics can play a greater role in providing insights into these processes for meaningful learning and knowledge construction.
Knowledge Building
Knowledge Building is a long-term process that equips communities of students to create knowledge in a way we can capture. Knowledge building is guided by the 12 principles of knowledge building and is a collaborative effort directed towards the creation and improvement of theories, designs, and ideas for problem solving.
Knowledge Forum web platform
The principles underlying the theory and pedagogy of knowledge building have been realized in a well-designed online platform, Knowledge Forum, purpose-built to support knowledge building in school age students, through synchronous and asynchronous discourse. Knowledge Forum provides students with scaffolds for their online contributions, such as "I need to understand” and “This idea doesn’t explain”. These sentence-starters help students to think through what and how they want to build on and share their ideas with their peers.
For example, this screenshot shows how students who have difficulty in starting off a note on the Knowledge Forum can choose one of the scaffolds. They’ve chosen Theory Building and “I need to understand” as the opening to their question.
In contrast to traditional forum posts that are undifferentiated in type and a simple linear sequence/branch, Knowledge Forum conversations are displayed as spatial networks to convey at a glance a sense of the context and depth of discussions. Some of the affordances from the Knowledge Forum are highlighted when we compare a single threaded discussion that can be found on most conventional forums and the potentially wider range of ideas and discussions that can be generated from a multi-threaded discussion.
Single threaded discussions on the Knowledge Forum may seem similar to other conventional forums’ discussions with a remark and response format, but students on the Knowledge Forum can continue building on each other’s ideas (represented with an arrow)
A seemingly contentious note is the centre of discussions from multiple authors with different build-ons and opinions leading to different kinds of responses and conclusions, with scaffolds (in angle brackets) being used by students as sentence-starters
In this example, primary school students were discussing whether a historical figure was a good or bad emperor. It is usually difficult to get students to discuss in-depth, especially when students are asked to continue online. The teacher in this lesson, however, captured conclusions from the classroom discussions and reposted them onto the Knowledge Forum (note the handwritten notes on the right of the ruler’s portrait), which students were able to reference and continue their exploration of the topic. This action helps to provide a seamless transition between offline and online discussions.
Knowledge Building Analytics
Knowledge Forum’s discourse structure and mechanisms for encouraging contributions (e.g., scaffolds and build-on) and consolidations (e.g., group summaries) are what distinguishes Knowledge Forum from other conventional platforms. A suite of novel Learning Analytics tools are integrated into Knowledge Forum to help teachers make sense of students’ work, e.g.
- how well are students resolving problems?
- where are the knowledge gaps in the learning community?
- how should this inform face-to-face lesson plans?
This barchart provides summary statistics of student activity, but goes beyond merely charting the number of posts. Now the teacher can see which scaffolds and buildons are being used, to inform interventions.
Curriculum-ideas-Analytics (CiA) show whether students’ contributions are mentioning syllabus concepts within and beyond their grade levels. In Knowledge Forum, it facilitates teacher insight of student discourse in a visual manner. The CiA firstly mines a wide range of textual syllabuses to obtain multiple sets of syllabus-related keywords from various grades, and then compares these sets of keywords with those that are found in the student discourse. Depending on the set of keywords chosen for comparison, the final result is a word cloud containing keywords highlighted with different colors, indicating key curriculum words that students are currently using, have yet to use, or are using beyond their current grade levels. This has proved to be very useful for teachers who are looking for an automated method of gauging curriculum coverage and the potential of ideas from student discourse.
In this third example, Knowledge Building is connected to online documents and discourse through direct annotation. Hypothes.is an open-source web annotation tool, now integrated into the Knowledge Forum platform in the “Idea Magnet” tool. First, the student annotates a website:
Next, the annotations are connected into Knowledge Forum for further inquiry and discussion:
Finally, Idea Thread Mapper connects multiple students’ ideas across multiple discourse spaces on Knowledge Forum. Students are able to share discussions and create “wondering areas” as major directions of inquiry, based on shared interests and emergent problems. Within each wondering area, students can then focus on idea threads as a line of inquiry. As students build-on each other’s notes, the responses are visualized as lines that criss-cross between different students on a timeline. Each row is a student, and each dot is a note on the timline.
How can I use Knowledge Forum?
Although the Knowledge Forum is user-friendly, accessible, and provides a psychologically safe environment for students to build knowledge, its full potential can only be tapped if teachers have an understanding of the underlying knowledge building approach and pedagogy that guides its design.
If you’re new to knowledge building and the Knowledge Forum, we’d encourage you to join the community, and start off using the built-in analytics (e.g., note contribution frequency and scaffold trackers) to track student activity, and get a feel of how the accompanying learning analytics can assist both teachers and students on a trajectory of knowledge building. You can also check out our website, where we showcase communities of knowledge builders and their practice in actual classrooms! We have detailed information about knowledge building stories, learning experiences, and starter kits for teachers and researchers.
Once you’re familiar with the functionalities of Knowledge Forum, and as you develop a better idea of how the knowledge building approach fits into your lessons, you can then join other experienced users and analysts around the world to try out other more advanced kinds of analytics that are described in this article, such as the Idea Thread Mapper and Idea Magnets.
Learn More:
Chen, B., Chang, Y. H., & Groos, D. (2020). Bridging Public Discourse and Knowledge Building Discourse in Science Classrooms with the IdeaMagnets Tool. Proceedings of the 14th International Conference of the Learning Sciences (ICLS ‘2020), Nashville, Tennessee (pp. 863-864). International Society of the Learning Sciences. https://repository.isls.org//handle/1/6802
Hong, H. Y., Scardamalia, M., Messina, R., & Teo, C. L. (2015). Fostering sustained idea improvement with principle-based knowledge building analytic tools. Computers & Education, 89, 91-102. https://doi.org/10.1016/j.compedu.2015.08.012
Lee, A. V. Y., & Tan, S. C. (2017). Promising ideas for collective advancement of communal knowledge using temporal analytics and cluster analysis. Journal of Learning Analytics, 4(3), 76–101. http://dx.doi.org/10.18608/jla.2017.43.5
Teo, C. L., Chan, C., & Ng, D. (2018). Engaging teachers in discussions around temporality measures from analytics to inform knowledge building discourse. Proceedings of the 13th International Conference of the Learning Sciences (ICLS ’18), London, UK (pp. 1677-1678). International Society of the Learning Sciences. https://repository.isls.org//handle/1/799
Zhang, J., Tao, D., Chen, M. H., Sun, Y., Judson, D., & Naqvi, S. (2018). Co-organizing the collective journey of inquiry with idea thread mapper. Journal of the Learning Sciences, 27(3), 390-430.