Designing dashboards to help students take action

Ioana Jivet is a postdoctoral researcher at the Leiden-Delft-Erasmus Center for Education and Learning based at the Technical University of Delft while also finishing her PhD at the Faculty of Educational Sciences at the Open University of the Netherlands. Her research focuses on learner-centred learning analytics dashboard designs and culturally inclusive learning analytics.

Follow Ioana on Twitter

Using technology to support teaching and learning has one undeniable advantage over traditional classroom learning that happened 30 years ago: the possibility to collect fine-grained data about the learners. Usually, these data are a set of timestamped interactions of the learners with the learning environment. For example, these data could describe when and which video-lecture learners chose to watch or when and which reading materials they downloaded from the platform.

As more and more teachers added to their teaching an online learning component that tracked what their students were doing, large amounts of data suddenly became available. Using these data, it became possible to measure how much time learners spend learning online, how often they log onto the learning platform, what learning activities they engage with or the students with whom they collaborate. This information was then turned into colourful graphs (usually using bar charts or line charts) and delivered back to learners on displays called learning analytics dashboards. Similar to a car dashboard, such displays were meant to provide learners with information that helps them make decisions about their learning.

It was expected by some that once learners have access to this kind of information, their learning performance would skyrocket. An easy win, right? Unfortunately, this didn’t turn out to be the case. Early dashboard designs, although considered useful and user-friendly by learners, were rarely used consistently by learners, had little impact on learners’ behaviour and did not significantly improve their learning outcomes.

Building such displays relied on the assumption that learners could correctly decode the information displayed on learning dashboards, put it into context, interpret and evaluate it and make decisions on what to do next. Moreover, most existing dashboard designs were built to foster awareness and reflection only, with the expectation that being aware of one’s behaviour implies that remedial actions are being taken.

We challenged these assumptions and asked 247 first- and second-year Dutch university students to interpret the information displayed on a flawed mockup dashboard as if it were the feedback they had received in an online course. We then asked them to reflect on what helps them to make sense of this information, what motivates them to use this feedback and what does not motivate them.

In their answers, students requested numerous dashboard features that do not directly help them decode information on the dashboard, but rather help them in taking the next steps. Students were expecting dashboards to provide support for action. Here are some features that were frequently requested:

• Support for study planning: On dashboards, students found extremely relevant information that helps them organise and plan their learning. For example, the time they needed to dedicate to cover each lesson or complete assignments, the ability to import deadlines into their personal calendars, and to create checklists.
• Recommendations: Students requested recommendations both related to content and what topics to tackle next or to revise, as well as recommendations on how to change their learning behaviour to learn more effectively. Recommendations are commonplace on music, video or shopping platforms, but not so common on online learning platforms. A recent literature survey found that only 17% of the existing student-facing learning analytics systems implement a recommendation component next to an information visualisation component.
• Help with setting goals: Numerous students rated highly the possibility of adding, viewing and editing learning goals directly within a dashboard. Not surprisingly, this feature was more popular with students who had higher goal-setting skills. As learning goals are known to be a crucial part of the learning process, intuitive dashboard designs could encourage and nudge even less experienced learners in creating the habit of setting learning goals.
• Navigational support: Being able to access the content of the course or the learning material that needs to be revised directly from the dashboard was also listed among the most supportive features. Minimising content access barriers and making the use of the dashboard part of the usual learning process increases the chances that learners are exposed to information displayed on the dashboard and benefit from it.

The list presented above is not exhaustive and there are numerous other ways in which learners could be supported. It is important to mention that the features described in this article were proposed by learners, and designers and learning analytics researchers should consider them as a starting point when building future student-facing systems. At the same time, not all these features need to be implemented in every dashboard. Design elements should be carefully selected taking into account the learners, the learning context, and the purpose for which the dashboard is built.

Jumping on the scales every day doesn’t make you lose weight. Similarly, checking a learning dashboard regularly without making any changes in your learning behaviour will not lead to better outcomes. Awareness is not enough. For learning dashboards to have an impact, interpreting the displayed information needs to be followed by decision-making and action. Learning dashboards should move beyond being passive displays of information and should incorporate features that help learners to make decisions and enable them to take action and change their learning behaviour. Adding such features to dashboards will ultimately increase the chances that closing the feedback loop to learners with dashboards will actually improve learning.

To learn more…

Jivet, I., Scheffel, M., Schmitz, M., Robbers, S., Specht, M., & Drachsler, H. (2020). From students with love: An empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education. The Internet and Higher Education, 100758. https://doi.org/10.1016/j.iheduc.2020.100758 (Open Access)

Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: pitfalls of learning analytics dashboards in the educational practice. In European Conference on Technology Enhanced Learning (pp. 82–96). Springer, Cham. https://doi.org/10.1007/978-3-319-66610-5_7 [Available online as preprint]

Bodily, R., & Verbert, K. (2017). Trends and issues in student-facing learning analytics reporting systems research. In Proceedings of the seventh international learning analytics & knowledge conference (pp. 309–318). https://doi.org/10.1145/3027385.3027403 [Available online as preprint]