At the end of May 2023 we were approached by NOLAI (National Education Lab AI). Starting September 2024 we have been a consortium partner of the NOLAI and are taking on the development work for one of their 7 pilot projects: ‘Increasing vocabulary through rich context‘. From the NOLAI website:
Increasing vocabulary through rich context
#primary_education #equal_opportunity
Together with Flores schools, we are researching whether virtual reality and AI can increase the vocabulary of preschoolers with educational disadvantages. VR could allow preschoolers at school to actively experience new contexts, such as mountains or dunes. AI could facilitate naturalistic interactions between a child and a virtual companion. What does such an experience require of the students and how can it contribute to their vocabulary development?
Language delays are best prevented at a young age, hence the school choice to concentrate on children aged 5-6 years. They see a lot of potential in using Virtual Reality to create a rich context so that language can gain more meaning. Understandable, but we are also quite tentative to work with VR headsets with such a young target group!
All partners are aware of the risks that Virtual Reality can entail. But at the same time, the benefits of this project for equal opportunities can be so great that we think it is worth investigating whether we can use this technology in a responsible manner.
We also see potential in immersive rooms in the classroom. We are experimenting with such setups for use in education. Above, you can see a video of our very first experiment.
However, in this this project we will focus on an application for VR headsets, because the immersive effects will be stronger and this type of application is more scalable.
Learning with Artificial Intelligence
AI plays an important role in this project and will be used in the software in various ways. We think the most exciting form is enabling conversation between the child and the ‘AI tutor’. Practicing your language in a rich context under expert guidance is extremely valuable for children’s language development. However, because a teacher has to divide his or her attention among thirty students, this important learning time is always limited.
However, speech recognition for children with language delays in a busy classroom is a huge technical challenge. In this project we will investigate what the possibilities are and whether we can make this useful for our objectives in a smart way.
We are looking forward to this three-year project!
Featured image: ‘ai generated beach world with sand castle and bucket and shovel water paint style’ by Bing.
The past year has brought one of the biggest challenges ever in education. To protect those who are vulnerable and to reduce the pressure on healthcare, we have all shown our most flexible side. The beginning of 2021 is not much different. In this blog, we look ahead. What role can Virtual Reality play in education after this crisis? And how do we contribute to a sustainable implementation of this and other technologies?
Online education is a huge challenge, for teachers as well as students. Every lecture, I miss that I am not in the same room as my students. What’s more, our workshops & masterclasses for education professionals are much more enjoyable in-person.
Digital education still has too many limitations, even though I often argue that new digital technology like VR offers possibilities that go beyond education in person.
Unfortunately, the field is not quite there yet. VR headsets are not readily available for everyone and teachers and students often do not have enough experience with the technology to use it effectively for education. Especially not when you suddenly have to switch to online education.
Moreover, you also want to combine digital education and in-person education here. Blended. This way, you can choose the educational platform that fits the best for the material. However, this is precisely what is not possible in these times.
From experiment to implementation
At the same time, let’s commit ourselves to emerge stronger from this crisis.
Let’s work to develop rich, asynchronous digital education that students can follow independently regardless of time and place. Combined with synchronous meetings focused on interaction and knowledge processing. Preferably in-person, but digital if necessary.
At the same time, we continue to experiment with new technology like Virtual Reality to further enrich education.
As this field matures with affordable hardware and a wider range of software, we can work towards sustainable deployment. Grants create room for innovation, but ultimately, the business side must also make sense; your innovation must solve a problem.
In the lecture (in Dutch) I recently gave to all i-coaches of vocational schools, I distinguished these three phases:
We look forward to contributing to exciting educational projects in the Netherlands and to bringing them to the next stage!
Research direction
We want to investigate how VR and other technologies can be responsibly and sustainably deployed in various domains. That is why we will be challenging the following questions in the coming period, together with our team, new interns, and students:
Can the shortage of internships in educational programs such as nursing be solved with VR & other technology?
Together with our intern Tess and educational organizations, we are investigating this urgent question for nursing programs.
What can we learn from games?
Together with our new intern Siem, we immerse ourselves in the creative world of game design and share our lessons.
How do you design an ‘infospace’?
How do you use a 3D environment to convey knowledge? How do you tell a story with it? Read the first blog post in this series here.
How will Augmented Reality change the world?
For a few years, I have been challenging the students of my Honors Class to think about how AR will become part of our world. With this, we explore many interesting opportunities and threats. We will further develop and share these findings through our blog!
We research the various applications of Virtual and Augmented Reality and other technologies and share the findings on this blog. Additionally, we put them into practice by designing our teaching materials based on our research and experiences. In this way, we are involved in all phases that must be completed to deploy VR sustainably.
You wake up in the year 2027. Right after getting up, you put on your Augmented Reality (AR) glasses. These smart glasses help you throughout the day. They remind you of what you need to do, make lessons more enjoyable, and translate what your new classmate says. Can you visualize this future? Thousands of students have thought about these and other questions. In this article, we share the results.
Update 2022: during the Week of Media Literacy in 2022, there were 8,014 individual players who answered all the questions. The questions remained the same. You can view the results here (note: the report is in Dutch). There are little differences compared with the results from 2021. What did stand out was that fewer students from grades 5 and 6 think that learning foreign languages is necessary if Augmented Reality glasses can be used to interpret other languages for us. We have also created a new English version.
ARe you ready?
Students have shared their opinions while playing the game ARe you ready?, created by our talented interns Daan Hobbel and Sanna Bashir. The game was developed as a ThemeMission for MediaMasters, an annual national competition held during the Week of Media Literacy in November.
Nearly 80,000 players had experienced a day in the life of a child from 2027, in which smart AR glasses play a significant role. With this new medium, you can see things in the real world that aren’t actually there; a virtual layer on top of the real world.
By presenting these questions to the students during the game, we were able to encourage them to think along and come up with their own opinions about Augmented Reality. Having an opinion allows you to support it and prevents experiencing the future passively. This makes sure that not only Meta, Apple, and Snapchat can determine what role AR will play in society, but also citizens.
80,000 plays, 17,000 participants
The game was played more than 80,000 times by the end of November 2021. We collected 17,037 complete answer sets. Of these, we used 5,385 in our analysis. This dataset consists of answers from players aged between 9 and 13, who came to our site via the MediaMasters game. This was measured with cookies, which were necessary to count points for the MediaMasters game. This way, we can be sure that these were all unique players. Players outside this age limit or who played the game through another route were excluded.
Results
The questions that were asked in between the game’s levels can be divided into two categories:
The potential of Augmented Reality
The societal consequences of this technology.
We will discuss the results per category.
The potential of Augmented Reality
Augmented Reality already has useful and entertaining applications today. For example, checking if a piece of furniture fits in your living room before buying it from the Swedish furniture giant, or using AR to overlay a virtual information layer on the real world. Think of animations and texts that help a novice car mechanic replace brake pads or virtual buttons on a real anatomical model that can be clicked for additional information about an organ.
In the game, players can experience such applications as well as futuristic ones. Are they open to the idea of using smart glasses in this way?
In the game, the AR glasses provide nutritional advice. Do children from grades 7 and 8 appreciate this? Most do (73%), although 32% of them would make the healthy choice anyway, regardless of the glasses’ advice (Figure 1). A large portion of children (61%) also find it helpful if the smart glasses help them remember daily.
Figure 1 (If the smart glasses propose a healthier alternative, would you pick this healthier food choice? A: Yes, it’s nice that the smart glasses can help me; B: Yes, but I already make healthy choices; C: No, I want to make the decision completely by myself)Figure 2 (Would you find it enjoyable if smart glasses helped you to remember your daily tasks? A: Yes, I could use that; B: Maybe; C: No, I don’t need that)
AR in Education
A smart pair of glasses as a dietitian and personal assistant. Would it also be an interesting addition to education? In the game, children look into the life of a primary school child. The lessons the child receives make use of AR. Only 7% of the children did not like the idea of having lessons with AR glasses (Figure 3). 58% of the children are convinced that this form of teaching is also beneficial for learning (also Figure 3).
There are many opportunities for AR glasses in education.
Figure 3 (Blue: Do you think it would be useful to implement smart glasses in education? A: Yes; B: Sometimes; C: No. Orange: Do you think it would be fun to implement smart glasses at school? A: Yes, I would like to use smart glasses everyday; B: I think it would be fun to use them sometimes; C No)
We think that smart glasses not only influence how we learn, but also what we need to learn. Robin’s graduation research in 2014 was already about this topic.
In the game, a character speaks Japanese to the player. The AR glasses pick this up and automatically start Dutch subtitles. Is it useful to learn other languages if the smart glasses can translate everything for you? We wanted to further elaborate on this discussion for the participants with this question. 11% of the children think that AR makes learning foreign languages unnecessary. But the majority (62%) would rather not depend on the glasses and still find it useful to learn other languages (Figure 4).
Figure 4 (Is it useful to learn languages if the smart glasses can translate everything for you? A: Yes, if you speak other languages, you don’t need the smart glasses!; B: No, it is not necessary to learn other languages, but it is fun and interesting!)
AR for navigation
A more obvious example is navigating with your smart glasses. Virtual arrows projected in the real world help you find your way. In this way, it becomes very difficult to go in the wrong direction. 93% of the children think that we will use smart glasses for navigation later (Figure 5).
Figure 5 (Do you think people will use their smart glasses to find their way? A: Yes, often; B: Only when they visit new places; C: No
The social consequences of Augmented Reality
The emergence of a new computer interface has a huge impact on society and our lives. Enough to get excited about, but also to worry about. That’s why we asked the children if they are concerned about the ethics and safety of AR glasses.
Exclusion
One of the problems that can arise is a kind of social gap between people who have smart glasses and people who cannot (temporarily) be in our new, enriched AR world. We highlighted a similar situation in the game. Is it bad if there are other children who do not have smart glasses and therefore feel excluded? Most children (87%) think this is (a bit) unfair (Figure 6).
Figure 6 (For children who cannot afford smart glasses, the world will look quite different. Do you mind that? A: Yes, that’s unfair; B: A bit; C: No, I don’t mind
Distraction
The virtual layer projected by an AR glass can be distracting from information in the real world. In fact, things can be hidden by projecting over them. This can be dangerous in traffic. In an AR mini-game in ‘ARe you ready?’, the player is chased by virtual monsters. To stay ahead, many players run through red lights. The use of a smartphone can be dangerous in traffic.
We asked children if they think smart glasses should be banned on bicycles. 60% agreed. 28% want a special “traffic mode” so that the glasses do not pose a danger on the road and can be kept on. And 12% think we should decide for ourselves whether to keep them on or take them off (Figure 7).
Figure 7 (Using Smart glasses in traffic can be dangerous. Do you think using smart glasses should be prohibited when you’ re biking? A: Yes; B: No, but there should be a special ‘traffic mode’ that is less distracting; C: No, people should be allowed to decide for themselves whether they want to wear smart glasses in traffic)
Disruption of public order
Almost everyone has played or heard of Pokémon Go, an AR game that led to large gatherings of people in various locations. We believe that this could happen much more often in the future. Virtual artworks at secret locations and massive AR tournaments in the town square. Are you allowed to just organize this? Or should you apply for permits, like for regular events? 69% of the participating children said that rules should be in place and permission should be requested to organize such events (Figure 8).
Figure 8 (In the game, a big AR tournament is held in the city center. Currently, there is no regulation around this kind of event. Do you think new rules need to be made? A: No, people should be allowed to choose themselves; B: Yes, you should ask permission first)
Privacy violation
Many companies collect our data without our conscious consent. This data collection can go a step further with the help of an AR headset, as it can see much more and track things other than just online activity. An obvious example is personalized advertisements. The smart glasses could use information about you, such as conversations and images, to project specific ads in the real world; this also happens in the game.
38% of players aged 9 to 13 believed that no information about them should be used to personalize advertisements. 39% thought it was acceptable, but only if they had given permission themselves. 23% had no problems with sharing information (Figure 9).
Figure 9 (Should it be allowed to use smart glasses to collect information about you for advertisements? A: Yes, I have nothing to hide; B: Only if I have given permission; C: No, I find my privacy very important)
Are we becoming too dependent on our smart glasses? Technology like an AR headset can easily enrich the real world, making people possibly unwilling or unable to live without it. About 90% of the children find it concerning that we may become too dependent on this technology in the future (Figure 10).
Figure 10 (Would you mind if people became increasingly dependent on smart glasses, and they would not be able to do some thing without them? A: Yes; B: A bit; C: No)
Mission accomplished
We conclude that most children see benefits in using smart glasses, but certainly do not want to become dependent on them.
Let’s nuance the results a bit. This is not scientific research. The answers may have been influenced by how AR was portrayed during the game. And it could be that the youngsters who chose to play this game are overly enthusiastic about new media. Fortunately, our goal was not to conduct an academic study. Our mission was to get children thinking, and we certainly succeeded!
Read (about) Robin’s graduation research (2013), which is about how Augmented Reality not only influences how we learn, but also what we need to learn through the interplay of our cognition and the computing power of computers.
Last November 29th, we organized an event together with Smart 071 and the Honours Academy of Leiden University all about the applications of AR (Augmented Reality) in the workplace.
Students and professionals looked together for meaningful and feasible applications of AR in the participating organizations. This event was part of a new course given by Robin at the Leiden Honors Academy. The Augmented Reality & Human-Computer Collaboration discusses how Augmented Reality will influence our society and the future of work.
This blog post looks back at:
our event about AR and the future of work
the prototypes built by the students as their final projects
We’ll also tell you how you can participate as a professional or student next edition!
