BMW Virtual Viewer is an augmented reality web application accompanied with an audible chatbot.
The BMW Virtual Viewer was developed for BMW UK during the pandemic for people who want to learn more about the manufacturer’s eco-friendly hybrid solutions and try out how a life-sized, customized model would look in their garage.
The app’s main attraction is that it enables users to see and interact with a chosen BMW model in augmented reality (AR), accompanied by a narration of a verbal chatbot. Upon model selection, the app presents the essential gesture commands it can be controlled with (position the car into a desired location, rotate, scale the car). Finally, the experience itself starts — with plenty of informative and innovative content to enjoy.
Technology/Stack
The app is built using Vue.js, a frontend framework inspired both by React and Angular. As such, it is mainly responsible for defining the visuals. Every element that composes the application is defined using Vue components. It is also a glue, handling user input and orchestrating the communication between the services used by the application. The most important one is arManager, to which the app provides a canvas to draw the augmented camera feed onto and to handle the incoming and outgoing events. The other service used is the chatbot.
Chatbot
Since the main point is to allow users to see the model constantly, we wanted to avoid too many buttons or lengthy infographics that may cover part of the visual. Thus, we used a virtual assistant to explain things to the user, just like it happens in real-world dealerships.
There is only one button, the use of which is explained by the chatbot. When it appears, a slight floating tooltip explains that pressing this button lets the user record her command, and the chatbot will reply to whatever it understands from it.
Users can ask for information about the model and the capabilities of the hybrid technology. Also, they can interact with the car through the chatbot, e.g., they can ask it to turn on the radio.
Backend communication
A separate Python server is used between the external services and the core, responsible for post-processing the responses given by Dialogflow. It converts and maps the data, enabling the application to work with immutable and ready-to-use objects. This layer handles the caching of the audio file responses as well.
To be continued…
This short introduction was only the beginning. Follow us so that you won’t miss our next posts, expanding further on what the frontend does, how the chatbot works, and what is under the hood.
This article originally appeared on our Medium blog as a three part series. The second and third part are coming to our blog soon.
