In the next years a completely new museum in bavaria will open it's doors for visitors. My campus and all classes are currently developing styleguides, installations and all kinds of projects for said museum called "Walderlebsniszentrum Mehlmeisel" short WEZ.
This is one of two projects I am doing together Marco König— another student who's a very talented 3d-Designer.
I'm taking the main part of development in terms of code, Marco's doing the 3d stuff and we both work on the UX.
That title probably wont make it into the museum, as the titles will probably all be in german — that's because the main target group consists of children and schoolclasses.
The subject we decided to work on is called "Selection of typical plants from the Fichtelgebirge".
So since our main class for this semester is interaction design, we choose to bring those plants in digital space.
We are building an digital experience running in the browser, controlled via gestures. The display will recognize the hands of the user and use these hands as inputs. That means no mouse, no keyboard. Shytech on a whole new level.
We are therefore building on two main libraries:
as it's in our opinion the most reliable and extendable 3D library for the web. It's easy to load and combine with additional code/plugins/libraries. Also we both wanted to learn threeJS, so we were both hyped doing our first project with it.
because it's lightweight, fast and reliable in recognizing gestures. Also we wanted to support the developer of handsfreeJS, as we think it's already a very powerful library and will become widely used in the future as it enables the use of touchless interfaces.
+ it seems like magic and that's nice.
We started by creating moodboards and gathering flowers, textures and other inspiration online to find a style we want to pursue.
As Marco's modeling the flowers and experimenting with textures I'm setting up a basic project to show the functionality of threeJS combined with handsfreeJS.
In the same time we meet regularly via Video (as there's still a very annoying global pandemic going on) to make sure we're not drifting apart. Also this is a great way to discuss and work on the UI.
Aside from programming and modeling we also had some very tough decisions to make regarding UX: How do you make an interaction that's new to probably every single user facing our display intuitive and easy to learn? How many gestures are too much? Are two hands too hard to keep track of, especially for children? Is additional facetracking useful?
The visual foundation of our experience are undeniably objects and their surrounding world.
Here are some renderings of our flowers:
We are trying to keep the interface as simple as possible without getting the user stuck. In terms of colors there's a styleguide from another team we tried to follow as far as possible, but we had to add some individual colors for our flowers.
We broke it down to one main screen, as this is basically sort of a proof of concept.
Now there are five tabs, the overview and the 4 flowers we choose to show. By pointing your fingers in the direction of your choice, you navigate through them.
There are also buttons as fallback or for users without webcam.
The difficult part.
We decided to go for one Hand, as it's A) less confusing for the user and B) not really necessary for us.
The plan was to have the left hand control the lights and to use the right one for interactions.
But as we choose to minimize our scene and not use some fancy lighting due to performance, we choose to just use one hand.
That made it possible for us to mirror the gestures to both hands, so that left-handed people don't have a hard time using our application.
For developement we basically wrote a custom gesture recognition, since the methods inside the library were still work in progress and not released or documented yet.
So here are some insights:
Meanwhile we are able to use the libraries functions to check for gestures. That pushes the code down in size and up in functionality.
Experience Meadow on your computer at meadow-wez.webflow.io. Due to performance issues the site is not useable on mobile.
Thanks Marco! Nice project 💯
and thanks Michael for guiding us in the right direction 👏