Would you like to draw by just using words?
The MUSE project seeks to turn the spoken word into computer generated visual narratives
Everybody loves stories. The challenge is to make them compelling, memorable and easy to understand for everyone. Virtual 3-D environments have great potential for exceptional immersive storytelling, as exemplified by the latest video games. However, creating complex 3-D environments is a technical and time-consuming process that is out of reach for most would-be storytellers.
How could 3-D storytelling be made available for everyone with a few clicks of a mouse? Enter MUSE (Machine Understanding for interactive StorytElling), an European FET (Future and Emerging Technologies) project based at the University of Leuven, Belgium that is developing an innovative text-to-virtual-world translation system capable of automatically converting written text in to 3-D virtual worlds.
The research team is currently working on advancing technologies that allows natural language – the way we use words that has evolved naturally in humans through use and repetition without conscious planning or premeditation – to be better understood and processed by computers. The algorithms under development are learning to make associations between natural language and 3-D visual data so that a real-time 'visual thesaurus' can be created. Think of it as a 'Google Translate for virtual drawings', where you can input a sentence that describes a scene, such as "a man waiting for a train at the station", and that scene would then be automatically rendered in a 3-D environment.
Making language corpora that can be associated to a database of virtual tools for rendering 3-D environments in an automated way is a monumental challenge. However, the project's methodology is currently being tested and showcased in two scenarios: one for creating immersive children's stories from text, and another that enables medical patients to interact with educational materials. See the demos at: http://www.muse-project.eu/
MUSE's technology has the potential for countless applications. Visualising language could be a particularly useful to help people understand complex instructions or specific medical treatments. And besides games, people with limited mobility can explore scenarios and places that they never had the chance to encounter in real life.
The technology could power a whole new realm of creativity and business. Like the first graphic interfaces empowered those who were less computer-literate to get to grips with computing, individuals with little graphic ability may one day become creative designers or visual storytellers. As author and technology visualisation expert Tom Wujec put it: "We are visual creatures. When you doodle an image that captures the essence of an idea, you not only remember it, but you also help other people understand and act on it – which is generally the point of meetings in the first place."
Provided by Youris.com
