Music played a pivotal role in tribal society and in folk culture. It was an everyday form of self-expression: sharing emotions, telling stories and building communal consciousness. In the age of broadcasted media, "to perform" became a centralized business and "common people" became passive audience only.

Thanks to interactive media these centralized structures are breaking down by an unprecedented pace. In our "global village" everybody craves for a more active role to reclaim the so-long-missed 15min fame (at least among their own social network :P ). The current challenge every contemporary artist faces is to open up their visual/sonic compositions and enable others to directly reflect, change and interact in a meaningful way.



The beauty of interactive media is the freedom we give to our audience: for a certain extent they become an equal partner in the final step of the creation. Their mood, emotions and state of mind will affect the actual art-work while they are "playing". As music is the natural element of self-expression, music-video is the best media to turn into an immersive interactive experience.

The great challenge here to set up a visual world consciously that vast enough to incorporate many different people's dreams and fantasies, and in the same time helps its users to navigate towards the very spaces and styles they will find themselves inside.



As we put our "interactive music video" concept into a wider cultural frame, the next step is to discuss all the research and development at "VJ Yourself", which enables to define "open compositions". This openness is nothing new to art: musical improvisation and freestyle rap/dance are the closest relatives, but how would a computer "appreciate" and understand these open-ended artistic states?



When creating interactive videos we have to rely more on algorithms to synchronize imagery to music, unlike classic video clips, where everything is planned precisely on the timeline.

Our aim to develop more human approaches to achieve the synch. Beside real-time waveform analyzes and beat detection, we build up meta-data around the soundtrack, to teach the computer of the artistic relationship between certain parts of the music and the corresponding colors / shapes and animations.

It's absolutely a human task to find the best visual associations to certain loops / lyrics, but after fed all these information into a Database, the computer can make good decisions how to build up the visuals in the very situation the user navigated.

Let's reach the perfect Synesthesia when you can't tell finally: Is the music controlling the visuals, or the visuals generating the music ? :D



Giving freedom to the audience, while preserving the original style?
The challenge here to find the right balance what a user can change, and what are the fixed constraints of our interactive world. We developed an associative language to define styles with lot's of meaningful variations. The experience never fails, but the players can make hundreds of free choices inside the artistic style.

To keep the navigation simple, and easy to understand, we map all these choices to spatial structures. The most natural interface, we all experienced, is navigating in space. Easy to grab the concept by this little example: in the 3D visuals you get to a cross-road, the path to right contains the same imagery as the path to left, but in a more minimal style. If the users prefers to get minimal, they will turn right, and continue their journey in the preferred variation of the same style.



To take the concept even further it's possible to pull in custom images from the actual user (e.g via Facebook) and make some parts of the clip "surprisingly" personal. As an extra motivational factor it's worth to introduce certain achievements: e.g. collecting artifacts or avoiding obstacles. Having a competitive element, and sharable actions will accelerate the virality of the project.



Soon we realized that classic programming languages and visual platforms are too rigid to facilitate all these dynamic ideas. So about 10 years ago we started to experiment with a new programming paradigm that can handle artistic data with high permutations. The ease and robustness behind VJ Yourself's projects is thanks to this platform. Our cross-disciplinary research and development includes fields like: Cognitive Psychology, Computing Theory, Game Design and Post Modern Art.