TACIT KNOWLEDGE IN COMPLEX MIND-ENVIRONMENT SYSTEMS

- Cross-modal temporal behavior modeled with artificial neural network societies

Summary

We propose that tacit knowledge can be accessed by simulating cross-modal bodily actions (i.e interactions between different senses like vision, hearing, and motion) with artificial neural networks (ANNs). Our hypothesis is that a temporal self-organizing ANN learns to communicate when embedded in a society of similar agents, and collective intelligence emerges. Human-like behavior results only when physical constraints of a virtual body are modeled and the agent communicates with a human through sensory-motor devices.

Specific research goals towards the general aim are to:

• demonstrate emergent timing by an ANN that generates rhythmic patterns without an explicit timer
• model temporal behavior based on expectation and anticipation
• model temporal behavior with a system of hierarchical time-frames, capable of recognizing and generating musical patterns
• develop models for coordination of sound and motion, applicable to tap-dance or percussion music
• develop mind-environment coordination models that take the physical constraints into account
• develop models for coordinated action of multiple agents in musical tasks, for example a group of mutually synchronizing players
• explore the possibility to construct ANN models, which simulate emotional arousal based on the violation of expectations
• simulate a band of musicians, or a theater company in virtual reality, as a demonstration platform for the above topics

The project is a continuation of previous research done by the involved teams. Each team has previously conducted research relevant to this proposal and has been supported by the Academy of Finland. This project arises from the recognition of shared interests in artificial neural networks and temporal behavior of complex systems.

The proposal addresses the Information Research Programme in three ways: (1) complex non-linear systems are studied both within agents (ANNs) and as societies formed by agents, (2) medialisation techniques (virtual reality) are extensively used, and (3) man as processor of information is the main objective of study.

Keywords: tacit knowledge, artificial neural networks, connectionism, virtual reality, artificial life, agent systems, mind-environment systems, ecologism, music perception and performance, music generation, future-oriented temporality, cross-modality, emotions, complex nonlinear systems

Project group

• Tapio Takala, Helsinki University of Technology, Department of Computer Science
• Göte Nyman and Mauri Kaipainen, University of Helsinki, Cognitive Science Program, see "Periodically Moderated Knowing in Environment"
• Jukka Louhivuori and Petri Toiviainen, University of Jyväskylä, Department of Music

Links

• coordination of the programme (in Finnish)