My research deals with applications of Machine Learning and Statistical Signal Processing to Music, with results that lead to creation of two new research disciplines in computer music, namely:
These works also fall under the general emerging field of research called Computer Audition.
My earlier works include investigation of musical timbre using Bispectrum and generalization of Spectral Flatness measure.
I'm mostly interested in novel cultural practices mediated by technology that use machine intelligence for artistic individual and group interactions. Music technology effectively changed the traditional music practices from delivering "canned" musical contents to a new model that offers an integrated experience of listening, interaction and targeted socialization. Future of digital music will go beyond delivery and recommendation systems to provide fine-grained descriptions of musical semantics, delegating aspects of musical creativity to machines equipped with notions of emotions, aesthetics, style and more.
My research includes combination of machine perception with generative music algorithms for creating systems that are more interactive through musical knowledge and aesthetic sensibility. In the field of machine perception I'm developing methods for analysis of audio signals that would correspond to human reactions of emotional force and familiarity. Results of this research have been published in Journal of the American Society for Information Science and Technology, Computer Music Journal, and more recently in IEEE Transactions in Audio, Speech and Language (TSALP). I’m also actively involved in the emerging new field of computer audition http://en.wikipedia.org/wiki/Computer_Audition - check out our lab at http://cosmal.ucsd.edu/cal/.
In the area of generative media my work is on machine improvisation and learning of musical style. These works have been implemented in several computer music systems, such as the popular computer improvisation OMax system http://recherche.ircam.fr/equipes/repmus/OMax/. Papers about this work appeared in IEEE Computers, Soft Computing, and Lecture Notes in Artificial Intelligence on Anticipatory Behavior in Adaptive Learning Systems. Also the 2007 a paper with Arshia Cont and Gerard Assayag on fast modeling of audio structure (Guidage) based on a new structure analysis called "Audio Oracle" has received the Best Presentation award by ICMA.
A newer version of the improvisation using the Audio Oracle that uses criteria of predictability and complexity (generally known as Musical Information Dynamics) for selecting the best features and automatically tuning the oracle to the sound has been developed by Greg Surges and can be freely downloaded from PiPY.
An ongoing project "Opera of Meaning" in CALIT2 / UCSD, started as a collaboration with several UCSD Faculty and the Kiyoki Lab in SFC KEIO, Japan, is developing mechanisms for public sharing of data in a context of a story. We created a participatory performance where social aspects of audience presence in a common physical or virtual space became an integral part of the artistic experience. The first artistic project realized according to "Opera of Meaning" concept was a show "Kamza and Bar-Kamza", performed in Calit2 and in ArtPower!Film programs and recently aired on UCSD TV with an accompanying interactive website. This project revived the Talmudic tradition of commentary and debate in a technological-artistic setting http://kamzaandbarkamza.wikidot.com. The idea was to create a new performance art form inspired by various scholar and artistic traditions (Jewish Misha, Tibbetian Budhism, Greek Theatre) using information technology for performance with public participation. In the project collaboratory we explored aspects of gestural music interfaces, use of video and text annotated commentaries, and relations between Talmudic argumentation and aspect of game theory.
Worth checking also is a new book that I co-edited “The Structure of Style: algorithmic approaches to understanding manner and meaning” http://www.springer.com/computer/database+management+%26+information+retrieval/book/978-3-642-12336-8