1. Performance Modeling - 10/13/2000

OBJECTIVES

This proposal addresses a problem fundamental to performance scholarship and pedagogy: how to represent an historical performance. Neither a written description nor a filmed recreation is capable of conveying the experience of attending a live performance, an experience that encompasses not only way the performance on stage looks and sounds from the perspective of spectators in different parts of the theatre, but also spectators' perceptions of and interactions with one another. One strategy to address this problem has been to build a physical reconstruction of an historic theatre and stage performances in it, as has been done with the Globe Theatre in London. This solution, however, requires an extraordinary investment of money and land, and so is feasible only in a very limited number of cases. Moreover, physical reconstructions are available only to people at one geographic location, and they implement only one interpretation and so cannot be used to evaluate conflicting scholarly interpretations of the historical evidence.

Our proposed solution to this problem is to recreate historical performances in a virtual reality environment. Our short-term objective is to develop and test a general-purpose set of techniques and technologies for producing such VR performances by completing a fully-functional prototype. The test case will be a performance of vaudeville in 1880 at the Tony Pastor Theatre, New York City.

Our goals for the content of prototype are:

1. To create a detailed, convincing, richly textured model of the physical space, including audience and backstage areas;
2. To present a selection of skillful and compelling vaudeville performances in that space, including comic routines, music, freak and animal acts (we intend to make extensive use of motion capture technology to create these performances);
3. To recreate the behavior and socio-economic composition of the audience and people backstage;
4. To link objects in the virtual world to the archival material (e.g. scripts, scores, photographs, ephemera) upon which the reconstruction is based, as well as to other useful historical and critical sources, both primary and secondary;
5. To allow the user to alter limited aspects of the reconstruction, for example by selecting from a pre-designed set of costumes or architectural features, to compare different scholarly hypotheses.

Our goals for the interface are:

1. To allow users to move through the environment at will, changing both their visual and aural perspective on the events;
2. To allow the user to be embodied in any of three ways: (a) as invisible camera that can see anything in the VR world from any angle; (b) as any character onstage or off (in which case the user has no control over the character's eye or body movements); or (c) as one of a set of pre-defined avatars whose movements are under the user's control;
3. To give characters in the audience and backstage (not onstage) limited autonomous agency, so that they execute their behaviors when the user is in their presence and keep track of what the user has already seen and heard, thereby optimizing the user's experience, and also allowing characters to behave in different ways depending on the class, gender or race of the avatar the user has assumed;
4. To provide an efficient and intuitive way to access and view the archival material and other sources.

Our goals for delivery are:

1. To make the project available to the public over the internet via broadband technology;
2. To make the design scaleable so that it can be experienced on a variety of platforms at different resolutions and frame rates, such as DVD-ROM, game stations or through VR goggles, and will not become outdated as technology improves.
3. To document a general development strategy and create a technical tool set, not exclusive to this specific performative event, that will allow other scholars, educators and artists to create their own virtual performance events.

RESEARCH PLAN

Year 1. Research

1. Computer scientists, computer animators and theatre artists will collaborate to address the integration of live performance, scenic spectacle and music into a virtual reality environment.
2. Computer scientists will address the integration of the various component computer technologies, such as virtual environment, dimensional audio, character animation and autonomous agents.
3. Performance scholars will research the performance elements of American vaudeville performance from the 1880s, and will collect archival material in digital form.

Year 2. Design

1. Theatre scholars and artists will collaborate to create a performance score, including both the onstage performances and offstage character interactions.
2. Composers, working with music scholars, and sound designers, working with performance historians, will create musical scores, compose ambient sounds, and identify any other audio requirements.
3. Theatre designers, working in close conjunction with the theatre director and scholars, will design spectacle elements, including architecture, scenic design, costume and lighting design.
4. Computer artists, working from the set and costume designs and also the technical specifications provided by the computer scientists, will create 3-D models of architecture and objects.
5. Computer artists, in close collaboration with the theatre scholars and director, will create character models for both performers (including animals) and offstage characters
6. Computer scientists, in close collaboration with all the project's investigators, will begin to design the end-user interface. (This process will continue through the middle of the next year.)

Year 3. Performance and Media Integration

1. Computer scientists will place the computer models into the VR environment and add interactivity where required.
2. Computer scientists will work with performance scholars to link research material to 3-D objects in the VR environment.
3. The theatre director will cast and rehearse actors, the musical director will assemble and rehearse musicians, and the choreographer will cast and rehearse dancers, all in close collaboration with the performance scholars.
4. The theatre director, musical director and choreographer, working closely with the computer animators, will supervise motion capture of the performances and recording of the digital audio.
5. The computer animators will manipulate the motion capture data as necessary, and create additional keyframe animations for models.
6. Computer scientists will integrate animated performances into the virtual reality environment.
7. Computer scientists will implement the user interface and compile the program for delivery over broadband internet (and possibly other platforms as well).

INVESTIGATORS AND CONSULTANTS

The following participants in the NINCH workshop expressed an interest in participating in this project: Dr. David Saltz, Dr. Susan Kattwinkel, Dr. Frank Mohler, Professor Gary Maciag, Jeff Burke, Dr. Mark Pizzato, Dan Zellner, Dr. Larry Worster, Dr. Frank Hildy, Dr. Melanie Blood, Dr. Bruce McConachie.

Professors Michael Hussey and David Koffman, specialists in 3D computer animation at the University of Georgia (in Drama and Art, respectively), have also expressed interest in working on the project.

With appropriate funding, this group is capable of accomplishing the required scholarly research, computer modeling, animation, motion capture, and musical and theatrical production (i.e. assembling designers and performers to stage the vaudeville performances, and scripting and staging the offstage behaviors).

We require computer science collaborators with expertise to add interactivity to the virtual environment, autonomous agents and to design the user interface. We have made contact with Edward Price, research director of the Interactive Media Technology Center at the Georgia Institute of Technology, who has expressed interest in working with us on this aspect of the project.

Possible consultants include Bill Jepson (PI of the Urban Simulation Lab at UCLA), Benny Sato Ambush (Director, Institute for Teledramatic Arts at California State University, Monterey Bay) and Daniel J. Sandin (Director, Electronic Visualization Laboratory, University of Illinois at Chicago).

2. Institute for Digital Scholarship in the Performing Arts [no change]

OBJECTIVE

To facilitate the creation of digital scholarship that integrates theatre, music, and dance.

Means to the end

Create a board of directors as outlined below, formed of members of existing scholarly societies in theatre, music, and dance. This board will develop guidelines for digital projects, promote the projects, adjudicate among applicants, and administer grants to individual scholars and humanities computing centers. In addition, the involvement of scholarly societies should help devise standards for scholarly recognition of digital projects. See preliminary details below.

Needed expertise:

On the Board:

9 representatives of scholarly and library societies representing music, theatre, and dance. The following societies are recommended for potential inclusion on the board: Association for Theatre in Higher Education, American Society for Theatre Research, Theatre Library Association, United States Institute for Theatre Technology, Congress on Research in Dance, Dance Library Committee, American Musicological Society, Society for American Music, Music Library Association, Society for Ethnomusicology, and Society for Music Theory. One computer scientist should also be on the board. Hardware and software specific to projects selected is included in the proposal below. Server space and maintenance of project on a the Internet will be the responsibility of both the computing center and the individual societies selected by the board, as outlined below.

Investigators and consultants:

Frank Hildy, Bruce McConachie, Mary Strow, Bob Keller, Kate Keller, Melanie Blood, Susan Kattwinkel, David Saltz, Frank Mohler, Mark Pizzato, Jeff Burke. Four international consultants from international scholarly societies in the performing arts will serve as consultants to the board.

Detailed proposal for the Institute for Digital Scholarship in the Performing Arts:

The Institute for Digital Scholarship in the Performing Arts will seek a three year grant of $750,000. per year for three years. The Institute will oversee up to 18 digital scholarly projects that use digital technology to enhance our understanding of the relationship between music, theatre, and dance. Near the end of three years, the Institute will evaluate its successes and failures and redesign the Institute accordingly.

A board of directors will solicit projects, adjudicate among them, and administer the grants. The board of directors will consist of a Primary Investigator/Executive Director who is directly responsible for grant moneys and overseeing the progress of the participating scholars, nine representatives of scholarly and library organizations representing theatre, music, and dance, and one computer scientist.

The duties of the board include:

1. Serving as grant administrator
2. Conducting a competition among scholar/artists and computer scientists to identify up to six projects per year. Collaboration among scholars will be encouraged. In most cases the scholar will work at a selected center (see #3) for the term of the grant. Scholars from smaller institutions, minority scholars, and scholars working in areas that engage issues of diversity will be particularly encouraged. Final delivery of projects must be clearly defined and preference will be given to projects that can remain widely , to other scholars and artists.
3. Winning scholar/artists/computer scientists will be appointed for one year terms. They will be awarded one half their annual salaries, a living stipend, and $20,000. in hardware and software to support the project. Regardless of collaborations, a total of six individuals will be supported each year. Scholars may reapply for continuation of their projects.
4. The board will conduct a competition among centers for computing in the humanities. Centers will apply for the full three years of the grant. The centers selected will receive up to $300,000 per year to support the scholars selected in #2. If more than one site is selected, centers will receive $50,000. per scholar they support.

Proposed timeline:

spring 2001: assemble board and delineate criteria for centers' and scholars' competitions.

spring 2002: conduct the competitions and select winners

fall 2002: first scholars and centers receive support.

fall 2003: second scholars (and centers) receive support

fall 2004: third scholars (and centers) receive support

fall 2004: Board reevaluates project, redesigns to meet new needs, applies for new funds

Assuming that, in order to receive grants as an organization, incorporation and 501c3 status will be required for the proposed Institute. There are three possible solutions:

1) NINCH assumes this function not only for this Institute but for a larger Institute representing the needs of all the humanities

2) The Institute for Digital Scholarship in the Performing Arts

3) Performance in History Associates, an organization already incorporated and not-for-profit, incorporates this organization under PIHA, with PIHA functioning as an umbrella and the Institute functioning as outlined here.

3. International Dissertations in Music Online [no change]

Thomas J. Mathiesen

David Crawford

Objective

To expand the existing Doctoral Dissertations in Musicology--Online (which includes records of dissertations in progress and completed) into a full digital archive of dissertations in all aspects of music.

The current database is updated regularly, and this project would commit to maintaining updates on a regular basis. Expanding the database will involve significant computer challenges: indexing, cross-linguistic searching, and creating digital media for dissertations. The problems include handwritten dissertations (mostly European) and index and display of non-Roman type and languages that read from right to left and top to bottom (e.g., Cyrillic, Greek, Japanese, Hebrew, Arabic). The American Musicological Society, the International Musicological Society, and the Center for the History of Music Theory and Literature at Indiana University (CHMTL) have supported this project to date. At Indiana University, this project currently benefits from an institution-wide commitment to digital and multimedia projects.

UMI currently does not and historically did not collect all dissertations. Coverage is needed for older dissertations and beyond North American universities. The project supports both research and teaching on an international scale and could also expand collection and payment of royalties to the authors.

Means to the End: Currently in existence (since 1996) is the basic database of 10,706 bibliographic records going back to the 1940s, fully browsable and searchable on the CHMTL's web site, DDM-Online (). The CHMTL employs a fully developed database thesaurus, an established financial system for managing income and expenses, personnel, and space, but will need an expanded server for database storage as the project expands.

Suggested partners/personnel:

Current partners: American Musicological Society, International Musicological Society, the Royal Musical Association, the Dissertationsmeldestellung (M¸nster), and the CHMTL. Suggested new partners: UMI, Center for Research Libraries, Moscow Conservatory, and comparable groups in other countries. Negotiations are currently underway to expand collaboration with organizations in other parts of the world. Principal investigator would be Prof. Thomas Mathiesen.

Needed expertise: Computer scientists to be involved with multilingual searching, full-text indexing, indexing of non-text objects (e.g. musical notation, graphics, sound files), OCR and indexing of manuscript text, and index and display of non-Roman type and languages that read from right to left and top to bottom (e.g., Cyrillic, Japanese, Hebrew, Arabic, Greek).

Investigators and consultants: Need to expand cooperation with Music Educators National Conference, Society for Ethnomusicology, Society for American Music, Society for Music Theory, American Studies Association, etc. Formal application will be submitted jointly by the American Musicological Society and Indiana University.

9. Center for Distributed Scholarly Resources in the Humanities

Robert M. Keller

The objectives of this project are to establish a database on the Internet which would function as a gateway for networked scholarly resources in the Humanities.

The objectives would be implemented by the Center for Distributed Scholarly Resources in the Humanities, established at a cooperating computer/humanities organization.

This plan envisions that the scholarly societies would be invited to nominate scholarly electronic publications for inclusion in the database. A Director would be designated to develop the database and make it available on the net. The scholarly societies would also be invited to serve on a Board of Directors to establish policies and oversee development of the project. Partnerships could also be developed between the Center and electronic publishers, for distribution of the resources, if desirable.

Expertise and computer equipment needed by the hosting organization would include:

• Database management, cataloging and metadata skills
• Webpage development and maintenance
• Internet servers for WWW access to database and publications

Proposed timelines and milestones for the project would be as follows:

Year 1: Establish organizational structure, hosting organization, Board, database, and initial

staff

Year 2: Establish partnerships with scholarly societies

Year 3: Establish partnerships with E-publishers

The Center could be hosted by the Digital Library and Archives Project at Virginia Polytechnic Institute and State University, which has developed the Electronic Thesis and Dissertation Collection as part of the Digital Library Project, and has expressed interest in participating in this project. Representatives of several scholarly societies have expressed interest in working on this project and serving on the Board of Directors.