Building upon earlier meetings in London  and Washington, DC , members of the Cultural Content Forum (CCF)  gathered earlier this year in the Italian town of Pistoia. During their meeting, partially held in the spectacular grounds of the Villa Celle, participants explored ongoing work to better understand audiences for online cultural content, and agreed on a programme of work for the coming year.
Views from around the world
The Cultural Content Forum was an invitational event that followed this year's EVA Florence conference , and the Forum began with a series of presentations to set the scene and stimulate discussion. Each of the presentations addressed an aspect of existing work on understanding audience, and together they went a long way towards highlighting many of the issues that need to be addressed in this field. Slides from the presentations are available on the CCF web site .
The first presentation, by John Perkins of CIMI, outlined the initial phase of a study in which Alice Grant has been commissioned by the Cultural Content Forum (CCF) to survey published and grey literature pertaining to audience or user research within cultural heritage institutions. The research has so far identified more than 80 individual items, of which around 25% were regarded as confidential. A database has been compiled which describes each of the contributed resources and either provides information on how to obtain the resource or contact details for an individual involved in its creation. This database is to be made available via the CCF website.
Kevin Sumption reported on a recent evaluation of the web presence of Sydney's Powerhouse Museum . As also reported by others, the method of survey comprised a range of techniques, including analysis of server logs, an online questionnaire, a Delphi panel, and in-depth interviews.
Looking at the server logs for 2002, some 674,787 visits were identifiable, with an average visit length of just over 9 minutes. Australia was the single largest source of visitors to the site, with 282,669 visits originating from within the country.
Broad statistics such as these from server logs were supplemented by more in-depth information from the questionnaire, panel and interviews. The questionnaire was completed by 1,048 individuals, of whom 75% were visiting the site primarily to plan a physical visit to the museum. The profile of those completing the questionnaire does not fully match the total population of visitors to the site, with 96.4% of those completing the questionnaire living in Australia.
Given the emphasis many cultural institutions and associating funding programmes are placing upon the creation and provision of educational interactive resources, it is possibly of concern to note that 91.3% of those surveyed said they would not be using an interactive resource on the site, and 94.7% said they would not be accessing a curriculum-focussed resource. Similarly, there is a trend to provide access to the online catalogue or collections management system, yet 87.4% said that they would not be accessing the Powerhouse's collection database.
In a presentation with many parallels to that from Australia, Tim Hart from the Getty spoke about the redesign of their website to meet the varied needs of visitors to the site . At the Getty, they have identified three main content forms on the site (Collections & Exhibitions, Visit Information, and Research Tools & Resources), and three main user profiles (Internet Browser, Getty Center Visitor, and Researcher). Each of these different types of user interacts with the site in quite different ways, and the Getty have recognised that it is not actually necessary to accommodate all of their needs within a single homepage to the Getty site; browsers are likely to enter the site from a variety of referrers which point to different areas of the site, visitors are likely to enter through the home page, and researchers are often repeat visitors who will go directly to the resource(s) of interest to them. As a result, and with input from surveys of physical visitors to the Getty Center, analysis of server logs, and detailed usability analyses for areas of the site, the Getty web site was extensively reworked. This reworking recognised the areas of the site likely to be used by the different user profiles in their search for the three main content forms, and tailored the interfaces to those functions and users. Deep pages of the site have been reworked so that users arriving on these pages from an external referrer such as Google have sufficient contextual information and navigation functions for them to browse around within the site relatively easily.
Turning to the UK, David Dawson reported on a range of activities, including the user research undertaken by projects funded under the New Opportunities Fund's nof-digi Programme , the UK Omnibus Survey, and a survey of Cultural Websites undertaken by McKinsey and Co. for the Department of Culture, Media & Sport . These bodies of research pointed to an interest in online cultural content, with the Omnibus Survey, for example, showing that 24% of the cross-section of the UK population surveyed had "visited web sites or used CD-ROMs on arts or culture in the past 6 months". The McKinsey research, which consulted 4,430 visitors to five UK cultural websites over a two week period in 2001 identified a significant interest amongst these visitors in having access to cultural content, but a notable lack of interest in multimedia or interactive content forms. This may, of course, reflect the technical limitations of modem-based communication in 2001, and a different picture might emerge as Broadband continues to take hold.
For Canada, Jean-Marc Blais and Kati Geber from the Canadian Heritage Information Network (CHIN) reported on results from the Virtual Museum of Canada . Here, the virtual exhibits are extremely popular, and the site has built up a solid brand with high levels of recognition. Innovative associations with commercial organisations have led to opportunities for advertising and awareness raising on billboards, on television (a 30 second advert, run over two weeks on a public television station, led to a doubling in visits to the site), and online.
The team have the notion of an 'engagement index', measuring the degree to which visitors to a page actually engage with its content, and attendees agreed that such a measure is worth developing further. CHIN are already working to extend the data they have, and are preparing a report which attempts to understand the quality of a user's experience on the site.
Joyce Ray from the US Institute of Museum & Library Services (IMLS) spoke about work that IMLS is funding in order to learn more about audiences for cultural content. A workshop held in the United States in March has identified a set of key research priorities, including a range of user studies, and these align well with work described from other countries.
Jim Michalko from the Research Libraries Group (RLG) talked about the needs of research users, quoting from the work of Howard Bloom. He illustrated that researchers expect more and more, increasingly expecting to be able to find whatever they want online, in the format and language of their choosing.
The remainder of the meeting was devoted to discussion that aimed to identify areas of common concern, where participants agreed that solutions might be achievable.
There was widespread recognition of the need to be able to compare results of audience and user research conducted across the cultural sector, and an agreement that it is difficult to do so at present, because of the different techniques deployed. Nevertheless, it was seen as valuable to undertake an analysis of the data already collected in the CCF study, and to publish this.
Participants felt that there was scope in collaborative work to identify shared measures to be used in analysing such things as server logs, perhaps building upon initiatives such as COUNTER . Further, it was seen as useful to explore harmonising the definition of user profiles and roles; what, broadly, is a 'student' or 'learner', for example?
Towards Common Solutions
There was a clear and strong commitment from participants to complete the current round of CCF research. Funding has been identified for this from a number of partners, and an evaluation of the gathered data is now underway, with Alice Grant Consulting once again undertaking the work. Results will be published on the CCF website when available.
Funding has also been allocated to work which begins to identify common metrics for use in evaluating server logs, audience surveys and the like the Pistoia Profiles? and this will also expand upon CHIN's notion of an engagement index. More information on this work, and on the scoping process for it, will shortly be made available via the CCF website.
And in 2004...
Members of the Cultural Content Forum are now progressing with the shared work on Audience, results of which will be made available via the CCF web site .
In addition, planning is already underway for the next meeting in 2004, the emphasis for which is likely to be the role of cultural institutions and their digitised content in education and learning. As with the last meeting, we will aim to issue a call for expressions of interest late in 2003, from which a list of invitees can be drawn up, ensuring participants bring a relevant set of skills and experience to the meeting that will enable us to make progress in this area of shared interest.
The authors wish to thank all of those who travelled to Italy to participate in this meeting. Without their attendance and ongoing participation, this initiative would be much diminished. Thanks are also due to the Bank of Pistoia-Pescia Foundation (President Prof. Ivano Paci), whose generous financial assistance provided for our needs whilst in Italy. Bernard Smith, Head of the European Commission's Preservation and Enhancement of Cultural Heritage Unit, Chaired proceedings, and kept us all on track. Vito Cappellini and colleagues from the EVA Florence conference handled local arrangements, and took excellent care of us. Finally, we extend our thanks to Dr. Giuliano Gori, who so generously opened his home to us, and allowed the stunning environment in which he lives to have such a positive effect upon our activities.
Participants at the meeting were drawn from Canadian Heritage (Canada), CERLIM (UK), CHIN (Canada), CIMI (Canada), Digital Library for Dutch Literature Foundation (The Netherlands), European Commission, French Ministry of Culture (France), Getty Trust (USA), IMLS (USA), JISC (UK), The Natural History Museum (UK), Patrimonio.es (Spain), The Powerhouse Museum (Australia), RLG (USA), Resource (UK), SCRAN (UK), State Museums of Berlin (Germany), UKOLN (UK), University of Florence (Italy), and Vasari UK/EVAN (UK).
The meeting was conceived and realised as a partnership between UKOLN, Resource and CIMI.
 EVA Florence Conference
 Presentations from CCF meeting in Pistoia
 Culture Online Background Research
 Counting Online Usage of NeTworked Electronic Resources (COUNTER)
Copyright © Paul Miller, David Dawson, and John Perkins