D-Lib Magazine
The Magazine of Digital Library Research
transparent image

D-Lib Magazine

November/December 2015
Volume 21, Number 11/12
Table of Contents


Collaborative Construction of Digital Cultural Heritage: A Synthesis of Research on Online Sociability Determinants

Chern Li Liew
Victoria University of Wellington, New Zealand

DOI: 10.1045/november2015-liew


Printer-friendly Version



The purpose of this scoping study is to answer the research question: What does the literature tell us about online sociability that could inform how participation in collaborative construction of digital cultural heritage (DCH) can be supported, motivated and sustained? A scoping review was conducted with the aim of building on the recent advances in research on online sociability and participatory culture. An extensive literature survey was undertaken across various disciplinary fields to gain a broad snapshot of the factors that have been found and suggested as having an influence on online sociability in collaborative projects. Relevant literature were analysed and weaved together to map a pathway with motifs that could be useful as a guide for projects working towards collaborative construction of DCH.


1 Background

The cultures and tools afforded by the social Web have offered cultural heritage institutions (CHIs) (archives, libraries, galleries and museums) the opportunities to transform the services they provide as well as their relationships with their user communities and the wider public. CHIs are still a long way from reinventing themselves through the social Web however. Most CHIs still make restricted use of social media for basic functions like communicating general information and news, and promoting events, collections and services. Some have gone a bit further — taking advantage of the participatory potential of the social Web to actively engage users and stakeholders in conversations about their collections and services. Even fewer have involved carefully coordinated efforts of soliciting and providing meaningful ways for collaborative construction and enrichment of cultural heritage knowledge.

The success and sustainability of such endeavour are likely to depend considerably on the presence of a platform and system that supports sociability as a start — one that facilitates social interaction and connectivity — which could then potentially lead to sustained engagement and collaboration to achieve a shared goal.

The value and sustainability of online participatory projects depends on many factors. Each participant's interaction with the program is facilitated by technology. Hence, the usability of the project platform is undoubtedly important. However, a broader view recognises that participants aim to interact not only with the project system, but with each other and with the professionals who are guiding the projects. Therefore the sociability of the project as a whole must be taken into account, and this is the focus of this research.

We define online sociability in the context of DCH as the extent to which the information and communication environment mediated by social media is perceived and/or believed to facilitate social interaction, engagement and meaningful participation.


2 Methodology

A scoping study worked well to answer the research question: What does the literature tell us about online sociability that could inform how participation in collaborative construction of DCH can be supported, motivated and sustained?

A scoping review was conducted with the aim of building on the recent advances in research on online sociability and participatory culture. Relevant literature were analysed and weaved together to map a pathway with motifs that could be useful as a guide for projects working towards collaborative construction of DCH.

An extensive literature survey was undertaken across various disciplinary fields (see Appendix 1) including digital cultural heritage crowdsourcing and citizen science, e-commerce, educational and organisational psychology, sociology, media studies, online participatory culture and human-computer interaction to gain a broad snapshot of the factors that have been found and suggested as having an influence on online sociability in collaborative projects.

There are challenges involved in collating and interpreting this literature. One issue is that although online sociability is a concept which has received increasing recognition and attention within the field of computer science over the last decade, it is often studied in other fields without being labelled as such. For example, the past decade has seen extensive literature published examining the determinants of the overall success of participatory online cultural heritage programmes and online voluntary communities of purpose such as Wikipedia or open source software development networks. However, there are limitations on the information available in this area. Most published quantitative information on participation rates derives from those projects which were successful (Noordegraaf et al., 2014). Hence, there may be gaps in our knowledge of what hinders online sociability.

The available information on less successful participatory initiatives is also often anecdotal in the sense that it is usually published in summaries of experts' deductions from observations in the field, as in Ridge (2013), or as case studies of single projects as in McClean (2011). Furthermore, most qualitative information on participants' perceptions is gathered from the small proportion of active participants who answer surveys or comment in forums (Eveleigh et al., 2014; Raddick et al., 2010) and thus does not tell the full story of the majority of participants, who are also valuable members of any project, not only because of their combined production capacity but also because they act as "the essential audience" for others, and as ambassadors for the program and the institution in the wider community (Groenewegen & Moser, 2014; Nonnecke & Preece, 2000; Shirky, 2010). Studies on participation in cultural heritage work also rarely use control groups and manipulation of variables to clearly establish causal relationships between variables, or validate participants' self-reported motivations with examination of each individual's actual participation rates under various conditions (Eveleigh et al., 2014; Nov, Arazy, & Anderson, 2011, 2014). These are some of the reasons why a net was cast across various disciplines to identify as many factors and determinants as possible, without neglecting the context-specific insights available from within the DCH field itself.

The breadth of this literature review has created further challenges. Participatory cultural heritage projects differ widely in their aims, content, structures and participant groups. Determinants of online sociability vary likewise. What motivates and encourages participants in one context can hinder sociability in another (Eveleigh et al., 2014; Ling et al., 2005; Nov et al., 2014; Ridge, 2012). However, some commonalities have emerged consistently across the fields of practice and disciplines and these common determinants of online sociability are discussed in the next section. Key concepts and motifs are highlighted in bold.


3 Online Sociability Determinants in Collaborative Projects

There are many ways to conceptualise the complex relationships between the myriad factors which affect social interaction, engagement and participation in collaborative projects. Overviews of success factors have previously been arranged according to phases of project development (Noordegraaf et al., 2014), phases of participants' connection with the project (Rotman et al., 2012), or as simple checklists of considerations for planning (Holley, 2009, 2010; Romeo & Blaser, 2011).

Determinants of online sociability according to Preece (Liao & Chou, 2012; Preece, 2001) fall under three areas:

  • Purpose — Factors which enhance a community's shared focus
  • People — Factors which satisfy participants' individual, social and organizational needs and allow participants to fulfil desired roles
  • Policies — Factors such as protocols, norms, or informal and formal policies that support community governance

Motifs and concepts in relation to participants' interactions with a project under each of these areas are organised according to: before and during initial contact; throughout participation and post-participation events. Many of these could actually be placed under more than one of these areas, as these facets of online sociability intersect with each other (Preece, 2001); and volunteer participation in collaborative programs is often cyclical rather than following a simple linear sequence of beginning, middle and end (Rotman et al., 2012). However, this structure of reporting will provide some order for the wide array of determinants identified in the scoping study and review of the relevant literature.


3.1 Purpose — community's shared focus

Before participation begins

Social ties and social capital which exist outside of a project have been identified as beneficial for online sociability of a collaborative project. Successful projects often identify a community with pre-existing social ties (Bak, 2012; Gao et al., 2010; Liao & Chou, 2012). Some institutions benefit from investing time in building social capital with a community before asking them to contribute (Ransom, 2008).

Pre-existing interest in the topic or process of the enterprise is highly motivating for participants, as is indication of the public good that could result from the project. Hence, it is important to connect with the right community of interest (Causer, Tonra, & Wallace, 2012; Holley, 2009b; Mankowski, 2011; NABPP, 2009; Noordegraaf et al., 2014; Raddick et al., 2010; Romeo & Blaser, 2011; Shirky, 2010). Communities with niche or local interests may be small, but they may also be passionate and socially well-integrated. Hence, they should not be neglected (Ransom, 2008).

As participants initially make contact with the project

The goals and underlying purposes of a collaborative project must be clearly presented and communicated to prospective participants, indicating the value and impact of the work and who could benefit from the project (Holley, 2009b; Liao & Chou, 2012; Ling et al., 2005; Locke & Latham, 2002; NABPP, 2009; Noordegraaf et al., 2014; Nov et al., 2011, 2014; Proctor, 2010; Ridge, 2012, 2013; Romeo & Blaser, 2011; Yan & Davison, 2013). A key phrase or image for instance may be helpful to focus the attention of first-time visitors to the project site (Bitgood, 2010).

It must also be clear whether the information contributed will be added to the official record or an existing repository and in what way (Liew, 2014; Rotman et al., 2012). Data should be shared openly as soon as possible and the project should inform volunteers when it has been made openly available. This has been found to increase participants' trust in the organisation and the project concerned. This has also been found to also appeal to volunteers' altruistic or collective motivations for contributing to a meaningful cause, as well as their desire to be recognised for their efforts (Eveleigh et al., 2014; Romeo & Blaser, 2011).

Throughout participation

Provision of a large and quantified challenge has been found to be effective in maximising participants' motivation and contribution rates (Ling et al., 2005; Locke & Latham, 2002; Romeo & Blaser, 2011). Indications of progress towards this big goal are integral to the success of this strategy, both for project productivity and for participants' satisfaction and wellbeing (Deci & Ryan, 2000; Holley, 2010; Locke & Latham, 2002; Romeo & Blaser, 2011). Showing that the community is active builds volunteers' confidence that their efforts will be reciprocated (Gao et al., 2010; Liao & Chou, 2012; Nov et al., 2014).

Establishing a shared vocabulary and 'common language' builds a sense of belonging to the project community, and thus, encourages participation (Liao & Chou, 2012).

Supporting offline interaction and integration of the project with the wider community have also been found to reinforce altruistic motivations and participation rates by demonstrating the value of volunteers' work (Liew, 2014; Rotman et al., 2012).

As the aims and purposes of the project evolve, the project must keep participants and potential participants informed, for instance of any changes in short-term goals and policies (Chung & Lampert, 2011).


3.2 People — individual, social and organizational needs and roles

Before participation begins

Engagement with cultural heritage exhibitions or projects has been conceptualised as a process in which a prospective volunteer balances the perceived costs of participation, such as effort and time, against perceived benefits, whether to self or others, such as learning, fulfilment of needs or alignment with values and principles (Bitgood, 2010; Haefliger et al., 2011; Ye & Nov, 2013). These benefits may be experienced during participation in a task, as intrinsic motivators such as fun or a sense of altruism which arise inherently from the task itself; or they may result indirectly from the completion of the task, as extrinsic motivators such as social status (Ye & Nov, 2013).

This decision-making process occurs both at the moment of initial interaction with the project, and cyclically throughout the volunteer's participation, most noticeably at certain common exit points, such as the end of a task (Rotman et al., 2012). Therefore, sustained involvement in a project relies on an environment in which participants generally experience low barriers and positive benefits from participation.

As participants initially make contact with the project

Most authors agree that entry barriers should be as low as possible (Phillips, 2013; Proctor, 2010; Ridge, 2012). Accreditation through required training may increase some volunteers' commitment to the project (Rotman et al., 2012), and initial registration may in some cases encourage first-time visitors that the community is trustworthy (Virtanen & Malinen, 2008). However these procedures may be ineffective or even counterproductive in attracting and retaining the majority of potential participants, and should be used with care.

The first task presented to a participant must be encouragingly easy (Chung & Lampert, 2011; Eveleigh et al., 2014; Noordegraaf et al., 2014; Phillips, 2013; Proctor, 2010; Ridge, 2012, 2013). Clear, detailed instructions should be provided (Holley, 2009b; Noordegraaf et al., 2014); but task completion should also be guided by scaffolding in the form of constraints and visual cues, as it is best to show as well as to tell (Liew, 2013; Ridge, 2013; Proctor, 2010; Locke & Latham, 2002). In addition, participants who can see others' contributions as examples reported feeling more confident and this has been found to lead to them producing higher quality work (Lee, Hsu, & Chang, 2013; Locke & Latham, 2002). Tasks should be divided into small chunks (Eveleigh et al., 2014; Mankowski, 2011; Nov et al., 2011; Rotman et al., 2012). These measures show a first-time participant that the cost of getting started is low.

At this decisive initial contact phase, a prospective volunteer also needs to see that the potential benefits are high. Feedback on successful task completion (Eveleigh et al., 2014; Noordegraaf et al., 2014; Ridge, 2013), and a choice of clearly graduated levels of difficulty (Eveleigh et al., 2014), demonstrate that even casual participants can quickly gain intrinsic motivation and confidence from their success. Participants have been found to be highly motivated by learning opportunities at this stage (Rotman et al., 2012). So it is essential to show what or how participants can learn from their input (Bitgood, 2010; Chou, 2010; Liao & Chou, 2012; Nov et al., 2014; Verhagen et al., 2012). However, Ling et al. (2005) and Oum & Han (2011) sound a word of warning here: volunteers have strong altruistic and intrinsic motivations, and may react negatively to assert their autonomy against messages which they perceive as manipulative or insincere marketing. As in many areas of project planning, the best ways to strike the right balance are to show, rather than to tell (Ling et al., 2005); and to test the project interface with a small group of pioneer volunteers from the target audience (Ridge, 2012, 2013; Romeo & Blaser, 2011).

Throughout participation

To maintain a participant's engagement with the project, the perceived cost of participation must remain low. An easy, quick, reliable interface with navigable information architecture has been found to limit friction and barriers (Bonastre & Granollers, 2014; Ridge, 2012; Romeo & Blaser, 2011), and supports confidence building and experimentation with the technology, improving users' impressions of the project and increasing participation rates (Chou, 2010).

Again, the perceived benefits must be high. In this regard, there are several types of factors which have been found to contribute to participants' sustained engagement with projects.

Participants derive intrinsic motivation from a sense of autonomy. Therefore, a choice of options and of patterns of participation is important (Deci & Ryan, 2000; Eveleigh et al., 2014; Oomen et al., 2013; Romeo & Blaser, 2011).

Ongoing learning and personal growth are also meaningful for participants and have been found important to sustain their engagement with a project (Deci & Ryan, 2000). Projects can maximise learning opportunities and make progress salient by continuing to supply new or increasing challenges (Holley, 2009b; Noordegraaf et al., 2014; Nov et al., 2014), and by enabling individuals to monitor their own progress and set their own targets (Eveleigh et al., 2014), especially within a session (Locke & Latham, 2002).

Connected with personal growth are issues of self-expression and identity building, which are important to members of many online communities. A program can allow self-expression by representation of identity via the use of unique names and avatar images, and of opinions via discussion forums (Chou, 2010; Zollers, 2007). Recognition of individuals' actual contributions is vital (Chou, 2010; Eveleigh et al., 2014; Gao et al., 2010; Holley, 2009b; Huang & Benyoucef, 2013; Liao & Chou, 2012; Romeo & Blaser, 2011), and there are many ways in which this can be offered. The key for project designers is to "recognise what you want to reinforce" — for example, measure and reward quality as well as quantity if this is what is required (Nov et al., 2014). Allowing participants to view each others' contributions can help them to build status or reputation (Groenewegen & Moser, 2014), and as has previously been noted, this is one of the reasons why low contributors are valuable to a project, as they form the majority of the "essential audience" which promotes motivation for high contributors. Therefore, all participants' input should be encouraged and appreciated (Groenewegen & Moser, 2014; Nonnecke & Preece, 2000; Shirky, 2010). The relationship between low and high contributors can be mutually beneficial when participants can see each others' work, as low contributors often feel more confident to attempt tasks if they can observe before acting. Other features which may help to retain low contributors or tempt them to return include choices, easy well-defined tasks and opportunities to set short-term goals (Locke & Latham, 2002). On the other hand, high contributors may be rewarded by competitive features such as leader tables, or social rewards such as opportunities to take leadership or moderator roles (Eveleigh et al., 2014; Holley, 2010; Liao & Chou, 2012). This type of reputation-based reward is usually classified as an extrinsic motivator, but these rewards have been found to have the potential to enhance volunteers' enjoyment of the task itself (Nov et al., 2014).

Participation in DCH construction can also enhance volunteers' sense of identity outside of the program. Participants may be motivated by accreditation of the skills developed or training undertaken within the program, by attribution on publications or other public-facing literature resulting from their contributions, or by messages showing when their data was used (Rotman et al., 2012). Simply making a project's site or results easy to share via other social media can encourage volunteers to show their wider social networks what good work they are doing in their spare time, as well as potentially attracting more participants (Romeo & Blaser, 2011).

Conversation between the volunteers and professionals involved in a program, and between participants, is widely identified as essential to participants' perceptions that their contribution is meaningful and rewarding (Bonastre & Granollers, 2014; Bray et al., 2011; Gao et al., 2010; Holley, 2009b; Lee et al., 2013; Oum & Han, 2011; Proctor, 2010; Ridge, 2012; Romeo & Blaser, 2011). Project staff should be clearly tasked with the responsibility of responding to participants' messages and actively inviting questions and suggestions from volunteers (Liew, 2014; Chung & Lampert, 2011), as well as responding to participants as equals in the endeavour (Mankowski, 2011; Rotman et al., 2012). Project organisers and participants alike can benefit especially from the insights of the most active volunteers, or those who make the highest quality contributions (Gao et al., 2010; Nov et al., 2014; Proctor, 2010).

Productivity, enjoyment and intention to continue are heightened when participants feel deeply immersed in an activity. High engagement can be engendered by factors such as telepresence, which is the feeling that a user is immersed in a virtual environment and is communicating with others within that environment (Kwon & Wen, 2010; Oum & Han, 2011); or a sense of playfulness (Oum & Han, 2011). This can be a challenge for designers, as what is fun and diverting for one individual may be annoying or distracting for another (Organisciak, 2010), but it is worth aiming for (Lee et al., 2013; NABPP, 2009; Verhagen et al., 2012) by taking into consideration the suitability of activities for the context concerned. Game-like activities suit some participant groups (Nov et al., 2014; Oomen & Aroyo, 2011), while simply allowing for the possibility of unexpected discoveries, and for conversation about those unexpected finds, is preferable for others (Mankowski, 2011; Romeo & Blaser, 2011). Interaction with and between participants in close to real time enhances a sense of sociable telepresence, especially when personal opinions and identities can be expressed (Lee et al., 2013).

Flow states were originally described by Csikszentmihalyi in 1975 as cases of maximal engagement in a task, and have received ongoing attention in the psychological literature (Deci & Ryan, 2000; Ridge, 2013; Yan & Davison, 2013). Individuals experiencing "flow" feel so highly focused and engrossed in a task that they can lose track of time; and flow states are instrinsically motivating, as the experience is so enjoyable. Flow states require a level of challenge which is optimally matched to an invidual's self-perceived ability, and clear immediate feedback on the success of the individual's attempts. To provide the potential for flow states, an interface should limit distractions and unnecessary difficulties (Bitgood, 2010; Bonastre & Granollers, 2014; Ridge, 2012; Romeo & Blaser, 2011), lead smoothly from one task or chunk to the next (Eveleigh et al., 2014), provide feedback on successful task completion (Ridge, 2013), and provide graduated levels of difficulty so that each task is challenging, but not too difficult as to be discouraging.

Post-participation, or between participation sessions

Former participants may return to active involvement and could function as ambassadors for the program and the institution concerned. Continued communications about the project, especially news of interesting new events or opportunities, are often both welcome and have been found to have an effect on continuous engagement and productivity, and maintaining a sense of community (Bonastre & Granollers, 2014; Liao & Chou, 2012).


3.3 Policies — protocols; norms; informal and formal policies; community governance

Before participation begins

Projects can identify themselves with their parent institutions by using the familiar visual language, web space and logos of the parent site (Bonastre & Granollers, 2014), but they will then rely on the goodwill and trust which has been created over time by the interactions of the organisation with its community (Ransom, 2008). As has previously been noted, clearly stated aims, purposes and policies of the project must be easily accessible so that participants can see how their contributions will be managed and used, i.e. are their contributions altered, merged with existing content, shared openly, archived and/or open for reuse? (Liew, 2014).

As participants initially make contact with the project

At initial contact, participants need to know that the project, the institution, and the volunteer community are trustworthy, especially if they will be asked to contribute personal information, such as family stories or photographs (Bonastre & Granollers, 2014; Chou, 2010). Demonstrating the trustworthiness of the volunteer community is less directly controllable by the project organisers. Requiring participants to register is one option, but it is no fail-safe guarantee of courteous behaviour in online communities (O'Neil, 2010; Reagle, 2012). A better strategy is to allow potential participants to view the community's recent activity, so they can judge for themselves whether the culture is one in which they can feel at home. Some correspondents in online communities also feel more secure when they can see each other's demographic details, such as home town or gender (Virtanen & Malinen, 2008). Clear statements of expectations can also help to set the scene (Holley, 2010).

Throughout participation

Ongoing moderation has been found helpful in making participants feel safe and allows them to spend their time on productive contributions or discussions, instead of on conflict resolution (O'Neil, 2010; Reagle, 2012). Participants perceive that an online community is sociable and successful when they believe that they will receive supportive feedback when they ask questions, share unique opinions, or try new things (Locke & Latham, 2002; Mankowski, 2011; McClean, 2011; Rotman et al., 2012; Yu, Lu, & Liu, 2010). However, many volunteer communities are highly trustworthy, requiring minimal moderation, and respond well to respectful recognition of this fact (Holley, 2009b). Each project must weigh its unique circumstances and consider how, and how much, the organisers will attempt to influence the project community (Haefliger et al., 2011). In large or ongoing DCH projects, organisers can progressively support and empower participants to co-moderate the community themselves in accordance with agreed guidelines (Liew, 2013; Nov et al., 2011, 2014; Ridge, 2012; Proctor, 2010). Experienced participants also derive satisfaction, motivation and a sense of community from opportunities to guide or mentor each other; and newer volunteers can gain confidence and improve their contributions through this process (Jahnke, 2010; Nov et al., 2011, 2014; Proctor, 2010).


4 A 'Pathway' Towards Collaborative Construction of Digital Cultural Heritage

This study is an attempt to gain a preliminary understanding of what factors could be important in facilitating and supporting online sociability in DCH projects that involve participation and collaborative construction of cultural heritage knowledge. An inter- and cross-disciplinary literature review was conducted with an aim to scope the literature to uncover main factors and motifs that influence online sociability of collaborative projects. These were discussed in the previous section and in this section a preliminary 'pathway' is presented (see Figure 1). The aim of this is to map and summarise the main motifs that have been found in previous studies to affect online sociability in collaborative and participatory projects. The pathway is expected to provide a reference for researchers, to serve as a research roadmap and to stimulate new ideas in future research on this topic. It is also expected to provide practitioners in DCH projects an overview of key motifs that influence online sociability that they could consider in their projects.

  Before Participation Initial Contact Throughout Participation Post-participation
  • Purpose
  • Social ties
  • Social capital
  • Public good
  • Communities of interest
  • Goals, underlying purpose(s)
  • Trust building
  • Altruistic, collective motivations
  • Indications of progress
  • Sense of belonging
  • Supporting offline interaction
  • Integration with wider community
  • Continuous communications and engagement
  • Maintaining community identity
  • People
  • Low barriers
  • Positive benefits
  • Clear, detailed instructions
  • Task scaffolding
  • Feedback on progress and task completion
  • Graduated levels of difficulty
  • Learning opportunities
  • Confidence building
  • Support for experimentation
  • Sense of autonomy
  • Ongoing learning and sense of personal growth
  • Self-expression and identify building
  • Enjoyment, sense of playfulness
  • Conversation
  • Telepresence
  • Flow states
  • Policies
  • Accessible and clear statements of project goal and purposes,
    protocols, norms, informal and formal policies
  • Trustworthiness of project and community
  • Clear statements of expectations
  • Ongoing moderation
  • Participants empowerment

Figure 1: Preliminary 'Pathway'

The scoping review shows that although there are many existing projects to look to as models, there is no single recipe for planning participatory and collaborative programs. Project managers and coordinators are encouraged to "plan for the audience you want" (Ridge, 2012), as no two projects or volunteer groups are exactly alike (Eveleigh et al., 2014; Ling et al., 2005; Nov et al., 2014). Results and reactions can be surprising, and many initiatives have achieved success by starting small and designing iteratively in response to participant feedback (Ridge, 2012; Romeo & Blaser, 2011). Open communication with participants about the development process is critical to maintaining trust in these cases and also to empower participants to take ownership in guiding the growth of the project.

Further research is needed to empirically validate the relevance of these determinants of online sociability and to uncover other determinants that may be applicable to DCH projects. It would be useful to develop an integrative research framework which can be used to inform institutional policy and to guide practice decisions when developing and nurturing sustainable cultures of participation in collaborative construction of DCH and other similar endeavours.



I would like to acknowledge the Victoria University of Wellington — Faculty Research Fund (FCA) Grant/Project 115007 for supporting the undertaking of this research and the research assistance provided by Melissa Bryant.



[1] Bak, G. (2012). Building the Crowd: Archives as Social Networks. Presentation at the CHS | High-Tech Heritage 2012 Conference; slides accessed courtesy of Greg Bak, University of Manitoba, Canada.

[2] Bitgood, S. (2010). An attention-value model of museum visitors. The Center for the Advancement of Informal Science Education.

[3] Bonastre, L., & Granollers, T. (2014). A Set Of Heuristics for User Experience Evaluation in E-commerce Websites (pp. 27-34). Presented at the ACHI 2014, The Seventh International Conference on Advances in Computer-Human Interactions.

[4] Bray, P., Chan, S., Dalton, J., Dietrich, D., Kapsalis, E., Springer, M., & Zinkham, H. (2011). Rethinking Evaluation Metrics in Light of Flickr Commons. Paper presented at Museums and the Web 2011, April 6-9, 2011, Philadelphia, USA.

[5] Causer, T., Tonra, J., & Wallace, V. (2012). Transcription maximized; expense minimized? Crowdsourcing and editing The Collected Works of Jeremy Bentham. Literary & Linguistic Computing, 27(2), 119-137.

[6] Chou, S.-W. (2010). Why do members contribute knowledge to online communities? Online Information Review, 34(6), 829-854. http://doi.org/10.1108/14684521011099360

[7] Chung, S. K., & Lampert, C. (2011). Strategic planning for sustaining user-generated content in digital collections. Journal of Library Innovation, 2(2), 74+.

[8] Deci, E. L., & Ryan, R. M. (2000). The "What" and "Why" of Goal Pursuits: Human Needs and the Self-Determination of Behavior. Psychological Inquiry, 11(4), 227-268.

[9] Eveleigh, A., Jennett, C., Blandford, A., Brohan, P., & Cox, A. L. (2014). Designing for Dabblers and Deterring Drop-outs in Citizen Science. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2985-2994). New York, NY, USA: ACM. http://doi.org/10.1145/2556288.2557262

[10] Gao, Q., Dai, Y., Fan, Z., & Kang, R. (2010). Understanding factors affecting perceived sociability of social software. Computers in Human Behavior, 26(6), 1846-1861. http://doi.org/10.1016/j.chb.2010.07.022

[11] Groenewegen, P., & Moser, C. (2014). Online Communities: Challenges and Opportunities for Social Network Research. Research in the Sociology of Organizations, 40, 463-477. http://doi.org/10.1108/S0733-558X(2014)0000040023

[12] Haefliger, S., Monteiro, E., Foray, D., & von Krogh, G. (2011). Social Software and Strategy. Long Range Planning, 44(5-6), 297-316. http://10.1016/j.lrp.2011.08.001

[13] Holley, R. (2009). Crowdsourcing and social engagement: potential, power and freedom for libraries and users. (Report).

[14] Holley, R. (2009b). Many Hands Make Light Work: Public Collaborative OCR Text Correction in Australian Historic Newspapers. Australia: National Library of Australia.

[15] Holley, R. (2010). Crowdsourcing: How and Why Should Libraries Do It? D-Lib Magazine, 16(3/4). http://doi.org/10.1045/march2010-holley

[16] Huang, Z., & Benyoucef, M. (2013). User-Centered Investigation of Social Commerce Design. In A. A. Ozok & P. Zaphiris (Eds.), Online Communities and Social Computing (pp. 287-295). Springer Berlin Heidelberg.

[17] Jahnke, I. (2010). Dynamics of social roles in a knowledge management community. Computers in Human Behavior, 26(4). http://doi.org/10.1016/j.chb.2009.08.010

[18] Kwon, O., & Wen, Y. (2010). An empirical study of the factors affecting social network service use. Computers in Human Behavior, 26(2), 254-263. http://doi.org/10.1016/j.chb.2009.04.011

[19] Lee, C.-C., Hsu, S. H., & Chang, J.-W. (2013). Factors influencing sociability in educational MMORPGs — a fuzzy AHP approach. Internet Research, 23(3), 298-315. http://doi.org/10.1108/10662241311331745

[20] Liao, S., & Chou, E. (2012). Intention to adopt knowledge through virtual communities: posters vs lurkers. Online Information Review, 36(3), 442-461. http://doi.org/10.1108/14684521211241440

[21] Liew, C. L. (2014). Participatory Cultural Heritage: A Tale of Two Institutions' Use of Social Media. D-Lib Magazine, 20(3/4). http://doi.org/10.1045/march2014-liew

[22] Liew, C. L. (2013). Digital cultural heritage 2.0: a meta-design consideration. Information Research, 18(3).

[23] Ling, K., Beenen, G., Ludford, P., Wang, X., Chang, K., Li, X., ... Kraut, R. (2005). Using Social Psychology to Motivate Contributions to Online Communities. Journal of Computer-Mediated Communication, 10(4), 00-00. http://doi.org/10.1111/j.1083-6101.2005.tb00273.x

[24] Locke, E. A., & Latham, G. P. (2002). Building a Practically Useful Theory of Goal Setting and Task Motivation. American Psychologist, 57(9), 705-717.

[25] Mankowski, T. A. (2011). An Interpretive Study of Meanings Citizen Scientists Make When Participating in Galaxy Zoo (M.S.T.). University of Wyoming, United States.

[26] McClean, G. (2011). Multicultural Sociability, Imperfect Forums and Online Participation. International Journal of Communication (1932-8036), 5, 1649-1668.

[27] NABPP. (2009). North American Bird Phenology E-Newsletter 2009.

[28] Naguib, S.-A. (2013). Museums, Diasporas and the Sustainability of Intangible Cultural Heritage. Sustainability, 5(5), 2178-2190. http://doi.org/10.3390/su5052178

[29] Nonnecke, B., & Preece, J. (2000). Lurker Demographics: Counting the Silent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 73-80). New York, NY, USA: ACM. http://doi.org/10.1145/332040.332409

[30] Noordegraaf et al. (2014). Modeling Crowdsourcing for Cultural Heritage | MW2014: Museums and the Web 2014.

[31] Nov, O., Arazy, O., & Anderson, D. (2011). Technology-Mediated Citizen Science Participation: A Motivational Model. In Fifth International AAAI Conference on Weblogs and Social Media.

[32] Nov, O., Arazy, O., & Anderson, D. (2014). Scientists@Home: What Drives the Quantity and Quality of Online Citizen Science Participation? PLoS ONE, 9(4), e90375. http://doi.org/10.1371/journal.pone.0090375

[33] O'Neil, M. (2010). Shirky and Sanger, or the costs of crowdsourcing.

[34] Oomen, J., Balthussen, L. B., Brinkerink, M., & van Exel, T. (2013). Sound of the Netherlands: Crowdsourcing the Dutch Soundscape. Paper presented at MW2013: Museums and the Web 2013, April 17-20, 2013, Portland, OR, USA.

[35] Oomen, J., & Aroyo, L. (2011). Crowdsourcing in the Cultural Heritage Domain: Opportunities and Challenges. In Proceedings of the 5th International Conference on Communities and Technologies (pp. 138-149). New York, NY, USA: ACM. http://doi.org/10.1145/2103354.2103373

[36] Organisciak, P. (2010). Why Bother? Examining the Motivations of Users in Large-Scale Crowd-Powered Online Initiatives. MA Thesis in Information Studies, Edmonton, Alberta, Canada.

[37] Oum, S., & Han, D. (2011). An empirical study of the determinants of the intention to participate in user-created contents (UCC) services. Expert Systems with Applications, 38(12), 15110-15121. http://doi.org/10.1016/j.eswa.2011.05.098

[38] Past appeals | Oxford English Dictionary Online. (n.d.). Oxford English Dictionary.

[39] Phillips, L. B. (2013). The Temple and the Bazaar: Wikipedia as a Platform for Open Authority in Museums. Curator: The Museum Journal, 56(2), 219-235. http://doi.org/10.1111/cura.12021

[40] Preece, J. (2001). Sociability and usability in online communities: determining and measuring success. Behaviour & Information Technology, 20(5), 347-356. http://doi.org/10.1080/01449290110084683

[41] Proctor, N. (2010). Digital: Museum as Platform, Curator as Champion, in the Age of Social Media. Curator: The Museum Journal, 53(1), 35-43. http://doi.org/10.1111/j.2151-6952.2009.00006.x

[42] Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P., Schawinski, K., ... Vandenberg, J. (2010). Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. Astronomy Education Review, 9(1), 010103-1. http://doi.org/10.3847/AER2009036

[43] Ransom, J. (2008). Kete Horowhenua: the story of the district as told by its people. Paper presented at the 2008 VALA conference, Mebourne, 5-7 February 2008.

[44] Reagle, J. (2012). "Free as in sexist?" Free culture and the gender gap. First Monday, 18(1).

[45] Ridge, M. (2012, April 30). Designing for participatory projects: emergent best practice, getting discussion started.

[46] Ridge, M. (2013). From Tagging to Theorizing: Deepening Engagement with Cultural Heritage through Crowdsourcing. Curator: The Museum Journal, 56(4), 435-450. http://doi.org/10.1111/cura.12046

[47] Romeo, F., and Blaser, L. (2011). Bringing Citizen Scientists and Historians Together. In J. Trant and D. Bearman (Eds). Museums and the Web 2011: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2011.

[48] Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., ... Jacobs, D. (2012). Dynamic Changes in Motivation in Collaborative Citizen-science Projects. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (pp. 217-226). New York, NY, USA: ACM. http://doi.org/10.1145/2145204.2145238

[49] Shirky, C. (2010). Cognitive surplus: creativity and generosity in a connected age. New York: Penguin Press.

[50] Verhagen, T., Feldberg, F., van den Hooff, B., Meents, S., & Merikivi, J. (2012). Understanding users' motivations to engage in virtual worlds: A multipurpose model and empirical testing. Computers in Human Behavior, 28(2), 484-495. http://doi.org/10.1016/j.chb.2011.10.020

[51] Virtanen, T., & Malinen, S. (2008). Supporting the Sense of Locality with Online Communities. In Proceedings of the 12th International Conference on Entertainment and Media in the Ubiquitous Era (pp. 145-149). New York, NY, USA: ACM. http://doi.org/10.1145/1457199.1457231

[52] Yan, Y., & Davison, R. M. (2013). Exploring behavioral transfer from knowledge seeking to knowledge contributing: The mediating role of intrinsic motivation. Journal of the American Society for Information Science and Technology, 64(6), 1144-1157. http://doi.org/10.1002/asi.22820

[53] Ye, C., & Nov, O. (2013). Exploring user contributed information in social computing systems: quantity versus quality. Online Information Review, 37(5), 752-770. http://doi.org/10.1108/OIR-05-2012-0091

[54] Yu, T.-K., Lu, L.-C., & Liu, T.-F. (2010). Exploring factors that influence knowledge sharing behavior via weblogs. Computers in Human Behavior, 26(1), 32-41. http://doi.org/10.1016/j.chb.2009.08.002

[55] Zollers, A. (2007). Emerging Motivations for Tagging: Expression, Performance, and Activism.

Appendix 1

Reference Domain(s)/Field(s) of study/practice and country of authorship(s) Relevant research question(s) Notes on methodology
Bak, 2012 archives and social media; Canada How are archives using commercially-provided social media platforms? case studies
Bitgood, 2010 museum visitor studies; USA What is "visitor attention"? To what do visitors pay attention while viewing exhibitions? Why do visitors attend? What is the motivation for attending? How do the processes or mechanisms that explain visitor attention work? What factors interfere with paying attention to important exhibit elements? conceptual model development
Bonastre & Granollers, 2014 heuristics development, usability, e-commerce; Spain How can heuristics be used to evaluate user experience in e-commerce websites? analysis of three studies recommending functional requirements for online bookstores; development of a set of heuristics to evaluate user experience in e-commerce websites
Bray et al., 2011 tagging, social media, DCH; Australia/USA How are institutions evaluating their participation in Flickr Commons? analysis of five case studies
Causer et al., 2012 DCH, crowdsourced manuscript transcription; England/USA How will the participatory project impact upon long-established editorial practices? Is crowdsourcing the transcription of complex manuscripts cost-effective? Is it exploitative? Are volunteer-produced transcripts of sufficient quality for editorial use and uploading to a digital repository, and what quality controls are required? Does crowdsourcing ensure sustainability and widen access to this priceless material? Should the success of such projects be measured solely according to cost-effectiveness, or also by considerations of public engagement and access? case study; participant survey; analysis of transcriptions produced
Chou, 2010 information management, online communities, computing; Taiwan What are the relationships between individuals' differences, intrinsic and extrinsic motivation, and knowledge contribution in online communities? proposal of a research framework based on a motivational model and social cognitive theory; case studies of two online communities: the Electronic Engineering Times in Taiwan and China were surveyed
Chung & Lampert, 2011 Sustainability and preservation in DCH projects, planning, libraries; USA What are the challenges inherent in efficiently managing social media and user-generated content? Who are the stakeholders? How can libraries develop flexible and supportive organizational frameworks to sustain and deliver on the promise of social computing? literature review, case study, proposal of lists of considerations for planning
Deci & Ryan, 2000 psychology; USA What is the self-determination theory of human motivation? How does it relate to needs, regulatory processes underlying goal pursuits, well-being, social contexts, individual differences and other contemporary motivation theories? literature review regarding the development of theory over decades
Eveleigh et al., 2014 citizen science, motivation and engagement; UK What are the experiences of the majority of contributors to the Old Weather project — the small-scale contributors? What are the implications for project design? literature review; case study; survey and interviews
Gao et al., 2010 human-computer interaction; China What factors affect perceptions related to the potential sociability of communication technologies such as SMS or email among undergraduate students? interviews; surveys
Groenewegen & Moser, 2014 organisational sociology, social network theories, online communities How does social network theory apply to the study of online communities, especially with regard to tie formation and network structures? literature review; case study of a cake decorating community in the Netherlands incorporating log file data
Haefliger et al., 2011 corporate management; Norway/France/Switzerland Relevant research issues for corporate social software planning include: What are the organisational conditions for long-term value co-creation? How can power relationships be made transparent and ethical? To what extent should organisations lead projects' communities, how, and when? literature review and proposal of a research framework
Holley, 2009 DCH crowdsourcing; Australia What are the commonalities between successful crowdsourcing projects? How can the lessons learnt be applied across the library and archive sector, and what is the future potential? comparison of case studies
Holley, 2009b DCH crowdsourcing; Australia Report on the beginnings of the Australian Newspapers Digitisation Program case study; participant surveys
Holley, 2010 DCH crowdsourcing; Australia How and why should libraries engage in crowdsourcing? examination of case studies; participant surveys; statistics on data produced by projects
Huang & Benyoucef, 2013 social commerce design; Canada Which design features on social commerce websites are most important to users? online survey of users' perceptions and preferences of social features on social commerce websites
Jahnke, 2010 human-computer interaction; Germany How do roles change over time among users of a knowledge management system for undergraduate students at a German university? long-term design-based research; iterative design and assessment by survey and interviews
Kwon & Wen, 2010 human-computer interaction; Korea How do social identity, telepresence and altruism affect users' intention to use social network services? survey; modification of Davis' technology acceptance model
Lee et al., 2013 human-computer interaction in educational MMORPGs /serious games; Taiwan What is the relative importance of factors affecting sociability in educational MMORPGs? hybrid methodology combining fuzzy logic techniques and analytic hierarchy process approach; interviews and survey
Liao & Chou, 2012 human-computer interaction in bulletin board services, psychology; Taiwan What are the social capital and technical determinants of knowledge adoption intentions in virtual communities? Are these factors viewed differently by posters and lurkers? questionnaire used to test hypotheses about correlations of factors in a model of motivations
Liew, 2014 participatory DCH; New Zealand What are some differences between New Zealand participatory DCH projects in terms of content, culture and participation rates? comparison of two case studies; analysis of user-contributed content such as images, stories and comments
Liew, 2013 meta-design of DCH projects; New Zealand In what ways can cultural heritage institutions respond to the opportunities and challenges brought on by the social Web which involves variability, mutability and uncertainty? conceptual analysis and development of conceptual framework
Ling et al., 2005 social psychology, computer-mediated communication; USA Can theories of social loafing and goal-setting predict participation rates in response to different explanations of the value of contributions from members of an online community? application of Karau and William's collective effort model and Locke and Latham's Goal-setting Theory; testing design principles in field experiments with members of an online movie recommender community
Locke & Latham, 2002 social psychology; USA What are the core findings of goal theory, moderators of goal effects, relation between goals and satisfaction, practical significance of goal-setting theory, and relationships of goal setting to other theories? meta-study/review of thirty five years of literature on goal theory
Mankowski, 2011 physics and astronomy, citizen science; USA What are the motivations of participants? interpretive research; grounded theory analysis of project forum discussions, including lexical analysis
McClean, 2011 communications/media studies, online community engagement in a public service broadcasting company; Australia What is multicultural sociability? Are conditions for multicultural sociability facilitated by the online chat and comments that follow a television current affairs forum on Australia's national public broadcaster the Special Broadcasting Service? case studies analysis of forum comments, with iterative coding of type of interaction reflecting the diversity of the data
NABPP, 2009 citizen science; USA What are the motivations of volunteers? participant survey; case study
Naguib, 2013 museum studies, sustainability of intangible cultural heritage; Norway What is the work of museums in constructing the intangible cultural heritage of migration and diasporas? How can collaborative museum projects keep intangible heritage alive for peoples who have immigrated to Europe? literature review; examination of case studies
Nonnecke & Preece, 2000 human-computer interaction; USA How prevalent are lurking and low contribution rates in each of two email discussion lists? Is there a difference between lurking rates in the health related and software support related lists? What are the relationships between lurking and overall traffic rates? statistical analysis of contributions to two email discussion list over three months
Noordegraaf et al., 2014 design and evaluation of effective DCH crowdsourcing projects; Netherlands What are the effects of project design properties on the success or failure of DCH projects? literature analysis; close study of two cases
Nov et al., 2011 citizen science, business, psychology; USA/Canada What are the effects of task granularity and of different types of motivation on participation rates and intentions? survey of participants and comparison with actual contributions; model building
Nov et al., 2014 citizen science, business, psychology; USA/Canada How are quantity and quality of contributions affected by different types of motivation and incentives, and by reputation? longitudinal study of a set of cases; comparison of survey results with actual contributions; modification of the social movement participation model by drawing on Self-Determination Theory
O'Neil, 2010 human-computer interaction, volunteering; online collaboration; France/Australia/USA How are experts defined in collaborative encyclopaedia projects? What are the relative costs of different types of systems for identifying expertise? comparison of case studies
Oomen et al., 2013 DCH, public contribution of content; Netherlands What are some issues in recruiting participants to contribute content to an online sound archive? case study
Oomen & Aroyo, 2011 DCH, crowdsourcing; Netherlands What types of crowdsourcing activities are undertaken by cultural heritage institutions? What are the relationships between the different types of crowdsourcing and the core activities of heritage organizations? What are the critical challenges in finding sufficient knowledgeable and loyal users and in maintaining quality of contributions? empirical study of a substantial amount of projects initiated by relevant cultural heritage institutions
Organisciak, 2010 DCH, e-commerce, private enterprise; Canada What are users' motivations to participate in e-commerce sites? theory-building about motivations, based on studying the websites and reports of private and DCH projects, plus interviewing a small number of internet users
Oum & Han, 2011 human-computer interaction; South Korea What factors affected intention to participate in an online user-contributed content system among undergraduate students in Jeonju University, South Korea? comparison of correlations between survey factors
Phillips, 2013 open-source software, DCH; USA How can museums' role as content providers be augmented with the role of platform provider? How does Wikipedia provide a model of open authority for museums? literature review
Preece, 2001 information systems, human-computer interaction; USA What are the similarities and differences between usability and sociability? literature review
Proctor, 2010 participatory DCH; USA Why should institutions do participatory work? What makes a participatory culture? literature review; case studies
Raddick et al., 2010 citizen science; USA What are the motivations of volunteers? participant survey
Ransom, 2008 DCH; New Zealand What factors contributed to the initiation, design and success of the Kete Horowhenua project and community? case study
Reagle, 2012 communication studies; free/libre and open-source software (FLOSS) How do unstated norms and unofficial power structures cause gender imbalance in FLOSS communities such as Wikipedia and Ubuntu? How can content and collaboration be enhanced by more balanced participation? case studies; discourse analysis
Ridge, 2012 DCH; Australia What is best practice in participatory project design? case studies
Ridge, 2013 DCH, citizen science, crowdsourcing, Games with a Purpose (GWAP); Australia What is the value of crowdsourcing DCH for participants, institutions and users of the resulting resources? How does scaffolding support participation? What motivates volunteers? How does participation enable deep engagement with heritage? What design factors contribute to the success of projects? literature review; case studies
Romeo & Blaser, 2011 DCH, citizen science; UK What motivates volunteers? What is best practice in participatory project design? literature review; case studies; construction of a model for best practice in crowdsourcing
Rotman et al., 2012 human-computer interaction, citizen science; USA How do participant motivations evolve during participation in a project? participant survey and interviews
Shirky, 2010 social media, psychology, commerce, media studies; USA What design and planning strategies enhance success in participatory projects? literature review; case studies
Verhagen et al., 2012 human-computer interaction, virtual worlds; Netherlands/Finland What motivates users to engage in the virtual world Second Life? literature review; empirical testing of a model of user motivations based on information system value and motivation theory
Virtanen & Malinen, 2008 social media, online communities, user-contributed content, human-computer interaction; Finland How can social interaction and sense of place in geography-based communities be supported by online communities, such as Facebook location groups or a local-community service provided by a Finnish newspaper? comparison of case studies; participant survey and interviews
Yan & Davidson, 2013 management, computer-human interaction, computer science, psychology; China/Hong Kong How can self-perception theory and internal motivation factors explain the transition from knowledge seeking to knowledge contributing among users of online business discussion communities in China? literature review, empirical testing by survey of a model of user motivations based on self-perception theory and internal motivation factors
Ye & Nov, 2013 human-computer interaction, user-contributed content; USA What is the relationship between quantity and quality of user contributed information in the online social computing system Flickr? How are these two performance outcomes associated with motivational, cognitive and social factors, such as reputation, social ties or expected rewards including self-development? literature review, empirical testing of a model using a survey
Yu et al., 2010 computer-human interaction, professional information-sharing networks; Taiwan What factors optimise knowledge sharing in online professional information-sharing networks in Taiwan? survey of twenty self-selected, self-described participants in each of three online professional information-sharing networks
Zollers, 2007 social tagging; USA How do users utilize tags for social purposes, rather than for folksonomic information organization? analysis of two free-for-all systems: a commercial site (Amazon.com), and a music site (Last.fm), using a snowball sampling technique to identify co-occurring tags with sociable purposes

About the Author


Chern Li Liew is a Senior Lecturer in the School of Information Management, Victoria University of Wellington (New Zealand). She holds a PhD in Information Studies from Nanyang Technological University (Singapore) and an MSc from Loughborough University (UK). Her research interests center on digital information services as socio-technical environments and she has published in the areas of digital libraries and digital cultural heritage: http://www.victoria.ac.nz/sim/about/staff/chernli-liew. She is on the editorial advisory boards of "International Journal of Digital Library Systems" and "Online Information Review". She is also a member of the Executive Committee of the Consortium of iSchools Asia Pacific.

transparent image