This paper analyzes the data now available in Australia's coordinated Electronic Theses and Dissertations (ETD) gateway to show the impact of high-level institutional policy decisions on population of the individual repositories. The paper shows that just like research article repositories, voluntary ETD deposition results in repositories collecting less than 12% of the available theses, whereas mandatory policies are well accepted and cause deposit rates to rise towards 100%. Modeling of the Ph.D. and Master process in Australia is also carried out to indicate the delays and liabilities to be expected if mandatory policies are applied only to newly enrolled candidates.
Participation and software
The Australian university system has a common gateway to electronic theses and dissertations (ETDs) produced in its universities. This gateway, the Australian Digital Theses Program (ADT Program, 2005), harvests from 29 (74%) of the 38 Australian universities (there are 30 repositories, but two of them belong to one university), and is in its turn harvested by Google, Yahoo! and scholarly search engines. The remaining 9 universities have not yet established local ETD repositories. Nevertheless, the Program has a potential coverage of 84% of Australian university research degrees, since many of the non-contributors are smaller universities without strong research programs.
The participation rate must be regarded as a significant success by the ADT Program in a short time. An agreement has recently been reached with CONZUL (the Committee of New Zealand University Libraries) to have the eight New Zealand universities also contribute ETDs, but only one has as yet established a repository. The current situation is shown in Figure 1. The ADT Program has been running since 2000, for a sufficiently long time that an analysis of its success can be undertaken as a work in progress.
The gateway harvests metadata from (local) leaf repositories in the universities for its own database. Searchers who find the gateway metadata pages either through a search engine or via a direct search of the gateway are directed to the leaf repository, which is the only place that the 'full-text' is stored. While the ADT Program is oriented around text documents that are required to be in pdf format, a few theses have associated materials such as software, again only stored in the leaf repositories.
Currently, 26 of the 30 active repositories run an old modified version of Virginia Tech software that is not harvested directly by any robot other than the ADT Program's harvester. Two universities (Western Australia and Curtin) use in-house solutions based around their library catalogue and one other (Melbourne) feeds ETDs from their OAI-PMH-compliant EPrints repository, which also holds research articles. The software used by Melbourne was written by Tasmania, which used it as the production repository for six months; however a strange decision by the University Librarian then resulted in a separate installation of the Virginia Tech software. Currently, Tasmania's two repositories mirror each other. Melbourne and Tasmania are the only two universities whose theses are directly harvested by general and scholarly search engines, though they may be shortly joined by others. The software situation is shown in Figure 2.
Content in the Repositories
The theses deposited with the ADT Program were analyzed by university (Figure 3), by searching ADT on 'university' and restricting output to open-access ETDs (in other words excluding metadata-only records). There are a couple of surprising high bars in the chart.
The six that exceed 300 ETDs (an arbitrary level) do not all appear to be obvious candidates. The league table of documents and the corresponding number of 2003 research degree completions is shown in Table 1; the rank is also based on the 2003 degree completions nationwide.
Table 1 Ranking of universities with highest ADT depositions
On analysis, the higher number of theses in these universities appears to derive from a higher rate of investment in digitizing theses that were first accepted and stored in paper form. A later section documents this behavior. This is to be welcomed, but it does not reveal how the universities are adapting to the 21st century and the Internet, or dealing with the imperatives that the open access movement imposes. Accordingly, a more detailed analysis of current deposition rates was undertaken.
The Australian Government collects data on the completions of graduate research degrees (Ph.D. and Master by research), which is publicly available (AVCC, 2005). It is therefore an easy matter to compare this with what is actually happening in the ADT gateway, except that the data is currently available only up to 2003. Accordingly 2003 was chosen as the benchmark year against which to compare ETD deposits. Surprisingly, Table 2 shows that so far only an estimated 12% of the current 2005 Australian thesis production has ended up as ETDs in the ADT system. What is the problem?
Table 2 Overall success rate in attracting deposits
It is probable that the problem is the same one that is the bane of research article repositories. If graduates are left to voluntarily contribute an ETD copy of their thesis, few do, and apparently usually at a level at or below 15% (Hajjem et al. 2005). Though the extra work to provide an electronic copy is trivial and the thesis is almost always produced electronically anyway, this is just avoidable work and consequently it is avoided (Swan & Brown, 2005). The reasons for trying to capture all theses at the point of submission are worth restating:
Accordingly it was decided to analyze thesis accumulations, comparing them with the known completions, and record whether deposit of an ETD was voluntary or mandatory (required in a Rule or Regulation). Twelve Australian universities had mandatory policies at the time of writing as shown in Table 3.
Table 3 State of mandatory policies in Australia
It will be noted that in six cases this policy applies to all submissions after a given date and thus have immediate effect. Such policies are rated as having a potential effectiveness of 1.0 (or 100%) after their cut-in date. Another six universities have chosen to have the policy apply to all candidates who enrolled after a given date. This causes the effectiveness to increase gradually with time. Few Ph.D. candidates complete in three years; by four years some but not all of the full-time candidates have completed, and eight years is the normal maximum for part-time candidature, but even this may be extended by suspensions. Master results appear more quickly. Accordingly, a model-based effectiveness is assigned to all enrolment date policies in force. The modeling is described in a later section.
Figure 4 shows the actual deposits into ADT in 2005, derived from searching the database by university and graduation date = '2005' again restricted to open-access ETDs, expressed as a percentage of 2003 completions (graduations). These are derived from the Australian Government's data on Ph.D. and research Master completions for 2003. The data was collected in March 2006, by which time all point-of-capture ETDs should have been deposited.
The results are the same as they are for research articles: a mandatory deposit policy pays off handsomely. Or to put it into a 2 line octosyllabic aphorism:
All five universities with effective policies in this chart (Griffith, Murdoch, Queensland University of Technology, Southern Queensland and Wollongong) show that a mandatory policy creates the climate for compliance, and is not overly resisted (Swan & Brown, 2005). Indeed all of the universities that have a deposit rate greater than 20% have a mandatory policy. Quite striking is the effect of QUT's policy, which only took effect in 2005; however, there is probably some synergy from a parallel research article self-archiving policy.
The results are quite clear: a mandatory policy causes deposits to rise to at least 50-80%, compared to the general voluntary policy rate of 5-15%. It is expected that over time a mandatory policy would rise towards 100% deposit, but attitudinal change takes some time.
Given this data it is surprising that the Australian Government has not specified that candidates who received a government-funded scholarship (APA) must deposit an electronic copy of their thesis with the degree-granting university. They are aware of the issue, but choose to not become involved in the maximizing the outcomes of taxpayer funds. Of course, deposit does not mean 'open access' it simply means that the ETD describing the research is captured in electronic form, with its metadata. Issues relating to subsequent exploitation by publication or commercial use are not affected.
Modeling of Enrolment-date Policies
In a previous section, the effective implementation rates of the two mandatory policies were mentioned. Here, the figures chosen as the probable target of a mandatory policy are justified.
If a mandatory policy applied to all thesis submissions after a given date, this is assumed to bring a 100% target into immediate effect. Such policies are rated as having a potential effectiveness of 1.0 (or 100%) relative to completions, if the decision year is the same as or predates the year under examination (in this paper 2004).
Other universities have chosen to have the policy apply to all candidates who enrolled after a given date, usually the date of the decision. This causes the effectiveness to increase with time. Data on the time distribution of completions was not readily available across Australia. Three years of completions (2003-2005 comprising 314 Ph.D.s and 82 Masters) from the University of Tasmania were therefore analyzed and tracked back to their enrolment date (Folvig, 2005). Figure 5 shows the distribution of the elapsed year of completion for these graduates.
There is no reason to suspect that the University of Tasmania is substantially different from other Australian universities, and accordingly, these distributions are used to model the effect of mandatory policies dated from enrolment.
The available evidence suggests that voluntary policies lead to 12% compliance (the ADT average over Australia, also close to the 15% average for research article repositories, see Hajjem et al. 2005). Accordingly a realistic target T for an enrolment-date policy was computed from
T = (i × 1.0) + ((1 i) × 0.15) = 0.15 + (i × 0.85)
This assumes that all candidates for whom the policy has cut in are recorded at 100%, while the other candidates with enrolment dates preceding the policy change will voluntarily deposit at 15%. This function is graphed for Master and Ph.D. degrees in Figure 6. It will be seen that an enrolment-date policy does not reach 80% effectiveness (for either kind of degree) until five years after the decision is made!
The value of T for 2004 for all universities with an enrolment-date policy is shown in Table 4.
Table 4 Target effect of mandatory policies
Costs of delays in instituting fully effective mandatory policies can also be estimated, although it might be better to describe them as liabilities, which will be called to account if 100% deposition rate is considered desirable (this should be indisputable) and made retrospective.
To meet such a target, theses that had not been deposited as electronic copies would need to be scanned and uploaded. It is assumed that a retrospective attempt to capture electronic copies of old theses would have only small success, since academic and graduate record keeping and response rates are likely to be poor. A recent ProQuest study estimates the average time required for scanning at 2hr (Talmacs, 2005), and the average cost at $130/thesis (estimate, unpublished correspondence).
It was asserted earlier in the paper that the current thesis counts in ADT were skewed by retrospective deposits. Accordingly, the data for the University of Western Sydney and Curtin University of Technology were analyzed in detail. These are the two largest numerical depositors in the ADT program but they rank 18th and 9th in the Ph.D.-producing universities. The results of the analysis are shown in Figure 7.
Since the ADT Program was only opened up to general deposition in July 2000, all theses from 2000 and before (and some thereafter) are derived from retrospective scanning or serendipitous capture at an earlier time. The fraction is significant. Deposit rates across other universities also suggest that there is a relatively high degree of retrospectivity and a correspondingly lower level of current ETD capture. The success in gaining thesis content is disguising a comparative failure to collect born-digital ETDs at the point of final lodgment.
For the Australasian Digital Theses Program
For the Australian Government
The author would like to thank:
This work was carried out in the School of Computing, University of Tasmania.
AuseAccess (2005). Australian Open Access Wiki available at <http://leven.comp.utas.edu.au/AuseAccess/>.
Australian Vice-Chancellors Committee (2005). HERDC Time Series Data 1992-2003. Available at <http://www.avcc.edu.au/documents/publications/stats/HERDCTimeSeriesData1992-2003.xls>
Folvig V, University of Tasmania (2005). Unpublished communication regarding completion data at the University of Tasmania.
Hajjem C, Harnad S, Gingras Y (2005). Ten-Year Cross-Disciplinary Analysis of the Growth of Open Access and Its Effect on Research Citation Impact. IEEE Data Engineering Bulletin 28(4) pp. 39-47. Accessed at <http://eprints.ecs.soton.ac.uk/11688/01/ArticleIEEE.pdf>.
Swan A and Brown S. (2005) Open access self-archiving: An author study.
Swan A and Brown S (2004). ISC/OSI Journal Authors Survey Report.
Talmacs K, University of New South Wales (2005). Unpublished communications regarding costs of thesis scanning.
Copyright © 2006 Arthur Sale