Technical Report
ECSTR-IAM03-005 (ISBN: 0854 32798 3)
School of Electronics and Computer Science, University of Southampton
Evaluating Citebase, an open access Web-based citation-ranked search and
impact discovery service
Steve Hitchcock, Arouna Woukeu, Tim Brody, Les Carr, Wendy
Hall and Stevan Harnad
Open Citation Project, IAM Group, Department of Electronics and Computer
Science, University of Southampton, SO17 1BJ, United Kingdom
Contact for correspondence: Steve Hitchcock sh94r@ecs.soton.ac.uk
Version history of this report
Version 1.0, official report to
JISC, released to selected users and evaluators December 2002
This version 2.0, edited for publication as a Technical Report,
July 2003 (official TR format pdf;
html)
Version 3.0, focus on
usability results, edited for journal publication, draft, July 2003
Abstract
Citebase is a new citation-ranked search and impact discovery service that
measures citations of scholarly research papers which are openly accessible
on the Web, i.e. papers that are assessable continuously online. Other
services, such as ResearchIndex, have emerged in recent years to offer
citation indexing of Web research papers. In the first detailed user evaluation
of an open access Web citation indexing service, Citebase has been evaluated
by nearly 200 users from different backgrounds. The paper details the procedures
used in the evaluation, and analyses the results of this study, which took
place between June and October 2002. It was found that within the scope
of its primary components, the search interface and services available
from its rich bibliographic records, Citebase can be used simply and reliably
for the purpose intended, and that it compares favourably with other bibliographic
services. It is shown tasks can be accomplished efficiently with Citebase
regardless of the background of the user. More data need to be collected
and the process refined before it is as reliable for measuring citation
impact of indexed papers. Better explanations and guidance are required
for first-time users. Coverage is seen as a limiting factor, even though
Citebase indexes over 200,000 papers from arXiv. Non-physicists were frustrated
at the lack of papers from other sciences. The principle of citation searching
of open access archives has thus been demonstrated and need not be restricted
to current users. Since the evaluation, Citebase has become a featured
service of the ArXiv physics eprint archives.
View full text: Official TR format pdf;
html