The Open Citation Project - Reference Linking and Citation Analysis for Open Archives

See also Papers produced by the project

The effect of open access and downloads ('hits') on citation impact: a bibliography of studies

Despite significant growth in the number of research papers available through open access, principally through author self-archiving in institutional archives, it is estimated that only c. 20% of the number of papers published annually are open access. It is up to the authors of papers to change this. Why might open access be of benefit to authors? One universally important factor for all authors is impact, typically measured by the number of times a paper is cited (some older studies have estimated monetary returns to authors from article publication via the role citations play in determining salaries). Recent studies have begun to show that open access increases impact. More studies and more substantial investigations are needed to confirm the effect, although a simple example demonstrates the effect.

This chronological bibliography is intended to describe progress in reporting these studies; it also lists the Web tools available to measure impact. It is a focused bibliography, on the relationship between impact and access. It does not attempt to cover citation impact, or other related topics such as open access, more generally, although some key papers in these areas are listed as jump-off points for wider study.

Last updated 03 August 2006; first posted 15 September 2004. Please email additions, corrections or comments to Steve Hitchcock.

"This ongoing chronological bibliography may be worth bookmarking and checking every few months. There’s very little annotation, but it’s a good brief bibliography on a narrow - but important - subject."
Walt Crawford, Cites & Insights (November 2004, p13) http://cites.boisestate.edu/civ4i13.pdf
"excellent bibliography"
Peter Suber, American Scientist Open Access Forum (28 September 2005) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/4805.html

Latest additions

Added 03 August 2006 Metcalfe, T. S. (2006)
The Citation Impact of Digital Preprint Archives for Solar Physics Papers
ArXiv, Astrophysics, astro-ph/0607079, 5 July 2006
"Most astronomers now use the arXiv.org server (astro-ph) to distribute preprints, but the solar physics community has an independent archive hosted at Montana State University. For several samples of solar physics papers published in 2003, I quantify the boost in citation rates for preprints posted to each of these servers. I show that papers on the MSU archive typically have citation rates 1.7 times higher than the average of similar papers that are not posted as preprints, while those posted to astro-ph get 2.6 times the average. A comparable boost is found for papers published in conference proceedings, suggesting that the higher citation rates are not the result of self-selection of above-average papers."

Added 03 August 2006 Henneken, E. A., Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Thompson, D., and Murray, S. S. (2006)
Effect of E-printing on Citation Rates in Astronomy and Physics
ArXiv, Computer Science, cs.DL/0604061, v2, 5 June 2006, submitted to the Journal of Electronic Publishing
"It has been observed that papers that initially appear as arXiv e-prints get cited more than papers that do not. Using the citation statistics from the NASA-Smithsonian Astrophysics Data System, we confirm the findings from other studies, we examine the average citation rate to e-printed papers in the Astrophysical Journal, and we show that for a number of major astronomy and physics journals the most important papers are submitted to the arXiv e-print repository first.

Added 03 August 2006 Kousha, K. and Thelwall, M. (2006)
Google Scholar Citations and Google Web/URL Citations: A Multi-Discipline Exploratory Analysis
E-LIS, 05 June 2006, also in Proceedings International Workshop on Webometrics, Informetrics and Scientometrics & Seventh COLLNET Meeting, Nancy (France), May 2006
"we built a sample of 1,650 articles from 108 Open Access (OA) journals published in 2001 in four science and four social science disciplines. We recorded the number of citations to the sample articles using several methods based upon the ISI Web of Science, Google Scholar and the Google search engine (Web/URL citations). For each discipline, we found significant correlations between ISI citations and both Google Scholar and Google Web/URL citations; with similar results when using total or average citations, and when comparing within and across (most) journals."

Added 03 August 2006 Mueller, P. S., Murali, N. S., Cha, S. S., Erwin, P. J. and Ghosh, A. K. (2006)
The effect of online status on the impact factors of general internal medicine journals
Netherlands Journal of Medicine, 64 (2): 39-44, February 2006
"becoming available online as FUTON (full text on the Net) is associated with a significant increase in journal impact factor."

Added 03 August 2006 Metcalfe, T. S. (2005)
The Rise and Citation Impact of astro-ph in Major Journals
ArXiv, Astrophysics, astro-ph/0503519, 23 March 2005
"I describe a simple method to determine the adoption rate and citation impact of astro-ph over time for any journal using NASA's Astrophysics Data System (ADS). I use the ADS to document the rise in the adoption of astro-ph for three major astronomy journals, and to conduct a broad survey of the citation impact of astro-ph in 13 different journals. I find that the factor of two boost in citations for astro-ph papers is a common feature across most of the major astronomy journals."

Added 03 August 2006 Havemann, F. (2004)
Eprints in der wissenschaftlichen kommunikation (Eprints in scientific communication)
Author eprint, 26 October 2004, presented at the Institute of Library Science, Humboldt University, Berlin, June 1, 2004
"the use of eprints can significantly accelerate the scientific communication. This was demonstrated by me with a small sample of articles in theoretical High Energy Physics published 1998 and 1999 in Physical Review D. Typically the eprints in this sample are available eight months before the printed issue is published. Three quarters of them are cited in eprints authored by other researchers before the journal issue appears (among them all highly cited eprints)."

Simple and practical example

Citation analysis is specialised and difficult. To make the case for, or against, a claim such as 'open access increases impact' requires a lot of the reader, who may not be a specialist but who wants to try and understand the point at issue and decide if it has any relevance to him or her. The following simple example is included for this reason, not as proof but as evidence of the effect within a particular domain. Draw your own conclusions, and then read the more detailed evidence of the bibliography if you are still interested.

"Measuring the effect for physics or astronomy is easy. This link returns the number of articles published in the Astrophysical Journal in 2003 and their number of citations.

"This next link shows the number of these papers which are available OA in the arXiv, and their citations.

"The result is that 75% of the papers are in the arXiv, and they represent 90% of the citations, a 250% OA effect.

"By replacing ApJ with the mnemonic for any other physics or astronomy journal one can repeat the measurement; for Nuclear Physics A (NuPhA) one gets that 32% of the articles are in the arXiv, and they represent 78% of the citations, a 740% OA effect."
From Michael Kurtz, American Scientist Open Access Forum, 28 September 2005 http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/4808.html
Note, the database links are 'live', i.e. they return the current database figures, not the exact figures on which Michael Kurtz would have based his calculations, but the percentages quoted are unlikely to change dramatically, in the short term at least.

Studies with original data

Highlights

Lawrence (2001) was the first to publish data recognising the trend for online publication to increase impact, confirmed for open access papers by the work of the Open Citation Project based on arXiv (e.g. Harnad and Brody, D-Lib, 2004), and by Kurtz et al. (2004a, 2003a) looking at the NASA Astrophysics Data System. Commenting on Harnad and Brody (D-Lib, 2004) in Open Access News, Peter Suber said:
This is an important article. It's the first major study since the famous Lawrence paper documenting the proposition that OA increases impact. It's also the first to go beyond Lawrence in scope and method in order to answer doubts raised about his thesis. By confirming that OA increases impact, it gives authors the best of reasons to provide OA to their own work (21 June 2004)
Broader collaborations have emerged to extend these findings (e.g. Brody et al. 2004).

Open access has become feasible because of the move towards online publication and dissemination. A new measure that becomes possible with online publication is the number of downloads or 'hits', opening a new line of investigation. Brody et al. have been prominent in showing there is a correlation between higher downloads and higher impact, particularly for high impact papers, holding out the promise not just for higher impact resulting from open access but for the ability to predict high impact papers much earlier, not waiting years for those citations to materialise (e.g. Brody and Harnad 2004, in prep.). The effect can be verified with the Correlation Generator (below).

(Note. The latest listings might include preprints, or even pre-preprints. This area of study is effectively a work in progress, and as such the list is intended to raise awareness of the most recent results, even where these may not be definitive or final versions. Check back for definitive versions.)

Added 16 May 2006 Eysenbach, G. (2006)
Citation Advantage of Open Access Articles
PLoS Biology, Volume 4, Issue 5, May 2006
Further evidence for the OA citation advantage, although quite critical of other studies with which its findings broadly agree. This example is based on a small, single journal sample (PNAS: Proceedings of the National Academy of Sciences). Since PNAS offers authors the choice of paying to provide open access to published papers and/or freely self-archiving, a 'Secondary analysis' considers the relative impact of each type of OA, although the number of papers involved is really too small to give this result the weight of the broader findings. The paper is accompanied by two editorials, one in the publishing journal, the other a self-published editorial by the author:
MacCallum, C. J. and Parthasarathy, H. (2006) Editorial: Citation Advantage of Open Access Articles, PLoS Biology, Volume 4, Issue 5, May 2006
Eysenbach, G. (2006) The Open Access Advantage, Journal of Medical Internet Research, 2006;8(2):e8

Comment on this paper:
Harnad, S. on Chronicle of Higher Education News Blog on this paper, 15 May 2006: "The Eysenbach study is certainly not “the first to compare open-access and non-open-access papers from the same journal”. See [this bibliography]"
Harnad, S. eLetter, PLoS, Pipe-Dreams and Peccadillos, on the PLoS editorial, 16 May 2006: "There can be disagreement about what evidence one counts as "solid," but there can be little dispute that prior evidence derived from substantially larger and broader-based samples showing substantially the same outcome can hardly be described as "surprisingly hard to find". The only new knowledge from this small, journal-specific sample was (1) the welcome finding of how early the OA advantage can manifest itself, plus (2) some less clear findings about differences between first- and last-author OA practices, plus (3) a controversial finding that will most definitely need to be replicated on far larger samples in order to be credible: "The analysis revealed that self-archived articles are also cited less often than OA articles from the same journal.""
Author's response eLetter, PLoS, 17 May 2006: "None of the previous papers in [this bibliography] employed a similar methodology, working with data from a "gold-OA" journal. ... Regarding "larger samples" I think rigor and quality (leading to internal validity) is more important than quantity (or sample size). Going through the laborious effort to extract article and author characteristics for a limited number of articles (n=1492) in order to control for these confounders provides scientifically stronger evidence than doing a crude, unadjusted analysis of a huge number of online accessible vs non-online accessible articles, leaving open many alternative explanations. ... contrary to what Harnad said, this study is NOT at all "showing substantially the same outcome". On the contrary, the effect of green-OA -- once controlled for confounders -- was much less than what others have claimed in previous papers."
Harnad, S. blog, Confirming the Within-Journal OA Impact Advantage, 18 May 2006: "with the large, consistent within-journal OA/NOA differences found across all journals, all disciplines and all years in samples four orders of magnitude larger than Eysenbach's, it is not at all clear that controls for those "multiple confounders" are necessary in order to demonstrate the reality, magnitude and universality of the OA advantage. That does not mean the controls are not useful, just that they are not yet telling us much that we don't already know. ... the true measure of the SOA (Self-Archived OA) advantage today (at its 15% spontaneous baseline) is surely not to be found in PNAS but in the statistically far more numerous, hence far more representative full-spectrum of journals that do not yet offer POA (Payed OA). (I would be delighted if those journals took the Eysenbach findings as a reason for offering a POA option! But not at the expense of authors drawing the absurd conclusion -- not at all entailed by Eysenbach's PNAS-specific results -- that in the journals they currently publish in, SOA alone would not confer citation advantages at least as big as the ones we have been reporting.)"
Author's response WebCite blog, The OA debate between an “archivangelist” and an OA researcher, 24 May 2006: "“Confounded” associations between two variables which falsely suggests causality can ONLY be ruled out if one controls for the confounder no matter how “strong, consistent” the effect appears ... Our “events” (citations) are determined by many different “causes”, of which “access” is only one variable – many other variables, including confounders, have to be taken into account.
Harnad, S. blog, The Epidemiology of OA, 26 May 2006: considers each of Eysenbach’s list of confounders, which ones might have an effect and which ones will be tested. "Stay tuned..."
Suber, P. SPARC Open Access Newsletter, issue #98, Gunther Eysenbach confirms the OA impact advantage, 2 June 2006: "There's some controversy about whether some earlier results, especially by Tim Brody, Chawki Hajjem, and Stevan Harnad, are the same or only similar to some of Eysenbach's results. But no one doubts that Eysenbach has new and valid results, or that he has persuasively advanced the case that OA helps authors and journals build their citation impact. While there have been many previous studies of the OA impact advantage, none has made the splash that Eysenbach's has."

Added 15 May 2006 Mayr, P. (2006)
Constructing experimental indicators for Open Access documents (pdf 9pp)
Author preprint, to appear in Research Evaluation, special Issue on 'Web indicators for Innovation Systems', 14, 2 (2006)

Added 15 March 2006 Davis, P. M. and Fromerth, M. J. (2006)
Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? (pdf 12pp)
draft manuscript, ArXiv.org, cs.DL/0603056, 14 March 2006, Scientometics, accepted for publication (2007)

Comment on this paper:
Authors Self-Selection bias "While our study confirms the same citation advantage reported by others, it does not attribute Open Access as the cause of more citations, but to Self-Selection. Open Access therefore may be a result, not a cause, of authors promoting higher-quality work."
Liblicense, 14 March 2006
Harnad, S. OAA a causal factor "I think your results are very interesting, but I don't think they have shown that the OA citation advantage (OAA) is all or mostly a self-selection Quality Bias (QB) correlate, rather than being causal. It is still quite plausible that the OAA is a genuine causal factor, but that it has a bigger effect on the high quality/citation end."
SIGMETRICS, 14 March 2006
Qualifying statements: Authors "I am however, troubled by individuals who make universal and unqualified statements like, "Open Access increases citations by 50-250%!"" Harnad, S. "I am such an individual, and I hold by that statement" Authors "The more precise answer is much more subtle, but I understand that a statement like, "open access may provide some citation benefit, but only for prestigious authors who publish in prestigious journals and whose article is already highly-cited", doesn't sound as convincing to administrators and policy makers." Harnad, S. "I don't think it is a correct statement of the thrust of the current body of findings on the OAA. It is merely your interpretation of the result of your own study in 4 maths journals!"
SIGMETRICS, 15 March 2006
Causation or association?: Antelman, K. "Data I collected for philosophy, political science, engineering and mathematics do not support this hypothesis that OA causes more citations for better articles only ... at that time I had not looked at the distribution of OA and non-OA articles by citations. Graphs of those results are posted at http://www.lib.ncsu.edu/staff/kantelman/OA_by_citations.xls. These data show OA citation advantage across all articles with more than zero citations. Authors "The data that Kristin illustrates do not show causation, only association." Harnad, S. The data Phil illustrates likewise do not show causation, only association. ... More fine-tuned causal tests are needed to decide.
Liblicense, from 20 March 2006

Added 16 March 2006 Harnad, S. (2006)
OA Impact Advantage = EA + (AA) + (QB) + QA + (CA) + UA
Author eprint, 14 March 2006, ECS EPrints repository, School of Electronics and Computer Science, University of Southampton

Added 03 August 2006 Mueller, P. S., Murali, N. S., Cha, S. S., Erwin, P. J. and Ghosh, A. K. (2006)
The effect of online status on the impact factors of general internal medicine journals
Netherlands Journal of Medicine, 64 (2): 39-44, February 2006
"becoming available online as FUTON (full text on the Net) is associated with a significant increase in journal impact factor."

Added 30 December 2005 Hajjem, C., Harnad, S. and Gingras, Y. (2005)
Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact (pdf 8pp)
IEEE Data Engineering Bulletin, Vol. 28 No. 4, December 2005
also Author eprint, 16 December 2005 http://eprints.ecs.soton.ac.uk/11688/
"In 2001, Lawrence found that articles in computer science that were openly accessible (OA) on the Web were cited substantially more than those that were not. We have since replicated this effect in physics. To further test its cross-disciplinary generality, we used 1,307,038 articles published across 12 years (1992-2003) in 10 disciplines (Biology, Psychology, Sociology, Health, Political Science, Economics, Education, Law, Business, Management). The overall percentage of OA (relative to total OA + NOA) articles varies from 5%-16% (depending on discipline, year and country) and is slowly climbing annually. Comparing OA and NOA articles in the same journal/year, OA articles have consistently more citations, the advantage varying from 25%-250% by discipline and year."

Added 30 December 2005 Hajjem, C., Gingras, Y., Brody, T., Carr, L. and Harnad, S. (2005)
Open Access to Research Increases Citation Impact (.doc 12pp)
Author eprint, 16 December 2005, Technical Report, Institut des sciences cognitives, Université du Québec à Montréal

Added 30 December 2005 Sahu, D.K., Gogtay, N.J. and Bavdekar, S.B. (2005)
Effect of open access on citation rates for a small biomedical journal
Author eprint, December 1, 2005, in Fifth International Congress on Peer Review and Biomedical Publication, Chicago, September 16-18, 2005
"We assessed the influence of OA on citations rates for a small, multi-disciplinary journal which adopted OA without article submission or article access fee. DESIGN The full text of articles published since 1990 were made available online in 2001. Citations for these articles as retrieved using Web of Science, SCOPUS, and Google Scholar were divided into two groups - the pre-OA period (1990-2000) and the post-OA period (2001-2004). CONCLUSIONS Open access was associated with increase in the number of citations received by the articles. It also decreased the lag time between publication and the first citation. For smaller biomedical journals, OA could be one of the means for improving visibility and thus citation rates."

Added 27 September 2005 Zhao, D. (2005)
Challenges of scholarly publications on the Web to the evaluation of science -- A comparison of author visibility on the Web and in print journals (abstract only)
Information Processing and Management, 41:6, 1403-1418, December 2005
Compares author visibility between the Web and print journals as revealed from citation analysis based on a search for the term "XML" or "eXtensible Markup Language" using NEC Research Institute’s CiteSeer, the entire ISI Science Citation Index (SCI) database, and journals indexed and classified in SCI as representing computer science research. The main finding: "The author ranking by number of citations that resulted from CiteSeer data is highly correlated with that obtained from SCI." i.e. it's not comparing OA impact vs non-OA but Web vs journal, and finds that authors, notably the top authors, are self-archiving and publishing papers in both places.

Added 13 July 2005 Adams, J. (2005)
Early citation counts correlate with accumulated impact (abstract only)
Scientometrics, 63 (3): 567-581, June 2005
Working towards earlier prediction of impact. This paper is not OA and has just appeared but was written before Brody and Harnad (2005) revealed a correlation to predict impact from even earlier data, i.e. download data for OA papers, before any citations.

Added 26 September 2005 Moed, H. F. (2005)
Statistical Relationships Between Downloads and Citations at the Level of Individual Documents Within a Single Journal (abstract only)
Journal of the American Society for Information Science and Technology, 56(10): 1088-­1097, published online 31 May 2005
"Statistical relationships between downloads from ScienceDirect of documents in Elsevier's electronic journal Tetrahedron Letters and citations to these documents recorded in journals processed by the (ISI) for the Science Citation Index (SCI) are examined. ... Findings suggest that initial downloads and citations relate to distinct phases in the process of collecting and processing relevant scientific information that eventually leads to the publication of a journal article." Does not investigate open access sources. Notes the need for caution in drawing conclusions on the frequency of paper downloads from formal citation patterns, and vice versa.

Updated 05 October 2005 Vaughan, L. and Shaw, D. (2005)
Web citation data for impact assessment: A comparison of four science disciplines (abstract only)
Journal of the American Society for Information Science and Technology, Vol. 56, No. 10, 1075 - 1087, published online 27 May 2005
appears to be an expansion of Can Web Citations be a Measure of Impact? An Investigation of Journals in the Life Sciences (abstract only)
ASIST 2004: Proceedings of the 67th ASIS&T Annual Meeting, Vol. 41 (Medford, USA: Information Today), pp. 516-526

Brody, T. and Harnad, S. (2005)
Earlier Web Usage Statistics as Predictors of Later Citation Impact
Author eprint, 18 May 2005, University of Southampton, School of Electronics and Computer Science, Journal of the American Association for Information Science and Technology, Volume 57, Issue 8, 2006, 1060-1072 (abstract)

Added 19 May 2005 Wren, J. D. (2005)
Open access and openly accessible: a study of scientific publications shared via the internet
BMJ, 330:1128, 12 April 2005

BMJ Rapid Responses to this article:
Carr, L., 15 April 2005 Open Access Misdefined "by misdefining Open Access this paper somehow ignores both the phenomenon that is being measured (Open Access-ibility) and the significant research that has risen around it."
Chan, C-h. and Ng, D. K., 13 May 2005 Technological problems in this study: PDF "The selection of only PDF reprint in this study made this correlation (between the time since initial publication and the probability of availability of reprint in non journal website) a little bit artificial."
All Rapid Responses

Wren's article also prompted this editorial
Suber, P. (2005)
Open access, impact, and demand
BMJ, 330:1097-1098, 14 May 2005

Added 13 April 2005 Belew, R. (2005)
Scientific impact quantity and quality: Analysis of two sources of bibliographic data (pdf 12pp)
Arxiv.org, cs.IR/0504036, 11 April 2005

Added 03 August 2006 Metcalfe, T. S. (2005)
The Rise and Citation Impact of astro-ph in Major Journals
ArXiv, Astrophysics, astro-ph/0503519, 23 March 2005
"I describe a simple method to determine the adoption rate and citation impact of astro-ph over time for any journal using NASA's Astrophysics Data System (ADS). I use the ADS to document the rise in the adoption of astro-ph for three major astronomy journals, and to conduct a broad survey of the citation impact of astro-ph in 13 different journals. I find that the factor of two boost in citations for astro-ph papers is a common feature across most of the major astronomy journals."

Updated 13 April 2005 Ongoing studies Hajjem, C. (2004-05)
Cover page for the range of studies highlighted below, Laboratoire de recherche en Sciences Cognitives, UQAM. (Text in French but graphs "self-explanatory"; see this comment for elaboration)

Updated 26 September 2005 Bollen, J., Van de Sompel, H., Smith, J. and Luce, R. (2005)
Toward alternative metrics of journal impact: A comparison of download and citation data (pdf 34pp)
Arxiv.org, cs.DL/0503007, 03 March 2005, in Information Processing and Management, 41(6): 1419-1440, December 2005

Added 5 January 2005 Ongoing study Brody, T., et al.
Citation Impact of Open Access Articles vs. Articles Available Only Through Subscription ("Toll-Access")
with downloadable graphs of '% Articles OA' and '% OA Advantage' by discipline and sub-discipline

Updated 31 January 2005 Schwarz, G. and Kennicutt Jr., R. C. (2004)
Demographic and Citation Trends in Astrophysical Journal Papers and Preprints (pdf 14pp)
Arxiv.org, astro-ph/0411275, 10 November 2004, Bulletin of the American Astronomical Society, Vol. 36, 1654-1663
See also a note from AAS Pub Board meeting, Tucson, November 3-4 2003
"Greg Schwarz (from the ApJ editorial office) reported some work he's doing tracking citation rates of papers published in the ApJ based on whether they were posted on astro-ph or not: ApJ papers that were also on astro-ph have a citation rate that is _twice_ that of papers not on the preprint server"
http://listserv.nd.edu/cgi-bin/wa?A2=ind0311&L=pamnet&D=1&O=D&P=1632

Added 03 August 2006 Havemann, F. (2004)
Eprints in der wissenschaftlichen kommunikation (Eprints in scientific communication)
Author eprint, 26 October 2004, presented at the Institute of Library Science, Humboldt University, Berlin, June 1, 2004
"the use of eprints can significantly accelerate the scientific communication. This was demonstrated by me with a small sample of articles in theoretical High Energy Physics published 1998 and 1999 in Physical Review D. Typically the eprints in this sample are available eight months before the printed issue is published. Three quarters of them are cited in eprints authored by other researchers before the journal issue appears (among them all highly cited eprints)."

Brody, T. (2004)
Citation Analysis in the Open Access World
Author eprint, October 4, 2004, in Interactive Media International

Added 9 November 2004 McVeigh, M. E. (2004)
Open Access Journals in the ISI Citation Databases: Analysis of Impact Factors and Citation Patterns
Thomson Scientific, October 2004

Added 29 September 2004 Antelman, K. (2004)
Do Open-Access Articles Have a Greater Research Impact?
College and Research Libraries, 65(5):372-382, September 2004
also Author eprint, E-LIS, 29 September 2004, http://eprints.rclis.org/archive/00002309/

Updated 5 January 2005 Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y., Oppenheim, C., Stamerjohanns, H. and Hilf, E. (2004)
The Access/Impact Problem and the Green and Gold Roads to Open Access
Author eprint, 15 September 2004, in Serials Review, Vol. 30, No. 4, 310-314 (free access to published version during 2005)
Shorter version: The green and the gold roads to Open Access
Nature, Web Focus: access to the literature, May 17, 2004

Updated 26 September 2005 Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner, M., Murray, S. S. (2004b)
The Effect of Use and Access on Citations
Author eprint, September 2004, in Information Processing and Management, 41 (6): 1395-1402, December 2005

Perneger, T. V. (2004)
Relation between online "hit counts" and subsequent citations: prospective study of research papers in the BMJ
BMJ, 329:546-547, 4 September 2004

BMJ Rapid Responses to this article:
Harnad, S. and Brody, T., 6 September 2004 Prior evidence that downloads predict citations "confirms what Tim Brody's online usage/citation correlator has been demonstrating for several years now across a number of areas in physics and mathematics.
All Rapid Responses

Prakasan, E. R. and Kalyane, V. L. (2004)
Citation analysis of LANL High-Energy Physics E-Prints through Science Citation Index (1991-2002)
Author eprint, E-LIS, 26 August 2004

Added 13 April 2005 Murali, N. S., Murali, H. R., Auethavekiat, P., Erwin, P. J., Mandrekar, J. N., Manek, N. J. and Ghosh, A. K. (2004)
Impact of FUTON and NAA Bias on Visibility of Research
Mayo Clinic Proceedings, Vol. 79, No. 8, 1001-1006, August 2004
Notes and comment: FUTON = full text on the Net; NAA = no abstract available
This is not an article on how Open Access increases impact but on how *Online* Access increases impact. The effects are related, but one is a licensing effect, not an OA effect.

Harnad, S. and Brody, T. (2004a)
Comparing the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals
D-Lib Magazine, Vol. 10 No. 6, June 2004
Replicates the Lawrence effect -- OA increases impact -- in physics.

Pringle, J. (2004)
Do Open Access Journals have Impact?
Nature, Web Focus: access to the literature, May 7, 2004

Testa, J. and McVeigh, M. E. (2004)
The Impact of Open Access Journals: A Citation Study from Thomson ISI (pdf 17pp)
Author eprint, 14 April 2004

Kurtz, M. J. (2004)
Restrictive access policies cut readership of electronic research journal articles by a factor of two (pdf 2pp)
Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA
Poster presentation at National Policies on Open Access (OA) Provision for University Research Output: an International meeting, Southampton, 19 February 2004

Brody, T., Stamerjohanns, H., Harnad, S., Gingras, Y. and Oppenheim, C. (2004)
The Effect of Open Access on Citation Impact (pdf 1pp)
Poster presentation at National Policies on Open Access (OA) Provision for University Research Output: an International meeting, Southampton, 19 February 2004

Updated 5 January 2005 Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner, M. and Murray, S. S. (2004a)
Worldwide Use and Impact of the Nasa Astrophysics Data System Digital Library
Author eprint, January 28, 2004, in Journal of the American Society for Information Science and Technology, Vol. 56, No. 1, 36-45, published online 20 September 2004

Hitchcock, S., Brody, T., Gutteridge, C., Carr, L. and Harnad, S. (2003b)
The Impact of OAI-based Search on Access to Research Journal Papers
Author eprint, 15 September 2003, in Serials, Vol. 16, No. 3, November 2003, 255-260

Hitchcock, S., Woukeu, A., Brody, T., Carr, L., Hall, W. and Harnad, S. (2003a)
Evaluating Citebase, an open access Web-based citation-ranked search and impact discovery service
Technical Report ECSTR-IAM03-005, School of Electronics and Computer Science, University of Southampton, July 2003

Added 28 October 2004 Bollen, J., Vemulapalli, S. S., Xu, W. and Luce, R. (2003)
Usage Analysis for the Identification of Research Trends in Digital Libraries
D-Lib Magazine, Vol. 9, No. 5, May 2003

Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Murray, S. S., Martimbeau, N. and Elwell, B. (2003b)
The NASA Astrophysics Data System: Sociology, Bibliometrics, and Impact
Author eprint, March 2003, Journal of the American Society for Information Science and Technology, submitted for publication

Updated 23 February 2005 Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner, M., Murray, S. S., Martimbeau, N. and Elwell, B. (2003a)
The Bibliometric Properties of Article Readership Information
Author eprint, March 2003, in Journal of the American Society for Information Science and Technology, 56 (2): 111-128, January 15, 2005

Darmoni, S. J., et al. (2002)
Reading factor: a new bibliometric criterion for managing digital libraries
Journal of the Medical Library Association, Vol. 90, No. 3, July 2002

Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Thompson, D. M., Bohlen, E. H. and Murray, S. S. (2002)
The NASA Astrophysics Data System: Obsolescence of Reads and Cites (pdf 8pp)
Library and Information Services in Astronomy IV, edited by B. Corbin, E. Bryson, and M. Wolf, July 2002

Bollen, J. and Luce, R. (2002)
Evaluation of Digital Library Impact and User Communities by Analysis of Usage Patterns
D-Lib Magazine, Vol. 8, No. 6, June 2002

Lawrence, S. (2001)
Free online availability substantially increases a paper's impact
Nature, 31 May 2001

Added 04 October 2005 Anderson, K., Sack, J., Krauss, L. and O'Keefe, L. (2001)
Publishing Online-Only Peer-Reviewed Biomedical Literature: Three Years of Citation, Author Perception, and Usage Experience
Journal of Electronic Publishing, Vol. 6, No. 3, March 2001
One of the first studies of the citation effect of online against offline publication, rather than of open access against non-OA. Provides data for one journal and a small number of articles over a three year period year. This paper was added to the bibliography following this correspondence:

Banks, P. "does not find the same citation advantage for online publications claimed by Harnad and his colleagues." Harnad, S. "Please weigh findings against the preponderance of evidence."
American Scientist Open Access Forum, Open access to research worth £ 1.5bn a year, 28 September 2005

Updated 04 October 2005 Odlyzko, A. M. (2000)
The rapid evolution of scholarly communication
PEAK 2000: Economics and Usage of Digital Library Collections conference, Ann Arbor, MI, March 2000.
Also in Learned Publishing, 15(1), 7-19, January 2002 here. Author eprint http://www.dtc.umn.edu/~odlyzko/doc/rapid.evolution.pdf
Notes the growing usage of information in electronic form (c.f. print forms) and of journal papers from non-journal sites (e.g. eprints), and presents evidence that usage increases when access is more convenient

Youngen, G. K. (1998)
Citation Patterns to Electronic Preprints in the Astronomy and Astrophysics Literature
Library and Information Services in Astronomy III, ASP Conference Series, Vol. 153, 1998

Youngen, G. (1998)
Citation Patterns Of The Physics Preprint Literature With Special Emphasis On The Preprints Available Electronically
Author eprint, UIUC Physics and Astronomy library, c. 5 November 1998, presented at ACRL/STS on 6/29/97

Correlation between research assessment ratings and citations (the financial imperative)

There is another dimension to this argument. If open access increases impact, then it will also increase research income and funding. It has been shown in the UK that there is a correlation between research assessment ratings and citation counts, and higher ratings means more money for the higher rated research groups. Of course, if all papers were made open access by their authors, the relative effect would disappear. First-mover advantage anyone?

Added 26 September 2005 Harnad, S. (2005)
Maximising the Return on UK's Public Investment in Research
Author eprint, September 14, 2005
Attempts to monetise 'lost' impact: "The online-age practice of self-archiving has been shown to increase citation impact by a dramatic 50-250%, but so far only 15% of researchers are doing it spontaneously. Citation impact is rewarded by universities (through promotions and salary increases) and by research-funders like RCUK (through grant funding and renewal) at a conservative estimate of £46 per citation. ... As a proportion of the RCUK’s yearly £3.5bn research expenditure (yielding 130,000 articles x 5.6 = 761,600 citations), our conservative estimate would be 50% x 85% x £3.5.bn = £1.5bn worth of loss in potential research impact (323,680 potential citations lost)."
See also Australia is not maximisng the return on its research investment (ETD2005, Sydney) for the same estimate applied to the potential lost return ($425M) there.

Comment on this paper:
Rowland, I. "If I am the one millionth author (or the 10,000th research group or the 100th nation) to publish open access, that comparative advantage must quickly decline, approaching zero as the last few laggards pile in". Author response: "Ian Rowland is exactly right that the OA impact advantage (currently 50-250%) will shrink as we approach 100% OA. Right now we are at 15% OA, and the advantage is in part -- no one can say how large a part -- a *competitive* advantage of the minority 15% OA -- the head-start vanguard -- over the laggard 85% non-OA majority. ... The OA impact advantage consists of at least the following 6 factors ..."
American Scientist Open Access Forum, OA advantage = EA + (AA) + (QB) + QA + (CA) + UA, 15 September 2005
Waters, D. "Mr. Harnad may well be on to an important subject and line of argument in suggesting that citations are a kind of return on investment. However, close inspection of the concepts and logic of his argument suggests that he is quite a bit further from proving his case than he seems to have convinced himself that he is." Author response: "No proof here: Just conservative estimates."
American Scientist Open Access Forum, Open access to research worth 1.5bn a year, 27 September 2005

Smith, A. and Eysenck, M. (2002)
The correlation between RAE ratings and citation counts in psychology (pdf 12pp)
Technical Report, Psychology, Royal Holloway College, University of London, June 2002

Holmes, A. and Oppenheim, C. (2001)
Use of citation analysis to predict the outcome of the 2001 Research Assessment Exercise for Unit of Assessment (UoA) 61: Library and Information Management
Information Research, Vol. 6, No. 2, January 2001

Garfield, E. (1988)
Can Researchers Bank on Citation Analysis? (pdf 10pp)
Current Comments, No. 44, October 31, 1988
attached (pp 3-10)
Diamond, Jr., A. M. (1986)
What is a Citation Worth?
J. Hum. Resour., 21:200-15, 1986
Garfield comments on studies that attempt to quantify the reward system of science in terms of monetary returns to author salaries from article publication and citations, reprinting one of those studies

Web tools for measuring impact

Citebase Search "Search and citation analysis tool for the free, online research literature" http://citebase.eprints.org/ User service, free
see
Brody, T. (2003) Citebase Search: Autonomous Citation Database for e-print Archives, sinn03 conference on Worldwide Coherent Workforce, Satisfied Users - New Services For Scientific Information, Oldenburg, Germany, September 2003
Hitchcock, S., et al. (2003a) Evaluating Citebase, an open access Web-based citation-ranked search and impact discovery service
Correlation Generator http://citebase.eprints.org/analysis/correlation.php
Generates a graph (or table) of the correlation between citation impact and usage impact from the Citebase database
see Brody, T. and Harnad, S. 2005 (in prep.)
Citeseer "Scientific literature digital library" http://citeseer.ist.psu.edu/ User service, free
see
Jacsó, P., (2005) CiteSeer, Thomson Gale, November 2005
Lawrence, S., Giles, C. L., Bollacker, K. (1999), Digital Libraries and Autonomous Citation Indexing, IEEE Computer, Vol. 32, No. 6, 67-71, 1999

Elsevier Scopus Bibliographic database covering 13,450 peer-reviewed titles http://www.scopus.com/ User service
see
Added 15 March 2006 Burnham, J. F. (2006) Scopus database: a review, Biomedical Digital Libraries, 3:1, 8 March 2006
Added 15 March 2006 Dess, H. M. (2006) Scopus, Issues in Science and Technology Librarianship, Winter 2006
Added 15 May 2006 Quint, B. (2006) Elsevier’s Scopus Introduces Citation Tracker: Challenge to Thomson ISI’s Web of Science?, Newsbreaks, January 23, 2006
Jacsó, P. (2004) Scopus, Thomson Gale, September 2004
see also Comparative reviews

Google Scholar Find articles from academic publishers, preprint repositories and universities, as well as scholarly articles across the web (presents citations as separate results) http://scholar.google.com/ User service, free
see
Added 15 May 2006 Sadeh, T. (2006) Google Scholar Versus Metasearch Systems, HEP Libraries Webzine, issue 12, March 2006
"thoughtful and informative ... altogether the best overview of Google Scholar, other large federated search systems such as Scirus, and library-based metasearch tools I've seen." Reviewed by Tennant, R., Current Cites, January 2006 issue
Added 15 March 2006 Burright, M. (2006) Google Scholar -- Science & Technology, Issues in Science and Technology Librarianship, Winter 2006
Added 28 February 2006 Noruzi, A. (2005) Google Scholar: the new generation of citation indexes (pdf 11pp), E-LIS, 11 February 2006, in LIBRI 55(4): 170-180
Jacsó, P., (2005) Google Scholar and The Scientist, commenting on his interview in Perkel, J., The Future of Citation Analysis (abstract only), The Scientist, October 24, 2005
Jacsó, P. (2005) Google Scholar (Redux), Thomson Gale, June 2005
Myhill, M. (2005) Google Scholar, Charleston Advisor, Vol. 6, No. 4, April 2005
Jacsó, P. (2004) Google Scholar Beta, Thomson Gale, December 2004

ISI Web of Science Cited reference searching of 8,700 high impact research journals http://www.isinet.com/products/citation/wos/ User service
see
Jacsó, P. (2004) Web of Science Citation Indexes, Thomson Gale, August 2004
see also Comparative reviews

Added 13 April 2006 Windows Live Academic Beta version. Indexes content related to computer science, physics, electrical engineering, and related subject areas, with more than 6 million records from approximately 4300 journals, 2000 conferences and ArXiv.org. In collaboration with Citeseer http://academic.live.com/ User service, free
see
Added 15 May 2006 Quint, B. (2006) Windows Live Academic Search: The Details, Newsbreaks, April 17, 2006
Added 13 April 2006 Sherman, C. (2006) Microsoft Launches Windows Live Academic Search, SearchEngineWatch.com, April 12, 2006

Added 15 May 2006 Citations in Economics not intended for direct user access; instead is made available to RePEc services such as Socionet, EconPapers and IDEAS. Uses Citeseer software http://citec.repec.org/ Data service, free
see
Barrueco Cruz, J. M. and Krichel, T. (2004) Building an autonomous citation index for grey literature: the economics working papers case (pdf 12pp), E-LIS, 01 February 2005, also in Proceedings GL6: Sixth International Conference on Grey Literature, New York, December 2004

CrossRef Forward linking service tool allows CrossRef member publishers to display cited-by links in their primary content, Data service
CrossRef and Atypon announce forward linking service (press release) June 8, 2004
Institute of Physics becomes first journals publisher to implement 'cited-by' links using CrossRef's Forward Linking service: Time travel with IOP journals (IOP press release) 14 March, 2005

Forthcoming ISI Web Citation Index User service
see
Added 15 May 2006 Martello, A. (2006) Selection of Content for the Web Citation Index: Institutional Repositories and Subject-Specific Archives, Thomson.com, undated
Pringle, J. (2005) Partnering helps institutional repositories thrive, KnowledgeLink Newsletter, February 2005
Citeseer's replacement? List server mailing, 18 March 2004

Comparative reviews

Added 03 August 2006 Bakkalbasi, N., Bauer, K., Glover, J. and Wang, L. (2006) Three options for citation tracking: Google Scholar, Scopus and Web of Science, Biomedical Digital Libraries, June 29, 2006

Added 15 May 2006 Wenzel, E. (2006) Google Scholar beta, ZDNet, May 2, 2006
Brief comparison of Google Scholar and Microsoft Live Academic Search

Bailey, C. W. Jr (2006) A Simple Search Hit Comparison for Google Scholar, OAIster, and Windows Live Academic Search, Digital Koans, author blog, April 13, 2006
A simple but revealing experiment: "It should be clear that a sample of one search term is a very crude measure".

Pauly, D. and Stergiou, K. I. (2005) Equivalence of results from two citation analyses: Thomson ISI’s Citation Index and Google’s Scholar service (pdf 3pp), Ethics in Science and Environmental Politics, 22 December 2005, 33-35

Comment on this paper: ""outperform" means many things. OK, in this study the citation counts were close but the searchability of material on WOS is much stronger." ResourceShelf, February 25, 2006

Roth, D. L. (2005) The emergence of competitors to the Science Citation Index and the Web of Science (pdf 6pp), Current Science Online, Vol. 89, No. 9, 10 November 2005

Jacsó, P. (2005) As we may search – Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases (pdf 11pp), Current Science Online, Vol. 89, No. 9, 10 November 2005

Bauer, K. and Bakkalbasi, N. (2005) An Examination of Citation Counts in a New Scholarly Communication Environment, D-Lib Magazine, 11(9), September 2005.
Compares citation counts provided by Web of Science, Scopus, and Google Scholar.

Comment on this paper:
Stegmann, J. Clearer picture? "the authors of this interesting paper should, perhaps, take into account the ISI Proceedings database because the conference papers indexed therein are included together with their references. ... This would give a clearer picture of what one gets from free services like Google Scholar and from products which have to be licensed." Sigmetrics listserv, 16 September 2005
Author response: Focused and cross-disciplinary "We decided to concentrate on three multi-disciplinary databases only, as a way of focusing our work. We wanted to lay the foundation for a study that will compare subject areas: hence our decision to look at databases that cross many subject areas." Sigmetrics listserv, 16 September 2005

LaGuardia, C. (2005) Scopus vs. Web of Science, Library Journal, 130(1); 40, 42, January 15, 2005

Deis, L. F. and Goodman, D. (2005) Web of Science (2004 version) and Scopus, Charleston Advisor, Vol. 6, No. 3, January 2005

Background

On the relationship between citation impact, open access (self-archiving) and research assessment

Day, M. (2004)
Institutional repositories and research assessment (pdf 29pp)
Author eprint (v. 0.1), 2 December 2004

Harnad, S. (2003)
Maximizing university research impact through self-archiving
Jekyll.com, No. 7, December 2003

Harnad, S. (2003)
Enhance UK research impact and assessment by making the RAE webmetric
Author eprint, in Times Higher Education Supplement, 6 June 2003, p. 16

Harnad, S., Carr, L., Brody, T. and Oppenheim, C. (2003)
Mandated online RAE CVs linked to university eprint archives: Enhancing UK research impact and assessment
Ariadne, issue 35, April 2003

Harnad, S. (2001)
Research Access, Impact and Assessment (longer version)
Author eprint, in Times Higher Education Supplement, 1487: p. 16., 2001

Citation analysis, indexes and impact factors

Notes. Important work in this area builds on Eugene Garfield's pioneering work in the 1950s. Although producing some remarkably successful tools for measuring the impact of the scholarly literature, this area is not without controversy. This short list presents a cross-section underlining these issues, with a view to understanding how such long-established approaches might adapt to online data, and how possible shortcomings might be overcome.

Added 03 August 2006 PLoS Medicine Editors (2006)
The Impact Factor Game: It is time to find a better way to assess the scientific literature
PLoS Medicine, Vol. 3, No. 6, June 2006

Added 15 May 2006 Altbach, P. G. (2006)
The Tyranny of Citations
Inside Higher Ed, May 8, 2006

Added 28 February 2006 Noruzi, A. (2006)
The Web Impact Factor: a critical review (pdf, 10pp)
E-LIS, February 9, 2006, in The Electronic Library, 24 (2006)
"Web Impact Factor (WIF) is a quantitative tool for evaluating and ranking web sites ... search engines provide similar possibilities for the investigation of links between web sites/pages to those provided by the academic journals citation databases from the Institute of Scientific Information (ISI). But the content of the Web is not of the same nature and quality as the databases maintained by the ISI."

Added 28 February 2006 Bollen, J., Rodriguez, M. A. and Van de Sompel, H. (2006)
Journal Status (pdf, 16pp)
Arxiv, 9 January 2006
"By merely counting the amount of citations and disregarding the prestige of the citing journals, the ISI IF is a metric of popularity, not of prestige. We demonstrate how a weighted version of the popular PageRank algorithm can be used to obtain a metric that reflects prestige. ... Furthermore, we introduce the Y-factor which is a simple combination of both the ISI IF and the weighted PageRank, and find that the resulting journal rankings correspond well to a general understanding of journal status."

Added 03 August 2006 Moed, H.F. (2005)
Citation analysis of scientific journals and journal impact measures
Current Science, 89 (12): 1990-1996, December 25, 2005

Added 28 February 2006 Dong, P., Loh, M. and Mondry, A. (2005)
The "impact factor" revisited
Biomedical Digital Libraries, December 2005
This is a review, so the findings are not new, but this is perhaps the first such paper to reflect on the effect of free and online availability on journal impact factors, among other IF-related issues.

Added 30 December 2005 Hardy, R., Oppenheim, C., Brody, T. and Hitchcock, S. (2005)
Open Access Citation Information (.doc, 105pp)
Author eprint, November 11, 2005, JISC Committee for the Information Environment (JCIE) Scholarly Communication Group, September 2005
Describes a proposal to increase the exposure of open access materials and their references to indexing services, and to motivate new services by reducing setup costs.

Added 30 December 2005 Monastersky, R. (2005)
Impact Factors Run Into Competition
Chronicle of Higher Education, October 14, 2005

Comment on this article:
Harnad, S. IFs: solution is obvious "Although Richard Monasterky describes a real problem -- the abuse of journal impact factors -- its solution is so obvious -- (a) wealth of powerful new resources are on the way for measuring and analyzing research usage and impact online" American Scientist Open Access Forum, 10 October 2005
Bensman, S. J. Good copy, bad science "I found his article to be unfair, since he concentrated on the shenanigans that are being played with impact factor and supposed errors of ISI in constructing impact factor. This makes for good copy but bad science." Sigmetrics listserv, 18 November 2005
Leydesdorff, L. Discipline-specific impact factor "Monasterky's article lists a number of problems with the ISI-impact factor. However, he fails to mention that the average impact factors vary among fields of science. For example, impact factors in toxicology are considerably lower than in immunology. ... A fix to these problems might be a discipline-specific impact factor. ... Using ISI's Journal Citation Reports, I created the raw materials to make maps of the citation neighborhoods of all the journals." Sigmetrics listserv, 16 September 2005

Added 28 February 2005 Garfield, E. (2005)
The Agony and the Ecstasy - ­The History and Meaning of the Journal Impact Factor (pdf, 22pp)
International Congress on Peer Review and Biomedical Publication, Chicago, September 16, 2005
Garfield's typically dry, data-filled but essential take on JIFs.

Publishers promote impact factors of OA journals
BioMed Central "Open access journals get impressive impact factors" 23 June 2005
Public Library of Science "The first impact factor for PLoS Biology - 13.9" 27 June 2005
See also this discussion of these announcements on SPARC Open Access Forum, prompted by Elsevier's response from Tony McSean, followed by David Goodman, Charles Bailey, (both 8 July) and Matthew Cockerill (10 July), or see this summary of the discussion: "BMC’s Impact Factors: Elsevier’s Take and Reactions to It", Digital Koans (Charles Bailey's Weblog), 11 July 2005, including Peter Suber's conclusion: "It’s important to distinguish the citation impact of an individual article from a journal impact factor. The BMC-Elsevier debate is about the latter. But OA is more likely to rise and fall according to the former."

Abbasi, K. (2004)
Let's dump impact factors
BMJ, Vol. 329, 16 October 2004
BMJ Rapid Responses to this editorial; also see this list response

Baudoin, L., Haeffner-Cavaillon, N., Pinhas, N., Mouchet, S. and Kordon, C. (2004)
Bibliometric indicators: realities, myth and prospective (abstract only, full paper in French)
Med Sci (Paris), 20 (10):909-15, October 2004

Jacsó, P. (2004)
The Future of Citation Indexing - Interview with Dr. Eugene Garfield (pdf 3pp)
Author eprint, in Online, January 2004

Cockerill, M. J. (2004)
Delayed impact: ISI's citation tracking choices are keeping scientists in the dark
BMC Bioinformatics 2004, 5:93, 12 July 2004

Added 26 September 2005 Shin E. J. (2003)
Do Impact Factors change with a change of medium? A comparison of Impact Factors when publication is by paper and through parallel publishing (abstract only)
Journal of Information Science, 29 (6): 527-533, 2003
"it is found that Impact Factors of (journals from the period) 2000 and 2001 were significantly higher than those of 1994 and 1995 in the journals published by parallel publishing (combination journals–simultaneous publication of paper and electronic journals). In particular, the Impact Factors of the combination journals increased after the journals transformed their available media from paper journals to combination ones."

Walter, G., Bloch, S., Hunt, G. and Fisher, K. (2003)
Counting on citations: a flawed way to measure quality
MJA, 2003, 178 (6): 280-281

Borgman, C. L. and Furner, J. (2002)
Scholarly Communication and Bibliometrics, author preprint (pdf 45pp)
Author eprint, in Annual Review of Information Science and Technology, Vol. 36, edited by B. Cronin, 2002

Guédon, J.-C. (2001)
In Oldenburg’s Long Shadow: Librarians, Research Scientists, Publishers, and the Control of Scientific Publishing
Creating the Digital Future, Proceedings of the 138th Annual Meeting, Association of Research Libraries, Toronto, Ontario, May 23-25, 2001

Garfield, E. (1999)
Journal impact factor: a brief review
CMAJ, 161 (8), October 19, 1999

Wouters, P. (1999)
The Citation Culture (pdf 290pp)
PhD Thesis, University of Amsterdam, 1999

Garfield, E. (1998)
The use of journal impact factors and citation analysis in the evaluation of science
Author eprint, presented at the 41st Annual Meeting of the Council of Biology Editors, Salt Lake City, UT, May 4, 1998

Seglen, P. O. (1997)
Why the impact factor of journals should not be used for evaluating research
BMJ, 314:497, 15 February 1997

Garfield, E. (1973)
Citation Frequency as a Measure of Research Activity and Performance (pdf 3pp)
Author eprint, in Essays of an Information Scientist, 1: 406-408, 1962-73, Current Contents, 5, January 31, 1973

Garfield, E. (1955)
Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas
Author eprint, in Science, Vol:122, No:3159, p.108-111, July 15, 1955

Open access

Notes. In printed form, little of the published research literature was free. With more material beginning to appear on the Web from the mid-1990s, more became freely available. Open access is in a sense a formalisation of that process, a recognition that all published, refereed scholarly papers could and should be freely accessible in some form to everyone online without compromising the quality and integrity of the literature. That is the goal. This simple idea, especially when focussed on this very specific literature, seems to have been quite difficult to grasp for many bound by the old, pre-online ways of thinking. Despite the often antithetical tone of the debate, progress has been rapid since the landmark of the Budapest Open Access Initiative in February 2002, even impinging on prospective government policies by 2003 (e.g. Martin Sabo's Public Access to Science Act; UK House committee releases its report on open access; Major development in providing OA to taxpayer-funded research). It has all been brilliantly logged by Peter Suber in Open Access News (http://www.earlham.edu/~peters/fos/fosblog.html), but for a very quick overview the following papers are sufficient.

Suber, P. (updated)
Open Access Overview

Swan, A. (2005)
Open Access
JISC, Briefing Paper, 1 April 2005

Suber, P. (2004)
A Primer on Open Access to Science and Scholarship
Author eprint, in Against the Grain, Vol. 16, No. 3, June 2004

Harnad, S. (2004)
The Green Road to Open Access: A Leveraged Transition
American Scientist Forum, January 07, 2004

Suber, P. (2003)
Removing the Barriers to Research: An Introduction to Open Access for Librarians
Author eprint, in College & Research Libraries News, 64, February, 92-94, 113

^Top  
<Home                                                                                                                              
The OpCit Project
This page produced and maintained by the Open Citation project. Contact us