Find you way through the bibliography
Latest additions
Studies with original data
Web tools for measuring impact | Comparative reviews
Background
The financial imperative: correlating research access, impact and assessment | Citation analysis, indexes and impact factors | Open access
Despite significant growth in the number of research papers available through
open access, principally through author self-archiving in institutional
archives, it
is estimated that only c. 20% of the number of papers published annually
are open access. It is up to the authors of papers to change this.
Why might open access be of benefit to authors? One universally important
factor for all authors is impact, typically measured by the number of times
a paper is cited (some older studies have estimated monetary returns to authors from article
publication via the role citations play in determining salaries). Recent studies have begun to show that open access increases
impact. More studies and more substantial investigations are
needed to confirm the effect, although a simple example demonstrates the effect.
This chronological bibliography is intended to describe progress in
reporting these studies; it also lists the Web tools available to measure
impact. It is a focused bibliography, on the relationship between impact
and access. It does not attempt to cover citation impact, or other related
topics such as open access, more generally, although some key papers in
these areas are listed as jump-off points for wider study.
Last updated 26 May 2008; first posted 15 September 2004. Please email
additions, corrections or comments to Steve Hitchcock.
"This ongoing chronological bibliography may be worth bookmarking and checking every few months. There’s very little annotation, but it’s a good brief bibliography on a narrow - but important - subject."
Walt Crawford, Cites & Insights (November 2004, p13) http://cites.boisestate.edu/civ4i13.pdf
"excellent bibliography"
Peter Suber, American Scientist Open Access Forum (28 September 2005) http://www.ecs.soton.ac.uk/~harnad/Hypermail/Amsci/4804.html
Latest additions
Added 26 May 2008 Norris, M., Oppenheim, C. and Rowland, F. (2008)
Open Access Citation Rates and Developing Countries (extended abstract)
12th International Conference on Electronic Publishing (ElPub 2008), Toronto, June 25-27, 2008
"the admittedly small number of citations from authors in developing countries do indeed seem to show a higher proportion of citations being given to OA articles than is the case for citations from developed countries."
Added 26 May 2008 Althouse, B. M., West, J. D., Bergstrom, T. C. and Bergstrom, C. T. (2008)
Differences in Impact Factor Across Fields and Over Time
eScholarship Repository, California Digital library, Department of Economics, University of California Santa Barbara, Departmental Working Papers, paper 2008-4-23, April 23, 2008
Added 26 May 2008 Cheng, W. H. and Ren, S. L. (2008)
Evolution of open access publishing in Chinese scientific journals (full text requires subscription; abstract only)
Learned Publishing, Vol. 21, No. 2, April 2008, 140-152
From the abstract: "Citation indicators of OA journals were found to be higher than those of non-OA journals."
Added 26 May 2008 Lokker, C., McKibbon, K. A., McKinlay, R.J., Wilczynski, N. L. and Haynes, R. B. (2008)
Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: retrospective cohort study
BMJ, 2008;336:655-657 (22 March), published 21 February 2008
"Conclusion: Citation counts can be reliably predicted at two years using data within three weeks of publication."
Comment on this paper:
Harnad, S. (2008), Predicting later citation counts from very early data, American Scientist Open Access Forum, 21 April 2008: "This finding reinforces the importance of taking into account as many predictor metrics as possible, though a number of the metrics do seem specific to clinical medical articles. ... We might perhaps make a distinction between static and dynamic metrics. This study was based largely on static metrics, in that they are fixed as of the day of publication. Dynamic metrics like early downloads (which have also been found to predict later citations) were not included, nor were early citation growth metics (also predictive of later citations). ... To my mind, the article reinforces the importance of validating all these metrics, not just against one another, but against peer evaluations, in all fields, as in the RAE 2008 database"
Added 26 May 2008 Turk, N. (2008)
Citation impact of Open Access journals (full text requires subscription; summary only)
New Library World, Vol. 109, No. 1/2, January/February 2008, 65-74
Review of the main research about citation impact of Open Access journals, focused on LIS journals.
Added 26 May 2008 Surya, M., D'Este, P. and Neely, A. (2008)
Citation Counts: Are They Good Predictors of RAE Scores? A bibliometric analysis of RAE 2001
Cranfield QUEprints, 31.01.2008
Comment on this paper:
RAE/REF research, JISC-REPOSITORIES, 15 February 2008
Oppenheim, C. "I've had a first look at it; it uses methods not previously employed in such studies and without a full explanation of how the research was conducted. That's not to say it is invalid, but it is a lost opportunity, having collected so much data, not to have followed standard methods or to explain things more fully."
Johnson, I. M. "shows an inadequate understanding of the contextual issues. Moreover, most of the discussion in section 5 appears to have little or no basis in the preceding data."
Harnad, S. "This pilot study has some methodological weaknesses. The remedy for the ostensible problems encountered in this study is for
the panel rankings in the parallel metric/panel RAE 2008 to be analysed
in a full-scale multiple regression study using as rich and diverse
as possible a spectrum of predictive metrics (not just citation
counts!), discipline by discipline."
Added 26 May 2008 Hardisty, D. J. and Haaga, D. A. F. (2008)
Diffusion of Treatment Research: Does Open Access Matter? (pdf 39pp)
Center for the Decision Sciences, Columbia University, in Journal of Clinical Psychology, Vol. 64(7), 1-19 (2008)
From the abstract: "In a
pair of studies, mental health professionals were given either no citation, a
normal citation, a linked citation, or a free access citation and were asked
to find and read the cited article. After one week, participants read a
vignette on the same topic as the article and gave recommendations for an
intervention. In both studies, those given the free access citation were
more likely to read the article, yet only in one study did free access
increase the likelihood of making intervention recommendations
consistent with the article."
Added 26 May 2008 Sharma, H. P. (2007)
Download plus citation counts - a useful indicator to measure research impact (correspondence, pdf 1pp)
Current Science, 92 (7): 873-873, April 10, 2007
Added 26 May 2008 Chapman, S., Nguyen, T. N. and White, C. (2007)
Press-released papers are more downloaded and cited (full text requires subscription; extract only)
Tobacco Control, 16 (1): 71-71, February 2007
Added 26 May 2008 Walters, G. D. (2006)
Predicting subsequent citations to articles published in twelve crime-psychology journals: Author impact versus journal impact (full text requires subscription; abstract only)
Scientometrics, 69 (3): 499-510, December 2006
"These results suggest that author impact may be a more powerful predictor of citations received by a journal article than the periodical in which the article appears."
Citation analysis is specialised and difficult. To make the case for, or against, a claim such as 'open access increases impact' requires a lot of the reader, who may not be a specialist but who wants to try and understand the point at issue and decide if it has any relevance to him or her. The following simple example is included for this reason, not as proof but as evidence of the effect within a particular domain. Draw your own conclusions, and then read the more detailed evidence of the bibliography if you are still interested.
"Measuring the effect for physics or astronomy is easy. This
link returns the number of articles published in the Astrophysical
Journal in 2003 and their number of citations.
"This next link shows the number of these papers which are available OA
in the arXiv, and their citations.
"The result is that 75% of the papers are in the arXiv, and they
represent 90% of the citations, a 250% OA effect.
"By replacing ApJ with the mnemonic for any other physics or astronomy
journal one can repeat the measurement; for Nuclear Physics A (NuPhA)
one gets that 32% of the articles are in the arXiv, and they represent
78% of the citations, a 740% OA effect."
From Michael Kurtz, American Scientist Open Access Forum, 28 September 2005 http://users.ecs.soton.ac.uk/harnad/Hypermail/Amsci/4807.html
Note, the database links are 'live', i.e. they return the current database figures, not the exact figures on which Michael Kurtz would have based his calculations, but the percentages quoted are unlikely to change dramatically, in the short term at least.
Elucidation of calculation (by Stevan Harnad, figures valid on 22 July 2007)
For ApJ:
TOT: articles 2592 citations 70732
Arx: articles 1943 citations 62586 c/a 32.21 (rounded to 32)
Non: articles 649 citations 8146 c/a 12.55 (rounded to 13)
Then 32/13 = 2.5 (250%)
For NuPhA:
TOT: articles 1134 citations 4451
Arx: articles 344 citations 3225 c/a 9.375
Non: articles 790 citations 1226 c/a 1.552
Then 9.375/1.553 = 6.041 (600%)
Michael Kurtz comments: "The differences in (NuPhA: 740% to 600% effect) results are because the database has changed over the past two years since I did it. There is a systematic error in the calculations for Nuclear Physics A
(Elsevier does not give us the references) so the results will be higher
than the true value. Physical Review C (Nuclear Physics) has an OA
advantage number of 221%, the systematic in this case is small and in the other
direction."
Studies with original data
Highlights
Lawrence (2001) was the first to publish data recognising the trend for
online publication to increase impact, confirmed for open access papers
by the work of the Open Citation Project based on arXiv (e.g. Harnad and
Brody, D-Lib, 2004), and by Kurtz et al. (2004a, 2003a) looking at
the NASA Astrophysics Data System. Commenting on Harnad and Brody (D-Lib,
2004) in Open Access News, Peter Suber said:
This is an important article. It's the first major study since
the famous Lawrence paper documenting the proposition that OA increases
impact. It's also the first to go beyond Lawrence in scope and method in
order to answer doubts raised about his thesis. By confirming that OA increases
impact, it gives authors the best of reasons to provide OA to their own
work (21 June 2004)
Broader collaborations have emerged to extend these findings (e.g. Brody
et al. 2004).
Open access has become feasible because of the move towards online publication
and dissemination. A new measure that becomes possible with online publication
is the number of downloads or 'hits', opening a new line of investigation.
Brody et al. have been prominent in showing there is a correlation
between higher downloads and higher impact, particularly for high impact
papers, holding out the promise not just for higher impact resulting from
open access but for the ability to predict high impact papers much earlier,
not waiting years for those citations to materialise (e.g. Brody and Harnad
2004, in prep.). The effect can be verified with the Correlation Generator
(below).
(Note. The latest listings might include preprints, or even pre-preprints. This area of study is effectively a work in progress, and as such the list is intended to raise awareness of the most recent results, even where these may not be definitive or final versions. Check back for definitive versions.)
Added 11 February 2008 Chu, H. and Krichel, T. (2008)
Downloads vs. Citations: Relationships, Contributing Factors and Beyond
E-LIS, 9 February 2008, in 11th Annual Meeting of the International Society for Scientometrics and Informetrics, Madrid, 25-27 June 2007
From the abstract: "In a nutshell, an infrastructure that encourages downloading at digital libraries would eventually lead to higher usage of their resources."
Added 11 February 2008 Kousha, K. and Thelwall, M. (2007)
The Web impact of open access social science research (full-text requires subscription; otherwise abstract only)
Library & Information Science Research, Volume 29, Issue 4, December 2007, 495-507, available online 15 October 2007
preprint http://www.scit.wlv.ac.uk/~cm1993/papers/OpenAccessSocialSciencePreprint.doc (.doc 12pp)
From the abstract: "The results suggest that new types of citation information and informal scholarly indictors could be extracted from the Web for the social sciences."
Added 17 December 2007 Dietrich, J. P. (2007)
The Importance of Being First: Position Dependent Citation Rates on arXiv:astro-ph
arXiv.org, arXiv:0712.1037v1 [astro-ph], 6 December 2007, in Publications of the Astronomical Society of the Pacific, 120 (864): 224-228, February 2008
From the abstract: "We study the dependence of citation counts of e-prints published on the arXiv:astro-ph server on their position in the daily astro-ph listing. ... cannot exclude that increased visibility at the top of the daily listings contributes to higher citation counts as well."
Comment on this paper:
Kurtz, M. J. (2007), Systematics in Citation Statistics: Implications for the "OA Advantage", American Scientist Open Access Forum, 14 December 2007: "shows that systematic increases in
citation rates on the order of the 2 to 1 "OA advantage" can be obtained
by other, systematic means, such as author bias, as pointed out by
myself, Moed and others. Note that OA plays no
role in the Dietrich paper, all the articles are from arXiv.
"
Added 13 September 2007 Kurtz, M. J. and Henneken, E. A. (2007)
Open Access does not increase citations for research articles from The Astrophysical Journal
arXiv.org, arXiv:0709.0896v1 [cs.DL], 6 September 2007
Abstract: We demonstrate conclusively that there is no "Open Access Advantage" for papers from the Astrophysical Journal. The two to one citation advantage enjoyed by papers deposited in the arXiv e-print server is due entirely to the nature and timing of the deposited papers. This may have implications for other disciplines.
Comment on this paper:
Harnad, S. (2007), Where There's No Access Problem There's No Open Access Advantage, Open Access Archivangelism blog, September 7, 2007: "K & H suggest: "[T]here is no 'Open Access Advantage' for papers from the Astrophysical Journal" because "in a well funded field like astrophysics essentially everyone who is in a position to write research articles has full access to the literature." This seems like a perfectly reasonable explanation for K&H's findings. Where there is no access problem, OA cannot be the cause of whatever higher citation count is observed for self-archived articles. K&H conclude that "[t]his may have implications for other disciplines." It should be evident, however, that the degree to which this has implications for other disciplines depends largely on the degree to which it is true in other disciplines that "essentially everyone who is in a position to write research articles has full access to the literature."
Added 22 August 2007 Sotudeh, H. and Horri, A. (2007)
The citation performance of open access journals: A disciplinary investigation of citation distribution models (full-text subscribers only; no abstract)
Journal of the American Society for Information Science and Technology, Vol. 58, No. 13, 2007, 2145-2156, published online August 17, 2007
From the conclusion: "To sum up, the similarity of the science system across OAJ and NOAJ boundaries has been confirmed. We see this as further evidence of OA's widespread recognition by scientific communities. However, because the magnitudes of the exponents found in this study are lower than what was previously observed for the whole system, OA may currently perform at a slightly lower level. According to the models used in this study, the citation distributions between fields are strongly disproportionate in Life Sciences and Engineering and Material Sciences, favoring larger fields in the former, but smaller fields in the latter. However, the distributions tend to be rather linear in the Natural Sciences."
Added 13 September 2007 Brody, T., Carr, L., Gingras, Y., Hajjem, C., Harnad, S. and Swan, A. (2007)
Incentivizing the Open Access Research Web: Publication-Archiving, Data-Archiving and Scientometrics
CTWatch Quarterly, Vol. 3, No. 3, August 2007
Added 22 August 2007 Lin, S.-K. (2007)
Editorial: Non-Open Access and Its Adverse Impact on Molecules
Molecules, 12, 1436-1437, 16 July 2007
The point of this short editorial is clear, that the difference between the OA and non-OA content in the journal Molecules is clearly reflected in higher citations for the former. The context could be clearer, however. The OA/non-OA history of the journal, especially prior to the period under review (2005-6), is not elaborated and familiarity with the journal is assumed.
Added 22 August 2007 Taylor, D. (2007)
Looking for a Link: Comparing Faculty Citations Pre and Post Big Deals
Electronic Journal of Academic and Special Librarianship, v.8 no.1 (Spring 2007)
Note. The Big Deal is where a library or consortium of libraries subscribes to a larger package of a publisher's journals than they would have if they had subscribed to journals individually. Big Deals are claimed to improve access for an institution's users. "Pre Big Deal, the percentage of citations to journals that are part of Big Deals but were previously not subscribed to was an average of 2.6%. Post Big Deal this increased to an average of 6.1%." There is no analysis or comment on how this result might be affected if it was considering open access.
Added 22 May 2007 Craig, I. D., Plume, A. M., McVeigh, M. E., Pringle, J. and Amin, M. (2007)
Do Open Access Articles Have Greater Citation Impact? A critical review of the literature
Publishing Research Consortium, undated (announced 17 May 2007), Journal of Informetrics, 1 (3): 239-248, July 2007
Comment on this paper:
Suber, P. (2007), Publishers doubt the OA impact advantage, Open Access News, 18 May 2007: "we shouldn't be surprised to see that good relevant literature that is easier to find and retrieve is cited more often than good relevant literature that is harder to find and retrieve. Or, if a careful study concluded that this view is false, then one might expect it to be more careful in summarizing the reasons why."
Harnad, S. (2007), Craig et al.'s review of the OA citation advantage, Open Access Archivangelism blog, 26 May 2007: "I also agree that not one of the studies done so far is without some
methodological flaw that could be corrected.
But it is also highly probable that the results of the methodologically
flawless versions of all those studies will be much the same as the
results of the current studies. That's what happens when you have a
robust major effect, detected by virtually every study, and only ad hoc
methodological cavils and special pleading to rebut each of them with.
But I am sure those methodological flaws will not be corrected by these
authors, because -- OJ Simpson's "Dream Team" of Defense Attorneys comes
to mind -- Craig et al's only interest is evidently in finding flaws and
alternative explanations, not in finding out the truth -- if it goes
against their client's interests...
Iain D.Craig: Wiley-Blackwell
Andrew M.Plume, Mayur Amin: Elsevier
Marie E.McVeigh, James Pringle: Thomson Scientific"
Added 10 May 2007 Tonta, Y., Ünal, Y. and Al, U. (2007)
The Research Impact of Open Access Journal Articles
E-LIS, 30 April 2007, also in Proceedings ELPUB 2007, the 11th International Conference on Electronic Publishing, Vienna, 13-15 June 2007
Comment on this paper:
Harnad, S. (2007), OA citation impact study: No conclusions possible, American Scientist Open Access Forum, 1 May 2007: "No comparison was made with non-OA journals in the same fields. Hence it
is impossible to say whether any of these differences have anything to
do with OA. Fields no doubt differ in their average number of citations.
Journals no doubt differ too, in subject matter, quality, and citation
impct. And it is not clear whether the OA journals in each field are the
top, medium or bottom journals, relative to the non-OA journals. No conclusions at all can be drawn from this study. The authors are encouraged to do the necessary controls."
Added 10 May 2007 Piwowar, H. A., Day, R. S. and Fridsma, D. B. (2007)
Sharing Detailed Research Data Is Associated with Increased Citation Rate
PLoS ONE, March 21, 2007
Principal Findings: "We examined the citation history of 85 cancer microarray clinical trial publications with respect to the availability of their data. The 48% of trials with publicly available microarray data received 85% of the aggregate citations. Publicly available data was significantly (p = 0.006) associated with a 69% increase in citations, independently of journal impact factor, date of publication, and author country of origin using linear regression."
Comment on this paper:
Suber, P. (2007), Open data and citation impact, Open Access News, 28 March 2007: " Many studies have shown a correlation between OA articles and citation impact. I believe this is the first study to document a similar correlation between OA data and citation impact. Spread the word to colleagues who are still hoarding data, waiting too long before releasing it, or unable to see any gain for themselves in data sharing."
Added 8 March 2007 Bergstrom, T. C. and Lavaty, R. (2007)
How often do economists self-archive?
eScholarship Repository, University of California, February 8, 2007
Added 22 January 2007 Harnad, S. and Hajjem, C. (2007)
The Open Access Citation Advantage: Quality Advantage Or Quality Bias?
Author blog, Open Access Archivangelism, 21 January 2007
Does the OA Advantage (OAA) occur because authors are more likely to self-selectively self-archive articles that are more likely to be cited (self-selection "Quality Bias": QB), or because articles that are self-archived are more likely to be cited ("Quality Advantage": QA)? Preliminary evidence based on over 100,000 articles from multiple fields, comparing self-selected self-archiving with mandated self-archiving to estimate the contributions of QB and QA to the OAA shows: "Both factors contribute, and the contribution of QA is greater." Includes comment on Moed, H. (2006), The effect of 'Open Access' upon citation impact: An analysis of ArXiv's Condensed Matter Section.
Comment on this paper:
Laloe, F. and Harnad, S., Re: Quality Bias vs Quality Advantage, SIGMETRICS, see installments of this discussion on 20, 21 and 22 February 2007
Added 17 January 2007 Harnad, S. (2007)
Citation Advantage For OA Self-Archiving Is Independent of Journal Impact Factor, Article Age, and Number of Co-Authors
Author blog, Open Access Archivangelism, 17 January 2007
Further comment on Eysenbach, G. (2006), Citation Advantage of Open Access Articles: "The OA-self-archiving advantage remains a robust, independent factor."
Added 17 January 2007 Brody, T. (2007)
Evaluating Research Impact through Open Access to Scholarly Communication
PhD, Electronics and Computer Science, University of Southampton, May 2006, in ECS EPrints, 14 January 2007
Added 8 March 2007 McDonald, J. D. (2007)
Understanding Online Journal Usage: A Statistical Analysis of Citation and Use
Journal of the American Society for Information Science & Technology, 58(1): 39-50, January 1, 2007, also in Caltech Library System Papers and Publications, 18 May 2006
Added 23 November 2006 Harnad, S. (2006)
The Self-Archiving Impact Advantage: Quality Advantage or Quality Bias?
Author blog, Open Access Archivangelism, 20 November 2006
Added 19 November 2006 Moed, H. F. (2006)
The effect of 'Open Access' upon citation impact: An analysis of ArXiv's Condensed Matter Section
ArXiv, Computer Science, cs.DL/0611060, 14 November 2006, in Journal of the American Society for Information Science and Technology, Vol. 58, No. 13, 2007, 2145-2156, published online August 30, 2007 http://dx.doi.org/10.1002/asi.20663 (subscriber access only to full text)
"This article statistically analyses how the citation impact of articles deposited in the Condensed Matter section of the preprint server ArXiv, and subsequently published in a scientific journal, compares to that of articles in the same journal that were not deposited in that archive. Its principal aim is to further illustrate and roughly estimate the effect of two factors, 'early view' and 'quality bias', upon differences in citation impact between these two sets of papers ... The analysis provided evidence of a strong quality bias and early view effect. Correcting for these effects, there is in a sample of 6 condensed matter physics journals studied in detail, no sign of a general 'open access advantage' of papers deposited in ArXiv. The study does provide evidence that ArXiv accelerates citation, due to the fact that that ArXiv makes papers earlier available rather than that it makes papers freely available."
Comment on this paper:
Harnad, S. (2007), The Open Access Citation Advantage: Quality Advantage Or Quality Bias?, Open Access Archivangelism, 21 January 2007
Moed, H. (2006) reply on OA to sigmetrics, SIGMETRICS list server, 8 December 2006: On Quality bias. (Harnad) wrote: "The fact that highly-cited articles (Kurtz) and articles by highly-cited authors (Moed) are more likely to be Arxived certainly does not settle the question of cause and effect: It is just as likely that better articles benefit more from Arxiving (QA) as that better authors/articles tend to Arxive/be-Arxived more (QB)". Citation rates may be influenced both by the ‘quality’ of the papers and by the access modality (deposited versus non-deposited). This is why I estimated author prominence on the basis of the citation impact of their non-archived articles only. But even then I found evidence that prominent, influential authors (in the above sense) are overrepresented in papers deposited in ArXiv. But I did more that that. I calculated Arxiv Citation Impact Differentials (CID) at the level of individual authors. Next, I calculated the median CID over authors publishing in a journal. How then do you explain my empirical finding that for some authors the CID is positive, for others it is negative, while the median CID over authors does not significantly differ from zero (according to a Sign test) for all journals studied in detail except Physical Review B, for which it is only 5 per cent? If there is a genuine ‘OA advantage’ at stake, why then does it for instance not lead to a significantly positive median CID over authors? Therefore, my conclusion is that, controlling for quality bias and early view effect, in the sample of 6 journals analysed in detail in my study, there is no sign of a general ‘open access advantage’ of papers deposited in ArXiv’s Condensed Matter Section. ... I hope that more case studies will be carried out in the near future, applying the methodologies I proposed in my paper.
Harnad, S. (2006) The Self-Archiving Impact Advantage: Quality Advantage or Quality Bias?, Open Access Archivangelism, 20 November 2006
Added 13 September 2007 Bollen, J. and Van de Sompel, H. (2006)
Usage Impact Factor: the effects of sample characteristics on usage-based impact metrics
arXiv.org > cs > arXiv:cs/0610154v2 [cs.DL], 26 October 2006, in Journal of the American Society for Information Science and Technology, 59 (1): 136-149, January 1, 2008
Updated 23 November 2006 Mayr, P. (2006)
Constructing experimental indicators for Open Access documents
E-LIS, 05 October 2006, to appear in Research Evaluation, special issue on 'Web indicators for Innovation Systems', 14, 2 (2006)
Author preprint, http://www.ib.hu-berlin.de/~mayr/arbeiten/mayr_RE06.pdf (pdf 9pp)
Added 19 November 2006 Henneken, E. A., Kurtz, M. J., Warner, S., Ginsparg, P., Eichhorn, G., Accomazzi, A., Grant, C. S., Thompson, D., Bohlen, E. and Murray, S. S. (2006)
E-prints and Journal Articles in Astronomy: a Productive Co-existence
ArXiv, Computer Science, cs.DL/0609126, 22 September 2006, in Learned Publishing, Vol. 20, No. 1, January 2007, 16-22
Added 19 November 2006 Zhang, Y. (2006)
The Effect of Open Access on Citation Impact: A Comparison Study Based on Web Citation Analysis (abstract only)
Libri, September 2006 (Full text for subscribers)
Added 03 August 2006 Metcalfe, T. S. (2006)
The Citation Impact of Digital Preprint Archives for Solar Physics Papers
Solar Physics, Vol. 239, No. 1-2, December 2006, pp. 549-553
also in ArXiv, Astrophysics, astro-ph/0607079, 5 July 2006 http://arxiv.org/abs/astro-ph/0607079
"Most astronomers now use the arXiv.org server (astro-ph) to distribute preprints, but the solar physics community has an independent archive hosted at Montana State University. For several samples of solar physics papers published in 2003, I quantify the boost in citation rates for preprints posted to each of these servers. I show that papers on the MSU archive typically have citation rates 1.7 times higher than the average of similar papers that are not posted as preprints, while those posted to astro-ph get 2.6 times the average. A comparable boost is found for papers published in conference proceedings, suggesting that the higher citation rates are not the result of self-selection of above-average papers."
Added 03 August 2006 Henneken, E. A., Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Thompson, D., and Murray, S. S. (2006)
Effect of E-printing on Citation Rates in Astronomy and Physics
Journal of Electronic Publishing, Vol. 9, No. 2, Summer 2006, also in ArXiv, Computer Science, cs.DL/0604061, v2, 5 June 2006 http://arxiv.org/abs/cs/0604061
"It has been observed that papers that initially appear as arXiv e-prints get cited more than papers that do not. Using the citation statistics from the NASA-Smithsonian Astrophysics Data System, we confirm the findings from other studies, we examine the average citation rate to e-printed papers in the Astrophysical Journal, and we show that for a number of major astronomy and physics journals the most important papers are submitted to the arXiv e-print repository first.
Added 03 August 2006 Kousha, K. and Thelwall, M. (2006)
Google Scholar Citations and Google Web/URL Citations: A Multi-Discipline Exploratory Analysis
E-LIS, 05 June 2006, also in Proceedings International Workshop on Webometrics, Informetrics and Scientometrics & Seventh COLLNET Meeting, Nancy (France), May 2006
"we built a sample of 1,650 articles from 108 Open Access (OA) journals published in 2001 in four science and four social science disciplines. We recorded the number of citations to the sample articles using several methods based upon the ISI Web of Science, Google Scholar and the Google search engine (Web/URL citations). For each discipline, we found significant correlations between ISI citations and both Google Scholar and Google Web/URL citations; with similar results when using total or average citations, and when comparing within and across (most) journals."
Added 16 May 2006 Eysenbach, G. (2006)
Citation Advantage of Open Access Articles
PLoS Biology, Volume 4, Issue 5, May 2006
Further evidence for the OA citation advantage, although quite critical of other studies with which its findings broadly agree. This example is based on a small, single journal sample (PNAS: Proceedings of the National Academy of Sciences). Since PNAS offers authors the choice of paying to provide open access to published papers and/or freely self-archiving, a 'Secondary analysis' considers the relative impact of each type of OA, although the number of papers involved is really too small to give this result the weight of the broader findings. The paper is accompanied by two editorials, one in the publishing journal, the other a self-published editorial by the author:
MacCallum, C. J. and Parthasarathy, H. (2006) Editorial: Citation Advantage of Open Access Articles, PLoS Biology, Volume 4, Issue 5, May 2006
Eysenbach, G. (2006) The Open Access Advantage, Journal of Medical Internet Research, 2006;8(2):e8
Comment on this paper:
Harnad, S. on Chronicle of Higher Education News Blog on this paper, 15 May 2006:
"The Eysenbach study is certainly not “the first to compare open-access and non-open-access papers from the same journal”. See [this bibliography]"
Harnad, S. eLetter, PLoS, Pipe-Dreams and Peccadillos, on the PLoS editorial, 16 May 2006:
"There can be disagreement about what evidence one counts as "solid," but
there can be little dispute that prior evidence derived from
substantially larger and broader-based samples showing substantially the
same outcome can hardly be described as "surprisingly hard to find". The only new knowledge from this small, journal-specific sample
was (1) the welcome finding of how early the OA advantage can manifest
itself, plus (2) some less clear findings about differences between
first- and last-author OA practices, plus (3) a controversial finding
that will most definitely need to be replicated on far larger samples in
order to be credible: "The analysis revealed that self-archived articles
are also cited less often than OA articles from the same journal.""
Author's response eLetter, PLoS, 17 May 2006:
"None of the previous papers in [this bibliography] employed a similar methodology, working with data from a "gold-OA" journal. ... Regarding "larger samples" I think rigor and quality (leading to internal validity) is more important than quantity (or sample size). Going through the laborious effort to extract article and author characteristics for a limited number of articles (n=1492) in order to control for these confounders provides scientifically stronger evidence than doing a crude, unadjusted analysis of a huge number of online accessible vs non-online accessible articles, leaving open many alternative explanations. ... contrary to what Harnad said, this study is NOT at all "showing substantially the same outcome". On the contrary, the effect of green-OA -- once controlled for confounders -- was much less than what others have claimed in previous papers."
Harnad, S. blog, Confirming the Within-Journal OA Impact Advantage, 18 May 2006:
"with the large, consistent within-journal OA/NOA differences found across all journals, all disciplines and all years in samples four orders of magnitude larger than Eysenbach's, it is not at all clear that controls for those "multiple confounders" are necessary in order to demonstrate the reality, magnitude and universality of the OA advantage. That does not mean the controls are not useful, just that they are not yet telling us much that we don't already know. ... the true measure of the SOA (Self-Archived OA) advantage today (at its 15% spontaneous baseline) is surely not to be found in PNAS but in the statistically far more numerous, hence far more representative full-spectrum of journals that do not yet offer POA (Payed OA). (I would be delighted if those journals took the Eysenbach findings as a reason for offering a POA option! But not at the expense of authors drawing the absurd conclusion -- not at all entailed by Eysenbach's PNAS-specific results -- that in the journals they currently publish in, SOA alone would not confer citation advantages at least as big as the ones we have been reporting.)"
Author's response WebCite blog, The OA debate between an “archivangelist” and an OA researcher, 24 May 2006:
"“Confounded” associations between two variables which falsely suggests causality can ONLY be ruled out if one controls for the confounder no matter how “strong, consistent” the effect appears ... Our “events” (citations) are determined by many different “causes”, of which “access” is only one variable – many other variables, including confounders, have to be taken into account.
Harnad, S. blog, The Epidemiology of OA, 26 May 2006: considers each of Eysenbach’s list of confounders, which ones might have an effect and which ones will be tested. "Stay tuned..."
Suber, P. SPARC Open Access Newsletter, issue #98, Gunther Eysenbach confirms the OA impact advantage, 2 June 2006:
"There's some controversy about whether some earlier results, especially by Tim Brody, Chawki Hajjem, and Stevan Harnad, are the same or only similar to some of Eysenbach's results. But no one doubts that Eysenbach has new and valid results, or that he has persuasively advanced the case that OA helps authors and journals build their citation impact. While there have been many previous studies of the OA impact advantage, none has made the splash that Eysenbach's has."
Davis, P. Citation advantage of Open Access articles likely explained by quality differential and media effects, PLoS Biology, Responses, 16 January 2007: "The fact that OA articles were more likely to be featured on the front cover of PNAS and covered by the media suggests that other causal explanations may explain the OA advantage. Open Access may be a result – not a cause of – a quality differential which is amplified by the media. While Eysenbach's attempt to control other explanatory variables was excellent, what is needed are true randomized controlled studies of OA publishing."
Harnad, S. blog, Citation Advantage For OA Self-Archiving Is Independent of Journal Impact Factor, Article Age, and Number of Co-Authors, 17 January 2007: "Chawki Hajjem has now done a multiple regression analysis jointly testing (1) article age, (2) journal impact factor, (3) number of authors, and (4) OA self-archiving as separate factors for 442,750 articles in 576 (biomedical) journals across 11 years, and has shown that each of the four factors contributes an independent, statistically significant increment to the citation counts. The OA-self-archiving advantage remains a robust, independent factor.
Added 15 March 2006 Davis, P. M. and Fromerth, M. J. (2006)
Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? (pdf 12pp)
draft manuscript, ArXiv.org, cs.DL/0603056, 14 March 2006, Scientometics, Vol. 71, No. 2. (May 2007)
Comment on this paper:
Authors Self-Selection bias "While our study confirms the same citation advantage reported by others, it does not attribute Open Access as the cause of more citations, but to Self-Selection. Open Access therefore may be a result, not a cause, of authors promoting higher-quality work."
Liblicense, 14 March 2006
Harnad, S. OAA a causal factor "I think your results are very interesting, but I don't think they
have shown that the OA citation advantage (OAA) is all or mostly a
self-selection Quality Bias (QB) correlate, rather than being causal.
It is still quite plausible that the OAA is a genuine causal factor,
but that it has a bigger effect on the high quality/citation end."
SIGMETRICS, 14 March 2006
Qualifying statements: Authors "I am however, troubled by individuals who make universal and unqualified statements like, "Open Access increases citations by 50-250%!"" Harnad, S. "I am such an individual, and I hold by that statement" Authors "The more
precise answer is much more subtle, but I understand that a statement like, "open access may provide some citation benefit, but only for prestigious authors who publish in prestigious journals and whose article is already highly-cited", doesn't sound as convincing to administrators and policy makers." Harnad, S. "I don't think it is a correct statement of the thrust of the current body of findings on the OAA. It is merely your interpretation of the result of your own study in 4 maths journals!"
SIGMETRICS, 15 March 2006
Causation or association?: Antelman, K. "Data I collected for philosophy, political science, engineering
and mathematics do not support this hypothesis that OA causes
more citations for better articles only ... at that time I
had not looked at the distribution of OA and non-OA articles by
citations. Graphs of those results are posted at
http://www.lib.ncsu.edu/staff/kantelman/OA_by_citations.xls. These data show OA citation advantage across all articles with more than zero citations. Authors "The data that Kristin illustrates do not show causation, only association." Harnad, S. The data Phil illustrates likewise do not show causation, only
association. ... More fine-tuned causal tests are needed to decide.
Liblicense, from 20 March 2006
Added 16 March 2006 Harnad, S. (2006)
OA Impact Advantage = EA + (AA) + (QB) + QA + (CA) + UA
Author eprint, 14 March 2006, ECS EPrints repository, School of Electronics and Computer Science, University of Southampton
Added 03 August 2006 Mueller, P. S., Murali, N. S., Cha, S. S., Erwin, P. J. and Ghosh, A. K. (2006)
The effect of online status on the impact factors of general internal medicine journals
Netherlands Journal of Medicine, 64 (2): 39-44, February 2006
"becoming available online as FUTON (full text on the Net) is associated
with a significant increase in journal impact factor."
Added 30 December 2005 Hajjem, C., Harnad, S. and Gingras, Y. (2005)
Ten-Year Cross-Disciplinary Comparison of the Growth of Open Access and How it Increases Research Citation Impact (pdf 8pp)
IEEE Data Engineering Bulletin, Vol. 28 No. 4, December 2005
also Author eprint, 16 December 2005 http://eprints.ecs.soton.ac.uk/11688/
"In 2001, Lawrence found that articles in computer science that were openly accessible (OA) on the Web
were cited substantially more than those that were not. We have since replicated this effect in physics.
To further test its cross-disciplinary generality, we used 1,307,038 articles published across 12 years
(1992-2003) in 10 disciplines (Biology, Psychology, Sociology, Health, Political Science, Economics,
Education, Law, Business, Management). The overall percentage of OA (relative to total OA + NOA) articles
varies from 5%-16% (depending on discipline, year and country) and is slowly climbing annually. Comparing OA and NOA articles in the same journal/year, OA articles have consistently more citations, the advantage varying from 25%-250%
by discipline and year."
Added 30 December 2005 Hajjem, C., Gingras, Y., Brody, T., Carr, L. and Harnad, S. (2005)
Open Access to Research Increases Citation Impact (.doc 12pp)
Author eprint, 16 December 2005, Technical Report, Institut des sciences cognitives, Université du Québec à Montréal
Added 30 December 2005 Sahu, D.K., Gogtay, N.J. and Bavdekar, S.B. (2005)
Effect of open access on citation rates for a small biomedical journal
Author eprint, December 1, 2005, in Fifth International Congress on Peer Review and Biomedical Publication, Chicago, September 16-18, 2005
"We assessed the influence of OA on citations rates for a small, multi-disciplinary journal which adopted OA without article submission or article access fee. DESIGN The full text of articles published since 1990 were made available online in 2001. Citations for these articles as retrieved using Web of Science, SCOPUS, and Google Scholar were divided into two groups - the pre-OA period (1990-2000) and the post-OA period (2001-2004). CONCLUSIONS Open access was associated with increase in the number of citations received by the articles. It also decreased the lag time between publication and the first citation. For smaller biomedical journals, OA could be one of the means for improving visibility and thus citation rates."
Added 27 September 2005 Zhao, D. (2005)
Challenges of scholarly publications on the Web to the evaluation of science -- A comparison of author visibility on the Web and in print journals (abstract only)
Information Processing and Management, 41:6, 1403-1418, December 2005
Compares author visibility between the Web and print journals as revealed from citation analysis based on a search for the term "XML" or "eXtensible Markup Language" using NEC Research Institute’s CiteSeer, the entire ISI Science Citation Index (SCI) database, and journals indexed and classified in SCI as representing computer science research. The main finding: "The author ranking by number of citations that resulted from CiteSeer data is highly correlated with that obtained from SCI." i.e. it's not comparing OA impact vs non-OA but Web vs journal, and finds that authors, notably the top authors, are self-archiving and publishing papers in both places.
Added 11 February 2008 Coats, A. J. S. (2005)
Top of the charts: download versus citations in the International Journal of Cardiology (full-text requires subscription; otherwise abstract only)
International Journal of Cardiology, Volume 105, Issue 2, 2 November 2005, 123-125, available online 7 October 2005
From the abstract: "We have recorded the 10 top cited articles over a 12-month period and compared them to the 10 most popular articles being downloaded over the same time period. The citation-based listing included basic and applied, observational and interventional original research reports. For downloaded articles, which have shown a dramatic increase for the International Journal of Cardiology from 48,000 in 2002 to 120,000 in 2003 to 200,000 in 2004, the most popular articles over the same period are very different and are dominated by up-to-date reviews of either cutting-edge topics (such as the potential of stem cells) or of the management of rare or unusual conditions. There is no overlap between the two lists despite covering exactly the same 12-month period and using measures of peer esteem. Perhaps the time has come to look at the usage of articles rather than, or in addition to, their referencing."
Added 13 July 2005 Adams, J. (2005)
Early citation counts correlate with accumulated impact (abstract only)
Scientometrics, 63 (3): 567-581, June 2005
Working towards earlier prediction of impact. This paper is not OA and has just appeared but was written before Brody and Harnad (2005) revealed a correlation to predict impact from even earlier data, i.e. download data for OA papers, before any citations.
Added 26 September 2005 Moed, H. F. (2005)
Statistical Relationships Between Downloads and Citations at the Level of Individual Documents Within a Single Journal (abstract only)
Journal of the American Society for Information Science and Technology, 56(10): 1088-1097, published online 31 May 2005
"Statistical relationships between downloads from ScienceDirect of documents in Elsevier's electronic journal Tetrahedron Letters and citations to these documents recorded in journals processed by the (ISI) for the Science Citation Index (SCI) are examined. ... Findings suggest that initial downloads and citations relate to distinct phases in the process of collecting and processing relevant scientific information that eventually leads to the publication of a journal article." Does not investigate open access sources. Notes the need for caution in drawing conclusions on the frequency of paper downloads from formal citation patterns, and vice versa.
Updated 05 October 2005 Vaughan, L. and Shaw, D. (2005)
Web citation data for impact assessment: A comparison of four science disciplines (abstract only)
Journal of the American Society for Information Science and Technology, Vol. 56, No. 10, 1075 - 1087, published online 27 May 2005
appears to be an expansion of Can Web Citations be a Measure of Impact? An Investigation of Journals in the Life Sciences (abstract only)
ASIST 2004: Proceedings of the 67th ASIS&T Annual Meeting, Vol. 41 (Medford, USA: Information Today), pp. 516-526
Brody, T., Harnad, S. and Carr, L. (2005)
Earlier Web Usage Statistics as Predictors of Later Citation Impact
Author eprint, 18 May 2005, University of Southampton, School of Electronics and Computer Science, Journal of the American Association for Information Science and Technology, Volume 57, Issue 8, 2006, 1060-1072 (abstract)
Added 19 May 2005 Wren, J. D. (2005)
Open access and openly accessible: a study of scientific publications shared via the internet
BMJ, 330:1128, 12 April 2005
BMJ Rapid Responses to this article:
Carr, L., 15 April 2005 Open Access Misdefined "by misdefining Open Access this paper somehow ignores both the phenomenon that is being measured (Open Access-ibility) and the significant research that has risen around it."
Chan, C-h. and Ng, D. K., 13 May 2005 Technological problems in this study: PDF "The selection of only PDF reprint in this study made this correlation (between the time since initial publication and the probability of availability of reprint in non journal website) a little bit artificial."
All Rapid Responses
Added 13 April 2005 Belew, R. (2005)
Scientific impact quantity and quality: Analysis of two sources of bibliographic data (pdf 12pp)
Arxiv.org, cs.IR/0504036, 11 April 2005
Added 03 August 2006 Metcalfe, T. S. (2005)
The Rise and Citation Impact of astro-ph in Major Journals
ArXiv, Astrophysics, astro-ph/0503519, 23 March 2005
"I describe a simple method to determine the adoption rate and citation impact of astro-ph over time for any journal using NASA's Astrophysics Data System (ADS). I use the ADS to document the rise in the adoption of astro-ph for three major astronomy journals, and to conduct a broad survey of the citation impact of astro-ph in 13 different journals. I find that the factor of two boost in citations for astro-ph papers is a common feature across most of the major astronomy journals."
Updated 13 April 2005 Ongoing studies Hajjem, C. (2004-05)
Cover page for the range of studies highlighted below, Laboratoire de recherche en Sciences Cognitives, UQAM. (Text in French but graphs "self-explanatory"; see this comment for elaboration)
Updated 26 September 2005 Bollen, J., Van de Sompel, H., Smith, J. and Luce, R. (2005)
Toward alternative metrics of journal impact: A comparison of download and citation data (pdf 34pp)
Arxiv.org, cs.DL/0503007, 03 March 2005, in Information Processing and Management, 41(6): 1419-1440, December 2005
Added 5 January 2005 Ongoing study Brody, T., et al.
Citation Impact of Open Access Articles vs. Articles Available Only Through Subscription ("Toll-Access")
with downloadable graphs of '% Articles OA' and '% OA Advantage' by discipline and sub-discipline
Updated 31 January 2005 Schwarz, G. and Kennicutt Jr., R. C. (2004)
Demographic and Citation Trends in Astrophysical Journal Papers and Preprints (pdf 14pp)
Arxiv.org, astro-ph/0411275, 10 November 2004, Bulletin of the American Astronomical Society, Vol. 36, 1654-1663
See also a note from AAS Pub Board meeting, Tucson, November 3-4 2003
"Greg Schwarz (from the ApJ editorial office) reported some work he's
doing tracking citation rates of papers published in the ApJ based on whether
they were posted on astro-ph or not: ApJ papers that were also on astro-ph
have a citation rate that is _twice_ that of papers not on the preprint
server"
http://listserv.nd.edu/cgi-bin/wa?A2=ind0311&L=pamnet&D=1&O=D&P=1632
Added 03 August 2006 Havemann, F. (2004)
Eprints in der wissenschaftlichen kommunikation (Eprints in scientific communication)
Author eprint, 26 October 2004, presented at the Institute of Library Science, Humboldt University, Berlin, June 1, 2004
"the use of eprints can significantly accelerate the scientific communication.
This was demonstrated by me with a small sample of articles in theoretical
High Energy Physics published 1998 and 1999 in Physical Review D. Typically
the eprints in this sample are available eight months before the printed
issue is published. Three quarters of them are cited in eprints authored by
other researchers before the journal issue appears (among them all highly
cited eprints)."
Brody, T. (2004)
Citation
Analysis in the Open Access World
Author eprint, October 4, 2004, in Interactive Media International
Added 9 November 2004 McVeigh, M. E. (2004)
Open Access Journals in the ISI Citation Databases: Analysis of Impact Factors and Citation Patterns
Thomson Scientific, October 2004
Added 29 September 2004 Antelman, K. (2004)
Do Open-Access Articles Have a Greater Research Impact?
College and Research Libraries, 65(5):372-382, September 2004
also Author eprint, E-LIS, 29 September 2004, http://eprints.rclis.org/archive/00002309/
Updated 5 January 2005 Harnad, S., Brody, T., Vallieres, F., Carr, L., Hitchcock, S., Gingras, Y., Oppenheim, C., Stamerjohanns, H. and Hilf, E. (2004)
The Access/Impact Problem and the Green and Gold Roads to Open Access
Author eprint, 15 September 2004, in Serials Review, Vol. 30, No. 4, 310-314 (free access to published version during 2005)
Shorter version: The green and the gold roads to Open Access
Nature, Web Focus: access to the literature, May 17, 2004
Updated 26 September 2005 Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner, M., Murray, S. S. (2004b)
The Effect of Use and Access on Citations
Author eprint, September 2004, in Information Processing and Management, 41 (6): 1395-1402, December 2005
Perneger, T. V. (2004)
Relation
between online "hit counts" and subsequent citations: prospective study
of research papers in the BMJ
BMJ, 329:546-547, 4 September 2004
Prakasan, E. R. and Kalyane, V. L. (2004)
Citation analysis
of LANL High-Energy Physics E-Prints through Science Citation Index (1991-2002)
Author eprint, E-LIS, 26 August 2004
Added 13 April 2005 Murali, N. S., Murali, H. R., Auethavekiat, P., Erwin, P. J., Mandrekar, J. N., Manek, N. J. and Ghosh, A. K. (2004)
Impact of FUTON and NAA Bias on Visibility of Research
Mayo Clinic Proceedings, Vol. 79, No. 8, 1001-1006, August 2004
Notes and comment: FUTON = full text on the Net; NAA = no abstract available
This is not an article on how Open Access
increases impact but on how *Online* Access increases impact. The
effects are related, but one is a licensing effect, not
an OA effect.
Added 10 May 2007 Davis, P. M. (2004)
For Electronic Journals, Total Downloads Can Predict Number of Users
portal: Libraries and the Academy, Vol. 4, No. 3, July 2004, 379–392
Harnad, S. and Brody, T. (2004a)
Comparing
the Impact of Open Access (OA) vs. Non-OA Articles in the Same Journals
D-Lib Magazine, Vol. 10 No. 6, June 2004
Replicates the Lawrence effect -- OA increases impact -- in physics.
Pringle, J. (2004)
Do
Open Access Journals have Impact?
Nature, Web Focus: access to the literature, May 7, 2004
Testa, J. and McVeigh, M. E. (2004)
The
Impact of Open Access Journals: A Citation Study from Thomson ISI (pdf 17pp)
Author eprint, 14 April 2004
Kurtz, M. J. (2004)
Restrictive access
policies cut readership of electronic research journal articles by a factor
of two (pdf 2pp)
Harvard-Smithsonian Centre for Astrophysics, Cambridge, MA
Poster presentation at National Policies on Open Access (OA) Provision
for University Research Output: an International meeting, Southampton,
19 February 2004
Brody, T., Stamerjohanns, H., Harnad, S., Gingras, Y. and Oppenheim,
C. (2004)
The Effect
of Open Access on Citation Impact (pdf 1pp)
Poster presentation at National Policies on Open Access (OA) Provision
for University Research Output: an International meeting, Southampton,
19 February 2004
Updated 5 January 2005 Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner,
M. and Murray, S. S. (2004a)
Worldwide Use and Impact of the Nasa Astrophysics Data System Digital Library
Author eprint, January 28, 2004, in Journal of the American Society for Information Science and Technology, Vol. 56, No. 1, 36-45, published online 20 September 2004
Hitchcock, S., Brody, T., Gutteridge, C., Carr, L. and Harnad, S. (2003b)
The
Impact of OAI-based Search on Access to Research Journal Papers
Author eprint, 15 September 2003, in Serials, Vol. 16,
No. 3, November 2003, 255-260
Hitchcock, S., Woukeu, A., Brody, T., Carr, L., Hall, W. and Harnad,
S. (2003a)
Evaluating
Citebase, an open access Web-based citation-ranked search and impact discovery
service
Technical Report ECSTR-IAM03-005, School of Electronics and Computer Science, University of Southampton, July 2003
Added 28 October 2004 Bollen, J., Vemulapalli, S. S., Xu, W. and Luce, R. (2003)
Usage Analysis for the Identification of Research Trends in Digital Libraries
D-Lib Magazine, Vol. 9, No. 5, May 2003
Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner,
M., Murray, S. S., Martimbeau, N. and Elwell, B. (2003b)
The NASA Astrophysics Data System: Sociology, Bibliometrics, and Impact
Author eprint, March 2003, Journal of the American Society for Information Science and Technology,
submitted for publication
Updated 23 February 2005 Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Demleitner, M., Murray, S. S., Martimbeau, N. and Elwell, B. (2003a)
The Bibliometric Properties of Article Readership Information
Author eprint, March 2003, in Journal of the American Society for Information Science and Technology, 56 (2): 111-128, January 15, 2005
Added 17 December 2007 Drenth, J. P. H. (2003)
More reprint requests, more citations? (subscriber access to full text)
Scientometrics, Vol. 56, No. 2, February 2003, 283-286, revised version published online August 2006
From the abstract: "This study aims to correlate the number of reprint requests from a 10-year-sample of articles with the number of citations. ... Articles that received most reprint requests are cited more often."
Darmoni, S. J., et al. (2002)
Reading
factor: a new bibliometric criterion for managing digital libraries
Journal of the Medical Library Association, Vol. 90, No. 3,
July 2002
Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C. S., Thompson, D.
M., Bohlen, E. H. and Murray, S. S. (2002)
The
NASA Astrophysics Data System: Obsolescence of Reads and Cites (pdf 8pp)
Library and Information Services in Astronomy IV, edited by
B. Corbin, E. Bryson, and M. Wolf, July 2002
Bollen, J. and Luce, R. (2002)
Evaluation
of Digital Library Impact and User Communities by Analysis of Usage Patterns
D-Lib Magazine, Vol. 8, No. 6, June 2002
Lawrence, S. (2001)
Free
online availability substantially increases a paper's impact
Nature, 31 May 2001
Added 04 October 2005 Anderson, K., Sack, J., Krauss, L. and O'Keefe, L. (2001)
Publishing Online-Only Peer-Reviewed Biomedical Literature: Three Years of Citation, Author Perception, and Usage Experience
Journal of Electronic Publishing, Vol. 6, No. 3, March 2001
One of the first studies of the citation effect of online against offline publication, rather than of open access against non-OA. Provides data for one journal and a small number of articles over a three year period year. This paper was added to the bibliography following this correspondence:
Banks, P. "does not find the same citation advantage for online publications claimed by Harnad and his colleagues." Harnad, S. "Please weigh findings against the preponderance of evidence."
American Scientist Open Access Forum, Open access to research worth £ 1.5bn a year, 28 September 2005
Updated 04 October 2005 Odlyzko, A. M. (2000)
The rapid evolution of scholarly communication
PEAK 2000: Economics and Usage of Digital Library Collections conference, Ann Arbor, MI, March 2000.
Also in Learned Publishing, 15(1), 7-19, January 2002 here.
Author eprint http://www.dtc.umn.edu/~odlyzko/doc/rapid.evolution.pdf
Notes the growing usage of information in electronic form (c.f. print forms) and of journal papers from non-journal sites (e.g. eprints), and presents evidence that usage increases when access is more convenient
Youngen, G. K. (1998)
Citation
Patterns to Electronic Preprints in the Astronomy and Astrophysics Literature
Library and Information Services in Astronomy III, ASP Conference Series,
Vol. 153, 1998
see also
Citation Patterns to Traditional and Electronic Preprints in the Published Literature
College & Research Libraries, September 1998
Youngen, G. (1998)
Citation
Patterns Of The Physics Preprint Literature With Special Emphasis On The
Preprints Available Electronically
Author eprint, UIUC Physics and Astronomy library, c. 5 November 1998, presented at ACRL/STS on 6/29/97
Web tools for measuring impact
Citebase Search "Search and citation analysis tool for the free,
online research literature" http://citebase.eprints.org/ User service, free
see
Brody, T. (2003) Citebase
Search: Autonomous Citation Database for e-print Archives, sinn03 conference on Worldwide Coherent Workforce, Satisfied Users - New Services For Scientific Information, Oldenburg, Germany, September
2003
Hitchcock, S., et al. (2003a) Evaluating Citebase, an open access Web-based citation-ranked search and impact discovery service
Correlation Generator http://citebase.eprints.org/analysis/correlation.php
Generates a graph (or table) of the correlation between citation impact
and usage impact from the Citebase database
see Brody, T. and Harnad, S. 2005 (in prep.)
Citeseer "Scientific literature digital library" http://citeseer.ist.psu.edu/ User service, free
Updated 26 May 2008 Now available as CiteSeerx, or "Next Generation CiteSeer" http://citeseerx.ist.psu.edu
see
Jacsó, P., (2005) CiteSeer, Thomson Gale, November 2005
Lawrence, S., Giles, C. L., Bollacker, K. (1999), Digital Libraries and Autonomous Citation Indexing, IEEE Computer, Vol. 32, No. 6, 67-71, 1999
Elsevier Scopus Bibliographic database covering 13,450 peer-reviewed
titles http://www.scopus.com/ User service
see
Added 15 March 2006 Burnham, J. F. (2006) Scopus database: a review, Biomedical Digital Libraries, 3:1, 8 March 2006
Added 15 March 2006 Dess, H. M. (2006) Scopus, Issues in Science and Technology Librarianship, Winter 2006
Added 15 May 2006 Quint, B. (2006) Elsevier’s Scopus Introduces Citation Tracker: Challenge to Thomson ISI’s Web of Science?, Newsbreaks, January 23, 2006
Added 13 September 2007 Goodman, D. and Deis, L. (2006) Update on Scopus, The Charleston Advisor, Vol. 7, No. 3, January 2006, 42-43
Jacsó, P. (2004) Scopus, Thomson Gale, September 2004
see also Comparative reviews
Google Scholar Find articles from academic publishers, preprint repositories and universities, as well as scholarly articles across the web (presents citations as separate results) http://scholar.google.com/ User service, free
see
Added 26 May 2008 Meier, J. J. and Conkling, T. W. (2008) Google Scholar’s Coverage of the Engineering Literature: An Empirical Study, Journal of Academic Librarianship, Vol. 34, No. 3, May 2008, 196-201 (full text requires subscription; abstract only)
Added 13 September 2007 Harzing, A.-W. (2007) Reflections on Google Scholar, Harzing.com, fifth version, 6 September 2007
about the citation analysis software Publish or Perish and its relation with Google Scholar
Added 13 September 2007 Quint, B. (2007) Changes at Google Scholar: A Conversation With Anurag Acharya, NewsBreaks, August 27, 2007
rare public interview with the low-profile 'designer and missionary' behind Google Scholar
Added 22 August 2007 Mayr, P. and Walter, A.-K. (2007) An exploratory study of Google Scholar, arXiv.org > cs > arXiv:0707.3575v1 [cs.DL], July 24, 2007, in Online Information Review, Vol. 31, No. 6 (2007), to appear. Author preprint also available from http://www.ib.hu-berlin.de/~mayr/arbeiten/OIR-Mayr-Walter-2007.pdf
Added 10 May 2007 Robinson, M. L. and Wusteman, J. (2007) Putting Google Scholar to the test: a preliminary study, author eprint, also in Program: Electronic Library and Information Systems, Vol. 41, Issue 1, February 2007, 71-80
Added 15 May 2006 Sadeh, T. (2006) Google Scholar Versus Metasearch Systems, HEP Libraries Webzine, issue 12, March 2006
"thoughtful and informative ... altogether the best overview of Google Scholar, other large federated search systems such as Scirus, and library-based metasearch tools I've seen." Reviewed by Tennant, R., Current Cites, January 2006 issue
Added 15 March 2006 Burright, M. (2006) Google Scholar -- Science & Technology, Issues in Science and Technology Librarianship, Winter 2006
Added 28 February 2006 Noruzi, A. (2005) Google Scholar: the new generation of citation indexes (pdf 11pp), E-LIS, 11 February 2006, in LIBRI 55(4): 170-180
Jacsó, P., (2005) Google Scholar and The Scientist,
commenting on his interview in Perkel, J., The Future of Citation Analysis (abstract only), The Scientist, October 24, 2005
Jacsó, P. (2005) Google Scholar (Redux), Thomson Gale, June 2005
Myhill, M. (2005) Google Scholar, Charleston Advisor, Vol. 6, No. 4, April 2005
Added 22 August 2007 Giustini, D. and Barsky, E. (2005) A look at Google Scholar, PubMed, and Scirus: comparisons and recommendations, Journal of the Canadian Health Libraries Association/Journal de l'Association des bibliothèques de la santé du Canada (JCHLA / JABSC) 26: 85-89 (2005) (pdf 5pp)
Jacsó, P. (2004) Google Scholar Beta, Thomson Gale, December 2004
see also Comparative reviews
ISI Web of Science Cited reference searching of 8,700 high impact
research journals http://www.isinet.com/products/citation/wos/ User service
see
Jacsó, P. (2004) Web of Science Citation Indexes, Thomson Gale, August 2004
see also Comparative reviews
Added 11 December 2006 Rexa.Info Covers the computer science research literature. Rexa is "a sibling to CiteSeer, Google Scholar, Academic.live.com, the ACM Portal. It's
chief enhancement is that Rexa knows about more first-class, de-duplicated, cross-referenced object types: not only papers and their citation links, but also people, grants, topics" http://rexa.info/ User service, free (login required)
Added 13 April 2006 Windows Live Search Academic Beta version. Indexes content related to computer science, physics, electrical engineering, and related subject areas, with more than 6 million records from approximately 4300 journals, 2000 conferences and ArXiv.org. In collaboration with Citeseer http://academic.live.com/ User service, free
see
Added 26 May 2008 Nadella, S. (2006) Book search winding down, Live Search, The official blog of the Live Search team at Microsoft, May 23, 2008
"Live Search Books and Live Search Academic projects ... will be taken down next week. Books and scholarly publications will continue to be integrated into our Search results, but not through separate indexes."
Added 15 May 2006 Quint, B. (2006) Windows Live Academic Search: The Details, Newsbreaks, April 17, 2006
Added 13 April 2006 Sherman, C. (2006) Microsoft Launches Windows Live Academic Search, SearchEngineWatch.com, April 12, 2006
Added 15 May 2006 Citations in Economics not intended for direct user access; instead is made available to RePEc services such as Socionet, EconPapers and IDEAS. Uses Citeseer software http://citec.repec.org/ Data service, free
Rank working papers series and journals in Economics http://citec.repec.org/s/
see
Barrueco Cruz, J. M. and Krichel, T. (2004) Building an autonomous citation index for grey literature: the economics working papers case (pdf 12pp), E-LIS, 01 February 2005, also in Proceedings GL6: Sixth International Conference on Grey Literature, New York, December 2004
CrossRef Forward linking service tool allows CrossRef member publishers to display cited-by links in their primary content, Data service
CrossRef and Atypon announce forward linking service (press release) June 8, 2004
Institute of Physics becomes first journals publisher to implement 'cited-by' links using CrossRef's Forward Linking service: Time travel with IOP journals (IOP press release) 14 March, 2005
Forthcoming ISI Web Citation Index User service
see
Added 15 May 2006 Martello, A. (2006) Selection of Content for the Web Citation Index: Institutional Repositories and Subject-Specific Archives, Thomson.com, undated
Pringle, J. (2005) Partnering helps institutional repositories thrive, KnowledgeLink Newsletter, February 2005
Citeseer's replacement? List server mailing, 18 March 2004
Quint, B. (2004) Thomson ISI to Track Web-Based Scholarship with NEC’s CiteSeer, Information Today Newsbreaks, March 1, 2004
Comparative reviews
Added 26 May 2008 Meho, L. I. and Rogers, Y. (2008) Citation Counting, Citation Ranking, and h-Index of Human-Computer Interaction Researchers: A Comparison between Scopus and Web of Science, E-LIS, 10 March 2008, in Journal of the American Society for Information Science and Technology, to appear
Added 26 May 2008 Kloda, L. A. (2007) Use Google Scholar, Scopus and Web of Science for Comprehensive Citation Tracking, Evidence Based Library and Information Practice, 2(3): 87-90, 2007, also in E-LIS, 21 September 2007 http://eprints.rclis.org/archive/00011437/
Added 26 May 2008 Schroeder, R. (2007) Pointing Users Toward Citation Searching: Using Google Scholar and Web of Science, portal: Libraries and the Academy, Vol. 7, No. 2, April 2007, 243-248 (full text requires subscription)
Added 13 September 2007 Goodman, D. and Deis, L. (2007) Update on Scopus and Web of Science, The Charleston Advisor, Vol. 8, No. 3, January 2007, 15-18
Added 10 May 2007 Meho, L. I. and Yang, K. (2006) A New Era in Citation and Bibliometric Analyses: Web of Science, Scopus, and Google Scholar, arXiv.org, Computer Science, cs/0612132, 23 Dec 2006, published as Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar, in Journal of the American Society for Information Science and Technology, Vol. 58, No. 13, 2007, 2105-2125
Added 11 December 2006 Fingerman, S. (2006) Web of Science and Scopus: Current Features and Capabilities, Issues in Science and Technology Librarianship, Fall, 2006
Added 17 January 2007 Neuhaus, C. and Daniel, H.-D. (2006) Data sources for performing citation analysis: An overview, ETH E-Collection, June 30, 2006, Journal of Documentation, accepted for publication
Reports the limitations of Thomson Scientific’s citation indexes and reviews the characteristics of the citation-enhanced databases Chemical Abstracts, Google Scholar and Scopus.
Comment on this paper:
Pikas, C. K. "I think this article will be very helpful, but the extension that seems
necessary right now is to CrossRef data. Many publishers such as the
Optical Society of America via Optics Infobase provide forward and
backward citations using CrossRef. When trying to *approach*
comprehensiveness, I felt I had to look there as well as Scopus, Web of
Science, Google Scholar, and CA. In fact, I found many citations there
that were unique -- but this is not scientific, merely anecdotal." Sigmetrics listserv, 14 Dec 2006
Bakkalbasi, N., Bauer, K., Glover, J. and Wang, L. (2006) Three options for citation tracking: Google Scholar, Scopus and Web of Science, Biomedical Digital Libraries, June 29, 2006
Added 8 March 2007 Bosman, J., van Mourik, I., Rasch, M., Sieverts, E. and Verhoeff, H. (2006) Scopus reviewed and compared, Igitur repository, Utrecht University, June 2006
The coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar
Wenzel, E. (2006) Google Scholar beta, ZDNet, May 2, 2006
Brief comparison of Google Scholar and Microsoft Live Academic Search
Bailey, C. W. Jr (2006) A Simple Search Hit Comparison for Google Scholar, OAIster, and Windows Live Academic Search, Digital Koans, author blog, April 13, 2006
A simple but revealing experiment: "It should be clear that a sample of one search term is a very crude measure".
Pauly, D. and Stergiou, K. I. (2005) Equivalence of results from two citation analyses: Thomson ISI’s Citation Index and Google’s Scholar service (pdf 3pp), Ethics in Science and Environmental Politics, 22 December 2005, 33-35
Comment on this paper:
""outperform" means many things. OK, in this study the citation counts were close but the searchability of material on WOS is much stronger." ResourceShelf, February 25, 2006
Added 23 November 2006 Jacso, P. (2005) Comparison and analysis of the citedness scores in Web of Science and Google Scholar (pdf 10pp), Digital Libraries: Implementing Strategies and Sharing Experiences, Lecture Notes In Computer Science, 3815: 360-369, 2005, Proceedings of the 8th International Conference on Asian Digital Libraries, ICADL 2005, Bangkok, Thailand, December 12-15, 2005
Roth, D. L. (2005) The emergence of competitors to the Science Citation Index and the Web of Science (pdf 6pp), Current Science Online, Vol. 89, No. 9, 10 November 2005
Jacsó, P. (2005) As we may search – Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases (pdf 11pp), Current Science Online, Vol. 89, No. 9, 10 November 2005
Bauer, K. and Bakkalbasi, N. (2005)
An Examination of Citation Counts in a New Scholarly Communication Environment, D-Lib Magazine, 11(9), September 2005.
Compares citation counts provided by Web of Science, Scopus, and Google Scholar.
Comment on this paper:
Stegmann, J. Clearer picture? "the authors of this interesting paper should,
perhaps, take into account the ISI Proceedings database because the
conference papers indexed therein are included together with their
references. ... This would give a clearer picture of what one gets from free services like
Google Scholar and from products which have to be licensed." Sigmetrics listserv, 16 September 2005
Author response: Focused and cross-disciplinary "We decided to concentrate on three
multi-disciplinary databases only, as a way of focusing our work. We
wanted to lay the foundation for a study that will compare subject areas:
hence our decision to look at databases that cross many subject areas." Sigmetrics listserv, 16 September 2005
LaGuardia, C. (2005) Scopus vs. Web of Science, Library Journal, 130(1); 40, 42, January 15, 2005
Deis, L. F. and Goodman, D. (2005) Web of Science (2004 version) and Scopus, Charleston Advisor, Vol. 6, No. 3, January 2005
Background
The financial imperative: correlating research access, impact and assessment
There is another dimension to the open access advantage. If open access increases impact, then it will also increase research income and funding. It has been shown in the UK that there is a correlation between research assessment ratings and citation counts, and higher ratings means more money for the higher rated research groups. Of course, if all papers were made open access by their authors, the relative effect would disappear. First-mover advantage anyone?
"Research impact translates into money: employment, salary, tenure money, as well as research-funding money: (1) RAE rank correlates with substantial top-sliced funding, (2) it also correlates highly (0.91) with citation counts, and (3) self-archiving increases citation counts by 50-250+%. Do you really think that any researcher who is *aware* of those three correlations is being rational if he doesn't self-archive?" Stevan Harnad
Added 11 February 2008 Armbruster, C. (2008)
Access, Usage and Citation Metrics: What Function for Digital Libraries and Repositories in Research Evaluation?
Social Science Research Network, February 05, 2008
From the abstract: "This systematic appraisal of the future role of digital libraries and repositories
for metric research evaluation proceeds by investigating the practical
inadequacies of current metric evaluation before defining the scope for libraries
and repositories as new players. Service reviewed include: Leiden Ranking, Webometrics Ranking of World
Universities, COUNTER, MESUR, Harzing POP, CiteSeer, Citebase, RePEc
LogEc and CitEc, Scopus, Web of Science and Google Scholar."
Added 10 May 2007 Harnad, S. (2007)
Open Access Scientometrics and the UK Research Assessment Exercise
ArXiv, Computer Science, cs.IR/0703131, 26 March 2007. Preprint of invited keynote address to 11th Annual Meeting of the International Society for Scientometrics and Informetrics, Madrid, 25-27 June 2007
also in ECS EPrints, 29 March 2007 http://eprints.ecs.soton.ac.uk/13804/
Added 19 November 2006 Steele, C., Butler, L. and Kingsley, D. (2006)
The publishing imperative: the pervasive influence of publication metrics
ANU Institutional Repository, 30 October 2006, also in Learned Publishing, 19(4): 277-290, October 2006
Added 19 November 2006 Houghton J. and Sheehan, P. (2006)
The Economic Impact of Enhanced Access to Research Findings
Centre for Strategic Economic Studies. Victoria University. July 2006
See also
Houghton, J., Steele, C. and Sheehan, P. (2006)
Research Communication Costs In Australia: Emerging Opportunities And Benefits
Department of Education, Science and Training (DEST), Australia, September 2006
Comment on The Economic Impact of Enhanced Access to Research Findings:
Harnad, S. Maximising the Return on Research "These estimates agree substantially with prior estimates that have been made (e.g., for the UK, Canada and Australia)." American-Scientist-Open-Access-Forum, 9 August 2006
Added 13 September 2007 Shadbolt, N., Brody, T., Carr, L. and Harnad, S. (2006)
The Open Research Web: A Preview of the Optimal and the Inevitable
ECS EPrints, 02 May 2006, in Open Access: Key Strategic, Technical and Economic Aspects, Jacobs, N., Ed., chapter 21 (Oxford: Chandos Publishing)
Added 26 September 2005 Harnad, S. (2005)
Maximising the Return on UK's Public Investment in Research
Author eprint, September 14, 2005
Attempts to monetise 'lost' impact: "The online-age practice of self-archiving has been shown to increase citation impact by a dramatic 50-250%, but so far only 15% of researchers are doing it spontaneously. Citation impact is rewarded by universities (through promotions and salary increases) and by research-funders like RCUK (through grant funding and renewal) at a conservative estimate of £46 per citation. ... As a proportion of the RCUK’s yearly £3.5bn research expenditure (yielding 130,000 articles x 5.6 = 761,600 citations), our conservative estimate would be 50% x 85% x £3.5.bn = £1.5bn worth of loss in potential research impact (323,680 potential citations lost)."
See also Australia is not maximisng the return on its research investment (ETD2005, Sydney) for the same estimate applied to the potential lost return ($425M) there.
Comment on this paper:
Rowland, I. "If I am the one millionth author (or the 10,000th research group or the 100th nation) to publish open access, that comparative advantage must quickly decline, approaching zero as the last few laggards pile in". Author response: "Ian Rowland is exactly right that
the OA impact advantage (currently 50-250%) will shrink as we approach
100% OA. Right now we are at 15% OA, and the advantage is in part --
no one can say how large a part -- a *competitive* advantage of the minority
15% OA -- the head-start vanguard -- over the laggard 85% non-OA majority. ... The OA impact advantage consists of at least the following 6 factors ..."
American Scientist Open Access Forum, OA advantage = EA + (AA) + (QB) + QA + (CA) + UA, 15 September 2005
Waters, D. "Mr. Harnad may well be on to an important subject and line of argument in suggesting that citations are a kind of return on investment. However, close inspection of the concepts and logic of his argument suggests that he is quite a bit further from proving his case than he seems to have convinced himself that he is." Author response: "No proof here: Just conservative estimates."
American Scientist Open Access Forum, Open access to research worth 1.5bn a year, 27 September 2005
Day, M. (2004)
Institutional repositories and research assessment (pdf 29pp)
Author eprint (v. 0.1), 2 December 2004
Harnad, S. (2003)
Maximizing
university research impact through self-archiving
Jekyll.com, No. 7, December 2003
Harnad, S. (2003)
Enhance
UK research impact and assessment by making the RAE webmetric
Author eprint, in Times Higher Education Supplement, 6 June 2003, p. 16
Harnad, S., Carr, L., Brody, T. and Oppenheim, C. (2003)
Mandated online
RAE CVs linked to university eprint archives: Enhancing UK research impact
and assessment
Ariadne, issue 35, April 2003
Smith, A. and Eysenck, M. (2002)
The correlation between RAE ratings and citation counts in psychology (pdf 12pp)
Technical Report, Psychology, Royal Holloway College, University of London, June 2002
Holmes, A. and Oppenheim, C. (2001)
Use of citation analysis to predict the outcome of the 2001 Research Assessment Exercise for Unit of Assessment (UoA) 61: Library and Information Management
Information Research, Vol. 6, No. 2, January 2001
Harnad, S. (2001)
Research
Access, Impact and Assessment (longer version)
Author eprint, in Times Higher Education Supplement, 1487: p. 16., 2001
Garfield, E. (1988)
Can Researchers Bank on Citation Analysis? (pdf 10pp)
Current Comments, No. 44, October 31, 1988
attached (pp 3-10)
Diamond, Jr., A. M. (1986)
What is a Citation Worth?
J. Hum. Resour., 21:200-15, 1986
Garfield comments on studies that attempt to quantify the reward system of science in terms of monetary returns to author salaries from article publication and citations, reprinting one of those studies
Citation analysis, indexes and impact factors
Notes. Important work in this area builds on Eugene Garfield's pioneering
work in the 1950s. Although producing some remarkably successful tools
for measuring the impact of the scholarly literature, this area is not
without controversy. This short list presents a cross-section underlining
these issues, with a view to understanding how such long-established approaches
might adapt to online data, and how possible shortcomings might be overcome.
Added 11 February 2008 Kosmopoulos, C. et Pumain, D. (2007)
Citation, Citation, Citation: Bibliometrics, the web and the Social Sciences and Humanities
Cybergeo, Science et Toile, article 411, mis en ligne le 17 décembre 2007, modifié le 18 janvier 2008
From the abstract: "The paper reviews the main (bibliometric) data bases and indicators in use. It demonstrates that these instruments give a biased information about the scientific output of research in Social Sciences and Humanities."
Comment on this paper:
Krichel, T. (2007), bibliometrics and open access solutions, Budapest Open Access Initiative: BOAI Forum Archive, 25 December 2007: "I am somewhat saddened to read that this survey does not discuss
CitEc, which I think is
the largest open-access citation index in the social sciences."
Added 22 August 2007 Citrome, L. (2007)
Impact Factor? Shmimpact Factor! The Journal Impact Factor, Modern Day Literature Searching, and the Publication Process
Psychiatry, 4(5):54-57, 2007
Added 17 January 2007 Bornmann, L. and Daniel, H.-D. (2007)
What do citation counts measure? A review of studies on citing behavior
Author eprint, undated, Journal of Documentation, accepted for publication
Added 17 January 2007 Meho, L. I. (2006)
The Rise and Rise of Citation Analysis
Author eprint, dLIST, 31 December 2006, Physics World, January 2007
"Provides a historical background of citation analysis, impact factor, new citation data sources (e.g., Google Scholar, Scopus, NASA's Astrophysics Data System Abstract Service, MathSciNet, ScienceDirect, SciFinder Scholar, Scitation/SPIN, and SPIRES-HEP), as well as h-index, g-index, and a-index."
Added 19 November 2006 Electronic Publishing Services and Oppenheim, C. (2006)
UK scholarly journals: 2006 baseline report: An evidence-based analysis of data concerning scholarly journal publishing, see Area 4: Citations, impact factors and their role
Research Information Network, Research Councils UK and the Department of Trade & Industry, October 3, 2006
Added 3 May 2007 Ewing, J. (2006)
Measuring Journals
Notices of the AMS, Vol. 53, No. 9, October 2006, 1049-1053
"in many respects usage statistics are even
more flawed than the impact factor, and once again,
the essential problem is that there are no explicit
principles governing their interpretation. ... while usage statistics are only slightly useful, their
misuse can be enormously damaging."
Comment:
Velterop, J. RE: UKSG Usage Factor Research - an Update, liblicense, March 9. 2007: "Ewing further says
that "Distrust of 'subjective' scholarly judgment is a modern
disease -- one that is profoundly anti-intellectual." I would add
that blind trust in 'objective' measurements is equally
profoundly anti-intellectual."
Davis, P. RE: UKSG Usage Factor Research - an Update, liblicense, March 10. 2007: "Like citations, usage statistics do not give us an absolute notion of value of journals or articles, yet they do provide us with a measure of utility, and for the sciences, utility is a very powerful measure for how ideas get transmitted through communities and are incorporated into current research. Unlike citations, usage statistics give us a sense of the community of readers (which include authors) and not just the author community. Article downloads provide a robust estimate of the size of user communities, and are also predictive of future citations. In fact, a single week of article downloads from BMJ can predict citations five years later."
Added 8 March 2007 Garfield, E. (2006)
Commentary: Fifty years of citation indexing
International Journal of Epidemiology, 2006 35(5):1127-1128, published online September 19, 2006
Added 03 August 2006 PLoS Medicine Editors (2006)
The Impact Factor Game: It is time to find a better way to assess the scientific literature
PLoS Medicine, Vol. 3, No. 6, June 2006
Added 15 May 2006 Altbach, P. G. (2006)
The Tyranny of Citations
Inside Higher Ed, May 8, 2006
Added 28 February 2006 Noruzi, A. (2006)
The Web Impact Factor: a critical review (pdf, 10pp)
E-LIS, February 9, 2006, in The Electronic Library, 24 (2006)
"Web Impact Factor (WIF) is a quantitative tool for evaluating and ranking web sites ... search engines provide similar possibilities for the investigation of links between web sites/pages to those provided by the academic journals citation databases from the Institute of Scientific Information (ISI). But the content of the Web is not of the same nature and quality as the databases maintained by the ISI."
Added 28 February 2006 Bollen, J., Rodriguez, M. A. and Van de Sompel, H. (2006)
Journal Status (pdf, 16pp)
Arxiv, 9 January 2006
"By merely counting the amount of citations and disregarding the prestige of the citing journals, the ISI IF is a metric of popularity, not of prestige. We demonstrate how a weighted version of the popular PageRank algorithm can be used to obtain a metric that reflects prestige. ... Furthermore, we introduce the Y-factor which is a simple combination of both the ISI IF and the weighted PageRank, and find that the resulting journal rankings correspond well to a general understanding of journal status."
Added 03 August 2006 Moed, H.F. (2005)
Citation analysis of scientific journals and journal impact measures
Current Science, 89 (12): 1990-1996, December 25, 2005
Added 28 February 2006 Dong, P., Loh, M. and Mondry, A. (2005)
The "impact factor" revisited
Biomedical Digital Libraries, December 2005
This is a review, so the findings are not new, but this is perhaps the first such paper to reflect on the effect of free and online availability on journal impact factors, among other IF-related issues.
Added 30 December 2005 Hardy, R., Oppenheim, C., Brody, T. and Hitchcock, S. (2005)
Open Access Citation Information (.doc, 105pp)
Author eprint, November 11, 2005, JISC Committee for the Information Environment (JCIE) Scholarly Communication Group, September 2005
Describes a proposal to increase the exposure of open access materials and their references to indexing services, and to motivate new services by reducing setup costs.
Added 8 March 2007 Perkel, J. M. (2005)
The Future of Citation Analysis
The Scientist, Vol. 19, No. 20, October 24, 2005
"The challenge is to track a work's impact when published in nontraditional forms"
Added 30 December 2005 Monastersky, R. (2005)
Impact Factors Run Into Competition
Chronicle of Higher Education, October 14, 2005
Comment on this article:
Harnad, S. IFs: solution is obvious "Although Richard Monasterky describes a real problem -- the abuse of
journal impact factors -- its solution is so obvious -- (a) wealth of powerful new resources are on the way
for measuring and analyzing research usage and impact online" American Scientist Open Access Forum, 10 October 2005
Bensman, S. J. Good copy, bad science "I found his article to be
unfair, since he concentrated on the shenanigans that are being played with
impact factor and supposed errors of ISI in constructing impact factor.
This makes for good copy but bad science." Sigmetrics listserv, 18 November 2005
Leydesdorff, L. Discipline-specific impact factor "Monasterky's article lists a number of problems with the ISI-impact factor.
However, he fails to mention that the average impact factors vary among
fields of science. For example, impact factors in toxicology are
considerably lower than in immunology. ... A fix to these problems might be a discipline-specific impact factor. ... Using ISI's Journal Citation Reports, I created the raw materials to make maps of the
citation neighborhoods of all the journals." Sigmetrics listserv, 16 September 2005
Added 28 February 2005 Garfield, E. (2005)
The Agony and the Ecstasy - The History and Meaning of the Journal Impact Factor (pdf, 22pp)
International Congress on Peer Review and Biomedical Publication, Chicago, September 16, 2005
Garfield's typically dry, data-filled but essential take on JIFs.
Publishers promote impact factors of OA journals
BioMed Central "Open access journals get impressive impact factors" 23 June 2005
Public Library of Science "The first impact factor for PLoS Biology - 13.9" 27 June 2005
See also this discussion of these announcements on SPARC Open Access Forum, prompted by Elsevier's response from Tony McSean, followed by David Goodman, Charles Bailey, (both 8 July) and Matthew Cockerill (10 July), or see this summary of the discussion: "BMC’s Impact Factors: Elsevier’s Take and Reactions to It", Digital Koans (Charles Bailey's Weblog), 11 July 2005, including Peter Suber's conclusion: "It’s important to distinguish the citation impact of an individual article from a journal impact factor. The BMC-Elsevier debate is about the latter. But OA is more likely to rise and fall according to the former."
Abbasi, K. (2004)
Let's dump impact factors
BMJ, Vol. 329, 16 October 2004
BMJ Rapid Responses to this editorial; also see this list response
Baudoin, L., Haeffner-Cavaillon, N., Pinhas, N., Mouchet, S. and Kordon, C. (2004)
Bibliometric indicators: realities, myth and prospective (abstract only, full paper in French)
Med Sci (Paris), 20 (10):909-15, October 2004
Jacsó, P. (2004)
The
Future of Citation Indexing - Interview with Dr. Eugene Garfield (pdf 3pp)
Author eprint, in Online, January 2004
Cockerill, M. J. (2004)
Delayed impact:
ISI's citation tracking choices are keeping scientists in the dark
BMC Bioinformatics 2004, 5:93, 12 July 2004
Added 26 September 2005 Shin E. J. (2003)
Do Impact Factors change with a change of medium? A comparison of Impact Factors when publication is by paper and through parallel publishing (abstract only)
Journal of Information Science, 29 (6): 527-533, 2003
"it is found that Impact Factors of (journals from the period) 2000 and 2001 were significantly higher than those of 1994 and 1995 in the journals published by parallel publishing (combination journals–simultaneous publication of paper and electronic journals). In particular, the Impact Factors of the combination journals increased after the journals transformed their available media from paper journals to combination ones."
Walter, G., Bloch, S., Hunt, G. and Fisher, K. (2003)
Counting
on citations: a flawed way to measure quality
MJA, 2003, 178 (6): 280-281
Borgman, C. L. and Furner, J. (2002)
Scholarly Communication and Bibliometrics, author preprint (pdf 45pp)
Author eprint, in Annual Review of Information Science and Technology, Vol. 36, edited by B. Cronin, 2002
Guédon, J.-C. (2001)
In Oldenburg’s
Long Shadow: Librarians, Research Scientists, Publishers, and the Control
of Scientific Publishing
Creating the Digital Future, Proceedings
of the 138th Annual Meeting, Association of Research Libraries, Toronto, Ontario, May 23-25, 2001
Garfield, E. (1999)
Journal impact factor: a brief review
CMAJ, 161 (8), October 19, 1999
Wouters, P. (1999)
The Citation Culture (pdf 290pp)
PhD Thesis, University of Amsterdam, 1999
Garfield, E. (1998)
The
use of journal impact factors and citation analysis in the evaluation of
science
Author eprint, presented at the 41st Annual Meeting of the Council of Biology Editors,
Salt Lake City, UT, May 4, 1998
Seglen, P. O. (1997)
Why
the impact factor of journals should not be used for evaluating research
BMJ, 314:497, 15 February 1997
Garfield, E. (1973)
Citation
Frequency as a Measure of Research Activity and Performance (pdf 3pp)
Author eprint, in Essays of an Information Scientist, 1: 406-408, 1962-73, Current
Contents, 5, January 31, 1973
Garfield, E. (1955)
Citation
Indexes for Science: A New Dimension in Documentation through Association
of Ideas
Author eprint, in Science, Vol:122, No:3159, p.108-111, July 15, 1955
Open access
Notes. In printed form, little of the published research literature
was free. With more material beginning to appear on the Web from the mid-1990s,
more became freely available. Open access is in a sense a formalisation
of that process, a recognition that all published, refereed scholarly papers
could and should be freely accessible in some form to everyone online without
compromising the quality and integrity of the literature. That is the goal.
This simple idea, especially when focussed on this very specific literature,
seems to have been quite difficult to grasp for many bound by the old,
pre-online ways of thinking. Despite the often antithetical tone of the
debate, progress has been rapid since the landmark of the Budapest Open
Access Initiative in February 2002, even impinging on prospective government
policies by 2003 (e.g. Martin Sabo's Public Access to Science Act; UK House committee releases its report on open access; Major development in providing OA to taxpayer-funded research). It has all been brilliantly logged by Peter Suber in
Open Access News (http://www.earlham.edu/~peters/fos/fosblog.html),
but for a very quick overview the following papers are sufficient.
Suber, P. (updated)
Open Access
Overview
Swan, A. (2007)
Open Access and the Progress of Science
American Scientist, April-June 2007
Swan justified open access in support of her 'progress' article in a list discussion. See blogged extracts from that discussion.
Swan, A. (2006)
Open Access: Why should we have it?
presented at "Zichtbaar onderzoek. Kan Open Archives daarbij helpen?" / Visible research. Can OAI
help? Organised by AWI (Flemish Ministry for Economy, Enterprise, Science, Innovation and Foreign Trade) and VOWB
(Flemish Organisation of Scientific Research Libraries), May 2006
Swan, A. (2005)
Open Access
JISC, Briefing Paper, 1 April 2005
Suber, P. (2004)
A Primer on Open Access to Science and Scholarship
Author eprint, in Against the Grain, Vol. 16, No. 3, June 2004
Harnad, S. (2004)
The
Green Road to Open Access: A Leveraged Transition
American Scientist Forum, January 07, 2004
Suber, P. (2003)
Removing
the Barriers to Research: An Introduction to Open Access for Librarians
Author eprint, in College & Research Libraries News, 64, February, 92-94,
113
The OpCit Project
This page
produced and
maintained by the Open Citation project. Contact
us
