The Open Citation Project - Reference Linking for Open Archives
Evaluation | V1.0 demo evaluation

V 1.0 demo: evaluation by arXiv authors: results

Totals

No sent out 94; bounced 6; total out 88
No. returned 19
% return 21.6
Est. error margin 22.5%
Period of evaluation: 27 July to 13 August 2000
Date of this report: 22 August 2000

Summary of author results and main findings.
 

Evaluation

(Note. The selected target group of users included three subsets. The results from
these subsets are indicated primarily for further analysis. The sum of the results from
these subsets, shown in bold, is the main interest here.)

1 How useful did you find the reference linking in the demo as it is now? (select
ONE by placing an X in the brackets)
(1.1) [1+6+3 10] Very
(1.2) [4+1 5] Moderately
(1.3) [2+1 3] Hardly
(1.4) [ ] Not

2 How useful would you find it if a larger proportion of the reference links
reached their targets? (select ONE)
(2.1) [1+7+3 11] Very
(2.2) [3+1 4] Moderately
(2.3) [2+1 3] Hardly
(2.4) [ ] Not

3 What feature would you most like to see added to the service? (rank from 1
highest)
(3.1) [2 4+49+9 62 points] Integration with the underlying (physics) archives
(3.2) [4 2+27+7 36 pts] Links to databases of secondary content
(3.3) [1 5+38+12 55 pts] Links to online journals
(3.4) [3 3+27+5 35 pts] Links to other archives
(3.5) [5 1+17+1 19 pts] Links to services/holdings in your institutional library
Other (3.6) [
- Links to old but famous articles (there is KEK scanned archive, but it is not
sufficient)

4 What features would you like to see modified? (select ONE OR MORE by
placing an X in the brackets)
(4.1) [3+1 4] Link presentation
(4.2) [5 ] Link reliability
(4.3) [1+5+1 7] More links
(4.4) [4+2 6] User interface, e.g. better support for browsing
Other (4.5) [
- In the hypertex code I wrote I also made links from the paper body to the reference
list and equations, figs, etc. These seem to disappear in your linked examples. As a
general comment, most browsers don't allow to get back the hyperlink to the original
location - for example, if I click on a link to an equation and check that equation, in
most cases I like to go straight back to the place where I was, by clicking back. Most
browsers don't support this (at least as far as I know) and this severely limits the
functionality (the one I use is the Nextstep dvi previewer which is the only one I know
that works satisfyingly).
- make links at the text instead in the references
]

5 Which source do you use most to access the full texts of papers? (select ONE)
(5.1) [1+10+5 16] arXiv.org
(5.2) [1 ] Electronic journals
(5.3) [ ] Print journals
Other (5.4) [(please specify) ]

6 Which service do you use most to discover new papers? (select ONE)
(6.1) [1+10+4 15] arXiv.org
(6.2) [1+1 2] SLAC/SPIRES
(6.3) [1 ] Web of Science
Other (6.4) [(please specify) ]

7 How often do you discover new works through citations? (select ONE)
(7.1) [2+2 4] Always
(7.2) [6+3 9] Frequently
(7.3) [1+4 5] Occasionally
(7.4) [ ] Not at all

8 Comments/suggestions
(8.1) [
- An opinion expressed by many people is that the service providing the number of
citations could be improved. In particular, papers that are referred in the replaced
versions are not being counted.
- I cannot view any of the links, I'm asked to supply a ``supported web browser``,
which I apparently do not have.
- I saw the links but I couldn't get them to work. But maybe this is just because I don't
know how to do web linking properly in PDF.
- Plz never include bitmapped fonts in the pdf files, as they are not very readable.
- The idea to use colours to discriminate between a linked paper, a non-archive paper,
etc., at first sight seems nice (though I would prefer an ensuring green for a linked one
and an alarming red for one out of reach). However, I realize that many of my
colleagues have to use screens which offer only black and white - no colours. Wouldn't it therefore be better to attach some symbol or letter - like "L" for linked,
"O" for out of range, etc. - to each citation?
- Good luck.
- The only problem I had was configuring my acroreader to use the netscape
browser. I had to find the path to our browser...
- The Acrobat PDF reader on all our local machines at our institute is helplessly slow.
For this simple reason, all the wonderful services you suggest to use are simply totally
inconvenient for us; it's MUCH faster to find papers "by hand" on the net. It's a pity,
because the idea is good.
- I think it is a good time to separate astro-ph into two or more categories
]


^Top  
<Home  Evaluation | V1.0 demo evaluation                                                                                                          
The OpCit Project
This page produced and maintained by the Open Citation project. Contact us