Citation Validation
Last updated September 07 2000 11:00:57.
The output from Spotcites
Comparing "orange" and "red" links with the total number of citations. For the opcit project certain classes of
identifiable citations have been colour coded (for development purposes):
Colour  Definition 
Red  A author has specifically cited a paper in arXiv, e.g. "J. Smith
et al, preprint hepth/9909999"). 
Orange  A author has cited a paper using its publication information
(journal, issue, page) which has been deposited in the archive and found using the journal reference in
arXiv. 
Using the output from Spotcites red links can be identified by lines that contain a arXiv reference more than
once. A table can be built of paper * cited papers * orangered. This table can then be used to build the number
of orange/red links for a given month.
awk '{ print $1,$3 }' ZZZ.type  ../arxiv2date  awk '{ print $3,$2 }'  grep R$  ../countcol v n
awk '{ print $1,$3 }' ZZZ.type  ../arxiv2date  awk '{ print $3,$2 }'  grep O$  ../countcol v n
This graph shows how many citations have been identified, retrospectively, for papers deposited each month. The
total number of "lines of citations" is shown in blue (not the total of red and orange), i.e. the
background population from which citations are being identified from. The blue line corresponds to the righthand
yaxis and is scaled down.
Excel Spreadsheet Source
The numbers of red and orange citations found can be compared against the actual number of citations in the
following graph. The green line corresponds to the sum of red and orange links scaled for comparison to the total
number of citations.
The total number of citations can be factored out by dividing the red, orange and sum by the total citations for
the equivalent month. This shows how increasingly spotcites is more effective at identifying citations, and that
more citations are to papers in arXiv.
A comparison can be made between red and orange links, i.e. the changing behaviour of authors from citing
postprint articles to citing eprint articles. The following graph shows how the number of direct citations has
increased at a greater rate than the number of identifiable citations to published articles.
The graphs so far have covered the entire archive's deposits. It would be useful to identify how well citations
are being identified within certain subject areas.
hepth
awk '{ print $1 }' SCOOT.HEPTH  ../arxiv2date  awk '{ print $2 }'  ../countcol v n
awk '{ print $1,$3 }' ZZZ.type  grep ^hepth  ../arxiv2date  awk '{ print $3,$2 }'  grep R$  ../countcol v
n
awk '{ print $1,$3 }' ZZZ.type  grep ^hepth  ../arxiv2date  awk '{ print $3,$2 }'  grep O$  ../countcol v
n
astroph
Why are 40% of citations redlinks  citations to LANL eprints?
condmat
hepph
In addition to analysing citation identification we can also add the ratio of citations to "antique" papers,
citations to papers that were published before the start of the archive. As would be expected the percentage of
citations to antique papers reduces over time (i.e. currently a paper would have to be more than 10 years old to be
antique").
When comparing the trend of antique citations to the increasing percentage of successfully identified red and
orange citations we can see that citations to papers that are potentially not in the archive has remained relatively
constant (from 50% to 30%), in contrast to the increase of identifiable citations to papers in the archive (from 0%
to 50%). If we assume that spotcites is capable of identifying 100% of citations to articles in arXiv this suggests
that the ratio of citations to articles not in arXiv has remained relatively constant  this could be because:
 these are citations to nonarXiv media (e.g. books)
 the cited author has decided not to deposit their articles in arXiv
 the numbers of articles outside of arXiv are constantly being added to because antiques are reducing
Negative Latency Citations
There are 6455 instances of citations having a negative latency (that is that a paper cites another paper before
the cited paper is deposited in the archive). These are from 4259 papers.
This number can be seperated into those that are citations to LANL preprints (3479) and those that are citations
to published articles (2976), but the published article has been deposited in arXiv. Negative latency citations
to LANL articles can only occur if an author updates the paper, and adds additional references. The total number
of papers that this has occured in is 2095 (1.58% of 132219 papers).
Checking a sample of citations
A sample of 100 citations (the last 100) were taken from papers deposited in 1999, 1997 and 1995. For each
citation the relevent line was found from the output of spotcites (the source citation text) and compared to the
abstract of the cited paper. Primarily the author's names were used for verification, as spotcites is based on
matching publication data.
Paper 
References 
Type 
Correct 

hepth/9912286 
hepth/9808140 
O 
1 

hepth/9912286 
hepth/9901023 
R 
1 

hepth/9912286 
hepth/9707163 
O 
1 

hepth/9912286 
hepth/9802074 
O 
1 

hepth/9912286 
hepth/9707163 
O 
1 

hepth/9912286 
hepth/9810068 
O 
1 

hepth/9912286 
hepph/9501213 
O 
1 

hepth/9912286 
hepth/9806074 
O 
1 

hepth/9912287 
hepph/9905221 
O 
1 

hepth/9912287 
hepth/9906064 
O 
1 

hepth/9912287 
hepth/9910093 
O 
1 

hepth/9912287 
grqc/9910076 
R 
1 

hepth/9912287 
hepth/9911055 
R 
1 

hepth/9912287 
hepph/9906513 
R 
1 

hepth/9912287 
hepph/9911406 
R 
1 

hepth/9912287 
hepph/9912266 
R 
1 

hepth/9912287 
hepth/9911043 
R 
1 

hepth/9912287 
hepth/9910149 
O 
1 

hepth/9912287 
grqc/9912002 
R 
1 

hepth/9912287 
hepth/9910219 
R 
1 

hepth/9912287 
hepth/9911165 
R 
1 

hepth/9912287 
hepth/9911181 
R 
1 

hepth/9912287 
hepth/9905210 
R 
1 

hepth/9912287 
hepph/9905487 
R 
1 

hepth/9912287 
hepth/9909053 
R 
1 

hepth/9912287 
hepth/9905012 
R 
1 

hepth/9912287 
hepph/9910498 
R 
1 

hepth/9912287 
hepph/9906523 
R 
1 

hepth/9912287 
hepph/9911406 
R 
1 

hepth/9912287 
hepph/9909481 
R 
1 

hepth/9912287 
hepth/9912118 
R 
1 

hepth/9912287 
hepth/9912001 
R 
1 

hepth/9912287 
hepth/9905111 
R 
1 

hepth/9912287 
hepth/9911230 
R 
1 

hepth/9912287 
hepth/9912233 
R 
1 

hepth/9912288 
hepth/9711162 
R 
1 

hepth/9912288 
hepth/9711165 
R 
1 

hepth/9912288 
hepth/9810179 
R 
1 

hepth/9912288 
hepth/9907166 
R 
1 

hepth/9912288 
hepth/9908134 
R 
1 

hepth/9912288 
hepth/9908142 
R 
1 

hepth/9912288 
hepth/9908056 
R 
1 

hepth/9912288 
hepth/9711200 
R 
1 

hepth/9912288 
hepth/9911057 
R 
1 

hepth/9912288 
hepth/9911031 
R 
1 

hepth/9912288 
hepth/9910083 
R 
1 

hepth/9912288 
hepth/9910092 
R 
1 

hepth/9912288 
hepth/9703040 
R 
1 

hepth/9912288 
hepth/9708005 
R 
1 

hepth/9912288 
hepth/9802109 
R 
1 

hepth/9912288 
hepth/9805140 
R 
1 

hepth/9912288 
hepth/9803023 
R 
1 

hepth/9912288 
hepth/9804095 
R 
1 

hepth/9912288 
hepth/9805050 
R 
1 

hepth/9912288 
hepth/9909059 
R 
1 

hepth/9912288 
hepth/9911193 
R 
1 

hepth/9912288 
hepth/9911183 
R 
1 

hepth/9912288 
hepth/9911137 
R 
1 

hepth/9912288 
hepth/9907024 
R 
1 

hepth/9912288 
hepth/9908092 
R 
1 

hepth/9912288 
hepth/9404071 
R 
1 

hepth/9912290 
hepth/9808042 
R 
1 

hepth/9912290 
hepth/9810022 
R 
1 

hepth/9912290 
hepth/9912092 
R 
1 

hepth/9912290 
hepth/0003188 
R 
1 

hepth/9912290 
qalg/9707029 
R 
1 

hepth/9912290 
hepth/9901099 
R 
1 

hepth/9912290 
hepth/9810087 
R 
1 

hepth/9912290 
hepth/9904044 
R 
1 

hepth/9912290 
hepth/9909185 
R 
0 

hepth/9912290 
hepph/9504352 
R 
1 

hepth/9912290 
hepth/9609128 
R 
1 

hepth/9912290 
hepth/9607174 
R 
1 

hepth/9912290 
errqa/9904055 
R 
0 

hepth/9912290 
hepth/9604128 
R 
0 

hepth/9912290 
hepth/9903249 
R 
1 

hepth/9912291 
hepth/9802149 
R 
1 

hepth/9912291 
hepth/9908149 
R 
1 

hepth/9912291 
quantph/9909084 
R 
1 

hepth/9912292 
hepth/9406217 
O 
1 

hepth/9912292 
hepth/9608177 
O 
1 

hepth/9912292 
hepth/9707061 
O 
1 

hepth/9912292 
hepth/9711203 
O 
1 

hepth/9912292 
hepth/9811232 
O 
1 

hepth/9912292 
hepth/9811130 
O 
1 

hepth/9912292 
hepth/9811244 
O 
1 

hepth/9912292 
hepth/9901003 
O 
1 

hepth/9912292 
hepth/9606139 
O 
1 

hepth/9912292 
hepth/9702202 
O 
1 

hepth/9912292 
hepth/9702163 
O 
1 

hepth/9912292 
hepth/9710129 
O 
1 

hepth/9912292 
hepth/9811022 
O 
1 

hepth/9912292 
hepth/9904109 
R 
1 

hepth/9912292 
hepth/9909098 
R 
1 

hepth/9912292 
hepth/9711116 
O 
1 

hepth/9912293 
hepth/9908142 
R 
1 

hepth/9912293 
hepth/9908105 
R 
1 

hepth/9912293 
hepth/9909132 
R 
1 

hepth/9912293 
hepth/9909081 
R 
1 

hepth/9912293 
hepth/9907211 
R 
1 











hepth/9712218 
hepth/9607076 
O 
1 

hepth/9712218 
hepth/9409037 
O 
1 

hepth/9712218 
hepth/9507003 
O 
1 

hepth/9712218 
hepth/9708036 
R 
1 

hepth/9712218 
hepth/9601162 
O 
1 

hepth/9712218 
hepth/9506102 
O 
1 

hepth/9712224 
hepth/9606033 
O 
1 

hepth/9712224 
hepth/9601029 
O 
1 

hepth/9712224 
hepth/9602051 
O 
1 

hepth/9712224 
hepth/9609026 
O 
1 

hepth/9712224 
hepth/9604166 
O 
1 

hepth/9712224 
hepth/9604179 
O 
1 

hepth/9712224 
hepth/9612095 
O 
1 

hepth/9712224 
grqc/9409013 
O 
1 

hepth/9712224 
hepth/9407118 
O 
1 

hepth/9712224 
hepth/9509093 
O 
1 

hepth/9712224 
grqc/9601032 
O 
1 

hepth/9712224 
hepth/9611095 
R 
1 

hepth/9712230 
hepth/9603167 
R 
1 

hepth/9712230 
hepth/9608024 
R 
1 

hepth/9712230 
hepth/9606165 
R 
1 

hepth/9712230 
hepth/9610140 
R 
1 

hepth/9712230 
hepth/9704151 
O 
1 

hepth/9712230 
hepth/9610043 
R 
1 

hepth/9712230 
hepth/9612126 
R 
1 

hepth/9712230 
hepth/9702203 
O 
1 

hepth/9712230 
hepth/9707019 
R 
1 

hepth/9712230 
hepth/9710178 
R 
1 

hepth/9712230 
hepth/9709220 
R 
1 

hepth/9712230 
hepth/9710009 
R 
1 

hepth/9712230 
hepth/9611214 
R 
1 

hepth/9712230 
hepth/9611137 
R 
1 

hepth/9712230 
hepth/9603127 
R 
1 

hepth/9712230 
hepth/9702203 
? 
? 

hepth/9712230 
hepth/9609017 
R 
1 

hepth/9712230 
hepth/9707093 
R 
1 

hepth/9712230 
hepth/9712049 
R 
1 

hepth/9712236 
hepth/9711005 
R 
1 

hepth/9712236 
hepth/9610043 
R 
1 

hepth/9712236 
hepth/9608137 
R 
1 

hepth/9712236 
hepth/9710217 
R 
1 

hepth/9712236 
hepth/9710226 
R 
1 

hepth/9712236 
hepth/9712202 
R 
1 

hepth/9712236 
hepth/9709091 
R 
1 

hepth/9712236 
hepth/9709108 
R 
1 

hepth/9712236 
hepth/9711003 
R 
1 

hepth/9712236 
hepth/9704080 
R 
1 

hepth/9712236 
hepth/9710009 
O 
1 

hepth/9712236 
hepth/9709220 
R 
1 

hepth/9712236 
hepth/9712063 
R 
1 

hepth/9712236 
hepth/9706072 
R 
1 

hepth/9712242 
hepth/9509074 
O 
1 

hepth/9712242 
hepth/9210021 
O 
1 

hepth/9712242 
hepth/9403060 
O 
1 

hepth/9712242 
hepth/9506027 
O 
1 

hepth/9712242 
hepth/9510117 
O 
1 

hepth/9712242 
hepth/9602087 
O 
1 

hepth/9712242 
hepth/9211021 
O 
1 

hepth/9712248 
heplat/9607068 
O 
1 

hepth/9712248 
heplat/9709042 
R 
1 

hepth/9712248 
heplat/9711009 
R 
1 

hepth/9712248 
heplat/9610005 
O 
1 

hepth/9712248 
heplat/9708023 
R 
1 

hepth/9712248 
heplat/9710039 
R 
1 

hepth/9712248 
hepth/9402066 
O 
1 

hepth/9712248 
hepth/9511015 
O 
1 

hepth/9712248 
hepth/9710128 
R 
1 

hepth/9712248 
hepth/9511104 
O 
1 

hepth/9712248 
hepth/9605116 
O 
1 

hepth/9712254 
hepth/9608082 
R 
1 

hepth/9712254 
hepth/9111036 
O 
1 

hepth/9712254 
hepth/9206002 
O 
1 

hepth/9712257 
hepth/9611064 
O 
1 

hepth/9712257 
condmat/9612187 
O 
1 

hepth/9712257 
hepth/9510017 
O 
1 

hepth/9712257 
hepth/9707068 
R 
1 

hepth/9712257 
hepth/9711029 
R 
1 

hepth/9712257 
hepth/9605042 
O 
1 

hepth/9712257 
hepth/9311183 
O 
1 

hepth/9712257 
hepth/9503207 
O 
1 

hepth/9712257 
hepth/9506014 
O 
1 

hepth/9712257 
hepth/9607229 
O 
1 

hepth/9712257 
hepth/9702037 
O 
1 

hepth/9712257 
hepth/9701190 
O 
1 

hepth/9712257 
hepth/9712186 
R 
1 

hepth/9712257 
hepth/9610066 
O 
1 

hepth/9712257 
hepth/9309105 
O 
1 

hepth/9712257 
hepph/9509246 
O 
0 

hepth/9712257 
hepth/9708141 
O 
1 

hepth/9712257 
hepth/9702195 
R 
1 

hepth/9712257 
hepth/9506135 
O 
1 

hepth/9712257 
hepth/9612148 
O 
1 

hepth/9712257 
hepth/9708166 
R 
1 

hepth/9712257 
hepth/9610192 
O 
1 

hepth/9712257 
hepth/9711201 
R 
1 

hepth/9712257 
hepth/9606112 
O 
1 

hepth/9712257 
hepth/9509171 
O 
1 

hepth/9712257 
hepth/9706057 
R 
1 

hepth/9712257 
hepth/9711069 
R 
1 

hepth/9712257 
hepth/9708040 
R 
1 






hepth/9512173 
hepth/9303057 
O 
1 

hepth/9512173 
hepth/9304154 
O 
1 

hepth/9512173 
astroph/9402057 
O 
1 

hepth/9512173 
hepth/9408099 
O 
1 

hepth/9512173 
hepth/9408155 
O 
1 

hepth/9512173 
hepth/9411149 
O 
1 

hepth/9512173 
hepth/9408074 
O 
1 

hepth/9512173 
hepth/9411102 
O 
1 

hepth/9512173 
hepth/9502013 
O 
1 

hepth/9512173 
hepth/9502057 
O 
1 
(conf. Paper refernced in citation) 
hepth/9512173 
hepth/9412200 
R 
1 

hepth/9512173 
hepth/9503121 
O 
1 

hepth/9512173 
hepth/9503234 
R 
1 

hepth/9512173 
hepth/9505004 
O 
1 

hepth/9512173 
hepth/9505006 
O 
1 

hepth/9512173 
hepth/9505051 
O 
1 

hepth/9512173 
hepth/9505135 
O 
1 

hepth/9512173 
hepth/9505186 
R 
1 

hepth/9512173 
hepth/9506035 
R 
1 

hepth/9512173 
hepth/9506148 
R 
1 

hepth/9512173 
hepth/9507040 
R 
1 

hepth/9512173 
hepth/9507112 
R 
1 

hepth/9512173 
hepth/9508021 
R 
1 

hepth/9512173 
hepth/9508091 
R 
0 

hepth/9512173 
hepth/9508140 
R 
1 

hepth/9512179 
hepph/9305224 
O 
1 

hepth/9512179 
hepth/9212009 
O 
1 

hepth/9512179 
hepph/9310330 
O 
1 

hepth/9512179 
hepth/9409019 
O 
1 

hepth/9512179 
hepth/9405176 
O 
1 

hepth/9512179 
hepth/9403038 
O 
1 

hepth/9512179 
hepth/9509020 
R 
1 

hepth/9512179 
hepth/9509027 
R 
1 

hepth/9512179 
hepth/9307146 
O 
1 

hepth/9512179 
hepth/9504026 
R 
1 

hepth/9512179 
hepph/9502214 
O 
1 

hepth/9512179 
hepph/9507450 
R 
1 

hepth/9512179 
hepth/9302119 
O 
1 

hepth/9512179 
hepth/9311108 
O 
1 

hepth/9512179 
hepth/9309128 
O 
1 

hepth/9512179 
hepth/9402015 
O 
1 

hepth/9512179 
hepth/9401153 
O 
1 

hepth/9512185 
hepph/9403224 
O 
1 

hepth/9512185 
hepth/9406081 
O 
1 

hepth/9512191 
hepth/9511030 
R 
1 

hepth/9512191 
hepth/9510017 
R 
1 

hepth/9512191 
hepth/9304104 
O 
1 

hepth/9512191 
hepth/9512053 
R 
1 

hepth/9512197 
hepth/9407027 
O 
1 

hepth/9512197 
hepth/9505084 
R 
1 

hepth/9512203 
hepth/9412184 
O 
1 

hepth/9512203 
hepth/9410167 
O 
1 

hepth/9512203 
hepth/9501068 
O 
1 

hepth/9512203 
hepth/9504095 
O 
1 

hepth/9512203 
hepth/9507048 
R 
1 

hepth/9512203 
hepth/9506126 
R 
1 

hepth/9512203 
hepth/9507158 
R 
1 

hepth/9512203 
hepth/9510086 
R 
1 

hepth/9512203 
hepth/9508154 
R 
1 

hepth/9512203 
hepth/9511159 
R 
1 

hepth/9512203 
hepth/9511213 
R 
1 

hepth/9512203 
hepth/9509070 
R 
1 

hepth/9512203 
hepth/9510209 
R 
1 

hepth/9512203 
hepth/9512181 
R 
1 

hepth/9512203 
hepth/9512196 
R 
1 

hepth/9512203 
hepth/9510017 
R 
1 

hepth/9512203 
hepth/9510135 
R 
1 

hepth/9512203 
hepth/9511030 
R 
1 

hepth/9512203 
hepth/9510161 
R 
1 

hepth/9512203 
hepth/9511222 
R 
1 
(two redlinks, only one picked up) 
hepth/9512203 
hepth/9510169 
R 
1 

hepth/9512203 
hepth/9511026 
R 
1 

hepth/9512203 
hepth/9512078 
R 
1 

hepth/9512203 
hepth/9511043 
R 
1 

hepth/9512203 
hepth/9511173 
R 
1 

hepth/9512203 
hepth/9511194 
R 
1 

hepth/9512203 
hepth/9512059 
R 
1 

hepth/9512203 
hepth/9512062 
R 
1 

hepth/9512203 
hepth/9512061 
R 
1 

hepth/9512203 
hepth/9512077 
R 
1 

hepth/9512203 
hepth/9305185 
O 
1 

hepth/9512203 
hepth/9407021 
O 
1 

hepth/9512203 
hepth/9501030 
R 
1 

hepth/9512203 
hepth/9503124 
O 
1 

hepth/9512203 
hepth/9504006 
R 
1 

hepth/9512203 
hepth/9504034 
R 
1 

hepth/9512206 
hepth/9401070 
O 
1 

hepth/9512206 
hepth/9408098 
O 
1 

hepth/9512206 
hepth/9410215 
O 
1 

hepth/9512206 
hepth/9304037 
O 
1 

hepth/9512206 
hepth/9312107 
O 
1 

hepth/9512206 
hepth/9506138 
R 
1 

hepth/9512212 
hepth/9410137 
O 
1 

hepth/9512218 
hepth/9507118 
R 
1 

hepth/9512218 
hepth/9507070 
R 
1 

hepth/9512218 
hepth/9508080 
R 
1 

hepth/9512218 
hepth/9510095 
R 
1 

hepth/9512218 
hepth/9511008 
R 
0 

hepth/9512218 
hepth/9511081 
R 
0 

hepth/9512218 
hepth/9511090 
R 
0 

