<--- 4 CHI-K. J. INTELL. PROP. 108 ---> 

 

 

PATENT GRANT RATES AT THE UNITED STATES PATENT AND TRADEMARK OFFICE

By Lawrence B. Ebert

Abstract

In recent proposals for patent reform made by the Federal Trade Commission and by the National Academy of Sciences, there has been discussion of the possibility that the grant rate of patents by the United States Patent and Trademark Office [USPTO] is high compared to that of other industrialized countries, including that of Japan and those of Europe.  This discussion began with papers of Quillen and Webster that suggested that the grant rate might be as high as 97% and more reasonably is at least 85%.  Although the actual grant rate at the USPTO is typically in the range 62% to 68%, Quillen and Webster suggested the higher numbers based on an analysis of continuing applications (including continuations, divisionals, and continuations-in-part).   The present paper suggests that the analysis of Quillen and Webster is flawed both legally and methodologically, and that recent work by Clarke, which places the corrected grant rate at less than 75%, is more accurate.

Introduction

There has been recent discussion of the need for reform in the United States Patent Office.[1]  In the draft of its recent report, "A Patent System for the 21st Century," the National Academy of Sciences, in discussing issues of patent reform, stated:

A second reason for concern about changes in quality is that according to recent estimates U.S. patent approval rates are higher than officially reported by the USPTO and higher than in Europe and Japan.  In a recent study, Quillen and Webster (2001) argued that calculations of allowance rates from USPTO reported numbers of applications filed, abandonments, and total allowances or issued patents have led to a consistent underestimate of actual allowance rates because the calculations did not take into account the effect of U.S. continuation practice. [2] 

 

The draft further stated:  “the committee believes that high acceptance rates, especially if increasing over time relative to comparable rates in other industrialized countries would be reason to look more closely at examination quality.”[3]

The concern about high allowance (acceptance) rates is misplaced because the reported enhanced grant rate is inaccurate, both from a legal and a methodological perspective.  Substantively, it is untrue from a legal viewpoint that all continuing applications reflect repeated attempts to claim the same invention.  Although Quillen and Webster, in papers in 2001[4] and 2002,[5] approached continuing applications (which include continuations, divisionals, continuations-in-part) as if they reflected repeated attempts to claim the same invention, divisionals and continuations-in-part, almost by definition, reflect attempts to claim different inventions.  Therefore, numerical enhancement of grant rates in terms of such continuing applications is improper.  As to methodology, a model creating a numerical enhancement of grant rate by subtracting continuing applications from abandonments is flawed.  The use of such an approach, as applied to patent families in which more than one continuing application issues as a patent, allows a grant rate in excess of 100%,  -- a nonsensical result. 

Discussion 

I.                     Grant rate issues

In the 2001 paper, Quillen and Webster fashioned a corrected patent grant from the expression (applications allowed/(applications allowed + applications abandoned) by subtracting from the denominator term the number of continuing applications.  If, hypothetically, it were true that each continuing application were a repeated attempt to patent the same invention, and if, hypothetically, it were true that only one patent could issue from such a sequence, this might be appropriate.  For example, if one had an initial application, and then filed nine continuing applications, of which only the ninth led to a patent, the Quillen and Webster approach would convert a 10% grant rate (1/(1 +9)) into a 100% grant rate (1/(1+9-9)). 

II.                   Legal issue:  The claims define the invention

The first assumption of Quillen and Webster, that each continuing application is a repeated attempt to patent the same invention, is false as a substantive, legal matter.  Divisional applications, which arise from restriction requirements issued by the USPTO, are by definition directed to claims corresponding to an invention different from those prosecuted in the parent.  The claims of divisional applications are insulated from obviousness-type double-patenting rejections under 35 USC 120ff, because they are directed to different inventions.  Similarly, the claims of continuation-in-part applications, which are typically directed to new matter, address an invention different from the claims of the parent application.  Furthermore, even in continuation applications, the claims of the continuation application are most frequently of different scope than in the parent, and thus are not directed to the same invention. In the 2001 paper, Quillen and Webster wrote at page 4:  "A continuation application is a second application for the same invention claimed in a prior copending nonprovisional application that claims the benefit of the filing date of the prior application."  As an empirical matter, other than with RCEs (37 CFR 1.114; RCE practice is different than that for continuations, divisionals, and continuations-in-part), it is rarely the case that one deals with the "same invention claimed" in a continuing application.  In summary, as to divisional and continuation-in-part applications, it is improper to fashion a numerical model to correct grant rate on the assumption that these continuing applications reflect repeated attempts to claim the same invention; even with continuation applications, one typically is not dealing with the "same invention claimed" in the parent.

Although much of the text of Quillen and Webster is directed to the disclosure of the application,[6] it is the claims, which are analyzed by the USPTO.  The claims of a patent define the invention.  35 USC 112 P 2; In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). The claims of a patent are the sole measure of the patent grant. Once the claims issue in a final particular form, the protected invention is, as a matter of law, that form. Bandag, Inc. v. Al Bolser's Tire Stores, Inc., 750 F.2d 903, 223 USPQ 982 (Fed. Cir. 1984).   If the claims are amended during prosecution, it is the amended, final claims that define the invention.  Purdue Pharma L.P. v. Faulding Inc., 230 F.3d 1320, 56 USPQ2d 1481 (Fed. Cir. 2000).  Even though the specification may offer support for various claims, it is the issued claims that define the invention that may be enforced. An essential purpose of patent examination is to fashion claims that are precise, clear, correct, and unambiguous. Only in this way can uncertainties of claim scope be removed, as much as possible, during the administrative process. In re Zletz, 893 F.2d 319, 13 USPQ2d 1320 (Fed. Cir. 1989).

III.                 Methodology issue:  Subtracting continuing applications is wrong

As noted, the methodology of Quillen and Webster is based on the mistaken assumption that every continuing application reflects a repeated approach to claim the same invention. The model is valid only for this assumption.  For example, as noted above, if one had an initial application, and then filed nine continuing applications, of which only the ninth led to a patent, the Quillen and Webster approach would convert a 10% grant rate (1/(1 +9) into a 100% grant rate (1/(1+9-9).

This methodology breaks down completely if more than one patent issues in a patent family, because, in fact, every continuing application does not reflect a repeated approach to claim the same invention.  To illustrate this numerically, consider the analysis of 5 original/ 5 continuing applications, wherein 1 original and 5 continuing applications are allowed.  In an uncorrected model, the grant rate is 60% (6 allowed/(6 allowed + 4 abandoned)).  However, in the "corrected" model, the grant rate is 120% (6 allowed/(6 allowed + 4 abandoned - 5 continuing)).  This is clearly nonsense.

In fact, in other calculations, Quillen and Webster reported in Figure 8 of their 2002 paper a grant rate in excess of 100% for the years 1995, 1999, and 2000.  12 Fed. Cir. B. J. at 47.  They stated:  "The calculated Grant Rate for those years is above 100%, which is not possible.  This impossibility undoubtedly occurs because some of the divisional applications claim subject matter that is genuinely independent and distinct from the invention claimed in the parent application that was not abandoned."  12 Fed. Cir. B. J. at 47.  Actually, Quillen and Webster make no assessment about independent subject matter of claims, so that the impossibility occurs because of a flaw in their basic numerical model and for no other reason.

This flaw is not corrected by the adjustments reported in the appendices of their 2002 paper.  Using correction factors based on a sampling of 1,000 patents, Quillen and Webster attempt to correct for the one case of a patent family wherein there is an issued patent for a parent and one continuing application.  They do not correct for the cases wherein there is a patent family with issued patents on more than one continuing application.  Thus, although the use of the 69% correction factor conceals the problem in the adjusted model relative to the unadjusted model, neither model is constructed such that the grant rate must be bounded to be no more than 100%, and as such both the adjusted and unadjusted models of Quillen and Webster are in error.

IV.                Empirical issue:  Continuations are generally not repeated attempts to patent the same invention

Although Quillen and Webster suggested that by filing a continuation, the patent applicant can avoid a final decision as to the patentability of the claims of his or her [initial] application, and continue seeking a patent as to that subject matter, one may question whether continuations reflect repeated attempts to obtain the same claims.  Consider actual examples from patent families comprising a patent on the parent application and on a continuation application. The first claim of parent U.S. Patent  6,546,378, entitled "Signal Interpretation Engine", is directed to an apparatus for providing an interpretation corresponding to a signal, and reads as follows:

An apparatus for providing an interpretation corresponding to a signal, the apparatus comprising a computer programmed to run a plurality of modules comprising:

a feature expansion module containing feature operators operating on a signal, expanding the signal to form a plurality of feature segments, each feature segment corresponding to a unique representation of the signal

 

across an epoch corresponding to an event occurring within a time segment, the event having a type;
a consolidation module for forming inner products between the plurality of feature segments and a weight

 

table comprising weight elements, and for applying aggregators to consolidate the inner products into a distribution function representing an attribute over a domain, the domain having one or more values;

a map generation module for producing an interpretation map reflecting a preferred weight table and aggregator to be applied to the signal data to characterize the event.

 

In contrast, the first claim of U.S. Patent 6,745,156 (entitled "Petroleum exploration and prediction apparatus and method," which issued from a continuation of the application yielding U.S. Patent  6,546,378) is to a method comprising seven steps, and reads as follows: 

A method comprising:

 

recording a first signal corresponding to a first wave response from a first object having a first status,

providing a first record to store information corresponding to the first signal and the first status;

recording a second signal corresponding to a second wave response from a second object having a second status distinct from the first status;

providing a second record to store information corresponding to the second signal and the second status;

providing a plurality of feature operators, each feature operator thereof comprising a mathematical manipulation to operate on a time segment of interest from either the first record or the second record to provide a feature segment;

processing the first and second records using the plurality of feature operators to produce a plurality of feature segments; and

selecting at least one feature operator from the plurality of feature operators that produces at least one feature segment from the plurality of feature segments having an attribute corresponding exclusively to one of the first record and second record.

 

No terminal disclaimer arose for U.S. Patent  6,745,156, suggesting that there were no issues of obviousness-type double-patenting for the claims of U.S. Patent 6,745,156 in view of the claims of U.S. Patent 6,546,378.  Unlike patents issuing on divisional applications, which are by statute immunized from obviousness-type double-patenting issues, such rejections can arise as to claims within continuation applications.  But this was not the case here, suggesting that this was not a repeated attempt to patent the same invention.

Also, the first claim of U.S. Patent  6,009,458, entitled "Networked Computer Game System with Persistent Playing Objects," is to an apparatus and reads as follows:

An apparatus comprising:

a plurality of processing devices, at least first and second ones of said processing devices including an associated display;

a network interconnecting said processing devices;

a first game program at least partially stored on one of said processing devices, said first game program having at least a first playing object having at least one attribute, said first playing object being identifiably associated with a user, said first playing object having a look and feel on said display corresponding to said at least one attribute of said first playing object;

a second game program at least partially stored on one of said processing devices, said second game program having at least a second playing object having at least one attribute, said second playing object being identifiably associated with said user said second playing object having a look and feel on said display corresponding to said at least one attribute of said second playing object; and

a mapping function, at least partially stored on one of said processing devices, for mapping attributes of said first playing object in said first game to said second playing object in said second game, wherein said mapping function maintains a substantially similar overall value of attributes between said first and second playing objects.

In contrast, the first claim of U.S. Patent 6,745,236, entitled Networked Computer Game System with Persistent Playing Objects and issuing from a continuation application, is to an apparatus accessible via a network, and reads as follows:

An apparatus accessible via a network comprising:

a processing device;

a memory coupled to said processing device;

a database on said memory storing a plurality of playing objects that are used in at least one video game, each playing object being associated with an ID field for designating a user, each playing object being unique and being associated with one user at any given time, each playing object being a data object with a collection of attributes; and

a marketplace program having instructions facilitating the sale or exchange of said playing objects between users, independent of a distributor of said video games; and

said marketplace program, upon a sale or exchange of a playing object, changing said ID field to designate an acquiring user in response to a verified message from a relinquishing user having an ID associated with said playing object.

 

No terminal disclaimer arose for U.S. Patent 6,745,236.

V.                  Double-counting issue: The paper by Clarke[7]

In April 2003, Robert A. Clarke of the USPTO published a paper criticizing the double-counting aspects inherent in the 2001 paper of Quillen and Webster.[8]  Clarke stated:  "By filing multiple continuing applications it is possible for an applicant to obtain a number of patents based on the same initial invention, subject to the need to file terminal disclaimers where claims conflict by an obvious difference… The flaw in the model of Quillen and Webster is that each of these second or subsequent patents was treated as if they were patents on new invention(s).  Thus, the Quillen and Webster model results in 15.5% double counting of original patents, which resulted in the Quillen and Webster model dramatically overestimating the likelihood of obtaining a patent from the USPTO."

Although Clarke mentioned the 2001 paper of Quillen and Webster, and not the 2002 paper, his remarks also suggest the invalidity of the 2002 paper.  Even though Quillen and Webster attempted a correction in 2002 for the one case wherein there was a patent on the parent application and on one continuing application, Quillen and Webster did not correct for cases wherein there was more than one patent on continuing applications, whether or not there was a patent on the parent application.  This failure immediately leads to an inflation in the grant rate, as described above.

Concrete examples of patent families wherein the correction of Quillen and Webster of 2002 fails are as follows and were taken merely from the set of patents that issued on June 1, 2004:

Patent families with more than two issued patents include:

U.S. Patent 6,745,191 issued as a continuation from U.S. Patent 6,584,547 which in turn issued as a continuation from U.S. Patent 6,169,982, which in turn issued as a continuation from U.S. Patent 5,987,462;

U.S. Patent 6,745,126 issued as a continuation from U.S. Patent 6,516,270, which in turn issued as a continuation from U.S. Patent 6,216,053, which is a continuation-in-part from U.S. Patent 5,878,402, which is a continuation from U.S. Patent 5,636,118, which arise from two abandoned applications;

U.S. Patent 6,745,060 issued as a continuation of a continuation from U.S. Patent  6,157,850,  which is a continuation from U.S. Patent 5,632,272, which is a continuation-in-part from U.S. Patent 5,490,505, which is a continuation-in-part from U.S. Patent Pat. No. 5,482,036, which is a continuation of an abandoned application;

U.S. Patent 6,745,010 issued as a continuation from U. S. Patent 6,522,867, which is a continuation-in-part from U.S. Patent 6,154,637, which is a continuation from U.S. Patent 6,047,165;

An entirely distinct issue with the approach of Quillen and Webster arises from the option in U.S. patent practice to file a continuation application directly from a PCT application without prosecuting the PCT application.[9]  Although denominated a continuation application, this type of application represents the first attempt at prosecution in the United States and is not properly characterized as a repeated attempt to obtain claims.  Some specific examples of these "continuation" applications are U. S. Patent 6,744,065 (a continuation of PCT/GB98/03429),  U. S. Patent 6,310,255 (a continuation of PCT/JP99/01583), and  U. S. Patent 6,745,032 (a continuation of PCT/FI99/00280).  Separately, the PCT issue can be intertwined with the multiple patent family issue, as in the case of  U. S. Patent 6,743,895, which patent family involves U.S. Pat. No. 5,880,094 and U.S. Patent 6,352,973, which application is a continuation of international patent application  PCT/CA 96/00401.  

In contrast to Quillen and Webster, Clarke dealt with patent families comprising multiple patents.  Using two different models, which roughly paralleled models of Quillen and Webster in description, but not in methodology,[10] Clarke concluded that the grant rate at the USPTO was less than 75%.  The adjusted model of Quillen and Webster in 2002 had asserted a grant rate of 85%.

VI.                Criticism of Clarke paper by Lemley and Moore

In February 2004, Lemley and Moore criticized the work of Clarke, and suggested that the 85% number of the 2002 paper by Quillen and Webster was more reliable.[11]  Lemley and Moore stated that Clarke erroneously assumed that every continuation resulted in a patent.[12]  The relevant text on page 338 of Clarke states: 

The USPTO models deduct from the overall number of original applications those that are also counted as continuing applications, to avoid double counting of applications (as does Quillen and Webster).  The USTPO models, however, also deduct from the total number of patents the percentage of applications that give rise to patents both from the original applications and from continuing applications (which Quillen and Webster does not).

 

There is no mention of any assumption that "every" continuation resulted in a patent.  There is a correction for cases in which BOTH an original (i.e., parent) application AND continuing applications give rise to patents.  In 2002, Quillen and Webster made such a correction, although less complete than that made by Clarke. There is no basis for the specific criticism by Lemley and Moore of Clarke.[13]  In fact, it appears that Clarke's approach is more accurate than that of Quillen and Webster, and, as such, the 74% number, rather than the 85% number, should be deemed more reliable.

VII.              Citation issues

The need to be "up to date" in Shepardizing cases is well recognized within the legal community.[14]  In the present situation pertaining to patent grant rates, the impact of the first paper by Quillen and Webster, which asserted a 97% patent grant rate, extended later in time than the second paper by Quillen and Webster which withdrew the 97% patent grant rate and posited (among other numbers) an 85% patent grant rate.  Thus, the Harvard Law Review[15]and the chief patent counsel of Intel[16] referred to the conclusions of the first Quillen and Webster paper in the year 2003, after the conclusions had been modified in 2002.[17]

In referring to approximation studies, the legal scholar both needs familiarity with the legal area, including the significance of the approximations, and requires an "up to date" knowledge of the status of the approximation studies, lest the scholar reference numbers which are no longer operative.[18]  As computing capabilities increase, one will have more direct access to statistics on families of continuing applications and to the significance of the variations in claim structure within such families, so that the approximations of Quillen and Webster and of Clarke will be unnecessary in the future.  The more enduring points of this discussion include the need to formulate clearly the assumptions of a model, to incorporate within the model appropriate and realistic boundary conditions, and to update promptly as new information becomes available.

Conclusion

The patent grant rate studies of Quillen and Webster are flawed and the published grant rate numbers should not be relied upon in making arguments about patent reform.  The models of Quillen and Webster erroneously assume that all continuing applications are repeated attempts to patent the invention of the parent application.  The methodology of Quillen and Webster, even as modified in 2002, involves double counting of patents and artificially inflates the patent grant rate.  The approximations of Clarke are more reliable than those of Quillen and Webster.

 

 

 

 

 



[1] Brenda Sandburg, Reinventing the patent system; National Academy of Sciences adds voice to chorus on fixing patent litigation, 128 The Recorder 1 (April 20, 2004); Neil Munro, Off-Limits, 36 The National Journal 1 (May 8, 2004) [Sandburg and Munro articles available on LEXIS]; Arti K. Rai, Engaging Facts and Policy:  A Multi-Institutional Approach to Patent System Reform, 103 Colum. L. Rev. 1035 (2003).

 

[2] Stephen A. Merrill, Richard C. Levin, and Mark B. Myers, A Patent System for the 21st Century, page 43, draft available at  http://books.nap.edu/catalog/10976.html.

 

[3]  Id.

 

[4] Cecil D. Quillen and Ogden D. Webster, Continuing Patent Applications and Performance of the U. S. Patent and Trademark Office, 11 Fed. Cir. B. J. 1 (2001-2002). 

 

[5] Cecil D. Quillen and Ogden D. Webster, and Richard Eichman, Continuing Patent Applications and Performance at the U. S. Patent and Trademark Office-Extended, 12 Fed. Cir. B. J. 35 (2002).  A proof of the 2002 paper appears as an appendix to the report of the Federal Trade Commission on patent reform.  See www.ftc.gov/os/comments/intelpropertycomments/continuingapp2002.pdf

 

[6] At page 42 of the 2002 publication, Quillen and Webster write: "As noted previously, continuing applications claim inventions that are described in earlier filed patent applications, and thus, to a considerable extent, represent 'rework' for the USPTO, since the inventions of the continuing applications were, or could have been, examined in the earlier parent applications."  Here, the emphasis of Quillen and Webster  is on inventions that are described  (as distinct from claimed) in earlier patent applications.  The invention of a continuation-in-part is not described in the earlier parent application.  As to divisionals, only the elected invention following a restriction requirement can be examined, so non-elected inventions were not, and could not have been, examined in the earlier parent application.

 

[7] In passing, observe a distinct paper by W. Lesser and Travis Lybbert, Do Patents Come Too Easy?, 44 IDEA 381 (2004).  The approach is quite different from that of Quillen and Webster or Clarke, and does not directly address the divergent results between Quillen and Webster compared to Clarke.

 

[8] Robert A. Clarke, U.S. Continuity Law and its Impact on the Comparative Patenting Rates of the U.S., Japan, and European Patent Office, 85 J. Pat. & Trademark Off. Soc'y [JPTOS] 335 (2003).

 

[9]  The filing of a continuation from a PCT application under 35 USC 111(a) is generally referred to as the "bypass route."  See Harris A. Wolin, Practical Considerations for Entry into the U.S. National Stage from the PCT, Intellectual Property Today, pp. 50-55 (October 2003); the options are compared in MPEP 1893-1896.

 

[10] Quillen/Webster and Clarke had a first model comparing patent applications in one year to patent grants two years later, based on the rough assumption of a two-year pendency.  This is actually a binning approach, so that the delay could be anywhere from one year (lower bound) to three years (upper bound).  Quillen/Webster and Clarke had a second model determining patent grants and patent disposals in one year.  For both the first and second model, Clarke handled the effect of continuing applications in a way entirely differently from Quillen/Webster.  Separately, there are some differences between the raw data (e. g., applications filed; patents granted) between Clarke and Quillen/Webster, so that they would not get exactly the same numbers for grant rate, even if they applied the same model.  However, these differences would not account for the difference between Clarke (74%) and Quillen/Webster (85%) for the second model.

 

[11] Mark A. Lemley and Kimberley L. Moore, Ending Abuse of Patent Continuations, 84 B. U. L. Rev. 63 (2004) at footnote 22.

 

[12] Footnote 22 of Lemley and Moore states: See Robert A. Clarke, U.S. Continuity Law and Its Impact on the Comparative Patenting Rates of the U.S., Japan and the European Patent Office, 85 J. Pat. & Trademark Off. Soc'y 335, 338 (2003) (erroneously assuming that every continuation resulted in a patent and concluding that the grant rate was 75%). The 85% number provided in the revised Quillen et al. study is based on actual data about the applications that issue based on continuations, and reflects the best estimate we have of how often applications mature into patents.

 

[13]  I contacted both Professor Moore and the Boston University Law Review about the basis for the criticism in footnote 22, but to date have not received a response.  In the area of scientific research, inquiries are not always answered, as Leon Jaroff has illustrated for the inquiries of Bruce Flamm about the paper by Kwang Y. Cha, Daniel P. Wirth, and Rogerio Lobo, 46 J.Repro. Med. 781 (2001); see

http://www.time.com/time/columnist/jaroff/article/0,9565,660053,00.html.

 

[14] For example, Rosenstiel v. Rosenstiel, 251 N.Y.S.2d 565, 578-579 (1964) (noting that "[t]he court was astounded to find that that case upon so much reliance was placed by defendant's counsel"
was reversed and the reversal had in turn been upheld). Also, Golden Eagle v. Burroughs, 103 F.R.D. 124, 129 (N.D. Cal. 1984)("Thus their failure to cite adverse authority is not excusable."); Cimino v. Yale University, 638 F. Supp. 952, 959 n.7 (D. Conn. 1986) ("diligent research, which includes Shepardizing cases, is a professional responsibility."); Fletcher v. Florida, 858 F. Supp. 169, 172 (M.D. Florida 1994) (noting plaintiffs' "neglect to Shepardize," and stating that they "should guard against future research failures");  Gosnell v. Rentokil, 175 F.R.D. 508, 510 (N.D. Ill. 1997)("It is really inexcusable for any lawyer to fail, as a matter of routine, to Shepardize all cited cases.")  The story of sheparditis of Supreme Court law clerk Edward Prichard pertaining to Oklahoma Packing Co. v. Oklahoma Gas & Electric Co., 309 US 4 (1940) is relevant and available at http://www.nybooks.com/articles/5459.

 

[15] Note, Estopping the Madness at the PTO: Improving Patent Administration Through

Prosecution History Estoppel, 116 Harvard Law Review 2164 (2003).  This article inferred that the numerically calculated bound of 97% of the 2001 paper by Quillen and Webster was the patent grant rate, rather than the empirically measured values of 62% to 68%, and thereby found madness at the PTO which needed to be stopped.  See Lawrence B. Ebert, Good Night, Gracie?, Intellectual Property Today, pp. 26-27 (August 2003) [available LEXIS].

 

[16] Testimony of David M. Simon, Chief Patent Counsel, Intel; July 24, 2003, available at www.house.gov/judiciary/simon072403.pdf

 

[17]  A more severe situation is found in the paper by William P. Whitely, Drummond Rennie and Arthur W. Hafner, The Scientific Community's Response to Evidence of Fraudulent Publication; the Robert Slutsky Case, 272 JAMA 170-173 (1994), wherein scientists continued to cite to the fraudulent papers of Robert Slutsky, even after the papers had been retracted.  A different study of citations to retracted findings is found in John M. Budd,  Mary Ellen Sievert, and Tom R. Schultz,, Reasons for Retraction and Citations to the Publications, 280 JAMA 296-297 (1998).

 

[18] Those relying on literature, whether legal or technical, must be able to assume that it is accurate, and that any earlier errors are corrected in a manner that  is readily ascertained by those reading the literature.  The consequences of uncorrected errors in the literature are surveyed both in Lawrence B. Ebert, There you go again, Intellectual Property Today, pp. 32-34 (July 2004)  and in a television analysis by NHK television of the Jan-Hendrik Schon affair (scheduled for broadcast in Japan, August 2004).