Navigation – Plan du site

An Application of Authorship Attribution by Intertextual Distance in English

Thomas Merriam


Une application d’attribution d’auteur au moyen de la distance intertextuelle en anglais

Le calcul de distance intertextuelle que C. et D. Labbé appliquent aux textes français peut être utilisé pour différencier les œuvres d’au moins deux auteurs dramatiques contemporains de l’époque élisabéthaine, William Shakespeare et Thomas Middleton. Bien que les 46 textes sous étude, transcrits avec une orthographe moderne, ne soient pas lemmatisés et que seuls des échantillons de textes de même longueur aient été utilisés, les indices de distance intertextuelle qu’on a pu ainsi établir empiriquement sont du même ordre de grandeur que ceux qu’ont établis C. et D. Labbé pour le français. Timon of Athens considéré comme étant pour deux-tiers de Shakespeare et pour un tiers de Middleton se place entre le groupe des œuvres de Shakespeare et celui des œuvres de Middleton dans une analyse multidimensionnelle de 1035 distances intertextuelles.

Haut de page

Texte intégral

1. Introduction

1Several months ago I received a copy of 'Inter-Textual Distance and Authorship Attribution : Corneille and Molière' by Cyril and Dominique Labbé, kindness of the latter1. With great interest I studied the article and asked myself the question, 'Could the Labbés' technique, so extensively tested in oral and written French, be applied in English ?'

2An initial trial, using the 36 Shakespeare First Folio plays and several works by other contemporary playwrights, proved encouraging in the light of previous work done with non-traditional methods of authorship discrimination. In particular, the intertextual distance between the verse parts of Shakespeare's Henry V and the body of the 36 Shakespeare plays was greater than generally expected, and in accordance with testing I had done previously.

3Besides the difference in language, there was a methodological divergence in my using a shorter preparation of texts than the Labbés'. I had available modern English spelling electronic texts of dramatic works originally written in the variable orthography of the 16th and 17th centuries. In the case of the 36 Shakespeare plays, these were edited texts in the public domain that were no longer protected by copyright. These dramatic texts would not have met the standards of modern textual study. Their use was thus conditional. All speech headings and stage directions were stripped from the texts to avoid repetitions of names and directions like 'Exit' and 'Enter'.

4Secondly, and more importantly, I did not lemmatize the texts. The past participle of 'have', 'had', would, for example, be counted as a word on its own, distinct from its lemma2. Modern spelling of all the texts was essential for standardization ; otherwise, one word, spelled differently, would be counted as two or three or more words.

5Thirdly, I chose to truncate plays to a standardized length of 11,371 words – commonly the first half, or a little more, of the plays used, excepting Henry V (verse) and Henry V (prose). This obviated the need to scale down larger texts to the size of smaller ones in order to give equal weight to their word distributions. But it meant that I could not use texts smaller than the length dictated by the key text with which I was originally most concerned, the verse parts of Henry V. Problems might arise in the case of a play with a first half written by one author and a second by another3.

2. The Purpose of the Present Investigation

6Some years ago I received through the good offices of Robert Matthews a set of electronic plays by Thomas Middleton, prepared for Gary Taylor in his capacity as editor of the Oxford Middleton. These plays are modern-spelling texts.

7Both Middleton (1580-1627) and Shakespeare (1564-1616) were contemporaries4. Would the Middleton plays, with intertextual distances, effectively distinguish themselves from the Shakespeare plays of the First Folio ? This was the essential question, and it turned out to be relatively simple to answer, as seen below.

8More complex was the question of how comparable my simplified methods in English were with those of the Labbés in French. To what degree did they harmonize ? It is to this that I turn first, as the related matter is dealt with in the opening two sections of 'Qui a écrit quoi ?' in this journal.

3. The Index

9In comparing 50 French literary texts, Dominique Labbé employed a data matrix of 50 rows and 50 columns. The number of separate distances between points is n*(n-1)/2 or 1225. The indices were found to range between 0.195 and 0.536. With an assumption of normality, the mean of 0.377 and standard deviation of 0.053 produced distribution tails (minus or plus two standard deviations) containing 32 and 43 indices respectively.

10Because there are four factors that govern the distancing of texts from each other (authorship, chronology, genre and content), it is possible to make a confident authorship identification or exclusion only with extreme index values, 0.2 for identification or inclusion and 0.5 for exclusion.

11I made use of a data matrix consisting of 36+1 Shakespeare texts (Henry V (verse) and Henry V (prose) split one of the 36 Folio plays in two.) and 9 Middleton plays. This matrix may be schematized thus :

12As the data matrix is square and symmetrical, the intertextual distances are duplicated, e.g., A to B = B to A, and so on. All cells along the diagonal are empty as A = A, B = B, and so on. The comparisons form three blocks, a Shakespeare within-set group - 666 comparisons, a Middleton within-set group – 36 comparisons, and a Shakespeare-Middleton between-sets group - 333 comparisons. The 46 texts are named in Appendix I.

13The mean of all 1035 distances (n*(n-1)/2) is 0.370, an index similar to that of the 1225 comparisons in 'Qui a écrit quoi', 0.377. Bearing in mind differences of language and chronology, the closeness of the two means, 0.007, is remarkable. The standard deviations are understandably more divergent, 0.031 for comparisons involving two contemporary playwrights Shakespeare and Middleton on the one hand, and 0.053 for the comparisons involving 11 French authors and 22 works over at least one hundred years on the other.

14The histogram below shows the overall distribution of the 1035 intertextual differences in the schema above. Although the configuration has the general appearance of statistical normality, the right-hand tail (with more greater-than-average intertextual distances than theoretically expected) is too extensive to fit a symmetric Gaussian bell curve.

15In addition, the overall mean > 0.035, suggests heterogeneity rather than a single statistical population.

16Of particular interest in assessing the comparability of my application with the Labbés', is the rectangle with 333 intertextual distances. As Middleton and Shakespeare are established as separate authors, the 333 distances should theoretically be distributed in a way agreeable with a pre-determined index, > 0.35, indicating different authors (or different registers).

17The mean of the 333 distances, 0.387, approaches the value of 0.40, the maximum index for texts by a single author with widely different genres and topics5. The distribution is notable because of its extended right-hand tail. If the 333 comparisons were between two authors only, the aggregate of intertextual distances should be roughly symmetrical. This is not the case in the histogram below.

18Suspicion arises that the 37 Shakespeare texts are heterogeneous. Bimodality may be checked for by separating eight Shakespeare texts previously suspected of part-Marlovian authorship from the remaining 296. The 333 intertextual distances are thus divided into two sets, one with 72 intertextual distances and the other with 261.

19The histogram below shows the distribution of distances between the Middleton plays and the bulk of the Shakespeare First Folio. Note that reasonable symmetry has been restored to the distribution with a mean between 0.37 and 0.38. We may provisionally assume that the pattern is that of the dispersion of intertextual distances between Middleton and one other author.

20This hypothesis of a heterogeneous Shakespeare body of plays is confirmed by examination of the histogram created by the 72 remaining distances.

21The distribution is not perfectly symmetrical due to the small number of distances concerned. What is revealing is the presence of a central clustering between indices 0.42 and 0.44, well in excess of the limit of 0.40 cited above, and quite distinct from the clustering between 0.37 and 0.38 in the previous histogram.

22Either the eight texts abstracted from the 37 Shakespeare Folio texts are significantly different from the remaining 29 plays with respect to date of composition, genre or topic, or they differ authorially from the bulk of the Shakespeare plays. Possibly two or more mutually reinforcing factors are at work7.

23Such considerations, however, are not central to the investigation at hand. The focus of interest is the limiting value of textual homogeneity. Are the indices derived from the examination of the 1035 intertextual distances in English at all comparable to those derived from extensive testing in French ? I believe that there are reasons for thinking that they are. Nevertheless, there are even better reasons for believing that the method of intertexual distances is transferable from French to English.

4. Multidimensional Scaling

24The entire square 46 by 46 matrix may be transformed into a two-dimensional best-fit 'map' by multidimensional scaling8. Imagine a mileage/kilometer chart of the sort shown in regional road atlases. The miles/kilometers are distances between geographical locations of towns on a limited spherical section of the globe. A two-dimensional map can be re-constructed from the matrix of distances to show the town locations as they would appear on a geographical map.

25The same technique may be applied to the matrix of intertextual, as opposed to geographical, distances. For the 'map' to be intelligible, two dimensions out of the potential 46 can be represented on flat paper. Multidimensional scaling mathematically orders the dimensionality of the matrix into a hierarchy of which the first two dimensions (Dimensions 1 and 2) account for more information than any other two dimensions combined. In effect, the remaining 44 dimensions are superfluous 'noise'.

26In the case at hand of the 46 by 46 matrix, the resulting 'map', if intelligible with regard to the four factors (authorship, chronology, genre and content) will indicate which one or two or three of these account for the lion's share of variation within the data matrix.

27I have assigned the 46 texts code numbers which indicate their chronological sequence. See Appendix I. If date of composition is the outstanding feature of the multidimensional scaling 'map', the visual clustering will reflect the numerical ordering of the plotted points. If, on the other hand, authorship is the major factor in the matrix, clustering will gather together texts by Shakespeare and texts by Middleton in distinct and separate clusters.

28In the diagram below, nine Middleton plays {28,32,25,36,41,43,44,45,46} are located in the upper left-hand corner. The majority of plays by Shakespeare form an irregular shape from lower left to upper right, while two plays 'float between' the Middleton and Shakespeare clusters. They are Timon of Athens {30} and Henry VIII {42}, both collaborative plays and recognized as such by a considerable number of scholars9. In addition, 3H6 {4}, 1H6 {5}, Tit {6}, are the First Folio plays seen as collaborative by Ward Elliott and Robert Valenza, and Gary Taylor10.

29The fact that the 9 Middleton plays form a cluster in the upper left-hand separate from the Shakespeare plays confirms that the intertextual distances are sensitive to the authorship difference between Shakespeare and Middleton. The 37 designated Shakespeare texts have an elongated shape with a semi-detached 'handle' of 9 plays to the right. This peculiar feature of the First Folio plays strengthens the impression received above that the aggregate distances between the Middleton plays and the Shakespeare plays are bimodal, bimodality being a feature of the Shakespeare corpus11.

30Although the plays in the 'handle' of the Shakespeare cluster are generally of early composition {3, 4, 5, 6, 7, 12, 13}, Macbeth 1606 {33} and Henry V (verse) 1598-99 {20} are not so. Two early Shakespeare plays, The Two Gentlemen of Verona 1590-91 {1} and The Taming of the Shrew 1590-91 {2} are relatively close to the Middleton plays, of which the earliest, The Phoenix 1603-04 {28}, was written some 12 years later. Chronology is not the predominant factor.

31Seven of the nine 'handle' plays are history plays {3, 4, 5, 7, 12, 13, 20}, the exceptions being Titus Andronicus {6} and Macbeth {33}. One history play, Henry VIII {42} stands by itself, and two other history plays, Henry IV, Part 1 {15} and Henry IV, Part 2 {17} lie within the main part of the Shakespeare cluster. Genre is not the predominant factor.

32Although unobtrusive, the positioning of Timon of Athens {30} is the strongest evidential support for the discriminating power of intertextual distances.

Developing a conjecture by Wells (1920) and Sykes (1924), Lake (1975), Jackson (1979), and Holdsworth have provided extensive, independent, and compelling evidence that approximately a third of the play was written by Thomas Middleton.12

33Timon of Athens is found between the Shakespeare and Middleton clusters ; it is the nearest Shakespeare play to the Middleton plays.

34The generally accepted collaboration of John Fletcher and William Shakespeare in Henry VIII (All is True) {42} stands by itself outside the cluster of Shakespeare plays13.

35My previously arrived at view is that the eight of the plays in the 'handle', {3, 4, 5, 6, 7, 12, 13}, are partly the work of Christopher Marlowe14.

Haut de page


Elliott W. & R. Valenza (1996). « And Then There Were None », Computers and the Humanities 30 : 191-210.

Farr D. M. (1973). Thomas Middleton and the Drama of Realism. New York : Harper Row.

Honan P. (1998). Shakespeare : a Life. Oxford : Oxford University Press.

Jackson M. P. (1979). Studies in Attribution. Salzburg : Salzburg University Press.

Kruskal J. B. & M. Wish (1978). Multidimensional Scaling. Beverly Hills & London : Sage.

Labbé C. & D. Labbé (2001). « Inter-Textual Distance and Authorship Attribution : Corneille and Molière », Journal of Quantitative Statistics 8 : 213-31.

Manly B.F.J. (1986). Multivariate Statistical Methods : A Primer. London : Chapman and Hall.

Merriam T. (1996). « Tamburlaine Stalks in Henry VI », Computers and the Humanities 30 : 267-80.

Merriam T. (1998). « Heterogeneous Authorship in Early Shakespeare and the Problem of Henry V », Literary and Linguistic Computing 13 : 15-28.

Shakespeare W. (1995). Titus Andronicus : J. Bate (ed.). London and New York : Arden.

|Shakespeare W. (1964). King Henry VIII : R. A. Foakes (ed.). London : Methuen.

Shakespeare W. (1999). Henry VIII or All is True : J. Halio (ed.). Oxford : Oxford University Press.

Shakespeare W. (2000). Henry VIII : G. McMullan (ed.). London and New York : Arden.

Taylor G. (1987). « The Canon and Chronology of Shakespeare's Plays », in S. Wells & G. Taylor with W. Montgomery & J. Jowett (eds.) On William Shakespeare : a Textual Companion. Oxford : Clarendon Press.

Vickers B. (2002). Shakespeare, Co-Author : A Historical Study of Five Collaborative Plays. Oxford : Oxford University Press.

Haut de page


1 Cf. Labbé, Cyril and Dominique Labbé (2001).
2. The absence of lemmatization does not entail the same drawbacks that it does in French, because of the fewer number of flexible endings and different words with the same spelling in English.
3. The algorithm used for reduction of the larger of the two texts in French is being adapted for use in English in the future.
4. See Park Honan, Shakespeare: a Life (Oxford : Oxford University Press, 1998), 347. D. M. Farr, Thomas Middleton and the Drama of Realism (New York : Harper Row, 1973).
5. Labbé and Labbé, 2001 : 219.
6. The eight texts are 2H6, 3H6, 1H6, Tit, R3, R2, Jn and H5v.
7. The dates of the eight texts, 2H6, 3H6, 1H6, Tit, R3, R2, Jn and H5v, are earlier than any of the Middleton plays. See Appendix I. However, according to the Oxford dating, TGV and Shr are the earliest Shakespeare plays, and they are closer in time to the Middleton plays than the eight texts.
8. For a detailed explanation of the technique, see Joseph B. Kruskal and Myron Wish, Multidimensional Scaling (Beverley Hills and London, Sage, 1978). Also Brian F.J. Manly, Multivariate Statistical Methods: A Primer (London, Chapman and Hall, 1986), 126-41.
9. Brian Vickers, Shakespeare Co-Author: A Historical Study of Five Collaborative Plays (Oxford, Oxford University Press, 2002), 244-290; 333-402. See also Gary Taylor, 'The Canon and Chronology of Shakespeare's Plays' in William Shakespeare: a Textual Companion, ed. Stanley Wells and Gary Taylor with John Jowett and William Montgomery (Oxford, Clarendon Press, 1987, 127-28, 133-34; Henry VIII or All is True, ed. Jay. L. Halio (Oxford, Oxford University Press, 1999), 16-24. Also M.P. Jackson, Studies in Attribution: Middleton and Shakespeare (Salzburg: Salzburg University Press, 1979.) For the opposite point of view on Henry VIII, see King Henry VIII, ed. R.A. Foakes (London, Methuen, 1964), xv-xxviii. Jackson views on the authorship of Timon have not, to my knowledge, been disputed in resent years.
10. W. Elliott and Valenza, R., 'And Then There Were None', Computers and the Humanities, 30 (1996), 191-210. Taylor, 111-115. See also Vickers, 148-243. For the opposite view, see Titus Andronicus, ed. Jonathan Bate (London and New York, Arden, 1995), 79-83.
11. See Appendix II for a table of distances between the Middleton plays and the non-Middleton and Middleton plays.
12. Taylor, 128.
13. In addition to the works cited above in notes 6 and 7, see the most recent edition of the play, King Henry VIII, ed. Gordon McMullan (London and New York, Arden, 2000), which carries on the title page 'William Shakespeare and John Fletcher'.
14. Thomas Merriam, 'Tamburlaine Stalks in Henry VI', Computers and the Humanities, 30 (1996), 267-80. Thomas Merriam, 'Heterogeneous Authorship in Early Shakespeare and the Problem of Henry V', Literary and Linguistic Computing, 13 (1998), 15-28.
Haut de page

Pour citer cet article

Référence électronique

Thomas Merriam, « An Application of Authorship Attribution by Intertextual Distance in English », Corpus [En ligne], 2 | 2003, mis en ligne le 15 décembre 2004, consulté le 21 janvier 2017. URL :

Haut de page


Thomas Merriam

King's College, London

Haut de page

Droits d’auteur

© Tous droits réservés

Haut de page
  • Logo Revues électroniques de l’université de Nice