[Journal of the Simplified Spelling Society, J26, 1999/2 pp24,25]
[See Journal and Newsletter articles by John Gledhill.]

Testing Readability: a small-scale experiment

John Gledhill

Dr Gledhill is Registrar of Coventry University, UK, and researched the history of Dutch spelling for his doctoral thesis. A shortened version of the following report appeared in the Society's newsletter Simpl Speling in July 1999 (p3).

Design of experiment.

Searching for something different for office staff to do at my University in the week before Christmas, 1998, I decided to seek their views on a variety of spelling systems sent to me for this purpose by members of the Simplified Spelling Society. The idea was to give participants the same text spelled in several different ways, to see which they felt was the easiest to read.

The full text was about one side of A4. It cannot be repeated here for reasons of space, but a sample is given in the lines below. The text was taken from a student handbook, so that the terminology would be familiar to the respondents.

1. Every module has a level attached to it. The main levels are 1, 2 and 3, corresponding roughly to the standards expected of a first year, second year ... (TO)

2. eVRE MoJXL HaZ a LeVeL aTacT TX iT. hu MAN LeVeLZ oR 1, 2 aND 3, KOReSPoNDig RuFLE TX hu STaNDoRDZ eKSPeKTeD uV a FRST YER, SeKoND YER ... (Bruce Beach)

3. Evry modul has a levl atachd to it. Th main levls ar 1, 2 and 3, corespondng rufly to th standrds expectd of a first year, secnd year ... (Allan Campbell)

4. Evri moduel haz a lev'l atacht too it. The maen lev'lz aar 1, 2 and 3, kor'sponding rufli too the stand'rdz ikspektid ov a feurst yeer, sek'nd yeer ... (Ron Footer)

5. Evri moduel haz a levl atacht too it. The maen levlz aar 1, 2 and 3, kor'sponding rufli too the stand'rdz ikspektid ov a feurst yeer, seknd yeer ...(Ron Footer)

6. Evry modul has a levl atachd to it. Th main levls ar 1, 2 and 3, corespondng rufly to th standrds expectd of a first year, secnd year ... (Chris Upward)

7. Evry module has a levl atachd tu it. Th main levels ar 1, 2 and 3, corespondng rufly to th standrds expectd of a ferst year, second year ... (Valerie Yule)

Text 1 is in current standard English spelling (Traditional Orthography or TO);
text 2 is in ANJeL (ANJ),
texts 3 and 6 are variants of Cut Spelng (CS),
texts 4 and 5 are variants of New Spelling (NS), differing only in the treatment of the schwa and the use of dh for voiced th in the second version;
text 7 is Surplus-Cut (SC).

Respondents were asked two questions: "How easy did you find it to understand?" and "How acceptable would it be to use as the normal form of spelling?" The texts were presented in the above order, which was deliberately chosen (a) to ensure that 'normal' spelling came first so that repondents were presented with the context and the terminology, (b) to put similar proposals near to each other to see if they got the same score (ie, to eliminate the tendency to give a high score merely because it was easier than the preceding version). The number of staff in the experiment was too low to be able to vary the order of the texts, which would have helped eliminate variation based on that order; instead they were deliberately given the texts in the same order so that this variation (if present) was a constant that could be ignored.

Sixteen staff were invited to take part and were given the texts; seven of them couldn't grasp what was intended and declined to take part. The figures below are therefore based on the 9 who could understand the basic assumption that there was such a thing as alternative spelling. That in itself is an interesting figure: nearly half the respondents simply took TO as an unchanging constant, even as an axiom.

As an optional extra, respondents were invited to report how long it had taken them to read the various versions. Not all offered timings. Of those who did, the length of time needed seemed to correlate well with their opinion of the acceptability.

Results.

The results were as follows (marks out of 10):


OrthographyEase of ReadingAcceptability Time (mins.)

TO9.9102.3
ANJ2.21.75.5
CS A8.65.32.4
NS A6.33.03.1
NS B6.43.13.3
CS B7.95.17.9
SC7.95.12.3



If we merge the variants we get a clearer picture, with Cut Spelng a clear leader at 8.1 for ease of use and New English Spelling at 6.4; with 'acceptability' at 5.2 (including two scores of 10/10) and 3.1 respectively. ANJeL was a complete shock to all those taking part, and elicited several puzzled comments. The time taken to read Cut Spelng was notably close to the time taken for TO.

Respondents were also invited to amend their scores after reading all the variants, so that they could moderate to some extent any influence in the order of texts. Five did this: one made CS 1 mark easier, but 3 made it 1 mark harder; one made NS 1 mark" harder; the fifth changed all "acceptability" scores to zero from the previous scores of 1 or 2. Overall that did not appear to change the relative ordering noted above.

Discussion and conclusion.

It is possibly discouraging that 3 of the 9 respondents found all alternatives unacceptable, and that all gave 10/10 for the 'acceptability' of current normal spelling (only one gave 9 for 'ease of reading' of TO; all the rest were 10/10).

As a further control respondents were asked whether they (a) knew shorthand, and (b) knew any foreign languages. This was to try to control for familiarity with different ways of writing sounds. There did not, however, appear to be any bias in the scores attributable to these other skills. Only one had admitted to not having either of these skills; and one volunteered that they knew the IPA - intriguingly that was the person who changed 'acceptability' to all zero.

The respondents consisted of the following, though it is impossible to check for any influence this may have had on the response: 1 male, 8 female; well educated (6 to graduate level); age mid-20s to mid-50s.

One respondent passed the test to a friend who is a language teacher. As well as making negative comments on the whole idea, this teacher also submitted comments on the consistency of each scheme. Most of these comments related to the treatment of the schwa, and inconsistency between, for example, facilitis: penltis, levl : level; most of these, to be honest, were simply typographical errors in the samples submitted by the originators.

This was a very modest attempt to see how people totally untutored in the idea of spelling reform would react to a range of changes, from the extreme to the straightforward. The outcomes probably reflect the extent to which each diverges from TO. Reactions to more extreme proposals were strongly negative.