Reliability and Validity of Latvian Sign Language Comprehension Test (LSLCT) for Deaf and Hearing Impaired Children

The aim of this study was to determine the reliability and validity of the Latvian Sign Language Comprehension Test (LSLCT). Thirty-four deaf and hearing impaired (DHI) children (boys: 50%: girls: 50%) aged 8 to 12 years participated in the study. The children were attending specialist schools for DHI children and did not have any officially diagnosed developmental disorders. Item difficulty indices and item discrimination indices were established and the internal consistency and inter-rater reliability of the scale were assessed. The one-factor structure of the scale was verified. Teachers’ ratings of children’s sign language perception skills were used to determine the concurrent validity of the LSLCT. Construct validity was assessed on the basis of the correlations between LSLCT score and children’s age, grade, gender, parental hearing status, age at exposure to sign language and signing age. LSLCT was shown to have high reliability and construct validity.


Introduction
Sign language plays a vital role in everyday communication and learning for many deaf and hearing impaired (DHI) children whose hearing impairment prevents them from mastering spoken language without assistance. Learning to communicate via spoken language is especially difficult for children with a severe hearing impairment. For DHI children sign language serves as a means of acquiring information and communicating in academic and social environments. Any language learning can be promoted more effectively if it is possible to assess the current level of language development. But in Latvia there is no one test on the evaluation of the LSL skills. The assessment of DHI children' sign language could help us to understand deeper the factors that affect LSL development.
Since 2008 Latvia has screened newborns for hearing impairment, but before there was often a delay before a child's impairment was diagnosed. This meant that some of these children did not receive the necessary specialist training at the appropriate time and their language skills remained poor.
There are no official statistics on the prevalence of deafness and hearing impairments among children in Latvia, but in United States, for example, about 0.2-0.3% of children are born deaf or with a hearing impairment (National Institute on Deafness and Other Communication Disorders, 2014). Annual data from preventive health examinations of children are available in Latvia and these distinguish children with hearing impairments (The Centre for Disease Prevention and Control, 2015). In 2014 1109 of 236,291 children aged 3 to 14 years (0.5%) were found to have impaired hearing.
In 1997 Latvia passed the State Language Law, which included a sub-clause about the use of the Latvian Sign Language (LSL) in communication with DHI people. It also made provision for DHI people to have access to the services of an interpreter and for state funding of LSL development. To date there has been no research into the assessment, development or acquisition of the LSL and hence there is no established, reliable and valid instrument for assessing LSL skills.
In Latvia DHI children learn LSL at various ages and by various means. DHI children of deaf parents learn it from birth but DHI children of parents with normal hearing tend to learn it at a later age, when they start attending a specialist pre-school educational establishment for DHI children. Fluency in sign language varies in families, schools and pre-school establishments for DHI children. Only a small proportion of teachers who use LSL are DHI. The sign language skills of teachers with normal hearing are variable and there are not special professional demands for teachers' skills of LSL.
There are two specialist schools for DHI children in Latvia which offer elementary education over a period of 10 or 12 years. In one school the main learning media are LSL and signed Latvian but the other uses spoken language. The children in both schools mainly communicate with each other in sign language. There are also some schools which provide separate classes for DHI children. In the remaining schools DHI children learn alongside their peers with normal hearing. No statistics are available on the proportional distribution of these types of school.
There are no bilingual curriculum for DHI children in Latvia although many specialists working with DHI people consider this to be essential. The LSL is also not taught to these children specially and so children's LSL skills are very variable. There is an urgent need for valid, reliable instruments which can be used to determine the sign language skills of DHI children to facilitate further development in this area.
In creating the Latvian Sign Language Comprehension Test (LSLCT) we took into account both the course of language development in children with normal hearing and data from other countries on the sign language skills of children with hearing impairments at specific ages. We involved DHI specialists with a university degree in education who research LSL and whose native language was LSL in the development of the instrument.
In other countries the first sign language assessment tests were developed at the end of the 1980s (Mayberry, 1989) but thus far no test of LSL skills has been developed although the Latvian Association of the Deaf (LAD) Sign Language Centre specialists have been actively researching and improving LSL. There has been only study of the grammar of the LSL (Bethere, 2004). Haug (2008Haug ( , 2012 has reviewed the development of sign language assessment tests for the sign languages of various countries and concluded that there is still a global lack of sign language assessment tests for which reliability and validity data are available (Haug & Mann, 2008). The Sign Language Assessment home page provides information on some of the tests currently available. It shows that the majority of currently available sign language assessment tests apply to American sign language, there are also some for German, British and Dutch sign languages, and one test each is available for Italian, French and Australian sign language. Some of the tests were adapted from tests developed for other sign languages and some from tests for assessing spoken languages (Haug, 2008). The BSL Receptive Skills Test, (Herman et al., 1999) is unusual in that it has been adapted for use with the sign languages of several other countries including USA, Australia, Germany, Italy etc. (Allen & Enns, 2013;Haug, 2012;Johnston, 2004) although the French sign language version of the test is not recognised as valid (Haug, 2008). Adaptation and translation of language tests (including sign language tests) is problematic since languages differ in their grammatical and lexical structures and are to some extent culture-dependent.
A lot of instruments for sign language assessment were developed for a specific research purpose. But many of them are not established norms for DHH children' s population, partly because of their poor reliability and validity.

Literature Review
Language comprehension is usually defined as the ability to understand speech or written text (Davies, 1999). In order to understand speech and narrated or written text completely, it is necessary to comprehend the smallest language units as well as individual words and sentences and the principles governing their formation (Weir, 2005;Daneman & Merikle, 1996).
Some aspects of language acquisition are specific to particular languages but there are plenty of features of language acquisition which are common to the majority of languages, including sign languages. Children start learning language from birth. First they learn individual sounds, then syllables, words and sentences. This sequence and the approximate age at which particular language milestones are reached are similar for different languages. (Field, 2004;Woll, 1998;Kess, 1992). Children master English phonology by the age of about five to seven years. Morphology (plural forms, verb tenses (past, future), cases, derivatives) is learnt depending on its complexity, however, at the age of about five children can form morphologically and syntactically correct sentences and narratives. Between the ages of about three and five years children start forming narratives that are based on personal interest (Morgan, 2006). The episodes in these narratives are often not mutually and consecutively connected. Children begin forming sequential narratives including the main narration elements at the age of about five to six years. One indicator of language acquisition is the average number of words in sentences produced by the child (Field, 2004). At the age of seven most children produce sentences containing an average of seven to eight words.
Language acquisition proceeds through similar stages in DHI children whose native language is a sign language. Research into sign language acquisition in other countries has shown that DHI children first master the 'phonology' of the sign language i.e. they learn how to produce the hand movements and signs and the locations in which they should be formed (Woll, 1998). By the age of two to three years these children have acquired the morphology of their native sign language. At the age of about four most DHI children can form meaningful sequences of signs although they have not yet learnt simultaneously formed signs, which are characteristic of sign languages (Morgan, 2006). Between the ages of six and ten years DHI children continue to learn the sign language skills that are necessary to form and understand narratives (Morgan & Woll, 2002;Woll, 1998). At this point they have acquired the phonology and morphology skills needed to form separate sentences, however, they carry on learning how to form sequential narratives without losing track of the main idea and main actors. To summarise, at the age of seven years DHI children have largely acquired the phonological, morphological and syntactic rules of a sign language and are capable of understanding other people's narratives and forming logical, sequential narratives themselves.
Although there are no development studies of LSL acquisition in Latvia, the similarities in the course of acquisition of various other languages and sign languages suggest that LSL acquisition will follow a similar developmental pattern. It should be noted that this study deals with mastery of LSL as a native language.
Most of the sign language tests developed to date assess users' grasp of morphology syntax and semantics (Haug, 2008). Some tests focus on a very narrow aspect of the language, for instance, phonological skills whereas others evaluate broader skills such as the comprehension and production of a sign language.
Various types of test have been developed to evaluate sign language comprehension. Some assess comprehension of individual signs, some the understanding of phrases or sentences and yet others understanding of narratives. Some were developed as multiple-choice tests for children, e.g. the Computer Test for German Sign Language (Haug, 2011) assesses comprehension of individual sentences and narratives using a multiple-choice format. When developing multiple-choice tests it is important to take into account the probability of the respondent guessing the correct answer.
Other tests require children to provide answers to prepared comprehension questions about narration, e.g. the Assessment Instrument for Sign Language of the Netherlands (Hermans et al., 2010). In these tests the stimulus material (a sign language narrative) and questions testing comprehension of the narrative are usually presented via video recording. Several countries use video sign language comprehension tests; the tests differ with respect to the number of narratives and comprehension questions.
To summarise, sign language comprehension like spoken language comprehension can be assessed from various perspectives. Comprehension of individual morphemes and comprehension of narratives can be evaluated. A narrative integrates a broader aspect of comprehension: it includes the comprehension of both individual language units (in a sign language -signs) and sentences. A good way to test sign language comprehension is to present a video recording of a sign language narrative, followed by open-ended questions assessing semantic comprehension.
Following the requirements and conditions of a test construction, we decided to design a LSL comprehension test based on two sign language narratives. The LSL is an instructional language during educational process of DHI in Latvia. Therefore it is important to determine whether DHI could understand the instructions or explanation of teachers in academic environment. Or do these children comprehend any other narration in LSL in the daily communication.
We aimed to develop a one-factor test, based on general test construction principles (Kline, 2005). The test would consist of 14 short-answer questions on the two narratives with an increasing degree of item difficulty. The number of items would be sufficient to develop a test with sufficient high reliability. Short answers are provided for these questions, which would be indicative of children's comprehension of sign language. The test was intended for primary school children (age 8 -12) whose language skills are still developing. It was designed to provide an early assessment of the sign language skills of children so that if necessary, the school could intervene to improve their language skills.
The objective of this study is to determine the reliability (split-half reliability and inter-rater reliability) and concurrent validity and construct validity of the Latvian Sign Language Comprehension Test (LSLCT).
The research questions: 1. What are the psychometric properties of the LSLCT (the item difficulty and discrimination indices, split-half reliability and inter-rater reliability)? 2. What is the concurrent validity of the LSLCT and the construct relationships with other variables: children's gender, chronological age, parental hearing status, age of LSL exposure, signing age? Construct validity can be determined by considering the specific relationship of the measured trait to other traits that have already been found during the previous studies (Kaplan & Saccuzzo, 2001) such as age, gender, etc. In the case of DHI children the hearing status of the parents and the age at which the child was first exposed to sign language may also be relevant. The specificity of this population shall be noted that not all children start learning the sign language from their birth as it is with sharp of hearing children, therefore it is important to consider the fact how long the child has actually used the sign language or his/her signing age. Several studies have reported that the language skills of DHI children are closely linked to their age and their parents' hearing status (Maller et al., 1999;Strong & Prinz, 1997;Haug, 2012;Allen & Enns, 2013). Children with deaf parents and girls tend to have better sign language skills (Herman & Roy, 2006). Beal-Alvarez (2014) reported that receptive language abilities were highly correlated in younger participants (6 to 13 years) but were not correlated in an older group (14 to 22 years).

Participants
The study involved 34 DHI children (50% girls, 50% boys) aged 8 to 12 years (M = 10.03; SD = 1.66); 0.04% from these ages children population in Latvia -94 000, statistics of Ministry of Education and Science (2015). These children were attending specialist schools for DHI children and had been diagnosed as having moderate to profound hearing impairment i.e. their hearing impairment was such that they could not perceive or learn spoken language without special training. None of the children had been officially diagnosed with any developmental disorder. Table 1 reflects detailed information about the children. All the children were attending one of two schools which differ in children's distribution to grades. At one of the schools children learn Grade 1 material for two years and so there were children differing in age by up to three or four years who were at the same school grade.

Instrument
Latvian Sign Language Comprehension Test (LSLCT; Umbraško, 2013). The stimulus material for the LSLCT consists of two narratives: 'Library' and 'In Riga'. In order to ensure the variation in children's answers, the narratives were designed to be easily comprehensible for 11-to 12-year-old children with LSL as their native language but more difficult for younger children (8 to 9 year old) to understand. Each narrative consisted of 11 sentences. Sentence length was variable, being eight to nine signs per sentence on average. The narratives included different morphological features that primary schoolchildren were expected to understand: verb tenses (present, past, future), cases, numbers, spatial verbs, adverbs etc. These narratives used vocabulary that was expected to be easy for children to understand and dealt with scenarios of interest to children in the target age range. When the narratives and questions had been prepared one of DHI specialists whose native language was LSL presented them in LSL for the video recording. The recording was made in a special studio. The videotaped narrative 'Library' lasts for 100s and the recorded narrative 'In Riga' for 93s. Seven comprehension questions were prepared to accompany each narrative. The questions were designed so that children's answers could be scored on a three-point scale. Two points are given for a correct and comprehensive answer, one point for a largely correct answer and no points for an incorrect or illogical answer. For instance, one narrative states that Liga (L ga) lives in a small town near Riga. One of the questions is: "Where does Liga live?" A response which specifies both the size of the town and its location e.g. "In a small town near Riga" scores two points. An answer which specifies either size or location e.g. "In a small town" or "Near Riga" scores one point but the answer "In Riga" scores no points. The assessment criteria of children's answers was established by the team of experts.
Teacher's rating of child'LSL. To determine concurrent validity, teachers were asked to evaluate their pupils' sign language skills on a 1 to 10 scale (10 high). Each participant's LSL was assessed by at least two teachers (in most cases by three to five teachers) who worked with the participant on a daily basis. At least one of these teachers was DHI and had LSL as his or her native language.

Procedure
The parents of the children provided consent to their participation. Parents received a letter telling them about the study and asking if they would agree to their child's participation. The parents who provided consent received the questionnaire to obtain additional information about the child: parental hearing status, child's language abilities, communication mode etc. Table 1 profiles the participants.
Five LSL specialists were involved developing the narratives. Three were people born with a hearing impairment whose native language is LSL. They have a university degree in pedagogy and work with DHI children in nurseries and schools on a daily basis. One of the DHI specialists was studying sign language linguistics at Reykjavík University at the time of the study. Another of the specialists was a teacher of Latvian who developed a hearing impairment in adulthood but is fluent in LSL. Another of the specialists was a LSL interpreter and a sign language teacher with normal hearing. The team of specialists was designed to have a majority of members whose native language was LSL yet were fluent in Latvian and had experience of working with DHI children. Because this was the first study of LSL acquisition it was felt to be important to include in the team people who work with primary school age children on a daily basis and thus have an understanding of their grasp of sign language and would be able to advise how to structure the narratives so that they would be comprehensible to children in the target age range. In constructing the narratives the team took into account the sign language lexicon of children in the target age group and their expected knowledge of sign language phonology and morphology.
The children were tested individually in their schools, usually during scheduled teaching time (9:00 am to 3:00 pm). It took approximately 15 minutes to administer the LSLCT to a child. The LSLCT, which consisted of the video recordings of two signed narratives and accompanying questions, was presented to children on a portable computer. The children watched each video recording once. None of the children wanted to watch the recordings more than once. After watching the narratives children watched the questions, answering each question immediately after it had been presented. The children answered in whatever language was most comfortable for them, LSL, signed Latvian or spoken language.
Children's answers were recorded in a protocol; because most answers were short and easy to understand it was not necessary to video them. Use of LSL grammar in responses was not taken into account; we were evaluating child's sign language comprehension skills and not their production skills.

Item Difficulty and Item Discrimination Indices for the LSLCT
The item difficulty index is calculated as the item mean score (Kline, 2005). The higher an item's difficulty index, the easier that item is. The discrimination index reflects the correlation between score on the item in question and total score for the test (Kaplan & Saccuzzo, 2001). Some researchers recommended use corrected item-total correlation (e.g. Ferketich, 1991). It is the correlation between each item and a scale score that excludes that item (uses all the other items, but not that one). According to literature, the corrected item-total correlation should be > 0.3 (Maltby et al., 2010). Table 2 shows that the item difficulty indices of LSLCT range from .29 to 1.03 (optimum limits 0.20 to 1.6) with the mean of item difficulty indices .64. It is lower than optimum mean level for this test items -1.00 (Aiken, 2000). All corrected item-total correlations are higher than .52 with mean value .63 (see Table2). Bartlett's Test of Sphericity: 2 = 224.07, p < .001 ** p < .0; Note: n = 34. Items no. 1-7: the items of the first signed narrative ("Riga"); no. 9-14: the items of the second signed narrative ("Library").

Reliability of the LSLCT
The split-half reliability coefficient calculated as the correlations between two parts of the test usually choose to determine the reliability for ability tests (Kline, 2000). The Spearman-Brown coefficient for LSLCT is .92, it meets the high level (DeVellis, 2003).
Inter-rater reliability is another important indicator of the reliability of test items in. It is expressed as the correlation between the scores given by two independent evaluators for responses to a given test item (Aiken, 2000). The raters assessed all participants' answers using the assessment criteria, prepared by DHI specialists beforehand. Table 2 shows that the inter-rater reliability coefficients were statistically significant and very high for all items (from .76 to 1.00, all ps < .01). The mean correlation coefficient was .88.
One-dimensional tests are usually more reliable than multidimensional tests. Principal component analysis was used to determine the factorial structure of the LSLCT. The results showed that all items belong to one component, which accounts for 49% of the variance in test scores. Kaiser-Meyer-Olkin Measure of Sampling Adequacy is .80 (should be greater than .70), Bartlett's Test of Sphericity is significant: 2 = 224.07, p < .001 (see Table 2).

LSLCT Validity
Convergent validity, which is a form of construct validity reflects the link between a given tests and other established tests measuring the same construct (Davies, 1999). Because there are no other instruments developed to assess LSL skills, LSLST scores were correlated with teachers' ratings of children' sign language skills to determine concurrent validity of the LSLCT. Table 3 shows teachers rated the LSL of the majority of respondents (n = 29) between good and outstanding (7 to 10 points). Children's scores on the LSLCT were highly correlated with teachers' ratings of their LSL skills (r = .75; p < .05). To assess the construct validity of the LSLCT we considered the correlation between children's scores on the test and their chronological age, gender, parental hearing status, age on exposure to sign language and signing age. LSLCT scores significantly and quite closely correlate with the signing age (r = .64, p < .01), parental hearing status (r = .53, p < .01), age on exposure to sign language (r = -.49, p < .01), the grade (r = .50, p < .01), and medium correlate with gender (r = -.38, p < .05). There is no significantly correlation with child's age.

Discussion
The aim of this study was to determine the reliability and validity of a newly constructed instrument for assessing comprehension of Latvian sign language, the LSLCT. The LSLCT is the test designed to assess the Latvian sign language receptive skills of DHI children. The LSLCT was found to have high split-half reliability and inter-rater reliability. It was also shown to have construct validity and concurrent validity. Although the test items had high reliability and good response variation they proved too difficult for the target age group; almost all items are below the optimum average value (1.00) (Aiken, 2000). In fact none of the items could be classed as 'easy' for our sample and most were classed as difficult, indicating that the children in our sample found two stories of the test difficult to comprehend when they were presented in LSL. Wherequestions and What-questions proved to be the most difficult. Correct responses depended on having a grasp of sign language syntax and the sequence of signs. Or it is possible that this was because it depended on understanding the longest sentences in the narrative. Incorrect answers may have occurred because children did not remember the story accurately and confused, for example, the animals mentioned in the two contexts. There were also many incorrect but logical answers to the questions. This suggests to us that sometimes children of this age do not base their answers on the narrative which has been presented, despite explicit instructions to do so; instead they offer what seem to them to be plausible answers to the questions. We conclude that the test is complicated enough for the target age group (8-to 12year-olds) and might also be suitable for older children.
Children's LSLCT scores were highly correlated with the teachers' assessments of their sign language comprehension skills. Although we did not assess the teachers' knowledge of sign language it seems plausible that they were capable of assessing the children's LSL skills.
Having verified other aspects of construct validity, we concluded that sign language comprehension was closely related to children's signing age, age on exposure to sign language, parental hearing status (children with deaf parents had better sign language skills) and grade but not so close with gender (girls LSL skills were better). Children who started learning the sign language at an early age and children with a higher signing age also achieved better scores on the LSLCT. These results are consistent with those of previous studies in other countries (Hermans et al., 2010;Herman & Roy, 2006;Haug, 2011), with the exception that LSLCT score was not correlated with age in months, as it was in other studies. This suggests that sign language development in DHI children in Latvia is not directly related to age. As the majority of Latvian DHI children live in boarding schools instead of with their families during school terms one would expect the school environment to have more impact on their language development than the family environment. These children mainly communicate with their teachers and classmates and we therefore assume that their language skills are more strongly related to their environment than their chronological age. This would explain why LSLCT score was correlated with school grade rather than with age.
Another possible reason why we did not detect a correlation between sign language comprehension and chronological age is that not all children learn LSL at an early age. Some of the participants in the study had only started learning sign language when they started school i.e. aged six or seven years, and their duration of exposure to LSL (signing age) is very short. Therefore the signing age of the children who come from DHI parent families and who have learnt the SL since birth, for instance, at the age of eight is eight years, whereas the signing age of the children, who start learning the sign language at school, at the age of eight is only one to two years. We found a strong association between sign language comprehension and signing age Although we did not observe a strong correlation between sign language comprehension and chronological age in our sample, presumably owing to the fact that some DHI children do not start to learn sign language until relatively late, our general conclusion is that construct validity of the test is sufficiently high.
The convergent validity of the LSLCT should be assessed in future research by comparing performance on the test with performance on a similar sign language test. The small sample is another important limitation of the study; it was due the small number of DHI children aged 8 to 12 years in Latvia. The sample was further restricted because we excluded children diagnosed with developmental disorders. The sample size was also influence by the fact that parents had to give permission for their child to take part. As this is one of the first studies of the population of DHI children in Latvia some parents may have been reluctant to allow their children to take part. About 25% of eligible children were excluded because their parents refused consent.
The values for the LSLCT with respect to the various reliability and validity indicators meet the criteria for a reliable and valid instrument. This study suggests that the LSLCT has good psychometric properties, but it has some limitations. First, the small sample. These findings should be replicated in a larger sample covering a wider age range to provide more evidence for the external validity of the test. Another LSL test should be constructed to enable the convergent validity of the LSLCT to be assessed. Other aspects of the validity of the LSLCT should also be assessed. Further studies could examine the comprehension skills of separate sentence of LSL or to check the vocabulary of LSL. Thus to determine the relationship between these variables and LSL comprehension skills.

Conclusion
The LSLCT, which was developed to assess sign language comprehension in DHI children, has good reliability and validity and is suitable for use in research on LSL and the assessment of DHI children' LSL comprehension after norms establishing.