Assessments

Description

The C&T 898 1D-Assessments artifact was the final project for C&T 821 Evaluation and it was titled Four Modalities of Cherokee Proficiency in Testing. The artifact was a description and review of a battery of tests that were performed over the coursework of the class. These four tests were focussed on the four modalities of listening, speaking, reading, and writing. The test subject was Cherokee language learner and an English L1 speaker called Chickadee in the artifact. Chickadee is described as the test taker throughout the artifact for C&T 821, and the background as well as the reasoning for her being chosen as the test subject. She is indicative of the type of student that we work with in the adult language immersion programs sponsored by the Cherokee Nation. The artifact described her educational, social, and cultural background as a preference to explain the decision to choose her as the test taker for the final project.

The artifact described the testing battery as being developed from the standards of American Council on the Teaching of Foreign Languages (ACTFL) NCSSFL-ACTFL (2015), “I can” but those original ACTFL standards were initially developed for a learner’s self assessment. Although Chickadee did take the self assessments in the four modalities covered by the tests in the artifact, the standards were used to develop the four tests that she took for the purposes of the final project. These four tests were administered by the same person that developed and scored the tests. The ACTFL proficiency levels from Novice low, mid, and high, through Intermediate low, mid, and high as well as Advanced low, mid, and high reached up into the highest proficiency levels of Superior followed by Distinguished (NCSSFL-ACTFL, 2015). Chickadee’s limited experience allowed for the lower proficiency ranges within both Novice and Intermediate to be the focus of the battery of tests. After she took the self assessment pretests, the proficiency ranges for the proficiency tests in the four batteries were finalized to assess Novice and Intermediate level proficiencies in speaking, listening, reading, and writing.

The modalities used for the test batteries were specifically designed to isolate each communicative modality. The first test was a pure test for interpretive listening. The second was a productive speaking skill addressing two aspects of the speaking modality, responsive speaking and directed response as described by (Brown and Abeywickrama, 2010, pp. 189 - 202); responsive speaking tests elicited responses to evaluate interpersonal communication, and directed response evaluations assessed presentational speaking. The third test, another pure test, focussed on the receptive reading skill, as Brown and Abeywickrama (2010, pp. 224 - 225) explained, skills associated with grapheme and phonological decoding as well as lexical recognition were important for emergent readers, like Chickadee. These skills were assessed to determine how well she could utilize the orthography, morphology, and lexical content of Cherokee within a short sample of language (Brown and Abeywickrama, 2010, p. 228). This test was described using italics translations in English to denote the original Cherokee language content and orthography. The final test was developed to select for the productive writing skill. This skill was based on four types of writing performances for testing purposes “Imitative,” “Intensive (controlled),” “Responsive,” and “Extensive” (pp. 261 - 261). Imitative writing evaluation assessed the accuracy of the language orthography’s reproduction. Intensive evaluation focussed on the test taker’s accuracy in orthography, morphology, and lexical content of Cherokee within context (Brown and Abeywickrama, 2010, p. 261). The last two categories, Responsive and Extensive writing tasks were beyond the skill set of Chickadee, an emergent reader and a novice to written Cherokee. These skills dealt with much longer discourse level writing forms and were related to rhetorical and stylistic strategies in paragraphs or essays (Brown and Abeywickrama, 2010, pp. 261 - 262). The original criteria used to create these questions was modified to meet the needs of the Cherokee language and culture. In some of the artifact’s test examples the criteria were kept separate from the test questions, and the test taker was not privy to the rubric. However in other examples, the criteria were incorporated into the test in parenthesis.

These four tests were described in detail within the artifact as samples of each were also provided. An analysis of Chickadee’s responses, and the score comparisons between the battery of tests that were designed to evaluate her proficiencies in the four modalities and her previous self evaluations on those same skills were used to help gauge the validity of the tests for listening, speaking, reading, and writing. The artifact suggested improvement to the tests, and the challenges in making the tests were described as being a result from the difficulty in translating the English skills into corresponding and comparatively difficult examples in Cherokee language grammar in context for the tests. The pure test specification skills that were developed by this process for the artifact helped to differentiate productive skills from receptive skills. The artifact described the greatest challenges as creating pure tests for both reading and listening modalities. Specifically, the process of isolating the receptive modalities were quite challenging and required a lot of creativity. The use of the test taker’s L1, English in all cases to ensure complete understanding of the tests' rubric and assessment competencies was a necessity to ensure the validity of each test. A more holistic assessment of these modalities would allow for a greater ability to assess the overall competency level, but at the cost of any one specific skill assessment in each modality. A test that would allow the use of multiple modalities to assess overall proficiency would allow a more authentic assessment of skills. Syntactic and discursive examples used in the artifact’s tests were aimed to be accurate reflections of authentic Cherokee language and context. The questions and prompts were targeted for the hypothesized proficiency level for assessment, and were restricted by the linguistic examples used in application of the test with simple sentences, short phrases, and low level lexical and morphological examples. The testing prompts and questions limited the authenticity of the context. A more integrated test, via multiple modalities would allow for the testing to be more linguistically natural and accurate to reproduce authentic Cherokee language communication.

Rationale

The rationale describes how completing the artifact has impacted the candidate’s thoughts, processes, or actions. The rationale is informed by the candidate’s program, completed readings, and/or relevant literature. The primary goal of assessments in the language programs at Cherokee Nation are to measure growth in the learners’ proficiency and are used to some degree for summative assessments at milestones within the language programs but primarily as formative assessments to help the learners and their instructors determine what needs to be taught in language sessions.

The artifact was selected because it demonstrated how assessments can be used to track and guide language proficiency growth using a target language adjusting the linguistic pattens of Cherokee using the rubric modified from the American Council on the Teaching of Foreign Languages (ACTFL) normal language standards. Cherokee is incredibly polysynthetic and there is limited understanding on how to teach polysynthetic languages, especially through literacy. Our Cherokee Nation programs have struggled with assessment tools for decades to accurately assess the second language Cherokee learner’s literacy development. Also the artifact is riddled with mistakes and suggestions about how to make a better test in subsequent efforts, so it provides other assessment writers with ideas about what worked well and why, as well as what didn’t and how those tests could have been improved, and further how teaching and learning can be made more effective.

ACTFL proficiency standards were chosen to be used for assessment purposes in the artifacts testing. The proctor and the test taker spoke in depth about using the ACTFL rubric modified for Cherokee language and agreed that the rubric reflected the sort of skills that should be assessed on a Cherokee language proficiency assessment, and further that this correlation may better help the use this kind of rubric in the future for further comparative Cherokee language proficiency assessments. In the artifact NCSSFL-ACTFL (2015), “I can” document, was primarily selected because it used explicit details in the self assessment tool to determine key aspects of language proficiency, it really broke down the specifics of each proficiency benchmark with real world examples of what skills or proficiency standards looked like from Novice low, the most beginning level of language learning to Distinguished, the most advanced.

Performance Based

According to Brown and Abeywickrama (2010), Performance-based assessments call on critical thinking that is necessary for a more conversational approach, and that they usually focus on oral production but also incorporate listening, reading, and writing modalities in a blending of formal and informal assessment (p. 16). The formal aspect of this assessment process is important because the rubric in the artifact’s test process gave clear and detailed accounts of the standards used to create the assessments as well as how those standards were applied to the interpretation of the assessment’s validity. Another assessment process that Brown and Abeywickrama (2010) outlined was the diagnostic test; to diagnose the proficiency of the learner's language with regard to a specific set of skills. The testing example used both assessment processes, and this allowed the artifact to further showcase the processes used to review the test taker’s proficiency level. In the artifact, the tests provided feedback through direct evaluation related to specific skill needs, or real life criteria for each modality.

By targeting the tests to specific proficiency ranges it was easier to cut down on the subjectivity and bias in the test making process. By trying to create a test that is valid, it was clear that bias and subjectivity slowly lessened as experience was gained in the process of creating tests and reading the academic articles and course materials in the program. The second version of the listening test gave evidence to that fact. The receptive listening and reading skills were easier to assess with a higher level of objectivity, because the tester was only required to respond to what they understood, but the productive skills of speaking and writing did lend themselves to more challenges. Bias or subjectivity came most prominently into play when the criteria for the proficiency levels of my testing questions were being determined. The proficiency levels on the questions created for some of the tests were over or underestimated in accuracy when it came to interpretation from English to Cherokee.

Diaz (2013) explained that the focus on a learner's humanity and the fact that this value should not be overlooked in instruction or assessment was a point of egress into allowing the test taker into the testing process. This can be supported with self testing when it comes to language proficiency, but it also is important to support their emotional needs in any learning environment whenever applicable. Krashen (1982[2009]) explained that emotional response is a potentially crippling factor in the process of language acquisition. This appears as an affective filter that compromises communicative competence, allowing communicative breakdown to occur in any language learning environment. Even allowing some of the learners first language into evaluation and instructional processes to lower the affective filter is reasonable, with some personal struggle. The struggle to use English, the test taker’s L1 in the artifact was not provided the description it deserved.

Revitalization

In many indigenous language revitalization efforts the focus on language immersion has become so popular that an expectation of strict adherence to this factor is after a given. This can be good if it is something that is valued by the learners and teachers alike, as it helps but focus and motivation to advance the target language as a primary goal in all language learning sessions. If the learners are motivated to use the target language, in an example, heritage languages like the test taker’s in the artifact, then often the students appreciate an effort to maintain an immersion environment to maintain momentum and a sense of commitment to the endangered language. If it can be achieved, and immersion with endangered languages is an explicit goal within the context of the learning environment, then the learners themselves will often push for rigor in that regard.

Endangered Indigenous languages are propagated in a context of ever present awareness of language loss. Restricting language learning environments to the target language in such cases seemed only natural to Cherokee language teachers and program staff for the last decade. This was something that has been called into question while working with the testing process described in the artifact. There was a clear benefit to using the test taker’s L1 in ensuring comprehension for test validity. In general immersion in the target language is preferable especially for indigenous languages that are endangered. In most examples, language acquisition serves any number of multiple purposes, including survival. However, for all languages, the learners’ needs must come first. When it comes to language learning, a student’s L1 is the home language, it is like a piece of their home, a piece of their lives that lives within their L1. They can retreat to the comfort of that language for any number of purposes, from cognitive function, to emotional comfort, but for some Indigenous language learners, including Chickadee, their L1 is more like a constraint than a benefit. When she was interviewed for the artifact in October of 2018, as a part of my class project in Curriculum and Instruction 820, it was expressed that she felt the need to use English was a form of sanction in itself. She felt that her inability to function in the target language at the same level as her L1 was a handicap. Chickadee said she valued her English proficiency as a tool, but for her it was not a home. She felt it was not a safe space or a place of emotional comfort or cognitive support, it was the result of intentional displacement of her true native (Indigenous) language.

For people that can use their L1 as an intellectual tool to acquire their target language without emotional distress, then it is fundamentally valuable, especially in high stress situations of assessment. It aids in tying the L2 to their life-experience and identity through the context of their L1 worldview and etymological cosmology. It can be a stalwart tool to build confidence, self esteem, and an intellectual understanding of the target language. When adult Cherokee learners choose to acquire their L2 as a target language, they know it is their Indigenous language. In the process of becoming interested in Cherokee, they have already intimately come to understand the tie between Cherokee language and Cherokee identity through culture, and they know that without that language the abstract “deep culture” from Cherokee thinking, cosmological and etymological thinking is not reachable. Cherokee L1 elders can’t express why, they just say, “you won’t understand it if I tell you in Cherokee, and I can’t say it in English”. This means that without the Cherokee language as a medium for abstract thought, our core values and beliefs are lost to us. There is no place other than our intentional Cherokee language communities, classrooms and intentional immersion environments to hear the language in context. It does not exist anywhere else in the universe. We cannot go to the homeland to become immersed. The elders that speak it are individuals, islands surrounded by English, that cannot interact in their language without leaving their homes or picking up their phones. Learners cannot go somewhere to get lost in a Cherokee world. It was gone when they were born. The youngest Cherokee speakers are already grandparents. Even those that spoke it as children have lost it in boarding or public school systems. Cherokee immersion environments are the only habitats that foster the cultivation of the target language, and elders so rarely use Cherokee that they are challenged by the effort to stay in the language. When English is spoken, for just a moment, in session, a dam breaks and English floods the room. It engulfs the entire moment, as they struggle to hold back the tide. Leaders in Cherokee language programs ask the speakers to stay in the target language, and demand that the learners respond to them and address them in Cherokee. The learners whisper among themselves in English and they will also whisper an English translation to help save one of their fellow learners when they are caught in the spotlight, but the goal is a focus on the target language. Cherokee language programs seek a Cherokee learning space, but English always happens. So when the teachers say, Ꮭ ᏲᏁᎦ (no English) they are begging for a place to keep Cherokee language alive. It is known that learners will use English as they may, as they need to do so, they can’t help it, and it is supported covertly. The Cherokee language programs can’t force anyone to speak Cherokee, it’s not the Cherokee way, not in the traditional values system. However, often, when English speaking Cherokees come into the program as learners, they often ask to be fed only Cherokee, because they have had their fill of English. However, even strong supporters of immersion for indigenous languages can find themselves overwhelmed when their affective filter is aroused to the point that conversational capacity is diminished (Krashen, 1982[2009]) and testing further raises the emotional sensitivity of learners.

Conflict and Support

In Moll, Amanti, Neff, & Gonzalez's "Funds of Knowledge for Teaching" (1992), they explored how to integrate the teacher into the process of ethnographic research by developing research teams to integrate new teaching pedagogy using the Funds of Knowledge teaching methodology. Then in another article, Cathy Amanti, one of the teachers in the previous article, writes in "Beyond Beads and Feathers Approach" (2005) an in depth exploration of the same methodology when she examines how she put the knowledge she had acquired into practice using the Funds of Knowledge technique with her students. These two articles, with supporting ideas from Chapter 14, by Diaz-Rico, and Mari Heneda's article "Becoming Literate in a Second Language" (2010) both deepened my understanding of how to use relationships with learners, that focus on home, family, and culture to enrich the value of learning for students and enhance their buy in and participation in the educational process. In the spirit of honoring the target language and respecting the L2 learns and test takers in their desire to revitalize their Indigenous endangered language, it is vital to both ensure that they are being supported in their Cherokee language learning goals, but it is equally important that we utilize their L1, English to support their funds of knowledge backgrounds and previous educational experiences, especially with assessment. These two contradictory goals can create no small amount of conflict in the learners and the language environment.

In Velasco and Garcia (2014) they pointed out that true bilinguals don’t just have two separate languages but have one translanguage that includes the cumulative knowledge of all of their languages, but use that language knowledge to target one language or another (pp. 7 & 8). Aside from their idiosyncratic translanguage that they grow as they learn, bilinguals have a comprehensive understanding of all the language that they know and they can use that language knowledge in context to select for and communicate in one language or the other, but they naturally learn by incorporating elements of both languages (Velasco & Garcia, 2014, p. 8). The use of Chickadee’s L1 and while maintaining a language environment respectful to Cherokee, the target language allowed her to use her L1 skills and background to assess and strengthen her Cherokee language growth. Despite the challenge, it is important to ensure that every tool available is used to reach the educational goal for each context. When the proctor spoke with the test taker in English, permission to speak English was always asked and granted, in an effort to respect the Cherokee learning environment. This is a new change in approach. In the past all efforts to maintain the target language came first, even at the cost of the L2 learners' affective filter challenges and comfort levels. The shift in philosophy is a fundamental change and was effective for the test taker in the artifact as well as subsequent Cherokee language program participants.

References

Amanti, C. (2005). Beyond a beads and feathers approach. In Funds of knowledge: Theorizing practices in households, communities,

and classrooms (pp. 131-141)

Brown, H. D., & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices (2nd ed.). White Plains, NY:

Pearson Education.

Chang, A. & Read, J. (2006). The effects of listening support on the listening performance of EFL learners. TESOL quarterly, 375-397.

Dίaz-Rico, L. (2013). Chapters 3 & 14. Views of teaching and learning. In Strategies for teaching English learners (3rd ed.). Boston:

Pearson.

Krashen, S. (1982[2009]). Principles and Practice in Second Language Acquisition. Pergamon Press, Inc. Internet edition.

Moll, L., Amanti, C., Neff, D., & Gonzalez, N. (1992). Funds of knowledge for teaching: Using qualitative approach to connect homes

and classrooms. Theory Into Practice, 31(2), 132-141.

NCSSFL-ACTFL. (2015). Can-do statements performance indicators for learners (2nd ed.). Alexandria, VA: Retrieved from:

www.actfl.org

Velasco, Patricia & García, Ofelia. (2014). Translanguaging and the Writing of Bilingual Learners. Bilingual Research Journal. 37. 6-23.

10.1080/15235882.2014.893270.