Q. What about NOELLA? How is that different from CAP/STAMP?
show answer
A. NOELLA is a proficiency-oriented test for students in grades three through six.
It is based on the same general principles as STAMP, but geared for younger children.
Q. Isn't the STAMP test developed with federal grant money?
How can you charge for a test that is essentially developed with taxpayer money?
show answer
A. Grant funding provides CASLS with the opportunity to research and develop language tests such as STAMP.
However, maintaining a complex, computer-based assessment is a costly endeavor that federal funding would not support.
CASLS licenses its technology and content to Avant Assessment through the University of Oregon Office of Technology Transfer.
This partnership is the best way to keep STAMP available after grant funding concludes.
Avant Assessment has made their own contributions to STAMP: innovative score reporting capabilities and raters in different languages to grade students' speaking and writing.
CASLS will always make the assessments available free of charge during the grant-funded development phase.
Q. How is CAP/STAMP related to ACTFL?
show answer
A. The American Council on the Teaching of Foreign Languages (ACTFL) is a national organization dedicated to the improvement of language teaching and learning.
CASLS is one of fifteen federally funded Language Resource Centers (LRCs) also dedicated to promoting the teaching and learning of foreign languages in the U.S.
There is no formal relationship between CASLS and ACTFL, although many CASLS staff members are also ACTFL members.
CASLS' assessment benchmarks and item specifications are based on the ACTFL Proficiency Guidelines, which many states also incorporate into their world language standards.
Q. What happened to the benchmarks? I can't seem to find them.
show answer
A. STAMP, which is currently available through Avant Assessment, was based on descriptions of proficiency called benchmarks.
These benchmarks described the types of tasks that students should master in order to achieve overall proficiency.
Reanalysis of data from our tests has shown that the benchmark levels do not always reflect the empirical difficulty of test items and that benchmarks alone are insufficient as a construct definition for assessment.
Many features of the benchmarks have been incorporated into our new
test and
task definitions.
Q. Why are the test items in English?
Research has shown that target language input is vital for language acquisition.
Why not have the entire test in the target language to better match "best practices" in language teaching?
show answer
A. Keeping the test questions in English helps us ensure that we are testing what we want to measure.
On the reading test, for example, we are testing students' ability to read a particular target-language passage.
If the test question is in the target language as well and the student answers incorrectly, we won't know if the student failed to understand the passage or the question.
Especially at the lower levels, reading the question in the target language could actually be more difficult than reading the passage itself.
Please note that CASLS' assessments assume a sufficient level of English literacy.
CAP is not appropriate for students who do not have a reasonable command of English.
Q. Why is there no video for the listening section?
We use visual information to help understand spoken messages in real life, so aren't we making the test difficult if we deny students that visual information?
show answer
A. In the past, we have piloted full-motion video for the listening section.
We found, however, that using full-motion video required too much bandwidth, and many schools had technical trouble when testing large groups of students.
We also found that the video would occasionally give away the answers to the test questions, so students could answer correctly without understanding what the speaker said.
For these reasons, audio-only items provide a better measurement of listening proficiency.
(For recent research on this issue, please see Ockey, G. [2007].
Construct implications of including still image or video in computer-based listening tests.
Language Testing, 24, 517-537.)
Q. Why is the audio played only once for the listening questions?
My students need to hear something several times before they understand it.
show answer
A. The number of times to play an audio passage has always been a somewhat contentious issue in language testing.
Since real-world listening takes place in real time, we want our test to mimic that situation.
For this reason, we only allow students to hear the audio once per question.
At the same time, we want the test to measure listening, not memory, so we give students the option of playing the audio once for each additional question.
Q. Who makes the items on the test?
show answer
A. We develop our tests in teams, and many people review each item before it appears on the final version of a test.
In many instances, language instructors or university professors initially create items that CASLS staff then review and edit.
NOELLA and some CAP tests used this process.
In other cases, CASLS directly employs native or near-native speakers of the target language to find authentic materials.
CASLS' test developers then write and edit the questions.
For information about the process used for a particular language, please consult the technical manual for the test.
Technical manuals will be available here as they become available.
Q. I don't think this test works.
I gave the test to my students, but some of my best students didn't score as well as some of my worst students.
Is there something wrong with the test?
show answer
A. First, CAP is in the development phase, and the scores shouldn't be interpreted as "official."
There may be some problematic test items that are contributing to the strange results.
CASLS uses information from the pilot process, as well as input from external language professionals, to make decisions about which items to include in the final version of the test.
If you are comparing percentage correct scores, look at the raw number of items taken as well as the percentage score.
A student who takes one item and gets it correct will score 100%, though we would not consider this evidence of proficiency.
Our tests are not an indication of achievement in a particular class.
Coming to class on time, always completing assignments, and working well in groups, for example, are not necessarily correlated with higher language proficiency.
A student who has some experience living in a place where the language is spoken may do well on the test even though they do not participate appropriately in your class.
Q. I want to use this test for placement purposes. Can you recommend an appropriate score for a 200-level language class?
show answer
A. Placement involves selecting the most appropriate course from course offerings for a particular student.
Because each program has unique goals and course contents, people directly involved in the program should determine context-appropriate placement procedures.
Q. Why should I use STAMP instead of the ACTFL OPI? What makes STAMP better?
show answer
A. A more appropriate question to ask is "Which test will best suit my purpose?".
Different tests have different uses and strengths.
Depending on your situation and needs, one test might be more appropriate than another.
Just as you wouldn't want to play an entire round of golf with only a putter, you shouldn't depend on one single test to fulfill all of the diverse assessment needs in a language program.
CASLS designed STAMP to help language teachers and programs evaluate the proficiency of their students for the purpose of improving instruction.
Q. My program gave tests in several languages, and my students did not do as well as students in some of the other languages. Are the tests for some languages more difficult than others?
show answer
A. The answer depends partially on how you define "difficult."
For English-speaking students, learning to read Hindi is more difficult than Spanish, because reading Hindi involves learning a new alphabet.
By the same token, learning to read Chinese is more difficult than Hindi, because Chinese is logographic and Hindi is alphabetic.
Because of this difference, the same language task, such as "reading a simple menu," will be harder in Chinese than in Spanish given that a native-English-speaking student needs more time to gain the language ability necessary to complete the task.
In this sense, some of the tests are "harder" than others for students with the same amount of seat-time in the language.
However, CASLS builds all of the tests to the same test and task specifications.
Tasks at the same proficiency level should be more or less comparable across languages.
In other words, we don't ask beginning-level Spanish students to read novels and beginning-level Chinese students to read menus just because Spanish is easier.
Students should be able to perform roughly the same real-world tasks at the same proficiency levels, even though students will need more time to reach those levels in some languages.
Q. Some of the items on the test do not seem authentic to me. We would never see something like that in my country. How can you claim that you are testing authentic language?
show answer
A. Our goal is to keep our test materials as authentic as possible while maximizing our ability to feasibly produce useful test material.
However, authenticity is a slippery concept, especially in situations in which a language is widely spoken in many different countries (such as Spanish or Arabic).
What is common in one area might be very rare in another.
CASLS strives to contextualize the items in realistic situations (e.g., reading a train schedule, looking at an advertisement, listening to a radio announcement).
Wherever possible, we try to base our situations around authentic or semi-authentic materials.
At the same time, we want to sample a range of language on the test, which sometimes entails creating our own materials.
In addition, we want to be sure that we test language, not knowledge of a particular culture.
For this reason, we may need to adapt even authentic materials so that students who have not had the chance to study abroad will not be unduly penalized.
Q. Why do you use multiple-choice questions for reading and listening items? Students can guess the correct answer.
show answer
A. There are benefits and limitations in any testing format.
A computer can automatically score multiple-choice questions as students take the test, which allows the test to adapt to the student's ability.
Although students can guess the correct answer, they will not be likely to "ace" the test just by guessing.
For this reason, longer tests tend to be more reliable than shorter ones.
CASLS is continuing to investigate new technologies and scoring options.
At the moment, multiple-choice testing gives us the most efficient way to deliver listening and reading tests.
Q. What does does an adaptive test mean? How does this work?
show answer
A. An adaptive test is one in which students receive a harder or easier item based on their response to the previous item.
If students correctly answer an easy item, they will receive a harder item next.
The most extreme form of this would be an "item" adaptive test in which the test adapts after each item.
The current CAP and NOELLA algorithm is based on "bins" (or "testlets") of several items.
Rather than adapting after each item, the test adapts after a series of items.
The technical documentation for each test will describe the particular algorithm for that test.
Some of CASLS' tests in the development phase may be fixed-form tests.
Questions About Pilot Test Registration
Q. Why should I participate in pilot testing?
show answer
A We believe that online assessment represents the future of testing.
By participating in this pilot, you and your students will experience state-of-the-art assessment technology free of charge.
Participation in the pilot will also help improve your students' test-taking skills and help them prepare for future online assessments.
CASLS conducts pilot to collect data from students at each proficiency level in order to offer the best possible language assessment tool to educators.
However, we will only be able to achieve this with your help and participation.
Q. What should I do next to participate in the pilot?
show answer
A. We're delighted that you are considering participating in an assessment pilot. To participate:
- Read this FAQ carefully. Contact us if you have any questions.
- Take the technology check to ensure that your school has the necessary infrastructure to support online testing.
- Register for the pilot test.
You can view a short screencast about registering.
- Once you have registered, a pilot coordinator will contact you to complete the registration process.
- You will be able to schedule your test dates after confirming human subjects compliance. Your test date will be confirmed when we send you a teacher login and a class login.
Q. What kind of human subjects procedures are in place? Does CASLS have permission from anyone to pilot the test??
show answer
A. As part of the University of Oregon, CASLS is required to follow certain procedures set by the university's
Office for Protection of Human Subjects.
CASLS annually updates its protocol.
Informed consent is one of the core human subjects concepts.
Test takers eighteen and older can give consent themselves.
For test takers under eighteen, we require passive consent of the parents.
Passive consent means that parents are informed that their children will be taking the test and are given the opportunity to opt out.
The pilot coordinator will send more information about this procedure once you have registered for the test.
Q. I want to take the test. What is the login password?
show answer
Q. Can teachers who want to preview the test get access? What about people who aren't in a language class, but want to take the test on their own.
How do they get a login password?
show answer
A. To protect test security, CASLS only assigns logins to teachers who have registered to take the test.
We sometimes grant access to language professionals in exchange for review and feedback.
Because our login system is based on classes and not individuals, we are unable to provide logins for individuals.
If you have specific questions about the test that are not answered in this FAQ, feel free to contact us.
Q. What are the optimal testing conditions?
show answer
A. We ask that teachers help us to ensure the quality of our test results by providing a controlled test setting.
Only one student should be assigned to each computer workstation during the test.
Q. How many questions will each student have to answer?
show answer
A. Note: The following information applies to various pilot tests and not all skills will be available in all languages.
Part of the pilot process involves finding the optimum number of questions to maximize useful information, but minimize test time.
For this reason, reading, listening, and grammar tests may contain more questions than students can answer in a typical class period.
Students taking a speaking or writing test will typically answer four questions.
There is no penalty for not completing all of the questions.
A student may receive two questions that, at first glance, look exactly alike.
Although the context and the image may be the same from screen to screen, a different question will be asked on the second one.
Students should be made aware of this so they don't disregard a question thinking they've already completed it.
NOELLA tests usually have about thirty items per skill for listening and reading with two prompts for speaking and writing.
CAP tests have between thirty and sixty items per skill and four prompts for speaking and writing.
Q. How long does the test take?
show answer
A. Our tests are NOT timed tests!
The amount of time necessary to complete the test is different for each student, depending on how quickly or slowly he or she proceeds.
Some students may finish
CAP for a single skill in a fifty-minute class period.
Students may finish
NOELLA in a twenty-minute class period for one set of skills.
In our experience, students perform at their best during the first hour of testing.
We recommend that you plan no more than sixty minutes of uninterrupted testing.
Some languages have tests for several different skills.
Students are not required to take all of the skills available.
If you are only scheduling a single testing session in a language that has multiple skills available, you may want to encourage your students to focus on a particular skill rather than allowing them to choose.
Q. I only have time to take one section of the test. Which test section should I take?
show answer
A. If you are interested in a particular skill, we encourage your students to take that section.
Please remember, however, that CASLS does not grade speaking or writing sections of pilot tests.
For most programs, we recommend prioritizing the reading or listening section (when available), as the analysis of these sections requires much empirical data.
Q. Does the entire department at my school have to take the pilot test at the same time?
show answer
A. No. Although we welcome the joint participation of multiple teachers in the same school or program, we treat each teacher's classes as individual groups.
Q. How will my students be placed for the pilot test?
show answer
A. For non-adaptive pilots, CASLS will pre-assign each class that a teacher registers based on seat time and school level (e.g., first-year students will be assigned the lowest level of questions).
For adaptive pilot tests, students will receive easier or harder questions based on their performance during the test.
Some test questions may appear too difficult or too easy for your students.
Because this is a pilot test, we need to collect data on a wide range of test items to empirically verify the appropriate level.
Please encourage your students to do the best they can on the questions they answer, even if they feel that the questions are not at their level.
Q. How will I find out the results of my students' tests?
show answer
A. Teachers can access the results using their login. Students can receive their results only from their teacher.
In most cases, the results will be shown with a percentage of each student's overall performance at a certain level (e.g., 75% correct for reading and 50% correct for listening).
Students may also be given an estimated proficiency level.
Because the test is still in pilot phase, the test results may not accurately reflect students' true proficiency and should be interpreted with caution.
Scores from pilot tests should NEVER be treated as authoritative or official.
Currently, CASLS cannot provide results for Korean NOELLA.
Q. How soon will I find out the results of my students' tests?
show answer
A. Students' results in listening and reading will be available the day after students complete the test.
CASLS will not provide any scoring for writing and speaking sections.
However, teachers can access the written or spoken student responses from the teacher report page.
In addition, teachers who choose to do so may informally score the results for their own students.
Please remember that we are continually updating and improving
CAP and
NOELLA, so some features (such as report pages) may not always be available.
Q. If CASLS doesn't score the writing and speaking sections, why should I bother piloting them?
show answer
A. We don't score the responses, but we do collect them. During the pilot phase, we do not yet have trained raters.
We collect student responses to speaking and writing items during the pilot phase so that they can be used for rater training in the future.
Q. I have a FLAP grant that requires me to test the proficiency of my students. Can I use the pilot test results for this?
show answer
A. Although caution must be used when interpreting scores from pilot tests, we recognize that proficiency tests are not always available in final form for all languages or desired skills.
If you are looking at scores at the classroom level (rather than tracking improvement in individual students), then the pilot test results might be sufficient for your funder.
Feel free to contact us to discuss your particular case.
Q. Does the use of a particular textbook and its content correlate to the assessment levels tested by CASLS' pilot tests?
show answer
A. CASLS bases its test on test and task specifications derived based on the ACTFL Proficiency Guidelines, not particular textbook or curricula.
Please refer to the
test specifications.
Q. Is CAP appropriate for my K-5 kids?
show answer
A. No.
CAP is for students in grades seven and higher. Children in grades three through six can take
NOELLA.
Q. I have a class that includes both grade 2 and grade 3 children. Can they still take NOELLA?
show answer
A. We recognize that classes sometimes have students from different grades. In these cases, teachers can register their entire class.
Remember, however, that NOELLA is a proficiency-oriented test, and students who have had very little language study may not have any measurable proficiency.
Contact us with Questions
Use the email addresses below if you have questions related to a specific pilot test.
If you have a general question about CASLS, please contact us.
