Analyzing and Improving a Test using Statistics

Analyzing and Improving a Test Using Statistics

Dawn Stevane

EDU645: Learning & Assessment for the 21st Century (MRB1231B)

Richard Newman

August 27, 2012

 

            The Mean Score for Each Question

            What is the definition of mean? Mean is the average of the numbers. A good question is how we find the mean score for each question? I had to research it because it has been a long time since I had a math course. You find the mean by adding up all the numbers, then dividing by how many numbers there are.  The questions provided had five questions each of the questions had a total based upon ten students. The first question had a total of 18, the second question had a total of 14, the third question had a total of 2, the fourth question had a total of 20, and the fifth question had a total of 2.  There were a total of 56; if you divided the total by 5 questions then you will get 11.2. The mean score for each question is 11.2. These scores reflect what the students may have scored on each question.

            The Mean Score for the Entire Test

The mean score for the entire test can be calculated by also finding the average for each question and then divided the sum by the number of students and questions. The sum for the scores of each question is 56.  There are ten students that may have been administered the test so we can add the tenth student to the sum of the score for the test of 56 and also add the number of questions and then divide the three (students, questions, and sum of score). The mean score for the entire test is 23.6. I came up with this score by adding the sum of 56 to the amount of students which was 10, and also adding the number of question, which is 5, and then divide the total by three entities of the questions, students, and sum of scores.

 

 

 

A Graph that Represents the Scores

 

 

 

 

 

            Qualitative and Quantitative analysis are used to find the bad items on a test. Quantitative item analysis is a numerical method for analyzing a test items utility, with data that can be measured (Kubiszyn and Borich, 2010). Away to remember Quantitative is quantitative equal’s quantity. Qualitative item analysis is non-numerical method for analyzing a test items utility, with data that can be observed not measured (Kubiszyn and Borich, 2010). A way to remember Qualitative is qualitative equals quality. In this chart it shows the scoring that each student received for the questions. The quantitative items in such chart show the analysis of the questions whereas the qualitative analysis reflects the amount of students that test was administered to. The qualitative analysis of the test shows the more subjective part of the chart, and the quantitative analysis was more objective. The quantitative analysis of the chart would be taking the ten students who have selected correct answers and divide by the total number of students who made an attempt. Student 5 answered all five questions correctly, whereas there were nine other students who did make an attempt to answer all of the answers correctly.

            The test questions that were given to each student were to short and did not give the test taker enough evidence to use qualitative and quantitative research. There was enough evidence to be able to evaluate the qualitative research. But, the quantitative item analysis assesses the quality of the chart and it was able to identify the distracters that are not doing what they should be doing. The Quantitative part of the test was the questions were multiple-choice questions. In using the mean score of the questions, I found that quantitative was also utilized as quantitative is taking the total number of students who select the correct answer and then divide that by the number of students who attempted the item.

 

            By observing the answers from students in this graph, we can determine if miskeying has taken place, did the student guess or was there ambiguity. This is where qualitative item analysis plays a part in determining if the problem is a lack of mastery or a poorly written item. Qualitative item analysis checks the validity of the item and any technical faults that may exist. You want to match items and objectives and edit any items that are poorly written. The benefits of these two types of analysis are that they must be used together to get the best results. They allow you to find out where the problem areas are and make sure your test questions past the validity test.

            In my opinion there is no test or score that is completely valid or reliable “the tests come with varying degrees of goodness” Kubiszn & Borich (Chapt. 18). If they are used together than the teachers can grade the tests effectively. Their benefits to learning and assessment are “they help us decide whether to retain or eliminate an item, which distracters should be modified or eliminated, whether an item is miss keyed, whether guessing occurred, and whether ambiguity is present—quantitative analysis” (Kubiszyn and Borich, 2010). The risks to learning and assessment if these types of analyses are not administered are as follows: Qualitative analysis is limited because it shows the items that have errors but it doesn’t help you to find them. Quantitative analysis is timely and the teachers can get very frustrated using type of analysis.

 

 

 

 

 

 

References:

Kubiszyn, T. & Borich, G. (2010). Educational testing & measurement: Classroom application and practice (9th ed.). John Wiley & Sons, Inc., Hoboken, NJ.

 

http://vizedhtmlcontent.next.ecollege.com/pub/content/82f194db-9e1f-4158-be19-c271217e5980/Copy_of_EDU645_Week_4_Assignment.pdf

http://chemistry.about.com/od/workedchemistryproblems/a/calculate-the-mean.htm

 

 

 

 

Response to Intervention

 

Response to Intervention (RTI) Approach

Dawn Stevane

EDU645: Learning & Assessment for the 21st Century (MRB1231B)

Instructor:  Richard Newman

August 6, 2012

 

           Response to intervention is a system that uses assessments, instruction, and decision making to help students with and without special needs. It is made up of three tiers of an intense intervention for those students who need help. The first and largest tier is where about eighty percent of the students, they will stay at this level throughout their schooling. The second tier is where those students who had trouble with tier one will be helped more and this is usually about fifteen percent of the students in the school. The third tier is when the student is still having trouble and will be refereed for a full evaluation to see if the student is qualified for special education services. The students will go back to tier one if they understand and if not then they will stay in tier two until they get the information, if not then they will move to the third tier and be evaluated.

            Tier one is the first is when all students are tested to see which children are need help in the specific academic areas. If after the children finish the testing and they are not in need of any extra help the will go back in to the regular class. The children that still need help will go in to the second tier. The second tier is for the students that need a little more help, they will receive more help and take tests again to see if they made any progress. If they made progress they will go back to the regular class. If they did not get it then they will go to the third tier of the program. At this stage the children have individualized help depending on what they need help with. Like for example reading, math, and speech.

            Implementing RTI in the classroom is a challenge and is not as hard as you think it is. First you have to figure out what the child needs by making a plan to figure out what services that child needs help with. It is also important to make sure that the implementations at culturally responsive and evidence based assessments that will be easy to understand. Making sure that you have ongoing meetings like ppt’s, to discuss the child’s progress and if they are getting it and need more help. If they need help then you can change the plan to help them understand it or help them move ahead. There is lot of benefits from using the RTI approach, one of them is the early detection of children with special needs. It also helps the children that enter school that have trouble reading and speaking English as well. Once they identify which students are struggling, they would be monitored to see if they understand and then they can move up in their education if they are understanding. RTI is an education reform that is starting to be used in the classrooms all across the world. RTI helps determine the difference in the poor instruction to a students learning delay. The RTI model will help suggest that a student’s disability may need special education programs and services.

            The challenges that the teacher face with the RTI approach is; that they think that some children will be misdiagnosed that they have a disability they not even have one. Another challenge is when the people that implemented the RTI approach move on to another school and the teachers just go back to what they used to do and say that every child had disability and the really didn’t when they used the RTI approach. Ways to overcome the challenges is to keep using the RTI approach and don’t stop using it when the children are doing better.  Also I would just make sure that the teachers understood that the RTI approach is a good thing and if we keep using it, it will help both students with disabilities and children without disabilities. The children without disabilities will be helped on areas that they are falling behind and that they don’t understand. For the students with disabilities they will get help and they will not be misdiagnosed while using the RTI approach.

            There are three main reasons why early intervention in a child’s education is very important to get a child ready to be able to do well in their society. A child learns more and develops quickly when they are in pre-school. This is the best opportunity to give them all the knowledge as possible because their brain is absorbing it. Timing is a very important part of early intervention in a child’s education. If a teacher doesn’t take advantage of this the child will have difficulty learning a skill at a different time.

            The children will also have a higher self-esteem when they learn and depend less on people helping them. Education is very important aspect in a child’s life as they grow older and become a productive part of the society. It will also open up a great a lot of great opportunities for you as well. RTI approach will help a child have more knowledge that will help them to solve any problem that comes their way. The teachers will also be able to have a smooth running classroom if every child feels comfortable in the classroom, they will learn better and not make the classroom as stressful.

            Is early intervention really cost effective? That is a good question some people will say yes and some would say that it is up what you resources you know and if you know how to use them properly to get what you need. The highly specialized, comprehensive services necessary to produce the desired developmental gains are often, on a short-term basis, more costly than traditional school-aged service delivery models (Kidsource). A study showed that children that start out as early as preschool will not need as much special education if they start early with help on the areas that they need help in.

             There are also three critical features to make sure that early inventions programs are effective. The three program features are: (a) the age of the child at the time of intervention; (b) parent involvement; and (c) the intensity and/or the amount of structure of the program model (Kidsource). Like I said before the earlier that you start with the diagnosis and the intervention the better the child will receive his/her education. Having the parents involved in the child’s education is very important and it also will help the parent because they can get ideas and suggestion on how to handle certain things at home. Also by doing this this will make the stress on parents and this will make the children more comfortable in the classroom.

             The RTI teachers should be open to change in how students are identified for intervention; how interventions are selected, designed, and implemented; how student performance is measured and evaluated; how evaluations are conducted; and how decisions are made, members of the observation team when the child’s learning problems involve reading, service providers in the RTI process, sharing expertise with the other professionals on the team, suggesting interventions that are integrally connected with the core reading program used

in the classroom, and be willing to adapt a more systemic approach to serving schools, including a workload that reflects less traditional service delivery and more consultation and collaboration in general education classrooms (reading.org).

            When the RTI specialist is serving the children they should be patient and provide them with a fun and relaxing environment to learn in. The specialist should also get together with the teacher and parent to start early intervention in the school as well as at home. It is also important to observe the children in the home as well as in the school. The ways to improve and keep RTI going is easy to do but you need the right people who care about the children. The first way is for the teacher and or specialist makes sure that he/she evaluates the achievement of the student. Make sure that the student receives the help that the need until they are not needed anymore. Make sure that the plan change as the children grows and doesn’t need that plan anymore.

           

           

           

References:

Kubiszyn, T. & Borich, G. (2010). Educational testing & measurement: Classroom application and practice (9th ed.). John Wiley & Sons, Inc., Hoboken, NJ.

http://reading.org/downloads/resources/rti0707_implications.pdf. Retrieved from http://reading.org on August 6, 2012

http://www.kidsource.com/kidsource/content/early.intervention.html. Retrieved from http://www.kidsource.comon August 6, 2012

 

 

 

 

Accuracy and Error

The authors of your textbook state that “all tests and scores are imperfect and are subject to error” (Kubiszyn & Borich, 2010). What do they mean by this? Why is it important to recognize that there is no “perfect test”? Knowing this, what can you do to ensure that you obtain the most accurate assessment results as possible?

I think that the authors mean that everyone is human and when we do something we are likely to make mistakes or errors. It is very important to realize that there is no perfect test, and once we do then we can look for the degree error. “Just as you can expect to make scoring errors, you can expect to make errors in test construction.” No test you construct will be ever so perfect; in fact these imperfect tests will include inappropriate, invalid or otherwise deficient items” (Kubiszyn & Borich, 2010 p.227). I remember reading in the beginning of this book that “there can be no one size fits all test or assessments” (p. 6). Knowing this I think that it is the responsibility of the teacher to take his/her time when they are making a test. I know that there will be mistakes because no one is perfect. The teacher can use assessments to help them as well as the children. The assessments can help the teachers find the students weaknesses and strengths. This can help the teachers help the children prepare for testing. There are four categories that errors are put in to: errors with the test takers, errors with the test, errors with the test administrator, and error with scoring. I think only way to I get the most accurate assessment results are to give the students one test. Having a second test will cause the children to make more errors. In conclusion I think that it is a learning process and that the teacher and the student will learn if they work together.

Marks and Marking System Reflections

Why do you think some teachers like to grade students “on a curve”? Do you see merit in the authors’ point about teachers being less motivated to improve when doing this? Why or why not? What effect do you think it might have on student motivation? Explain.
I think some teachers grade on a curve because sometimes it is easy for them. I don’t think it is fair to the students because there are some that understand the material and some that don’t. For the ones that understand it they are not getting the grade that they deserve. The children that don’t understand they are being based on a percentage of a grade instead of just getting the grade that they deserve. If they get the grade that they deserve then the teacher can help them because they will se that they don’t understand the material. I see the reason about the teachers being less motivated to improve this because it is just easier to grade on a curve than to go over the material until the students understand it. I think that would be better for the students because then they will have a better tie learning and a better self esteem because then they could be proud of their grade because they earned it and it was just not handed to them. The effect that it will have on the students will be bad because, the students that understand the information will just not exceed because they will just take the grade. The students that don’t understand the information will just be content with the grade given to tem and they will not try to lean more.

Norm-refrenced test and Criterion-refrenced test

A norm- referenced test is a test that determines a student’s placement on a normal curve. This is known as being graded with a curve. An example of a norm-referenced test would be the SAT’s that you take in middle school and in high school. These types of testing can also be used for representing an age group that; is used for diagnosis and or placement. A criterion-referenced test will determine how a child has learned a specific skill. An example of a criterion-referenced test is the multiple choice test like you take when you are getting your license. I took a criterion-referenced test when I got my license. I also took a norm-referenced when I took my SAT’s in high school. I think that the criterion-referenced test helped me to know more about driving and the ways to be safe while driving. When I took the norm-referenced test when I took my SAT’s did not support my learning to help me learn anything. It just told me how much I knew in each subject with a percentage of how good I did verse the total of students that took the same test. I think that it would support my learning if I did badly and then the teacher gave me work to understand that subject more and also do better the next time I take the test.

Summative and Formative Assesments

Summative assessments are when students are evaluated when they are finished with the work and the focus is the final product. They are used to know what the students know and what they do not know. Some examples of this assessment are final exams, entrance exams like SAT, and nationwide test.

Formative assessments are when students are evaluated during the work progress and also during this they try to improve the process as well. Some examples of this assessment are observations, discussions and questions.

A teacher can use these types of assessments to see where the children are and what they need to change to help the children learn better and understand. They are used to gather information, record, and interpret about the students to make a lesson plan that helps all the students in the classroom. If we use an analogy to describe the, it would be something like this; “If we think of our children as plants; Summative Assessment of the plants is simply measuring them. It might be interesting to compare and analyze measurements, but these do not affect the growth of the plant. Formative Assessment is the equivalent to feeding and watering the plants according to their needs, this will affect their growth.”

Rationale of Testing and Essay Items

First Outcome: Students will be able to identify the primary colors by pointing to the color when asked.

Second Outcome: Students will be able name the primary colors when a certain color is being pointed to.

Third Outcome: Students will be able to tell the difference when you hold two colors side by side.

Learning Outcomes for Pre-K

Knowledge

Students will identify the primary colors by pointing to them.

Test Item

Showing the children flash cards, the students will be asked to identify each color that they see.

Comprehension

Students will name the primary colors.

Test Item

 Using the flash cards again, students will find and identify the primary colors by name.

Application

Students will be able to identify different colors when they are side by side.

Test Item

On a piece of paper with several colors, student will circle the correct color of the shape on the paper.

Essay Item

Showing student several flash cards of colors every day, he/she will identify the primary colors on their own throughout the classroom and wherever they go.

Learning Outcomes

The three measurable outcomes for teaching children their colors.

Grade level- Preschoolers

First Outcome: Students will be able to identify the primary colors by pointing to the color when asked.

Second Outcome: Students will be able name the primary colors when a certain color is being pointed to.

Third Outcome: Students will be able to tell the difference when you hold two colors side by side.