I just returned from the MS State Department's MCT2 Data Review Committee in Jackson yesterday. My job, along with about 300 other people, was to review results, and incidentally test items, from each of the questions piloted in the assessment given to the students of Mississippi last May. Recently, the news has had a field day presenting evidence that Mississippi's State Curriculum Test is far too easy and doesn't align with test results from National tests. It was absolutely fascinating... And, I'm not just being my usual sarcastic self. I really enjoyed the experience and met a bunch of wonderful people along the way. Of course, the trip provided many bloggable moments, so I hope to share those with you! Bear in mind, I am bound by a confidentiality agreement, but honestly, the test wasn't the most interesting thing to share anyway. You know I am all about an experience!
I arrived at the Downtown Marriot Wednesday night after driving for three hours. Map Quest ensured my safe arrival, however, and I was delighted to find Bath and Body Works products in the room. Having received no agenda prior to arrival, I wandered around downstairs until I ran into one of the facilitators who told me to be downstairs at about 7:45 for a lovely continental breakfast. I returned to my room for a fretful night's slumber, as I am sincerely not accustomed to sleeping away from my husband.
The next morning, upon checking in with registration and figuring out that I would be serving on the 5th grade Language Arts committee, I ran into Lucy, another teacher from my district. We were indeed treated to a delicious continental breakfast of fresh fruit, bagels, croissants and little pats of real butter. After everyone had eaten and settled into the main ballroom, the Director of Curriculum for our state and a data analysis expert from the Pearson testing company gave a brief Power Point presentation. Well, actually, it wasn't really brief, but I just think it serves me better to say it was really brief since I really didn't understand much of what the data analysis expert said at all. I sat there listening to terms like "item response theory" and "point bi-serial", looking around hoping Dave would materialize from somewhere and tell me what in the heck the woman was talking about. My husband has a degree in statistics from the Naval Academy, not me.
Thus, I was feeling particularly forlorn and out of place until Lucy leaned over and said, "I feel like I'm listening to Charlie Brown's teacher... WawawawaWAH!"
The relief I felt after Lucy's admission that she didn't understand a word the woman was saying was short-lived, however... After breaking into our groups, it became apparent that we were going to have to use the data to review the items. My Gawd! The State Department of Education is serious about this depth of knowledge stuff! They were expecting me to take an hour long presentation about incredibly difficult mathematics concepts, synthesize the information and apply it to my review of individual test items! Briefly, I felt like one of my kids... Maybe I should pretend to be sick so nobody will guess that I don't have a clue what any of this is about.
But, fear not... My arrogance and innate ability to exude confidence, when I really have no idea what I'm doing, took right over allowing me to sit in a room with about twenty-five other ladies, giving my opinion about test items as they were presented. I sat between teachers from Madison County School District and Greenwood School District, forging fast friendships. Now, you can't get a large group of women together without claws coming out. Further, it appeared that the administrators who chose teachers for the committees made it a point to choose the most out-spoken, child-advocating types they could find. Thus, it made for a really interesting group.
Robin, my new friend from Madison, noticed that one woman in particular looked as if she was going to have a spontaneously created rain crowd erupt over her head. I always wonder about people like that... What happened in her life to make her so angry and morose? My feelings of sympathy quickly dissipated when she tattled to the facilitator because we were laughing and having a good time and she couldn't concentrate on reading. I try to laugh and have a good time wherever I go. And, honestly, at that point we had delved heavily into the various test items. Two choices presented: laugh hysterically or cry. Given that our 30ish to 50ish-year-old selves were spending copious amounts of time debating answers and items, imagine what the 10 and 11-year-olds were going to do when presented with the new test. I promise not to punish any student if he/she suddenly bursts into fits of tears or hysterical laughter. We'll just all have to pause and allow a little time to get it back together.
Our Pearson facilitator, Marnie, was from Ontario. She worked hard to keep us on track. She also insisted that we have a data reason for eliminating items. So, "this is terrible and we hate it" wasn't a good enough reason... (Sorry to my friends who were counting on that...) We had to make one of three choices about each item: accept, reject or revise. The choices had to be based upon our analyzing the data. So, I brought out my notes from the presentation and attempted to decode the mean, B-par, r-tots, and point bi-serials. The data was further divided into item results for the top 1/3, the median 1/3 and the bottom 1/3 performing students. Thus, I found myself looking primarily at three points of data: B-par score (ease or difficulty level of the question), and the results for the median and bottom performing students. If the overall data indicated a problem with the item, we looked at the item, how it was worded, what it measured, and how it compared to what was expected within the new MS curriculum framework. Further, I focused on how the "least of our brothers" performed to see if there might be a problem directly related to a disability that may have caused them to miss the question. It was quite a process.
On the second day, the Director of Assessment gave a strange speech after lunch. Educational administration is famous for these types of speeches... Evidently, there were issues within a couple of the review groups. Rather than discussing these issues with the people to whom they applied, she addressed the entire group. Therefore, there were approximately 250 of us who had no clue what she was talking about. When I inquired to my professional teaching counterparts as to what the speech was about, I received several summaries of the speech. Several of us thought she was saying that our input would be considered, however, the state has already spent hundreds of thousands of dollars on this assessment and some items would be used despite our negative evaluation of the items. Others thought she was talking about test alignment to new curriculum frameworks and how it would take several years to develop a truly accurate measure of this new framework. One teacher thought the Director of Assessment (a lady) was publicly proclaiming her undying, everlasting love for the Direct of Curriculum (a man). Probably, the speech was a slight combination of all of these items... Except the everlasting, undying love part. That just gave us all a good chuckle.
On the final day, I spoke to Marnie, the facilitator and she gave me insight into how tests are developed with teams of people focusing specifically on how to make it most accessible to people with disabilities. I heard what she was saying, and truly appreciate the effort; however, I really wish she could meet sweet little "Jo", my student with cerebral palsy. No matter how many people have reviewed it, this 5th grade test will not be friendly to my student. It was interesting to learn that Canada takes a more common sense approach to testing than is outlined in the No Child Left Behind act... Their law requires 70% of students reach a level of proficiency, versus the requirement of 100% by 2014 as stated in NCLB. Further, students with reading disabilities are allowed to use a Kurzweil reader on the reading test. This would absolutely not be allowed on state assessments in the U.S. But, it gave me hope that perhaps someone in America will come to their senses in the near future!
My adventure ended by missing my exit at Hattiesburg and driving 40 miles out of the way on Highway 59. I stopped in McNiell, MS when my bladder was about burst. Thankfully, I was talking to my friend on the cell phone, so I could give specific directions about my location. I was concerned it was going to be a Bates Motel experience, and he would need to come rescue me from banjo-playing backwoods inbreds. Fortunately, McNiell turned out to be a nice little stop with a lady who could direct me to I-10 via Slidell, LA. At least I was finally able to find my way home. My MCT2 Data Review Committee experience was definitely worthwhile. I gained great insight into how assessments are developed. But, I can't really say that it will change the things I'm doing with my students. I'm not sure this test will help prepare my students for life. Still, I think I'm a little better prepared for the teaching profession. I have added some new terminology to my vocabulary and enhanced my ability to dazzle others with my uncanny ability to present a front of absolute confidence in the face of incredibly unrealistic requirements.
~Namaste'~
I arrived at the Downtown Marriot Wednesday night after driving for three hours. Map Quest ensured my safe arrival, however, and I was delighted to find Bath and Body Works products in the room. Having received no agenda prior to arrival, I wandered around downstairs until I ran into one of the facilitators who told me to be downstairs at about 7:45 for a lovely continental breakfast. I returned to my room for a fretful night's slumber, as I am sincerely not accustomed to sleeping away from my husband.
The next morning, upon checking in with registration and figuring out that I would be serving on the 5th grade Language Arts committee, I ran into Lucy, another teacher from my district. We were indeed treated to a delicious continental breakfast of fresh fruit, bagels, croissants and little pats of real butter. After everyone had eaten and settled into the main ballroom, the Director of Curriculum for our state and a data analysis expert from the Pearson testing company gave a brief Power Point presentation. Well, actually, it wasn't really brief, but I just think it serves me better to say it was really brief since I really didn't understand much of what the data analysis expert said at all. I sat there listening to terms like "item response theory" and "point bi-serial", looking around hoping Dave would materialize from somewhere and tell me what in the heck the woman was talking about. My husband has a degree in statistics from the Naval Academy, not me.
Thus, I was feeling particularly forlorn and out of place until Lucy leaned over and said, "I feel like I'm listening to Charlie Brown's teacher... WawawawaWAH!"
The relief I felt after Lucy's admission that she didn't understand a word the woman was saying was short-lived, however... After breaking into our groups, it became apparent that we were going to have to use the data to review the items. My Gawd! The State Department of Education is serious about this depth of knowledge stuff! They were expecting me to take an hour long presentation about incredibly difficult mathematics concepts, synthesize the information and apply it to my review of individual test items! Briefly, I felt like one of my kids... Maybe I should pretend to be sick so nobody will guess that I don't have a clue what any of this is about.
But, fear not... My arrogance and innate ability to exude confidence, when I really have no idea what I'm doing, took right over allowing me to sit in a room with about twenty-five other ladies, giving my opinion about test items as they were presented. I sat between teachers from Madison County School District and Greenwood School District, forging fast friendships. Now, you can't get a large group of women together without claws coming out. Further, it appeared that the administrators who chose teachers for the committees made it a point to choose the most out-spoken, child-advocating types they could find. Thus, it made for a really interesting group.
Robin, my new friend from Madison, noticed that one woman in particular looked as if she was going to have a spontaneously created rain crowd erupt over her head. I always wonder about people like that... What happened in her life to make her so angry and morose? My feelings of sympathy quickly dissipated when she tattled to the facilitator because we were laughing and having a good time and she couldn't concentrate on reading. I try to laugh and have a good time wherever I go. And, honestly, at that point we had delved heavily into the various test items. Two choices presented: laugh hysterically or cry. Given that our 30ish to 50ish-year-old selves were spending copious amounts of time debating answers and items, imagine what the 10 and 11-year-olds were going to do when presented with the new test. I promise not to punish any student if he/she suddenly bursts into fits of tears or hysterical laughter. We'll just all have to pause and allow a little time to get it back together.
Our Pearson facilitator, Marnie, was from Ontario. She worked hard to keep us on track. She also insisted that we have a data reason for eliminating items. So, "this is terrible and we hate it" wasn't a good enough reason... (Sorry to my friends who were counting on that...) We had to make one of three choices about each item: accept, reject or revise. The choices had to be based upon our analyzing the data. So, I brought out my notes from the presentation and attempted to decode the mean, B-par, r-tots, and point bi-serials. The data was further divided into item results for the top 1/3, the median 1/3 and the bottom 1/3 performing students. Thus, I found myself looking primarily at three points of data: B-par score (ease or difficulty level of the question), and the results for the median and bottom performing students. If the overall data indicated a problem with the item, we looked at the item, how it was worded, what it measured, and how it compared to what was expected within the new MS curriculum framework. Further, I focused on how the "least of our brothers" performed to see if there might be a problem directly related to a disability that may have caused them to miss the question. It was quite a process.
On the second day, the Director of Assessment gave a strange speech after lunch. Educational administration is famous for these types of speeches... Evidently, there were issues within a couple of the review groups. Rather than discussing these issues with the people to whom they applied, she addressed the entire group. Therefore, there were approximately 250 of us who had no clue what she was talking about. When I inquired to my professional teaching counterparts as to what the speech was about, I received several summaries of the speech. Several of us thought she was saying that our input would be considered, however, the state has already spent hundreds of thousands of dollars on this assessment and some items would be used despite our negative evaluation of the items. Others thought she was talking about test alignment to new curriculum frameworks and how it would take several years to develop a truly accurate measure of this new framework. One teacher thought the Director of Assessment (a lady) was publicly proclaiming her undying, everlasting love for the Direct of Curriculum (a man). Probably, the speech was a slight combination of all of these items... Except the everlasting, undying love part. That just gave us all a good chuckle.
On the final day, I spoke to Marnie, the facilitator and she gave me insight into how tests are developed with teams of people focusing specifically on how to make it most accessible to people with disabilities. I heard what she was saying, and truly appreciate the effort; however, I really wish she could meet sweet little "Jo", my student with cerebral palsy. No matter how many people have reviewed it, this 5th grade test will not be friendly to my student. It was interesting to learn that Canada takes a more common sense approach to testing than is outlined in the No Child Left Behind act... Their law requires 70% of students reach a level of proficiency, versus the requirement of 100% by 2014 as stated in NCLB. Further, students with reading disabilities are allowed to use a Kurzweil reader on the reading test. This would absolutely not be allowed on state assessments in the U.S. But, it gave me hope that perhaps someone in America will come to their senses in the near future!
My adventure ended by missing my exit at Hattiesburg and driving 40 miles out of the way on Highway 59. I stopped in McNiell, MS when my bladder was about burst. Thankfully, I was talking to my friend on the cell phone, so I could give specific directions about my location. I was concerned it was going to be a Bates Motel experience, and he would need to come rescue me from banjo-playing backwoods inbreds. Fortunately, McNiell turned out to be a nice little stop with a lady who could direct me to I-10 via Slidell, LA. At least I was finally able to find my way home. My MCT2 Data Review Committee experience was definitely worthwhile. I gained great insight into how assessments are developed. But, I can't really say that it will change the things I'm doing with my students. I'm not sure this test will help prepare my students for life. Still, I think I'm a little better prepared for the teaching profession. I have added some new terminology to my vocabulary and enhanced my ability to dazzle others with my uncanny ability to present a front of absolute confidence in the face of incredibly unrealistic requirements.
~Namaste'~
