Just Finished with 2024 AP Calculus Reading!

WOW! I think that’s a wrap for the 2024 AP Calculus exam reading! This was my first year as a reader for CollegeBoard, and overall it was a great experience. Here’s the Custom Ink t-shirt all readers were able to purchase ahead of the reading - this year’s version was fun!

I was chosen to grade remotely, and all remote readers used the ETS Online Network for Evaluation (ONE) application to score exams securely.

I’ve been working on grading since last week on Tuesday, June 11, and I’ve used just about every free chunk of time I've had to log in and score. Over the past week, I’ve had final exams week with my own students, my daughter’s first on-stage dress rehearsal, her first recital, Father’s day, my son’s half-day last day of school with a pool party I helped host, and a broken pool pump I was dealing with. It was an extremely jam-packed week to say the least!

Today, I still graded for an hour when I woke up, but when I logged in just now after lunch, I got the following message.

This could mean that more exams still need to be scanned in for grading, so I will check again later on today, but either way, AP readers are close to the end.

Working as an exam reader remotely was great professional development - it was eye-opening, taught me to look at the scoring guidelines through a closer lens, showed me how efficiently I could grade when I was locked in and on a time-crunch, and helped me learn exactly what CollegeBoard expected for student work: what did earn the point and what wasn’t enough to get the point? In 2021, CollegeBoard changed their scoring rubric guidelines to the much more detailed, multi-page solutions which initially annoyed me because I liked the efficiency and conciseness of having a single page. One page per FRQ with both the question and the worked out solution was easier to keep track of in my FRQ Bin and in my curriculum binders. One page is also easier to quickly project during class when I want students to self-score their work after they attempt an FRQ . However, having been a reader, I can now see just how helpful those detailed bullet points are when deciding what is acceptable or not.

This year, I scored the following two FRQ’s: first #5 (BC only) and then #4 (AB/BC).

Readers get the detailed scoring guidelines while grading, but these are not released publicly online until September.

Since it was my first year, I had to spend most of Day 1 (Tuesday, 6/11) completing the ONE general training modules and the AP Calculus specific training modules. Then I carefully read through the six Benchmark problems with explanations of why that student earned the score they did (Benchmarks are another resource I had seen before but had never utilized much; they really are so helpful!). Next I completed the six Practice FRQs with feedback: after each FRQ I graded, I got a pop-up window that told me what score I assigned and what score it should have received. In general, any score that is off by 1 is called ‘Adjacent’ and any score that is off by 2 or more points is ‘Discrepant.’ I had to go through this training twice, once for each FRQ I graded, and I found that I got more accurate as I went, as can be seen in one of my feedback pop-ups below:

After that, I did six more Practice FRQs without feedback: I scored six FRQs in a row without knowing how I was doing, and at the end I got another summary of how I did.

The final step before being cleared to grade is Calibration, which is six more FRQs you grade in a row. After scoring all six, you either pass Calibration or need to redo it until you are grading in alignment with the scoring guidelines. I ended up getting new Calibration problems to do at the beginning of each day - this was to help readers get back into the groove of grading and to make sure they are still properly applying the guidelines.

I had about eight other readers in my ‘table’ with one table leader. The table leader backreads the FRQs of everyone in the table to ensure everyone is correctly grading the FRQs, and they return any FRQ to the reader if they see an error in the score given. Every 15 FRQs or so, you get a Validity FRQ, which used to ensure you are grading properly. It’s like quality control, and if you score that incorrectly, then you check in with the table leader to see which point(s) you may be misinterpreting.

Each FRQ got a score of 0-9, or was marked as NR (no response), Defer (to another reader), or was put on Temporary Hold. I put a handful of FRQs on hold each day whenever I wasn’t sure on a detail of student work - should they get the point or not? That FRQ on hold was sent to the table leader, who would respond with how to grade it and send it back.

Every year during the reading week, CollegeBoard hosts an Open Forum on Wednesday evening. They share a presentation on the AP exams in general, any upcoming changes, and any concerns they are noticing. This year they confirmed which additional exams would be going online next year. The following exams were already digital this year:

I really hope AP Calculus AB/BC does not go digital in the near future, although that looks like something CollegeBoard is seriously considering.

Another thing we learned was that instead of readers choosing a score 0-9 for the overall FRQ, they would grade each FRQ point-by-point (PBP) so that teachers can see exactly which parts of which FRQs they students did well on or struggled with. So if I score next year, instead of choosing a single overall score for each FRQ, I would click a sub-score for each part of the FRQ . AP Calculus is an exam moving to PBP for next year.

While in the ONE grading system, each reader has a sidebar where they can see how many FRQs they put on hold, how many hours they’ve been scoring (since their last login), and how many responses they’ve scored. On June 16, after five days, I was at 27 hours and 475 responses.

The one thing I wish ONE still showed us was our performance. In the ONE training video, we saw the following snapshot and were told we would see our performance as we went, an indication of how accurately we were grading. The highest evaluation you can get is 4 stars. This year, ONE did not show us this, but our table leader did let us know that we were all scoring at the 4 star-level at our table.

Since it was my first year, I had to spend my entire first day training and only got a single FRQ done right before I logged off for the day. Then on Friday early afternoon, the FRQ I had been working on was fully graded, so I had to retrain for the second FRQ, which took a couple more hours. Some years and for some AP subjects, readers grade the same FRQ for six days straight, so they get highly efficient and quick at grading them. I was hoping to get more FRQs done (I know many experienced readers get past 2,000 during the scoring week), but next year if I am able to score again, I know I will get through more.

This is when I hit 500, which was pretty exciting:

This is where I ended on the original final day of scoring:

There were still thousands of FRQs to score by the end of Monday, June 17 (the initial final day of scoring), so we were invited to keep scoring, something I had agreed to do. I ended up not being able to score at all on Tuesday, which was my son’s half-day last day of school and pool party, but I did get some scoring done on Wednesday and today. Right now I am here - so close to 800!

As a beginning reader, I wanted to make sure I was keeping track of scoring carefully and had the ability to easily refer back to an FRQ if there was an issue with the score I gave. I found that using a paper packet to keep track of each FRQ ID number and points earned was extremely helpful. It probably slowed me down a bit, but I would rather have this and know that I wasn’t messing things up. I found that writing ‘0’ was quicker than writing ‘X’ if a student didn’t earn a particular point, so I switched to that about halfway through the reading week.

Now I just have to wait until scores are released early July to see how my own students did!