On the Road with Jake: Evaluating Intervention Programs

Jake Firman is DSST's Manager of Education Technology. The following post is the latest in a series chronicling his trip west this spring to attend the Pearson Digital Learning Executive Forum and explore the wide, wide world of blended learning as it exists at other education orgs.

During lunch on Wednesday at the forum, I had an interesting conversation with the director of curriculum of the St. Vrain Valley school district, and we got to talking about something that has been at the forefront of the work that Mariah and the DSST academic achievement SWAT team have been working on: intervention programs (both digital and traditional) and how to separate the good stuff from the crap. We discussed the phenomenon of schools often selecting intervention programs without truly evaluating their effectiveness. It was nice to know that this is an issue at other districts.

Kel described that at St. Vrain, they have developed a process by which the district monitors and controls the intervention programs that schools use. They have a committee at the district level that meets semi-weekly to hear proposals from schools for new programs. When a school identifies a new program that they want to use, they have to submit an application, and have a representative (sponsor) physically attend the meeting to propose the tool. In that proposal, the sponsor must present scientific research of effectiveness of the program, along with a detailed plan of implementation and support. From there, the committee votes the program up or down, and requires the school to execute at least a one-semester pilot.

Later that evening, I spoke with John Farnam from the Morgridge Family Foundation (who, by the way, has a CRAZY past with ActivProgress that was super interesting to learn about), and he described a similar process that he had seen at another district. The difference in his example was that the school utilized a peer review committee. So the sponsor would present the program to a group of peers, and the peer review committee would make a recommendation to the district, who would ultimately make the final decision.

I'm hoping to get more details on both of these scenarios from both John and Kel, but the overarching takeaway is that it seems as though a review committee is the way to go when we talk about reviewing potential intervention programs.