The author argues that Rothstein’s alleged ‘replication’ of her results was in fact a deliberate attempt to find conflicting results, and that his alterations to her specification are both flawed and motivated by the same desire to contradict her findings.
Hoxby points out that Rothstein’s later (2004-5) comment has almost no overlap with his earlier (2003) one – as though one set of arguments had failed, and he had searched for new ones.
Key results:
Hoxby argues that Rothstein’s differing results when including private school pupils in the sample arise due not to superior sample/controls, but due to substituting error-prone data for error-free data. Rothstein changes the method of assigning locations to all pupils (not just public school pupils) when he adds in the private school children, introducing significant measurement error.
She states that the four school districts in Ohio (which Rothstein pointed out had been assigned to the wrong state) do not actually contain any students from the sample, and thus literally cannot affect the results.
Regarding her use of larger and smaller streams as instruments, Hoxby points out that large rivers may attract commerce and hence cities. Thus the smaller streams are the more credible instruments, and the differing coefficients on smaller and larger streams should not be surprising.
Hoxby takes Rothstein to task for proposing the number of school districts in 1942 as an alternative instrument for the number of districts in 1990. The number of districts is likely to be serially correlated and is therefore not an appropriate instrument.
She also savages Rothstein for proposing SAT scores as an alternative measure of student achievement (rather than the representative survey [NELS] data used in Hoxby [2000]). She points out that in some states as few as 3% of students take SATs, and “there are no statistical techniques that allow an econometrician to deduce what is happening in 97% of a distribution from looking at the top 3% of it.”
Hoxby concludes that Rothstein’s comments are “not replications in the scientific sense and are without merit.”
