I’m speaking at a conference at Berkeley sponsored by the American Political Science Association on “Democracy Audits and Governmental Indicators”. In getting some remarks together — on the the reliability of country-level measures of democracy etc — I wanted to compare the performance of measures of democracy against things like GRE scores, legislative ideal points.
ETS has a background document providing some technical data on GRE scores. The standard deviation of GRE-V scores issued in the 2003-08 period is 121 points, while the GRE-Q scores have a standard deviation of over 150 points. The standard errors of measurement are pretty small, relative to this cross-subject variation in the scores, and surprisingly uniform over the range of scores.
Usually you get a U-shape relationship between standard errors of measurement (or — if you are a Bayesian — standard deviations of marginal posterior densities of latent scores) and the scores; we have greater uncertainty about test subjects in the tails of the ability distribution, since the test items tend to be less informative about those subjects (as they rack up a lop-sided pattern of right/wrong answers).
The administered-by-computer, adaptive, version of the GREs helps smooth out that U-shape, with the computer administering items that have “cut-points” close to the running estimate of the subject’s ability.
To look at this I plotted the “conditional standard errors of measurement” for the GREs (as reported by the ETS) against scores; see below.
There is something of an inverted U, which is weird. We’re actually getting less precision in the middle of the scales than in the tails. The other thing is that we’ve got standard errors of measurement that are about 20%-35% of the between-subject score variation, which tails away to about 5-15% in the upper tails.
I wish those standard errors of measurement were smaller, and that is really only a function of the length of the test, given that ETS has near-perfect knowledge of the item parameters. So, does the GRE need to be longer?
The New York Times reports on salaries at Carnegie Hall:
The men — Dennis O’Connell, properties manager; James Csollany, carpenter; John Goodson and John Cardinale, electricians; and Kenneth Beltrone, carpenter — were identified on Carnegie’s tax return for the 2007-8 season as being the hall’s leading five earners after its top executive, Clive Gillinson. Their annual compensation ranged from Mr. O’Connell’s $422,599 (with an additional $107,445 in benefits and deferred compensation) to Mr. Goodson’s $327,257 (with $76,459 in benefits and deferred compensation), the return showed.
Into Bayes? Into music? Into wicked coding? Want to live in France for a couple of years? Read on below the fold. (more…)
Fake Steve Jobs had this up on his site yesterday. Nice work by the kiddies at UQAM. Perfect for a Friday.
Terry Durack gets it about right re the current state of Sydney dining and cuisine…
Friday night at LAX. I’m headed back to the Bay Area. Our departure gate was next to the united Sydney flight. It is weird to look over at those faces and immediately see the Australian stock. You know immediately you are looking at a predominantly Australian crowd, but it takes a little while for the cognitive part of the brain to figure out why… Predominantly white, close to zero Latino faces, perhaps a tad ganglier relative to the other people in the terminal, in LA, in the USA.
On taxi I spied 4 QF 747s and an A380 plus two V Australia 777s. That’s a lot of inventory headed to Oz from here.
The responses are in, and summarized below. The question went out in the form of an informal poll of some methodological types, but I’ll keep the “pollster” anonymous…
You know the job has finished running when the fans on the computer get quieter…
Powered by WordPress
Bad Behavior has blocked 3036 access attempts in the last 7 days.
.