Where Do Densities Cross?

Friday April 27, 2007

Filed under: computing,statistics — jackman @ 2:19 pm

Sarah Anderson and Laurel Harbridge stopped by with an interesting question: how to compute where two densities cross. This is what I came up with: a simulated case where data come from a t distribution with 4 df. We fit a non-parametric density estimator to the data, and we overlay a normal density, using the mean and variance of the data as the sufficient statistics. A few lines of R later, we have graph de jour, code below the fold.



Comments Off

Will Therese and $58M become an issue?

Monday April 23, 2007

Filed under: Australian Politics — jackman @ 2:38 pm

Today’s Sydney Morning Herald reports that Kevin Rudd’s wife, Therese Rein, heads up a business that has contracts with the (Commonwealth) Department of Employment and Workplace Relations worth about $58 million a year. The article pursues the obvious question to ask under the circumstances: what about a conflict of interest should husband Kevin become Prime Minister? According to the Herald, Rein said:

“What Kevin has said, and it’s for him to do, is that he will seek advice from Dr [Peter] Shergold [of the Prime Minister's department] and that advice is about what probity processes can be put in place to make sure that there is accountability, transparency and fairness in the administration of these contracts. And then we’ll see what the advice is, and I don’t want to prejudge that.”

I doubt it, Therese. Frankly, I don’t know how there is anything other than a conflict of interest here, or (perhaps more to the point) anything other than a perceived conflict of interest. Maybe Rudd thinks that he is so far ahead in the polls that he can have it both — being Prime Minister and retaining his (indirect) interest in his wife’s business. But with Howard looking for chinks in the armour, stand by for more on this. Mind you, it is not exactly that the Coalition has a wonderful record when it comes to conflicts of interest (although I can’t recall an analogous case, at least in terms of magnitude, for any Coalition minister etc).

One of my colleagues made an excellent point on this issue. He was aghast that Rein could say “Kevin will seek advice…” (i.e., future tense). An election is perhaps 6 months away? Rudd’s been Opposition Leader since before the New Year. And they are yet to get advice on how Rudd can be PM while his wife’s firm lands contracts from the Commonwealth worth $60 million a year? And the implicit tone of the quoted passage from Rein jars politically as well: i.e., “What Kevin has said, and it’s for for him to do, is that he will seek advice….”. I mean, isn’t that the way Howard talks about his ministers when they get in ethical trouble? Not a good look…

Again, I wonder if they are just not taking this seriously, betting on voters not caring, or the government unwilling or unable to credibly attack them on this issue?

Comments Off

cool poster by Jim Winters

Tuesday April 17, 2007

Filed under: type — jackman @ 12:53 pm


The lobby of our building attracts lots of posters and flyers for events on campus. A particularly eye-catching poster went up over the weekend, with the lavish, high-contrast, use of DIN helping to attracting my attention (click on the thumbnail below). Closer inspection reveals it to be the work of Jim Winters; his web pages have other samples.


Comments (1)

who is Googling me?

Filed under: computing,general — jackman @ 12:43 pm

I found an apache module that writes apache log entries to a mysql database. I then hacked together some PHP to grab all the referrals to my web pages that appear to have come from Google searches. The results are here. It turns out that my blog post on the Palo Alto caterpillar infestation is generating a few hits. Weird….

Comments (1)

craziest rollover effect ever?

Sunday April 15, 2007

Filed under: general — jackman @ 11:05 pm

Ralph Ferone to Mal Enright to me to you: this mad Japanese web-awards site.

Comments Off

testing for overdispersion in count models

Thursday April 12, 2007

Filed under: computing,statistics — jackman @ 2:08 pm

I have a function called odTest in my R package, pscl. John Fox spotted an error, which I’ve fixed and will push up to CRAN in pscl version 0.76 tonight. John wrote:

Dear Simon,

I took a look at the odTest function in the pscl package and have a question
about how you compute the p-value for the test. As I understand it, the
critical value of chisquare for a test at the alpha level should be qchisq(1
- 2*alpha, 1) rather than the usual qchisq(1 – alpha, 1). This seems the
opposite of what’s implied in ?odTest. Moreover, in computing the p-value,
odTest() returns 1 – pchisq(d, df = 1); shouldn’t this be (1 – pchisq(d, df
= 1))/2 — or a bit better, pchisq(d, df = 1, lower.tail=FALSE)/2 ?

Maybe I’m not thinking about this correctly, in which case my apologies.


I had this a little bit backwards in my implementation, but all should be well now (or soon, when the pscl update gets propagated through the CRAN).

Comments Off

OpenBUGS version 3.0 is released

Filed under: computing,statistics — jackman @ 12:49 pm

From the OpenBUGS list:

I’ve just had the excitement of putting up the latest version of OpenBUGS: version 3.0. The zip file is available from here: http://www.math.helsinki.fi/openbugs/OpenBUGS.zip and there are notes on what’s new here:

Just to whet your appetite, with this version, you can now save a BUGS run (externalise), and re-load it later (internalise), if you want to continue. Jolly useful when you have a long run, you can save the run in the evening, so you don’t loose everything when Windows decides to upgrade and reboot itself.

Enjoy the new version: anyone want to volunteer to mend the multivariate CAR?


Bob O’Hara

Dept. of Mathematics and Statistics
P.O. Box 68 (Gustaf Hällströmin katu 2b)
FIN-00014 University of Helsinki

Comments Off

Income Inequality Over the Long Run (Scheve and Stasavage)

Wednesday April 11, 2007

Filed under: politics,statistics — jackman @ 1:47 am

David Stasavage (NYU) was here on Monday, giving a talk based on work with Ken Scheve. It was a compelling talk, with some interesting issues; statistically, politically, normatively…

David and Ken have cross-national data on wealth shares (the proportion of a country’s GDP going to the top 1% or top 10% of the income distribution; the bigger this number, the more inequality), spanning most of the 20th century and about 14 advanced, industrial democracies. David had a very interesting graph, simply displaying the country-specific trends in the variable, all overlaid on each other; graphs like this are among the most simple yet most informative things you can do with these kinds of panel/repeated measures data, and I wish people would display them more often when working with that kind of data. The lack of color and the small size made it hard to see the specific country patterns that well, but in no small measure, that is precisely the point of the graph (i.e., the common time trend soaks up a lot of the variation in the data): I cut-and-paste this graph from the PDF of the paper as a JPG, below. We kept coming back to the graph in the discussion and for good reason: the inequality data start off high and relatively dispersed, but trend down from about the late 1930s onwards, hitting a more or less steady state through a lot of the 1970s and 1980s, with remarkably little dispersion; the recent, large, increases in inequality in the United States are clearly apparent, as well as a little more dispersion than elsewhere in the series, and a slight trend upwards from what looks like a global minimum around about 1980.


The JPG above is a thumbnail: you can click on it to get it up to about 780px wide in a separate window.

As the graph makes clear, the most compelling feature of the data is the strength of the common time trend: in response to a question I asked, David told us that about 70% of the variation in the data is attributable to a common time trend (i.e., the [tex]r^2[/tex] you get from simply regressing the dependent variable on a set of year-specific dummy variables is about .7). That is clearly apparent from the graph David showed us, but was really quite large relative to what I expected a priori. The cross-national variation is relatively small (and was really quite small in the 1960s-1980s). The covariates David had available for analysis largely pick up cross-national variation and display very little longitudinal variation, and so it is not surpising that they do very little in the analysis, with the possible exception of union density, which does trend around within countries (but the other controls are quite serially persistent, like dummies for presence/absence of universal suffrage, PR electoral system, centralized wage bargaining). It was interesting that tax rates and transfers weren’t in the analysis, or at least not that I remember.

Jim Fearon and I were quite struck by the conclusion that David gave us, and was vividly demonstrated by the graph I referred to above: that the cross-national variation in inequality observed in recent decades is small relative to the longitudinal variation one sees when taking a longer, historical view at the phenomenon. Political scientists’ interest in institutions (constitutional arrangements, electoral systems, etc) doesn’t seem to have much to do with the phenomenon at all, at least not relative to the strong, common time trend in the data.

Josh Cohen pointed out that even when the cross-national variation is at its minimum, we’re talking about a doubling of the level of the dependent variable across the countries in the data set: i.e., those kinds of differences across countries mean a lot, both normatively and politically. Fair enough, but if one was interested in trying to do something about levels of inequality, then the long-run historical analysis presented by David shows that it is more likely to be massive, supra-national processes that will generate shifts in inequality, rather than institutional or political innovations within any single country. An important exception is the recent, large increase in inequality in the United States over the last 15-25 years: the U.S. has clearly broken away from the pack, on a trajectory somewhat at odds with the rest of the data.

The data also had some other interesting features: missingness that they said they handled via multiple imputations. For all the reasons listed above it sounds like a great teaching data set and I hope to get my hands on it eventually.

Comments Off

/usr is not the same thing as /User

Tuesday April 10, 2007

Filed under: computing — jackman @ 9:32 pm

So someone deletes /usr on their Mac, not understanding its significance. Some 3rd party piece of software changed the permissions on /usr (so this user could see /usr for the 1st time); he deletes /usr and meltdown ensues… See the post on Apple Matters and the Digg firestorm.

Comments Off

No Future

Saturday April 7, 2007

Filed under: general — jackman @ 9:36 pm

We went up to San Francisco this afternoon to the De Young to check out the Vivienne Westwood show (inter alia). So weird to be paying $15 a head for the reverential museum presentation of the Punk stuff I saw virtually every day on the street, at Uni, at gigs, back in the early 1980s etc. Lovely to see the mug of Sarah Stockbridge large and luminious on the walls, on the catalog:

The punk stuff aside, Dame Westwood’s more recent output (which you encounter further into the retrospective) seems more in keeping with what you’d expect to find in a grand musée like the De Young; acres of fine dark taffeta, over the top bejeweled jackets, as much haute as you can handle.

Incidentally, the Museum Store at the De Young runs across two stories, and probably occupies as much square footage as the sizeable museum store at MOMA in NYC. There was also a separate store at the De Young dedicated to the Westwood display. I’m trying to remember my first visits to museums as a kid; did museums even have stores in the 1970s?

Comments Off
Next Page »

Powered by WordPress

Bad Behavior has blocked 3734 access attempts in the last 7 days.