Posts filed under General (580)

November 20, 2014

Not a good look

Clinical trials involve experimenting on humans, and so you want them to involve the minimum number of people and use the information as efficiently as possible. Part of that is committing in advance to what you expect the benefits of a treatment to be (the ‘primary endpoint’). If you got to look at the data first, and search for a favorable difference between the treated and untreated people you’d need a lot more evidence to convince people. That’s why clinical trial registration is important.

Derek Lowe, at In the Pipeline, has an unfortunate example of from biotech company studying stem cells in heart disease. Last year, the registration information at said

To determine safety and the effect of intracoronary infusion of AMR-001 on myocardial perfusion (RTSS), measured by gated SPECT MPI at baseline and six months in subjects post-STEMI


primary endpoint includes safety of bone marrow procurement (measured by adverse events) and AMR-001 cell infusion (including incidence of re-stenosis and stent thrombosis in addition to other adverse events) as well as efficacy measured by quantitative by gated SPECT MPI specifically looking at resting total severity score)

On November 17 this year it changed

To determine safety and efficacy of intracoronary infusion of NBS10.


The primary endpoint includes the occurrence of AE’s, SAE’s and Major Adverse Cardiac Events (MACE) and the assessment of myocardial perfusion measured by quantitative gated SPECT MPI specifically looking at resting total severity score.

From the company’s press release

  • A statistically significant mortality benefit (p<0.05) in patients treated with NBS10 (also known as AMR-001) as compared to the placebo group; there were no deaths in the treatment group.
  • A statistically significant dose-dependent reduction in SAEs (p<0.05).
  • Observation of a dose-dependent numerical decrease in MACE. MACE occurred in 14% of control subjects, in 17% of subjects of who received less than 14 million CD34 cells, in 10% of subjects who received greater than 14 million CD34 cells, and in 7% of subjects who received greater than 20 million CD34 cells.
  • No meaningful difference in perfusion, as evidenced by SPECT imaging, between the treatment and the control group from baseline to 6 months in resting total severity score (RTSS) suggesting this may not be a future suitable tool to assess NBS10, which is consistent with U.S. Food and Drug Administration (FDA) guidance that mortality and MACE are the appropriate approvable endpoints to determine efficacy of a cellular therapy for cardiac disease as opposed to imaging endpoints. 

That is, the trial failed to show a change where there were looking for one,  but found evidence for a reduction in other things — apparently after they knew the results.

November 18, 2014

What do statisticians do all day?

As usual about this time of year, our Honours and M.Sc. students are giving talks on their research projects

  • Modelling the natural inflows to New Zealand lakes
  • Refactoring the xtable package
  • Invertible reproducible documents
  • Bootstrap goodness of fit tests
  • Convex regression
  • Using the Robust Covariance Matrix Estimator to improve the precision of principal component eigenvectors in the orthogonal multivariate test
  • Orthogonalised multivariate survey-weighted linear models for medical data
  • Factors affecting catch composition in NZ scampi fisheries
  • Factors affecting tagging mortality in snapper
  • Assessment of rapid eradication assessment
  • Factors explaining the low income return for education among Asian New Zealanders
  • A comparison of methods used to reconcile forecasts in hierarchical time series
  • Numerical methods for drawing piecewise smooth curves
  • An evaluation of the Christian Broadcasting Association’s Appeal and Donorcom campaigns
  • Multi-choice and true/false assessments in introductory statistics: What can they tell us about student understanding?
  • Bayesian computation for exoplanet data
  • Modelling an Ophthalmology Clinic Booking List System: Assumptions and Implementation
  • Reinforcement Processes on Graphs
  • Influence analysis on phylogeny inference
  • Statistical analysis of chemical soil composition in the Wairau Valley
  • Describing the world’s nations
  • Parameter estimation of the coalescent in continuous space
November 13, 2014

School deciles

New Zealand has a national school funding system that allocates money to schools based on socio-economic data about students.  This isn’t self-reported individual-level data, but is at the level of Census meshblocks (details here.) Schools are divided into ten deciles, and more funding given to lower-decile schools.  Despite the higher funding, lower-decile schools, on average, have poorer results on standardised assessments.  You can see good visualisations of this from Luis Apiolaza,


There are less-good ones at Stuff: dot plots aren’t ideal for this, and it really is better to look at cumulative categories (‘at standard or better’) rather than individual categories (‘at standard but not better’). Unfortunately, these graphs tell you almost nothing about the policy question of whether there are better ways to target the funding. There might be; there might not be.

One advantage of the current system is its automatic stabilisation. The Herald, earlier this week, had a good story about changing ethnic profiles of schools, with the sort of combination of data and individual stories it would be nice to see more often.  It turns out the low-decile schools are seeing fewer students of European ethnicity, and more Māori and Pasifika students. The phrase ‘white flight’ was used, but because of the funding system this isn’t the same sort of problem as the original ‘white flight’ from US inner cities.

In the US, a lot of public school funding comes from local government. When more-affluent families leave an area, the government funding for education goes down.  In New Zealand, when more-affluent families leave an area, the government funding for education goes up.  There’s still a concern about diversity, but not the same sort of vicious circle that was seen in the US.

November 7, 2014

Measuring what you care about

From the Herald

According to co-founder Jackson Wood, many workplaces today use drug testing as a proxy for impairment testing. However, these are generally arbitrary or ineffective and not always reflective of potential employee impairment at the workplace.

Wood’s startup, Ora, is aiming to build a system that tests reliably for impairment. If it can be done, this would be valuable in NZ industries, and might well also attract interest from the US.  With the increasing number of states legalising cannabis, it is increasingly a problem that there is no simple and reliable proxy for driving impairment.


November 5, 2014



October 29, 2014


  • The Herald reports on a genetic study in Finland that found a couple of rare genetic variants which were about 2.5 times more common in people who had committed multiple violent crimes.  I don’t have anything criticise about the story, just a point about genetics. When you’re trying to interpret an association like this one from a philosophical or policy point of view, it’s helpful to note that roughly 95% of their extremely violent criminals carried a genetic variant present in only 50% of the population — an odds ratio more like 25 than 2.5.
  • A story and interactive tool at Fusion, showing how changes in youth turnout would affect the US election results next week (if they happened, which they probably won’t).
  • From Anthony Tockar at Neustar, how anonymised taxi ride data from New York could be used to track passengers, not just drivers.
  • And the same taxi data being used for good, via
October 24, 2014

Something in the air

There’s a story “Pollution can cause lung problems in unborn baby – research” in the Herald, which I’m not  convinced by, but the reasons are relatively subtle.

The researchers compared levels of traffic-related air pollution exposure for different pregnant women, and looked at the lung function of the children at age four and a half (press release).  The story gets the name of the main pollutant (nitrogen dioxide) wrong in two different ways, but is otherwise a good summary.  It’s all correlation, but weaker associations than this are fairly reliably estimated for short-term exposures to air pollution. Long-term exposure is different, and that’s what’s interesting.

Studies of short-term effects of air pollution compare the number of people dying or going to hospital on days when pollution is high to the number on days where pollution is low.  That is, the comparisons of pollution are for the same people and for the same air pollution monitors. There are a fairly limited selection of other factors that could explain the association — the main ones being related to weather.

Studies of longer-term effects compare people with high exposure to pollution and people with low exposure to pollution.  Actually, they don’t quite do that, because air pollution monitoring is expensive in labour and equipment. They compare people with high estimated exposure and low estimated exposure. Since we’re comparing different people, any factor that affects health and also affects where people live could cause a bias, and it’s very well established that poorer people tend to get exposed to more pollution, at least in cities. Also, since we’re comparing different air pollution monitors, there can be biases from how representative the monitors are of the local area.

These problems mean that it’s much harder to be confident about effects of longer-term air pollution exposure, even though these effects are likely to be bigger than the short-term ones. Fortunately, we don’t need to be sure of these effects in setting public policy. The main source of the pollution is traffic, and there are other independent reasons why we want to have fewer cars burning less fuel.

On the statistical generalisability of personal experience

Going by people I know in real life or on Twitter, you would think the majority of people brought up in the Mormon church become scientists. though I am informed this is not actually the case.

There’s an interview with one of them, Heather Hendrickson, in the Herald.

October 23, 2014

Official Information and Open Data

In recent years it has become much easier to just go and get routine government data. It’s now easy to put data up online, and organisations do it. We might whinge about how often the URLs and layouts change, but you can get and reuse information in ways that used to be impossible. For examples in just one field, see the blog of the NZ geodata company Koordinates.

On the other hand, non-routine requests seem to be increasingly difficult. David Fisher, of the Herald, gave a talk in Wellington last week on the Official Information Act. The talk has been published at Public Address

When I started, if I wanted to know about something, I would ring and ask. For example, if I want to know about how Kauri stumps were exported, I would ring up the equivalent of the MPI and ask how Kauri stumps get exported. I would then spend half an hour on the phone to the guy who oversaw the exporting – often the guy who was physically down at the docks – and I would be informed.

It seems a novel idea now. I can barely convey to you now what a wonderful feeling that is, to be a man with a question the public wants answering connecting with the public servant who has the information.

Things have changed, he says.

October 22, 2014

Currie Cup Predictions for the Currie Cup Final

Team Ratings for the Currie Cup Final

The basic method is described on my Department home page. I have made some changes to the methodology this year, including shrinking the ratings between seasons.

Here are the team ratings prior to this week’s games, along with the ratings at the start of the season.

Current Rating Rating at Season Start Difference
Lions 7.39 0.07 7.30
Western Province 5.87 3.43 2.40
Sharks 3.00 5.09 -2.10
Blue Bulls 1.17 -0.74 1.90
Cheetahs -3.53 0.33 -3.90
Pumas -8.20 -10.00 1.80
Griquas -10.10 -7.49 -2.60
Kings -14.91 -10.00 -4.90


Performance So Far

So far there have been 42 matches played, 31 of which were correctly predicted, a success rate of 73.8%.

Here are the predictions for last week’s games.

Game Date Score Prediction Correct
1 Lions vs. Sharks Oct 18 50 – 20 6.70 TRUE
2 Western Province vs. Blue Bulls Oct 18 31 – 23 10.00 TRUE


Predictions for the Currie Cup Final

Here are the predictions for the Currie Cup Final. The prediction is my estimated expected points difference with a positive margin being a win to the home team, and a negative margin a win to the away team.

Game Date Winner Prediction
1 Western Province vs. Lions Oct 25 Western Province 3.50