Posts filed under Just look it up (216)

November 9, 2014

The world’s most profitable crop?


This chart is from a beautiful infographic about cash crops.  I don’t believe the cannabis revenue number. That’s partly because I read Keith Humphreys and Mark Kleiman on the subject.

Keith Humphreys takes apart a claim of $120 billion for the total value of the US marijuana market, showing that it can’t be anything near that much.

Current pot smokers report that they use marijuana an average of 60 days a year. Using our current example, 40 ounces/60 days of use means that the average user would have to go through 2/3 of an ounce of marijuana on each day that they used marijuana. That’s .67 X 50 or 33.5 joints per day of use. And there’s a terrific bridge for sale in Brooklyn too.

Even then, the purported $120 billion was the price to the consumer.  That’s not what was used for the legal crops, and it makes a big difference.

Suppose we agree use consumer price rather than farmer revenue because the data are slightly more reliable. I don’t really believe a number above about $12 billion for the US.  The US has about 1/5 of the world GDP. If the US spent $12 billion/year on cannabis, the rest of the world would need to spend almost $300 billion, or more than six times as much as a fraction of their income.  A lot of the world would need to spend more on pot than on basic carbohydrates.

It’s not inconceivable that the number is right — maybe cannabis is really big in, say, Brazil or India and I just don’t know about it — but it’s surprising enough that I’d want a lot more detail to justify it.

August 29, 2014

Getting good information to government

On the positive side: there’s a conference of science advisers and people who know about the field here in Auckland at the moment. There’s a blog, and there will soon be videos of the presentations.

On the negative side: Statistics Canada continues to provide an example of how a world-class official statistics agency can go downhill with budget cuts and government neglect.  The latest story is the report on how the Labour Force Survey (which is how unemployment is estimated) was off by 42000 in July. There’s a shorter writeup in Maclean’s magazine, and their archive of stories on StatsCan is depressing reading.

August 21, 2014

Auckland rates arithmetic

In today’s Herald story about increases in rates and impact on renters it’s not that the numbers are wrong, it’s that they haven’t been subjected to the right sorts of basic arithmetic.

The lead is

Auckland landlords are hiking rents amid fears of big rates increases next year on the back of spiralling property values.

and later on

Increases in landlords’ expenses, including rates, mortgage interest rates and insurance premiums, could push up rent on a three-bedroom Auckland house by between $20 and $40 a week, he said.

Including‘ is doing a lot of work in that sentence. The implications are particularly unfortunate in a story targeted at renters, who don’t get sent rates information directly and are less likely to know the details of  the system.

The first place to start is with a rough estimate of how much money we’re looking at. One of the few useful things the Taxpayers’ Union has done is to collate data on rates, hosted now at Stuff. The average Auckland rates bill was $2636.  That’s all residences, not three-bedroom houses, but the order of magnitude should be right. An annual bill of $2636 is $50/week. If the average total weekly rates payment is around $50, the average increase can’t reasonably be a big fraction of $20-$40/week or there’d be a lot more rioting in the streets.

Anyone who owns a house in Auckland or checks the Council website should know there is a cap on rates increases to cover the neighbourhoods where prices are increasing fastest. The cap is 10%/year; no rates increase faster than that, and most increase slower.  To get more detailed information you’d need to look at the website describing 2014/2015 rates changes, and find that the average increase for residential properties is 3.7%, then calculate that 3.7% of $50/week is about $2/week.

According to the Reserve Bank, both floating and two-year-fixed mortgage interest rates have gone up 0.5% since last year.  That’s $9.60/week per $100,000 of mortgage, so it’s likely to be a much bigger component of the rental cost increase than the rates are.

The average increase in rates is a lot slower than the increase in property prices (10% in the year to July), but you’d expect it to be. The council doesn’t set a fixed percentage of value from year to year and live with real-estate price fluctuations. It sets a budget for total rates income, and then distributes the cost using a combination of a fixed charge and a proportion of value. In other words, the increase in average real-estate prices in Auckland has no direct impact on average increase in rates — it’s just that if your house value has gone up more than average, your rates will tend to go up more than average.   Increases in average real-estate price obviously do lead to increases in rental price, but rates are not the mechanism.

The Council is currently working on a ten-year plan, including the total rates income over that period of time. It will be open for public comment in January.


August 14, 2014

Breast cancer risk and exercise

Stuff has a story from the LA Times about exercise and breast cancer risk.  There’s a new research paper based on a large French population study, where women who ended up having a breast cancer diagnosis were less likely to have exercised regularly for the past five year period.  This is just observational correlation, and although it’s a big study, with 2000 breast cancer cases in over 50000 women, the evidence is not all that strong (the uncertainty range around the 10% risk reduction given in the paper goes from an 18% reduction down to a 1% reduction).  Given that,  I’m a bit unhappy with the strength of the language in the story:

For women past childbearing age, a new study finds that a modest amount of exercise — four hours a week of walking or more intensive physical activity such as cycling for just two hours a week — drives down breast cancer risk by roughly 10 per cent.

There’s a more dramatically wrong numerical issue towards the end of the story, though:

The medications tamoxifen and raloxifene can also drive down the risk of breast cancer in those at higher than average risk. They come with side effects such as an increased risk of deep-vein thrombosis or pulmonary embolism, and their powers of risk reduction are actually pretty modest: If 1000 women took either tamoxifen or raloxifene for five years, eight breast cancers would be prevented.

By comparison, regular physical activity is powerful.

Using relative risk reduction for the (potential) benefits of exercise and absolute risk reduction for the benefits of the drugs is misleading. Using the breast cancer risk assessment tool from the National Cancer Institute, the five-year breast cancer risk for a typical 60 year old is perhaps 2%. That agrees with the study’s 2000 cases in 52000 women followed for at least nine years.  If 1000 women with that level of risk took up regular exercise for five years, and if the benefits were real,  two breast cancers would be prevented.

Exercise is much less powerful than the drugs, but it’s cheap, doesn’t require a doctor’s prescription, and the side-effects on other diseases are beneficial, not harmful.

August 6, 2014

Income statistics

The Herald has a story headlined “Where to work if it’s money you’re after,” giving estimated median incomes across a range of job areas.  Sadly, if you read to the end, two of the sources are summaries of advertised salaries for advertised jobs on Seek and TradeMe.  That is, they are neither actual incomes, nor for the country as a whole.

Rather than just whinge about unrepresentative data, I looked at StatsNZ. They divide things up differently, so there was only one job group in the story that exactly matched one on NZ.Stat. People working in construction have a median weekly income of $840 and mean weekly income of $956 according to the NZ Income Survey. If most people in construction worked all year, without periods of unemployment, this would come to a median annual income of  $43,680 or a mean of $49,712.

The Herald thinks the median annual income in construction is $60,000-$78,000.



July 28, 2014

Misleading maps

This map, from Reddit, shows the most common name in each county of England and Wales in 1881, based on the 1881 census.


Matthew Yglesias at  says what’s remarkable is how nearly perfectly the Smith/Jones divide lines up with the political boundary between England and Wales”.  I think it’s remarkable that he think’s it’s remarkable — I think of ‘Jones’ as the stereotypical Welsh name — but obviously associations are different in the US.  It is worth pointing out that the line-up isn’t as good as you might think if you weren’t careful: three of the light-green counties are actually in England, not in Wales. 

Yglesias also says that the names seem to show pretty distinctively what part of the British Isles your male line hails from.” That’s an example of how maps are systematically misleading — the conclusion may be true, but the map doesn’t support it as strongly as it seems to.  The map shows the most common name in each county, and most of the counties where Jones is the most common name are Welsh. However, that doesn’t mean most people called Jones were in Wales. In fact, based on search counts from, Lancashire had more Joneses than any Welsh county, and London had more than all but two Welsh counties. Overall, only 51% of Joneses were in Wales, going up to 60% if you include the three English counties coloured light green on the map.

In this particular case, many non-Welsh Joneses probably did have Welsh ancestors who had left Wales well before 1881, but not all of them — according to Wikipedia, the name came from Norman French and the first recorded use was in England.

July 1, 2014

Does it make sense?

From the Herald (via @BKDrinkwater on Twitter)

Wages have only gone up $34.53 annually against house prices, which are up by $38,000.

These are the findings of the Home Affordability Report quarterly survey released by Massey University this morning.

At face value, that first sentence doesn’t make any sense, and also looks untrue. Wages have gone up quite a lot more than $34.53 annually. It is, however, almost a quote from the report, which the Herald embeds in their online story

 There was no real surprise in this result because the average annual wage increase of $34.53 was not enough to offset a $38,000 increase in the national median house price and an increase in the average mortgage interest rate from 5.57% to 5.64%. 

If you look for income information online, the first thing you find is the NZ Income Survey, which reported a $38 increase in median weekly salary and wage income for those receiving any. That’s a year old and not the right measure, but it suggests the $34.53 is probably an increase in some measure of average weekly income. Directly comparing that to the increase in the cost of house would be silly.

Fortunately, the Massey report doesn’t do that. If you look at the report, on the last page it says

Housing affordability for housing in New Zealand can be assessed by comparing the average weekly earnings with the median dwelling price and the mortgage interest rate

That is, they do some calculation with weekly earnings and expected mortgage payments. It’s remarkably hard to find exactly what calculation, but if you go to their website, and go back to 2006 when the report was sponsored by AMP, there is a more specific description.

If I’ve understood it correctly, the index is annual interest payment for an 80% mortgage  on the median house price at the average interest rate, divided by the average weekly wage.  That is, it’s the number of person-weeks of average wage income it would take to pay the mortgage interest for a year.  An index of 30 in Auckland means that the mortgage interest for the first year on 80% mortgage on the median house would take 30 weeks of average wage income to pay. A household with two people earning the average Auckland wage would spend 15/52 or nearly 30% of their income on mortgage interest to buy the median Auckland house.

Two final notes: first the “There was no real surprise” claim in the report is pretty meaningless. Once you know the inputs there should never be any real surprise in a simple ratio. Second, the Herald’s second paragraph

These are the findings of the Home Affordability Report quarterly survey released by Massey University this morning.

is just not true. Those are the inputs to the report, from, respectively, Stats New Zealand and REINZ. The findings are the changes in the affordability indices.

June 26, 2014

Slightly too Open Data

  1. The Atlantic published some visualisations of taxi rides in New York
  2. Chris Whong asked for the data under Freedom-of-Information laws, and got it. Of course, the taxi and driver ids were anonymized
  3. Vijay Pandurangan noticed that the driver id and taxi id were really, really weakly anonymised.
  4. You can find out a lot once you know the taxi id.


The NY Taxi & Limousine Commission had run the ids through a cryptographic hash function, MD5. Hash functions are designed so that if you don’t know anything about the input you can’t reconstruct it from the output, but if you know the input exactly, you can verify easily that it gives the same output.  The problem comes when you know a lot about the input, but not everything.  In this case, there are only about two million possible id numbers, and you can just try them all. Once you have the ids, you can look up.

Even if the taxi authorities had done the anonymisation correctly — replacing each id with a random number — it would inevitably have been possible to extract some of the ids with a bit of work.  That’s not the same as being able to extract all of them with a few hours’ computer time.

June 11, 2014

But did he ever return?

An excellent visualisation of very detailed data from the Boston subway system:

Boston’s Massachusetts Bay Transit Authority (MBTA) operates the 4th busiest subway system in the U.S. after New York, Washington, and Chicago. If you live in or around the city you have probably ridden on it. The MBTA recently began publishing substantial amount of subway data through its public APIs. They provide the full schedule in General Transit Feed Specification (GTFS) format which powers Google’s transit directions. They also publish realtime train locations for the Red, Orange, and Blue lines (but not Green or Silver lines). The following visualizations use data captured from these feeds for the entire month of February, 2014. Also, working with the MBTA, we were able to acquire per-minute entry and exit counts at each station measured at the turnstiles used for payment.

[No, he never returned]

June 8, 2014

Foreign drivers

From the ChCh Press

Foreign drivers cause more fatal and injury crashes in the South Island than the national average – and the West Coast is the worst spot.

They don’t actually mean “more,” they mean “a higher proportion of”.

New Zealand Transport Agency (NZTA) safety directions chief adviser Lisa Rossiter said its crash statistics for the past 10 years showed foreign drivers were involved in about 6 per cent of all fatal or injury crashes in New Zealand, and were at fault in about 2 per cent.

On average, short-term visitors make up roughly 2.5% of people in New Zealand (2.78 million visitors in the year to April 2014, median visit of 9 days, so I’m guessing mean visit about two weeks). About another 2% of people in New Zealand are international students, who are at least sometimes counted as foreign drivers.

So, the risk seems to be a bit higher for foreign drivers, but probably not twice as high. Some of the excess can probably be explained by age: international students, backpackers, and drunk Australians in Queenstown are younger than the population average.

It’s different in parts of the South Island

The tourist hot spots of Otago and the West Coast fared worst.

A foreign driver was identified as a factor in 13 per cent of fatal crashes on the coast, and 5 per cent of fatal crashes in Otago from 2004 to 2013.

A lot of this must be because tourists are over-represented in tourist hot spots: that’s what ‘tourist hot-spot’ means. The proportion of short-term visitors is about 2.5% nationwide, but it’s probably rather lower that than in Gisborne and rather higher on the West Coast.

It’s also worth noting that “identified as a factor” is fairly weak. If you go to the Ministry of Transport reports and add up the percentage of times different factors were involved in a crash, you get a lot more than 100% (for the 2010 report I get 225% for fatal crashes and 185% for injury crashes)

For crashes involving a tourist driver and more than one car, the foreign driver was fully or partly responsible two out of three times.

This at least gets rid of the denominator problem, but the “partly” responsible is still a problem. We aren’t told what proportion of the time the local driver was fully or partly responsible — based on the information given, that could also be two out of three times.

It’s quite likely that foreign drivers are at higher risk, especially those from countries that drive on the right, but the problem is not a big fraction of the NZ road toll. It’s worth considering things that can sensibly be done to reduce it — which doesn’t include withdrawing from the U.N. Convention on Road Traffic — but if you’re trying to stop road deaths it may be more effective to concentrate on interventions that don’t just affect foreign drivers.  Clearer signage, guard rails and median barriers, separated bike lanes, improved public transport… there are many things that might knock a percentage point off road deaths more easily than targetting foreign drivers.