The following originally appear on Twitter as an epic thread. It’s been adapted and lightly edited for this website.

A new report is out on @GRTCTransit‘s year-old system redesign, and it shows benefits that starkly contradict a prior report by @Wilder_Planning‘s CURA. The CURA report is so flawed that the authors should immediately retract it. Let’s take a look.

First, the topline: the new system increases the number of jobs the average resident can reach in 45 minutes by 6%, and the number of jobs the average low-income and minority resident can reach by 10%. Better on average, in particular for disadvantaged populations.

So how did CURA find that access for disadvantaged populations decrease by 22%. Well, besides using methodology that doesn’t make sense and is not generally accepted in the field, they used the methodology incorrectly! 😲

CURA took all the bus stops (red) and calculated all the areas within a half a mile (blue) and considered loss of access for areas over half a mile (white). As you can see, some of the areas considered over half a mile from bus stops are bus stops themselves! 🤔

Obviously, this makes no sense, but sometimes data spits out weird results. That’s why you need to do a sanity check and, if you find something wrong, fix your analysis. Unfortunately, CURA didn’t, and just published, and when this was brought to their attention they did nothing.

What’s the result of this lack of simple checking? Take a look at Creighton Court. It is two large parcels. The parcel on top, in yellow, has 0.2% of it more than 1/4 mile from a bus stop. So CURA excludes the entire parcel from those having <1/4 mile to a bus stop.

Now you would think, “Well, if 0.2% is outside 1/4 mile, you should count 99.8% within a quarter mile, so of the 356 dwellings, include 355, especially since the 0.2% is grass.” CURA disagrees, and excludes the whole parcel, so all 356 units “lost access” in their report.

“Well, using this methodology is bad, but at least it affected both systems equally, right?” you might ask. Wrong again. You see, CURA noticed that some parcels, especially housing courts, were being excluded, so they manually included them…only for the old system!

[w]e were concerned that many multi-family and public housing courts were not completely contained in the polygons due to their size even though they overlapped with the coverage polygons, and made an effort to accurately include them in both old and new polygons.

Yes, the CURA report manually fudged the numbers while analyzing the old system but not the new one! At best it’s sloppy; at worst, dishonest. Note that when trying to replicate the results (CURA vs QGIS), the new network is the same, but the old is different, due to fudging.

Don’t be scared by the math, but the formula CURA used isn’t right either. The first one, from a peer-reviewed paper, has seven variables, while CURA’s, below, has five. They just eliminated two variables! One is activity density, the other is speed. Yes, speed disappeared!

It doesn’t take a transit expert to realize that the faster a bus goes, the better the service is. While the peer-reviewed experts incorporated speed as a key element, CURA just lops it off. 🤷‍♀️ It’s not even hard to figure out: distance/time. The schedules are all online.

CURA messes up the frequency factor too! It’s easy to look up, but they instead pulled the data from a source that isn’t updated, so they got a ton of weird results, including saying a training platform with no regular service gets a bus every 7 minutes!

Want more sloppy data handling? CURA comingled weekday and weekend times when calculating how often a bus came. This shows two buses coming one minute apart, and then they just averaged it, producing a wait time that is half of reality. This mistake is EVERYWHERE.

CURA also just assumed all buses ran 12 hours a day, even though some run only at rush hour and some run 18 hours a day, making the most frequent and useful routes seem worse and the commuter ones (mostly in rich areas) seem better.

As a result, for peak-only services like Routes 26, 27, 28, 29, and 64, which actually operate about 6 hours per weekday, the hours of operation have been overstated by more than double. And for routes that run later in the evening, like The Pulse, 1, 2, 3, 4A, 4B, 5, 12, 13, 14, the hours of operation have been understated by a third: those long-span routes run 18-19 hours per weekday.

For the distance factor instead of looking at how long a route is (more things to go to), CURA looked at how far a stop was from other stops (long trip). Thus Short Pump had some of the best service, while downtown had the worst. They multiplied when they should have divided!

CURA then decided to grade more positively a stop if it has more routes serving it. So a stop with two routes that run once a day is better than one with one route that runs every 15 minutes. A quick thought experiment and you should easily spot the error. CURA didn’t.

That’s right: of the 7 variables in the equation CURA took from peer-reviewed researchers, they used 5 incorrectly and didn’t use the other 2 at all. There’s not a single variable CURA used correctly. Not one! How can we trust any of this? Garbage in, garbage out.

I could go on, as there’s lots more, but it’s clear that CURA has no idea what it’s doing here. They do really good work in a lot of fields, but their expertise is not in transit, and anything they put out in this field should clearly be ignored.

We all have specialties and weak spots, and when venturing into another field it’s good to speak to those experts before publishing. Unfortunately, it seems they didn’t do this, and when I and others brought up these concerns after publication, they were kind but didn’t change.

It’s really disappointing that such shoddy research is sullying such a respectable organization. If CURA has any honor, they’ll immediately retract their report. This doesn’t live up to their academic tradition.

It’s worth adding that @VCU put the @Wilder_Planning CURA report on their front page. I hope, in the vein of intellectual honesty, when this paper gets retracted they offer up a mea culpa that is similarly prominent, as it is to reporters. cc: @ByRobertoR @Suarez_CM /fin