A couple of weeks ago, I posted a link to the new nyc.gov Citywide Performance Reporting (CPR) system. In that post, I summarized the rather spotty performance of the Department of Buildings, at least as measured by the criteria of CPR. One number that jumped out at me was the 6% rate of violations issued against complaints for after hours work. It seems rather remarkable that citywide, 94% of complaints regarding after-hours construction could be unfounded. There are many reasons why complaints might not pan out - bad addresses, delays in inspection, malicious 311 dialing, and so on - but that all those reasons add up to 94% of the complaints registered for after-hours construction work is a tad mind boggling.

And as I noted then, because CPR does not report the number the total number of complaints, we can't tell what the magnitude of the after-hours work problem is. Thus, it is impossible to tell what that 6% is 6% of. Are there dozens of complaints per year? Hundreds? Thousands?

Well as it turns out, DOB does issue reports on the number of complaints that it receives, and the number for Community Board #1 is 436. Citywide, the number of complaints per month (in all categories) is in the thousands - close to 5 figures in some months. For CB1, there were 3,854 DOB complaints in calendar 2007, which means that just over 11% of the complaints registered were for after-hours work (weekends and evenings). Wading through the data, it soon becomes clear that 6% is not the most pathetic figure (and if anyone from DOB wants to tell me how I might have waded astray, by all means drop me an email).

DOB only sort of reports on all of the complaints it receives. What DOB issues is weekly and monthly lists of all complaints received, with a coded resolution for each complaint. In terms of tracking complaints, the weekly reports are great; in terms of tracking complaint resolution, they are near useless. That is because DOB doesn't always respond to complaints in the week in which they occur, and complaints that are resolved in following weeks do not show up in the weekly lists. The same things happens for the monthly list, but there is a bit less noise in the data, as DOB has more time in which to enter resolutions into their database. (You can, of course, look up complaints individually on BIS, which will tell you the resolution, regardless of whether it occurred in the same week or month as the complaint. Until I get an intern, I'm not looking up the status of 3,800+ complaints in CB1.)

This flaw in the data reporting is irrelevant when it comes to complaints for after-hours work, which makes this complaint category ripe for analysis. It is irrelevant because after-hour complaints are particularly time sensitive. If you call about illegal construction at 5:00 p.m. on a Sunday, an inspection at 9 a.m. the following morning is tantamount to no inspection at all. So unresolved complaints can be pretty safely lumped in with complaints that were inspected more than a day after the infraction - in either case, by definition an inspector will not see the infraction (unless those Priuses can travel back in time). This, of course, is not news.

There are other methodological problems. One is that the monthly reports list the date of the last inspection. So if an inspector does happen by, and issues a violation or stop work order, and that order is then lifted a week later, it is the lifting of the order that appears on the monthly report. And the date of the lifting is the date of the last inspection, making it appear that DOB took a week to get to the site. But because the number of resolved complaints is relatively low, its actually fairly easy to weed these problems out.

CHART 1: CB1 after-hours complaints, 2007
(Source: DOB, unadjusted numbers based on monthly complaint disposition reports)

So of the 436 complaints in calendar 2007, how many wound up being inspected? A rather disappointing 53% (see Chart 1). That's right, almost half the complaints logged were not inspected in the month in which the complaint was made. Which means they might as well stay uninspected.

Further, 38% (166) of the complaints resulted in no action. If you look up the complaints, you'll see that this was usually because the inspector found no illegal work in progress at the time of inspection. But you'll also see that the vast majority of these complaints were not inspected for days. Which means, once again, they might as well have not been inspected at all.

When you combine thsee uninspections with the noninspections, fully 77% of the complaints for after-hours work turn out not to have been inspected in a timely manner. Is it any wonder that only 6% of complaints citywide result in violations? At a maximum, 25% of all complaints are inspected in a timely manner (i.e., in a time frame in which its reasonable to expect that the violation might still be occurring). On that basis, DOB has a slightly more impressive rate of violations of 30% (sure, its 30% of 25%, but still, it looks better). Although at least in CB1, the majority of the violations issued actually have nothing to do with after hours work. Instead, inspectors wind up issuing violations for other infractions. All told, only 11 stop work orders were issued in all of 2007 for after-hours work. In CB1, chances are there are at least 11 sites operating illegally on any Sunday.

What of the other complaints that were inspected? Another 30 cases (31% of "good" inspections) appear to have been inspected on the date that the complaint was logged and no illegal work was found. Based on a random sampling, about a third of those wind up being sites that had a variance for after-hours work, and were miscoded in the system. So in reality, there are probably 20 or so complaints that were inspected on a timely basis and turned out to be truly unfounded. Only 5 cases (5% of "good" inspections) turn out to have variances for after-hours work (add to this the misreported variances mentioned above, and you get up to about 15%). Another 22% (21 cases) are coded as "previously inspected". In many cases these represent multiple calls on the same day, but (again, based on a random sample) in about 1/3 of those cases, the complaint was for a completely different day. So add those (7 or so) cases to the list of "noninspections". The remainder (about 10% of "good" inspections) fall into "other" categories - complaints referred to other agencies, bad addresses, sites where there was no access).

To recap, in CB1, where there is a construction site on every block, there were 436 complaints for illegal after-hours work in calendar 2007. Just over 75% of those complaints were not inspected in a timely fashion (see Chart 2), and roughly half were not inspected at all (or at least not in the same month as the complaint was made). As a result, the vast majority of the inspections that are made result in resolutions that say "No illegal work at the time of inspection", though clearly the (alleged) illegal work was going on the weekend before (or the night before) said inspection. Just under 7% of the inspections result in violations or stop work orders (not always for after-hours work), and a like number are inspected in a timely manner and turn out to be (truly) unfounded complaints. Just over 3% of the complaints were for sites with legitimate after-hours variances. About 5% of complaints are recorded as duplicates (although many of then aren't); just under 3% fall into the "other" categories.

In this light, 6% doesn't seem so pathetic.

CHART 2: CB1 after-hours complaints, 2007
(Source: DOB, numbers based on monthly complaint disposition reports, adjusted to
reflect timeliness of inspections)