By Darrell Oyer, Darrell J. Oyer & Company:
A canned DCAA presentation contends that the quality of audits has improved as evidence by the fact that in FY2003 DCAA examined $265b and questioned only $8b (3%); but in FY2011 DCAA examined only $128b but questioned $12b (9%). So they are doing less but doing a better job!
Don’t you believe it! As most people will tell you, DCAA is questioning more costs, not because of better work, but because of a more outrageous questioning of costs. For example, I have had one situation where DCAA would take credit for over $4m in corporate allocations but never did an audit—they merely rejected the entire allocation based on the wording in the company’s annual report. In another instance, DCAA questioned $2m based on the position that direct labor should not be included in the contractor’s G&A base. And then there are many dollars of compensation questioned that will not stand up based on recent ASBCA rulings.
The test of quality is the percentage of cost questioned sustained—as was the agency metric in 1981. Now, the DCAA annual report no longer even mentions such a metric.
In addition to Darryl Oyer’s comments (discussing flaws in DCAA’s claim to increased cost questioned as a percentage of dollars examined), even if the FY2011 data was valid, the comparison to prior years is a DCAA distortion of comparable results or trends. In particular, FY2011 is predominantly bid proposals where audit exceptions as a percentage of dollars examined approach 10%; however, prior years have significantly more incurred cost audits where audit exceptions were less than 1% of dollar examined. The use (by DCAA) of composite percentages (cost questioned as a percentage of total dollars examined) is intentionally ignoring the fact that the types of audit or “mix” will significantly impact the composite results. Nothing like presenting data as if it is comparative (apples to apples) when it is anything but comparative. Analogous to a company reporting comparative revenues across multiple years without disclosing that later years reflect higher revenues as the result of an acquisition. DCAA is also hiding the fact that audit hours now required to examine dollars is probably 5-times the hours to examine dollars in 2008 and before. Hence, the supposedly improved audit results are actually worse than prior years when measured against the cost of completing the audits (when they actually complete an audit).
I agree with Michael. Questioned cost percentages mean nothing. Questioned cost sustained is a better measure of success. However, even this statistic is suspect because in order to figure out what was sustained, the auditor must analyze Government Price Negotiation Memorandums (PNMs) written by someone in the contracting officer’s shop. These PNMs are often written in such a way that it is difficult to determine whether reductions from a contractor’s proposal were the result of audit input or other factors (such as technical evaluations or price analysts’ reports. Given that void, auditors tend to take credit for reductions even when not readily evident.
Comparing 2003 with 2011? Really?
Ask an auditor who has been around awhile (who’s willing to be candid) whether they think DCAA is doing a better job now days and you won’t get an answer anywhere close to the “canned DCAA presentation”.