When a Good Rating Doesn't Mean Much
By Allan Holmes | Thursday, March 06, 2008 | 02:20 PM
By Allan Holmes | Thursday, March 06, 2008 | 02:20 PM
The Risk Factor blogger Bob Charette, a risk management expert who consults with federal agencies on risk management, picked up yesterday's story on the deep trouble that the Census Bureau's handheld computer contract is in. In his blog post, he questions the credibility of the Capability Maturity Model Integration (CMMI®). Harris Corp.'s Government Communications Systems Division, which is the prime contractor on the $600 million handheld contract (now likely much more than $650 million after all the costs from changes, errors and delays are included), has a Maturity Level 3 rating. "The Level 3 rating denotes superior process maturity within the division's program management, engineering, quality assurance, and other disciplines, and achievement of this rating has become a competitive differentiator on many government programs," Charette quotes.
Charette wants to know: "At the very least, I think the division's CMMI rating may need to be re-evaluated, or maybe better, the U.S. government better start looking at what, if anything, SEI CMMI Level 3 actually means in practice."
Or it could mean, the customer, the Census Bureau, put too many demands on Harris -- so many, in fact, that no maturity designation, no matter how high, could have avoided the very problems that now threaten the viability of the U.S. census.
Comments
Two things:
First, no matter the level of maturity of the contractor, their performance will be no greater than the maturity of the agency who manages the contract. I heard Watts Humphrey speak to a group of DCMA QA people after the CMM for software was released. He described the "lowest common denominator" problem.
Second, because so many procurements have minimum maturity requirements to be able to bid, corporations are encouraged to be assessed, and have that assessment apply to as large a portion of their company as possible. The broader the coverage, the less meaningful the assessment. Perhaps crafting an award fee plan that evaluted evaluted compliance with targeted specific and generic practices in process areas most applicble to that procurement would be a better approach than merely asking the bidders to show "they got the CMMI coffee mug and the T-shirt". This assumes, though, that the government has the ability to write an effective award fee plan that accurately measured applicable areas. If U.S. Census were operating at the maturity level they expect of their contractors, that should not be an issue.
Michael | Friday, March 14, 2008 | 11:18 AMCMMI certification isn't unlike ISO with regards to actual practice. "say what you do and do what you say" It's the do what you say part that needs to be verified.
Mike | Friday, March 07, 2008 | 01:52 AMABOUT THIS BLOG
Allan Holmes on what's happening and what's being discussed in the world of federal information technology.
feed