Liu: 911 Upgrade Cost City Millions
Audit Finds Negligence; Asks D.A. To Probe
Because of the severity of the findings and potential for fraud in both the vendor selection and billing processes, Liu has referred the matter to the Manhattan District Attorney’s Office for further review.
This follows a previous audit released by Liu in March that found the Emergency Communications Transformation Program (ECTP) had only one component up and running, was seven years behind schedule, and a billion dollars over budget.
Liu said the audit found that severe mismanagement by the Department of Information Technology and Telecommunications (DoITT) has resulted in the city taxpayers not getting what they paid for. In fact, the systems integration portion of the much needed 911 upgrade could cost an additional $362 million, with taxpayers entitled to as much as $163 million in restitution.
“We have long feared that City- Time was not an isolated incident, and unfortunately what we have learned today is that our fear has become a reality,” Liu said last Wednesday. “With one month to go before the city’s budget is ratified, and with devastating cuts on the table, taxpayers should be outraged at the fleecing that transpired under City Hall’s watch.”
The audit uncovered a lengthy se- ries of missteps, starting with the selection of an unqualified company as lead contractor and continuing throughout the life of the ongoing project.
Some of the findings include:
• Hewlett Packard (HP) was unqualified because it failed to meet DoITT’s minimum technical requirements for selection and should have not been hired as lead contractor on the project, calling into question the solicitation process and resulting in delays and cost overruns;
• DoITT allowed HP to drastically mark up subcontractor bills resulting in questionable billing;
• Project consultants were unqualified and billed at higher rates than allowed;
• Despite numerous failures and unsatisfactory reviews, City Hall failed to assess any damages against HP, in accordance with the contract.
“As the end users of the info pro- duced by the UCT, fire officers have known right from the beginning that the system did not function,” said Uniformed Fire Officers Association President Al Hagan. “Fire officers refer to UCT as ‘U Can’t Tell’ because we never knew whether the information we were getting was accurate. Now, thanks to the diligent work of the comptroller’s office, we know we were overcharged for a shoddy system."
In 2004, the city received two proposals from iXP Corporation and Hewlett Packard for the systems integration portion of ECTP. iXP passed the test for the required technical standard and was selected. However, the company unexpectedly withdrew its proposal and rather than re-soliciting the contract, the DoITT selected HP, which had twice failed to meet the minimum technical score to manage and complete the project.
In addition, iXP was hired by HP as a subcontractor on the project, which raised suspicion.
The DoITT justified awarding the contract to HP by stating that the company had stellar recommendations from the NYPD and the US Air Force, and was at the time contracted to perform CAD system upgrades at NYPD. Furthermore, DoITT stated that re-soliciting the contract would delay the important project. Both explanations have proven hollow.
In fact, DoITT had no record of NYPD or Air Force recommendations. Furthermore, as far back as 2004—before the project was solicited— the NYPD gave HP an unsatisfactory review on the quality and timeliness of HP’s performance in a VENDEX performance evaluation.
It also turns out that HP paid the NYPD a $33 million settlement in 2008 for failing to deliver the very CAD system that served in part as justification for their selection on ECTP.
Finally, DoITT stated that it selected HP, instead of soliciting additional applicants, in order to avoid delays. Nevertheless, the ECTP project is seven years behind schedule, incomplete, and $1 billion over budget, according to Liu’s March audit.
HP was selected as lead vendor to complete three major components of the ECTP system integration, under a contract that was not to exceed $380 million. Although HP failed to deliver two of the three components (a unified-CAD and PSAC 2), the estimated cost for the completion of PSAC 1 is $346 million.
In addition, the city has had to engage a second vendor, Northrop Grumman, to complete what HP has failed to do, driving the systems integration costs to $632 million.
There is no disputing the fact that HP has failed to deliver what they were contracted for, Liu said. This failure is due to a lack of oversight on behalf of the city and has resulted in another IT project drastically increasing in cost.
Despite documented dissatisfaction with HP’s failure to deliver on the ECTP project, the comptroller noted, DoITT paid the company $113 million during the first three years of the contract. City Hall refused to heed warnings from high-level agency commissioners and made no effort to recoup expended funds after HP was found to be unsatisfactory, in accordance with the contract.
In 2007, the then-DoITT commissioner, in conjunction with NYPD and FDNY officials, wrote to the then-deputy mayor and outlined numerous problems with HP and recommended termination. No action was taken.
In 2008, DoITT stated HP “needs improvement” or was “unsatisfactory” in response to all 20 questions on a VENDEX performance evaluation. And in 2009, a “lessons learned” report by the quality assurance monitor found the same issues within the same time frame.
In the face of the failures by HP, Liu noted, the city failed to assess damages to the company, adding that City Hall stood idly by and took no action.
The DoITT and HP did not properly ensure that project consultants were qualified for their titles, that hours were billed appropriately and accurately, and that timesheets were approved in a timely manner. As a result DoITT should move to recoup as much as $50 million for questionable timesheets (in addition to the aforementioned $113 million).
Billing errors included:
• Thirteen consultants were overpaid $1.5 million during the audit period. HP billed eight consultants at a higher paying title than they were working at and an additional five consultants were not qualified for their titles. Auditors were unable to determine if an additional 71 consultants were qualified and paid appropriately
• Three consultants who were qualified to be administrative staff were billed as Project Managers or Project Analysts (Engineers) for $396,000.
• HP billed the city at a rate of $192 per hour for non-allowable tasks such as opening the door for visitors and handling restroom garbage.
• Consultants submitted electronic timesheets indicating work was reported prior to actually performing the work, sometimes as far as a month ahead.
Liu said that the DoITT’s poor management of HP and heavy reliance on subcontractors drove up costs and allowed the lead vendor to take advantage of taxpayers. For example:
• HP was allowed to bill DoITT for the cost of leasing a management office, including payments for rent, City taxes, coffee, paper, postage, and office supplies
• DoITT never reduced the contract amount when it was clear that HP could not finish all three components of the project
• DoITT allowed HP to bill between nine and 195 percent mark ups for subcontracted services and allowed HP to change the payment structure on eight contract components from deliverable (paid upon completion) to time and material (hourly), thus incentivizing additional billing delays
The audit also noted that there were lengthy delays in timesheet approval by HP, ranging from two to 885 days and that more than half waited two months or more for approval. As a result of egregious timesheet errors HP may have overbilled the city as much as $106 million. Of this $106 million, $56 million is included in the previously mentioned $113 million. The remaining $50 million is for work billed after 2008.
As a result of the audit, Liu’s office made 11 recommendations to DoITT, of which the agency agreed to four.
Liu credited Deputy Comptroller for Audits Tina Kim and her team for their hard work on the report, which can be found online atwww.comptroller.nyc.gov/bureaus/a udit/audits_ 2012/06-30-12_ FM11- 107A.shtm
The ECTP was initiated in 2004 to transform and consolidate the city‘s 911 Emergency Dispatch System. The objective of the ECTP is to centralize and integrate the call-taking and dispatch operations between the NYPD and FDNY, including its Emergency Medical Dispatch division, into two fully integrated Public Safety Answering Centers (PSAC 1 and PSAC 2) equipped with state-ofthe art hardware and software communications systems.
On Apr. 1, 2005, the DoITT contracted with Hewlett-Packard Compan as system integrator for the ECTP. The contract includes, but is not limited to, development for PSAC facilities, organizational transformation of the call-taking process, application integration for all Computer Aided Dispatch systems and the Automated Vehicle Location system, and the communication infrastructure supporting the ECTP.
Liu has put strict payment requirements in place for the city’s second systems integration contract to complete this important project to ensure that this is not repeated.
In March, Liu announced audit findings on the overall project management of ECTP by DoITT. This report addresses a more detailed facet of that management, DoITT’s contract with HP. The March audit can be found at www.comptroller. nyc.gov/press/2012_ releases/pr12- 03-026.shtm
On May 14, a delegation of Congressional representatives sent a letter to the Comptroller General of the United States requesting an investigation into Mayor Bloomberg’s alleged lack of oversight of spending on private contractors funded by the City of New York with help from federal funds.