A report released by the Department of Homeland Security (DHS) Office of the Inspector General (OIG) highlights deficiencies in the Federal Emergency Management Agency’s (FEMA) information technology (IT) systems and integration with overall DHS strategic objectives.

While the report does not cover the 2005 hurricane season, it does cover FEMA’s responses to the 2004 hurricanes, and is illustrative of the problems FEMA faces, which are caused by its IT infrastructure.

The report (PDF) shows that FEMA relies on four major IT systems to perform its response and disaster recovery functions.

The first, National Emergency Management Information System, or NEMIS, “is the backbone IT system for response and recovery operations. FEMA uses NEMIS to electronically enter, record, and manage information regarding registered applicants for disaster assistance, obligations and payments, mission assignments, and grants,” according to the report.

“Integrated Financial Management Information System (IFMIS) forwards financial information to the Department of Treasury for payment of disaster assistance claims.

“Logistics Information Management System III (LIMS III) maintains the inventory of equipment and supplies.

“Automated Deployment Database (ADD) is used to identify and deploy personnel to disaster sites.”

The systems have inadequate reporting and tracking functions, according to the report.

For example, “LIMS III provides no tracking of essential commodities, such as ice and water, needed by disaster victims. As a result, FEMA cannot readily determine its effectiveness in achieving DHS’ specific disaster response goals and whether or not there is a need to improve.”

One obvious result of this is that ice needed in New Orleans winds up in Maine.

In addition, the four systems are not integrated and do not share data with each other.

“Specifically, NEMIS — the system for managing mission assignments — does not share information with the ADD or LIMS III deployment systems. When a disaster occurs, FEMA and state officials must quickly identify the people and other resources needed to respond to the incident. Information on the disaster is established in NEMIS, including requests for assistance, requisitions and commitments for services and supplies, and the initial allocation of funds. However, FEMA is unable to match automatically the mission assignments in NEMIS to either the personnel deployed through ADD or to the equipment and supplies dispatched through LIMS III. To compensate, regional staff create and maintain ad hoc databases, spreadsheets, and paper records to manage deployments. During the [2004] Florida hurricanes, for example, the regional office in Atlanta, Georgia received from 300 to 400 requests for personnel and supplies within a three-to-four-day period. To respond to these requests, the region improvised by creating a mission assignment spreadsheet that showed the dates and status of the requests, as well as the dates and times that the resources would be received. The spreadsheet enabled the region to coordinate the response and identify those requests that had been filled and those which remained open,” according to the report.

“Further, the lack of integration between ADD and LIMS III hinders FEMA from providing the appropriate number and combination of people and supplies to meet the level of need at disaster locations. Without adequate coordination, personnel might arrive at a disaster site and be unable to begin work because the supplies and equipment they need have not yet arrived, or the supplies may arrive without the necessary people to accept and distribute them. Generally, to achieve the right mix, FEMA’s Emergency Operations Center staff laboriously searches through ADD to identify available personnel. Likewise, the Agency Logistics Center must search through LIMS III to identify available supplies. This approach was not effective during the Florida hurricanes when 600 to 800 tractor-trailer trucks of supplies arrived at one staging area within a 24-to-36-hour period. There were only five people at the staging area to accept the supplies because their arrival had not been effectively coordinated with personnel deployments. The truck drivers were forced to wait at the staging area for hours until the goods could be unloaded and processed, a costly delay which hampered disaster assistance. . . .

“Because of the unintegrated IT environment, during the 2004 hurricanes, EP&R systems did not effectively handle increased workloads, were not adaptable to change, and lacked needed capabilities. Accordingly, FEMA field personnel developed manual workarounds, adjusted processes, and created alternative IT methods to supplement existing response and recovery systems and operations. Consequently, this created operational inefficiencies and hindered the delivery of essential disaster response and recovery services.”

Multiply this by a few orders of magnitude and you see what happened during FEMA’s response to Hurricane Katrina.

FEMA management culture also contributes to the problem. “FEMA’s disaster response culture has supported the agency through many crisis situations, such as the 2004 hurricanes. However, its reactive approach encourages short-term systems fixes rather than long-term solutions, contributing to the difficulties that FEMA encounters in efficiently and effectively supporting response and recovery operations. Without taking the time to fully define and document systems requirements, it is difficult for FEMA to evaluate effectively viable alternatives to its custom designed systems. Further, the reactive manner in which IT systems are funded and implemented has left little time for proper systems testing before they are deployed,” according to the report.

For instance, the NEMIS system, designed in 1995, was not designed for the way FEMA actually handles disasters. As a result, officials often find it faster and easier to fax hardcopy damage assessment reports than to enter them into NEMIS.

FEMA has also not evaluated alternatives to its existing systems, including existing commercial off the shelf systems, which might perform better or be better suited to its needs, because the existing “systems have carried FEMA through its responsibilities over the years.” Uh huh, sure they have. We see how well they’ve performed to date.

“EP&R’s tendency to rush systems acquisition to meet immediate needs has encouraged ad hoc development and implementation of IT programs, which has contributed to many systems integration and performance problems,” the report says.

Not only that, the systems are rushed into production so quickly, without proper testing and integration, that serious problems aren’t found until after the fact.

“For example, nine months prior to the 2004 hurricanes, FEMA’s recovery program offices provided funding to develop and implement an online registration capability for NEMIS. The online system is to allow disaster victims to submit claims via the internet without having to call the National Processing Service Centers. When the 2004 hurricanes occurred, the number of disaster victims registering for assistance increased significantly, thus overloading systems and staff of the National Processing Service Centers. The EP&R CIO was able to deploy the online registration system a full three months earlier than initially planned. However, an EP&R CIO official involved in development of the online systems said that its implementation did not follow standard change management or configuration management processes. Failing to follow such processes ultimately leads to systems availability problems. . . .

“Additionally, according to regulations, agencies are responsible for ensuring effective and efficient operation of IT equipment before it is implemented. This entails proving that new systems function in a ‘production-like’ test environment to ensure that the IT applications work properly and contain needed safeguards. However, in addition to its rushed systems acquisition approach, the EP&R CIO does not have a test environment to match the real systems environment, and does not always adequately test systems prior to release.

“For example, the online NEMIS registration capability did not have a name check function to ensure the validity and existence of the individuals filing claims. Also, the online system did not have controls to prevent one individual from generating multiple claims at the same time, even though the technology to prevent this from occurring already exists. One FEMA official was aware of six false claims made online. Proper testing of the online system likely would have disclosed this lack of system controls, leaving FEMA less susceptible to such fraud,” the report says.

Going for the quick fix, as opposed to the long-term workable solution, leads to these sorts of breakdowns in disaster response and recovery that we saw in 2004 and especially in 2005.

There’s much more to this report, including many more examples of how FEMA failed in its mission in the 2004 hurricane season, that I could go on for days just going over the case studies. I will say, though, that FEMA entered the 2005 hurricane season with much the same procedures and systems in place, and we see how well they did: not well at all, and not nearly as well as they could have, with a better IT infrastructure.