Initial Report of Breach
On April 4, 2015, the US Office of Personnel Management (OPM) discovered that the personnel data of 4.2 million current and former Federal government employees had been stolen. Information such as full name, birth date, home address and Social Security Numbers were affected.
In late June 2015, in what it is characterizing as a separate breach, OPM announced that additional information had been compromised: Background investigation records of current, former, and prospective Federal employees and contractors. OPM announced that sensitive information, including the Social Security Numbers (SSNs) of 21.5 million individuals, was stolen from the background investigation databases. This includes 19.7 million individuals that applied for a background investigation, and 1.8 million non-applicants, primarily spouses or co-habitants of applicants. Some records also include findings from interviews conducted by background investigators and approximately 1.1 million include fingerprints. Usernames and passwords that background investigation applicants used to fill out their background investigation forms were also stolen.
What Really Happened?
OPM continues to call the breach of federal government employee database and the breach of the background investigations database as two separate but related breaches. Calling these two separate breaches doesn’t seem to be supported by OPMs own incident timeline, dated July 14, 2015. Analysis of the OPM timeline shows that:
- May 7, 2014: Local-area-network credentials compromised and malware introduced into OPM network, backdoor for exfiltration of data created.
- July 3, 2014: Exfiltration of background investigations database begins.
- Data exfiltration of background investigations database continues through August, 2014. Data for 21.5 million background investigations are compromised.
- October 2014: Attackers shift focus to the U.S. Department of the Interior data center that houses OPMs personnel database.
- December 15, 2014: Data for 4.2 million government employees and retirees from the personnel database are compromised. OPM claims that credentials stolen from background investigations contractor KeyPoint Government Solutions were used to breach OPM.
- April 15, 2015: OPM discovered SSL traffic and a decryption tool installed on network in December 2014 that related to the background investigations database breach. US-CERT investigation begins.
- April 17, 2015: US-CERT, after loading data from the April 15 discovery onto the Einstein intrusion detection system, it is discovered that the personnel database records were at risk.
- April 17, 2015: OPM deploys a “malware prevention” capability to block malicious network access.
- April 23, 2015: US-CERT found evidence of the December 15 personnel database compromise in Netflow data. OPM notified congress of breach.
- April 23, 2015: Intruders evicted from OPM network.
- April 24, 2015: Malware is discovered, quarantined and eliminated from OPM network.
The OPM breach and response timeline provides a lot of interesting details but there are some events missing from its timeline that could have significance:
- March 2014: OPM believes that it successfully blocked an attempted network breach from China targeting information on individuals who have applied for top-secret security clearances. No Personally Identifiable Information (PII) was thought to have been exposed and no public announcement was made about the breach attempt.
- March 2014: United States Investigation Services (USIS), an OPM contractor that conducted background checks, was breached and lost the PII of 25,000 government employees.
- March 2014: It is announced that OPM security documentation and system manuals were stolen, possibly in November 2013. This theft represented a security breach, and that attackers could use the information to “learn about the platform, the infrastructure of our system”, according to OPM CIO Donna Seymour in a letter to congress.
- June 17 2014: United States Investigations Services (USIS) breach from March 2014 is announced. Personal information for 25,000 government employees is compromised.
- August 6, 2014: First KeyPoint Government Solutions breach announced. Breach affects up to 390,000 current and past DHS employees and contractors.
- August or September 2014: Second KeyPoint breach is announced. This breach has possibly resulting in the loss of the PII of more than 48,000 government employees. This breach likely occurred in December 2013. OPM claimed that credentials stolen from KeyPoint were used to breach OPM. During testimony to congress in June 2015, OPM Director Katherine Archuleta said that there is a “direct line” between KeyPoint and the intrusion of the two OPM databases.
- June 4 2015: OPM announces that the personnel records of 4.2 million current and former government employees was compromised.
- June 23 2015: OPM announces that the background investigations database was compromised, affecting 21.5 million people.
The first breach to be announced, the breach of the personnel database, began in December of 2014 while the second breach to be announced, the breach of the background investigations database, began six months earlier.
There is also some evidence that the OPM breach was carried out by a group that is also related to the breach at Anthem, Inc. See Anthem Breach Report
What Was the Outcome?
- 4.2 million individuals including every federal employee, every federal retiree, up to 1 million former employees had their personal information disclosed.
- 19.7 million individuals, which is everyone who has had a background investigation, or applied for a background investigation since at least 2000 had their personal information disclosed.
- 1.8 million individuals who did not apply for a background investigation but whose personal information was included in another’s background investigation application had their personal information disclosed. This included primarily spouses and co-habitants of applicants but also individuals included as employment and personal references on applications.
- Phishing attacks targeting individuals whose PII was compromised have been seen, usually offering identity-theft repair services that are actually malicious schemes to steal identities.
- At least seven class action suits against OPM and KeyPoint have been filed including two suits by government employee unions.
- Katherine Archuleta resigned her position as OPM Director on July 10, 2015.
- While the true and complete costs of the OPM breach will not be known for years, former OPM Director Katherine Archuleta told Congress in June 2015 that the estimate to provide credit monitoring services for five years to the 4.2 million victims of the personnel database breach would range from $19 million to $21 million.
- Extrapolating that estimate to a cost of $5 per victim, the total cost of credit monitoring services alone for 25.7 million victims could cost $128.5 million.
- OPM plans to raise fees to other agencies for its background investigation services in order to raise money to pay the costs of breach recovery. The Department of Defense has estimated the cost to them for credit monitoring services to be $132 million.
- Using a typical cost of a breach of $150/record, the total cost of the OPM breach could exceed $3.5 billion.
Indicators of Compromise
The best indicators of compromise in this case were the data breaches that were already occurring to OPM and its contractors.
- November 2013 OPM attack-no loss of PII detected, but loss of security docs and manuals detected.
- OPM contractor USIS suffered an intrusion with loss of PII.
- OPM contractor KeyPoint was intruded upon twice, with hundreds of thousands of records compromised.
OPM was likely already under attack when these intrusions occurred-but did they realize it?
How Could This Breach Have Been Prevented? (Lessons Learned)
There are a number of factors that could have contributed to the OPM breaches. There may have been an over-reliance on the DHS-provided Einstein intrusion detection system to actually detect intrusions. Intrusion detections systems like Einstein rely upon a database of attack signatures to provide alerts of malicious activity. That means for previously unknown attacks for which there are no attack signatures available, intrusion detections systems, including Einstein, do not provide any intrusion detection or alert capability. Einstein or other intrusion detection devices could and should have been able to see and report on exfiltration of data to foreign (outside of government) IP addresses. Einstein is just one device and one layer in what should be a comprehensive defense-in-depth security program. (A discussion on defense-in-depth security will be offered in a later blog post.)
The volume of data heading out of the network should have been noticed, perhaps using NetFlow or IPFIX analysis of data streams and bandwidth utilization analysis. In fact, after US-CERT was notified of the breach, it found evidence of the intrusion in “historical NetFlow data”. That means that it should have been possible for OPM to detect the attack itself. As it was, it took about a year for the intrusion to be detected. Perhaps that lengthy interval between intrusion and detection could have been shortened with a regular program of network monitoring, ingress and egress filtering and bandwidth utilization analysis. Remember, there can be no incident response until the incident is detected.
The OPM databases were hosted in a U.S. Department of the Interior (DOI) datacenter. Is it possible that neither agency was looking closely enough at what was happening? Perhaps OPM figured that security services was something that they were getting from DOI. Did DOI assume that OPM was taking care of its own data and application security? Was there a misunderstanding or lack of agreement between OPM and DOI over the security services that were to be provided?
OPM points directly to KeyPoint Government Solutions as the source of the credentials that were used to access both OPM databases. That actually just points back at OPM as not properly managing its contractors. Clearly, management of contractors and their access is required. Almost half of major OPM systems are run by contractors with OPM having limited visibility and control over those systems.
OPM had implemented multi-factor authentication only for its privileged staff. There was no multi-factor authentication for users accessing systems from outside OPM. None of OPMs major applications utilized multi-factor authentication or encryption. Stolen credentials used from outside OPM could access anything inside OPM.
According to OPMs inspector general, OPM had a “long history of systemic failures to properly manage its IT infrastructure” with identified security deficiencies going back to 2007 not being addressed. Until 2013 OPM had no internal IT staff with security experience and credentials. OPM did not maintain an inventory of its servers, databases or network devices.
Conclusion (More Lessons Learned)
As the human resources department of the United States Executive Branch, OPM was the repository of all personnel data on every current and past federal employee. As the agency charged with performing background investigations, OPM was responsible for enormously detailed information on every person who has every applied for a background investigation as part of a government job application or security clearance.
There is much to learn from the breach of the U.S. Office of Personnel Management. OPM had numerous problems from asset management and configuration and change management to a lack understanding of network operations and a lack of qualified internal security staff. Too much of OPMs data operations were controlled by contractors and out of sight of OPM operational and security control. At least one OPM contractor had Chinese nationals employed in positions with access to OPM data. 65% of OPM data was stored on systems with no authorization to operate.
Chief among the lessons to be learned is the need for a strong management with a focus on security and the will to bring it about.
OPM Director Katherine Archuleta was a long-time political executive who was the National Political Director of Barak Obama’s 2012 re-election campaign. President Obama appointed her as Director of OPM in May 2013. Trained as an elementary school teacher but with a background of political appointments, the White House declared its vision for her as sharing “President Obama’s vision for diversity and inclusion in the Federal workforce”. Even today (September 1, 2015), months after the discovery of the intrusions, the OPM website lists “IT Improvement” as fourth in a list of seven “Top Priorities” for its current acting-Director.
Katherine Archuleta was sworn into office on November 4, 2013. Ironically, OPM was under attack virtually from Director Archuleta’s first day on the job. As the new Director of OPM, Katherine Archuleta inherited a security disaster from day one of her tenure. OPM leadership didn’t seem to comprehend the extent of its security problems with Director Archuleta stating in mid-2014 that she didn’t believe that a security problem existed in the wake of an attack from China in late 2013.
But it was a disaster that was a long time in the making and was well understood to be a disaster waiting to happen. OPMs cyber vulnerabilities were well documented and well understood. Starting in 2009, OPMs Inspector General, reported: “The continuing weakness in OPM information security program results directly from inadequate governance. Most if not all of the [information security] exceptions we noted this year result from a lack of leadership, policy, and guidance.” That’s a documented wake-up call that has gone unanswered for over six years.
But OPM is not alone. OPM is perhaps just the worst example to date of the Federal Executive Branch failing in cyber security. In June 2015, the U.S. Government Accountability Office (GAO) reports that there were 67,168 information security incidents reported by federal agencies in fiscal year 2014. The GAO also reports that “19 of 24 major agencies declaring cybersecurity as a significant deficiency or material weakness for financial reporting purposes”.