VIRTUALISATION
Seven steps to a secure virtual environment Gene Kim, Chief Technology Officer, Tripwire For better or worse, virtualisation is here. Tripwire’s survey of 219 IT organisations revealed that 85% were already using virtualisation, and half of the remaining 15% were planning to. VMware has stated that 85% of customers are already using virtualisation for mission-critical production services. However, analysts have published some startling predictions on the information security implications. Gartner predicts that through 2009, 60% of virtual servers will be less secure than their physical counterparts and that 30% of virtualised servers will be associated with a security incident. Indeed, the information security controls that seemed to adequately control risks before virtualisation may no longer be adequate. Virtualisation allows rapid deployment, enabling insecure IT infrastructure to quickly spread throughout the organisation and silently circumvent security and compliance controls. Unfortunately, this security weakness is only discovered when it is too late and a security breach, audit finding, or loss of critical functionality has already occurred.
Securing virtualised environments There is nearly universal agreement that information security and IT operations must properly manage virtualised servers the same way as physical servers. Gartner states that “security must be ‘baked in’ from conception, not addressed later as an afterthought.”1 But, if virtualisation is already here, what steps can be taken to implement effective security controls? Where to start, and in what order? And how can this be done in a way that delivers value, instead of creating bureaucratic barriers to getting real work done? How these information security organisations achieved their ‘good to great’ transformation has been codified in the Visible Ops Security Handbook: Achieving 14
Network Security
Common Security and IT Operations Objectives in 4 Practical Steps.2 Although Visible Ops Security is not dedicated solely to the topic of virtualisation, it describes and examines the core chronic conflict that exists in every IT organisation, and helps explain why virtualisation is so compelling. Visible Ops Security also describes why security has so much to gain or lose through virtualisation, and how security can meaningfully integrate into an IT organisation’s virtualisation strategy.
Seven prescriptive steps To replicate the observed high-performing attributes identified in Visible Ops Security, an organisation must gain control of the virtualisation cycle. Critically, the processes must address three criteria: business alignment, tight operational integration, and adding value.
“Although Visible Ops Security is not dedicated solely to the topic of virtualisation, it describes and examines the core chronic conflict that exists in every IT organisation, and helps explain why virtualisation is so compelling” The first three steps in delivering a secure virtualised environment focus on gaining awareness of the situation, and controlling configurations and changes at the virtualisation layer. Once achieved, the organisation can then focus on gaining control over configurations and changes at the virtual machine layer.
Gene Kim
Step One: Gain situational awareness Building situational awareness is defined as “the ability to identify, process and comprehend the critical elements of information about what is happening to the team with regard to the mission.” In the virtualisation context, this requires visibility into where virtualisation technologies are being used, what they are being used for, and who is responsible for their management. The goal is to gain an adequate understanding to create a clear view of what IT services are being enabled by virtualisation; to identify business and IT units and how they are organised; to clarify the relevant regulatory and contractual requirements for the business process enabled by virtualisation; and define the technologies and IT processes in use. It is also important to take a historical view to understand past high-risk indicators, such as repeat audit findings or frequent outages. This information enables organisations to define business and technology risks, enabling prioritisation of effort. Key questions organisations should ask include: • What IT services are being enabled by virtualisation (e.g., e-commerce, point of sale, financial reporting, order entry, etc.)? • Who are the business and IT units, and how are they organised (e.g., the centralised IT services group, an IT outsourcer, etc.)? • What are the relevant regulatory and contractual requirements for the business process enabled by virtualisation (e.g., SOX-404, PCI DSS, FISMA, etc.)?
August 2008
VIRTUALISATION configuration settings, and virtual machine policies.
“In order to ensure that VMM controls are functioning correctly, it is essential to have a detective control to measure variance against the configuration policies”
• What are the technologies and IT processes being used (e.g., VMware Fusion, Citrix XenServer, Microsoft Virtual Server, etc.)? • Are there any high-level risk indicators from the past (e.g., repeat audit findings, frequent outages, etc.)?
Step Two: Reduce and monitor privileged access Once an organisation has clarified the use of virtualisation across the business, the next step is to integrate information security into the access management procedures. The goal is to reduce access wherever possible and to ensure that some form of effective access control exists. However, excessive access and privileges create the potential risk that people will make uncontrolled changes to critical systems. This not only exposes IT systems to human errors that can disrupt service, but also creates unnecessary vulnerabilities for malicious and criminal acts that could jeopardise the organisation. These risks are especially evident in the Virtual Machine Manager (VMM). Often the VMM resides on a host operating system, which has privileged user accounts that can modify security configuration settings and virtual machines, as well as activate and deactivate virtualised computing environments. Consequently, we must document all the virtualisation administrators who
August 2008
have privileged access to the VMM, and ensure the access can be reconciled back to authorised staff. Any ghost accounts that cannot be reconciled to authorised staff should be disabled or deleted. The security team must work with virtualisation managers to reduce the number of administrators to the minimum and ensure that when there are relevant personnel changes, access is appropriately revoked. These policies are preventive controls and need to be supported by a number of detective controls. These include monitoring additions, deletions and changes to privileged accounts, including service accounts, and ensuring they can be reconciled with authorised users and authorised change orders from virtualisation managers. Accounts must be routinely re-accredited, preferably quarterly, to ensure management can reconcile privileged accounts to reports from HR and payroll.
Step Three: Define and enforce virtualisation configuration standards As with any complex application, VMMs have configuration and logical security settings that are designed to limit the risk of human errors, and fraud and security incidents by ensuring that the technology only performs as designed. Examples include proper password settings for the system BIOS, hypervisor host operating system settings and permissions, network
The goal in this step is to ensure all these VMM configuration settings are properly defined, implemented, and verified. Guidance is available from third-parties and vendors, including the Centre for Internet Security: “VMware ESX Server 3.x Benchmark Version 1.0” and VMWare: “VMWare Infrastructure 3, Security Hardening”. 3, 4 To embed these processes within operations, organisations must work with IT management and virtualisation managers on a policy that defines which virtualisation security standards should be used. It is also essential to mandate that all virtualisation technologies use these secure configuration settings; create a plan for deployment; and define a time limit for initial implementation as well as expectations around any corrective actions when configurations are non-compliant. In order to ensure these VMM controls are functioning correctly, it is essential to have a detective control to measure variance against the configuration policies, and verify that corrective actions are properly implemented in the required time.
Step Four: Integrate and enforce change management processes Once VMMs are in a known and trusted state, all changes made to the VMM should be authorised, scheduled and substantiated by change management. This helps to assess the potential information security and operational impact of changes; improves procedures for change authorisation, scheduling, implementation, and substantiation, and ensures that change requests comply with information security requirements, corporate policy, and industry standards.
Network Security
15
VIRTUALISATION people circumventing the change management process.' To substantiate compliance with change management processes, the organisation needs to prepare in advance of the audit evidence of the following: change requests and their approvals, changes detected on all relevant IT systems, reconciliations of detected changes to approved changed requests, and any corrective actions undertaken for unauthorised changes. This approach integrates information security into the necessary preventive change management processes, as well as creating detective controls to ensure that those controls are working. It also proactively creates evidence to substantiate for auditors that change controls exist and are effective.
In order to treat changes in this way, the security team needs to become embedded in operational IT processes and ensure that security considerations are an integral component of the review and operational process for VMM changes. This requires involvement with Change Advisory Board (CAB) meetings – the forums for assessing the risks of proposed changes, approving or denying change requests, reviewing the status of changes being planned, agreeing on implementation schedules, and reviewing the success of implemented changes. In order to ensure the relevant changes have been appropriately approved, an organisation must put in place a detective change control. This provides visibility of
changes made in the VMM and enables the security team to determine whether or not the change was properly authorised and conforms to required standards, and, in the event of a security breach, gather relevant forensics data to support an investigation. It is also important to create the right attitude toward security, which will require support from senior management. Auditors use the term 'tone from the top' to express the fact that words and actions from the boardroom on down set the tone for the behaviour of everyone in the enterprise. The message required is: 'The only acceptable number of unauthorised changes is zero and senior executives will not tolerate
Step Five: Create library of trusted virtualised server builds Virtualisation makes it easier to deploy infrastructure on demand, faster than ever before. The obvious risk is that insecure virtual servers and applications are deployed without adequate controls, which results in security breaches, compliance, and audit findings. In this step, the goal is to create a library of known, trusted and approved virtual images that can be used and re-used. The library will make it easier to deploy an authorised, secure configuration rather than an unauthorised, insecure configuration. Secure builds are a combination of mandatory and recommended configurations that reduce the likelihood of operational and information security failures that create vulnerabilities and can be exploited by intruders.
“Secure builds are a combination of mandatory and recommended configurations that reduce the likelihood of operational and information security failures.” To overcome this type of risk, organisations must document standards for the creation and maintenance of secure builds. This requires the development of standards that specify how to secure and harden the 16
Network Security
August 2008
VIRTUALISATION builds released into production or checked into the definitive software library (DSL). As these internal standards evolve, existing documents must be revised or new ones created to be used across the enterprise. The Centre for Internet Security (CIS) and the SANS Institute are external organisations that publish information security configuration standards. The security team must also work with the server provisioning and virtualisation team to build a library of standardised and secure virtualised server builds. The builds will include the integration of independent configuration standards and checklists, as well as standard steps towards reducing security risks, including ensuring that passwords must be changed after development. “Secure builds are a combination of mandatory and recommended configurations that reduce the likelihood of operational and information security failures.” The preventive controls in this step are the definition of policies and standards that create the library of approved virtual image builds. These detective controls monitor the approved virtual image library to ensure that all adds, removes and changes are approved, and to verify the virtual images to ensure that configurations are in the approved state.
Step Six: Integrate into release management testing and acceptance procedures To better safeguard the production environment, information security requires standardisation and documentation, implementation controls such as checklists, and continual control of production variance. These are also some of the key objectives of release management. In this step, the goal is to engage with release management to ensure that release packages are tested and information security requirements are taken into account. Development, on the one hand, often focuses on specific components and release management, on the other hand, focuses on collections of components and whether or not the components work together. To ensure security requirements are added to the typical release management checklists and templates, the security team must
August 2008
develop templates and interface with release management, QA, and project management. This will ensure that information security and regulatory compliance requirements are methodically collected at the start of each project. This process requires the creation of an agreed protocol that specifies when and how to engage information security. It is also advisable to integrate automated security testing tools to run against code, builds, and releases into the release testing process. Even if the use of vulnerability scanning and management testing tools may potentially crash applications in the testing process, they should still be used. It is better to find the vulnerabilities in preproduction instead of during production. Moreover, it is important to use the same tools in preproduction that will be used in production environments so that IT operations can be prepared for potential problems in the production environment when these tools are run. In some situations, the security testing conducted by QA will be sufficient; in other cases the security team will want to conduct independent security testing. In either situation, arming QA with the same set of tools reduces findings and allows more time for correction – at lower cost, with less stress, and with higher success rates. The preventive controls are the release testing protocols, including checklists and test procedures. The detective control is the notification that new releases have been deployed, and verification that it matches the approved and tested builds.
mission-critical application for a revenuegenerating business process that is in scope for SOX-404, and the potential implications of an unauthorised deactivation of the computing environment. Consequences could include jeopardising financial reporting objectives, as well as revenue and information security objectives. Clearly, this is a change that must be authorised and scheduled before being implemented. In addition to stating this policy requirement, information security must work with IT management to ensure ‘tone at the top’ that establishes that the only acceptable number of unauthorised changes is zero.
“Information security must work with change management and virtualisation managers to ensure that activating and deactivating a virtual computing environment is defined as a change” The preventive controls are the policies that define how virtualisation actions should interface with change management processes. The organisation will also need a corresponding detective control, to substantiate that the policy is being followed. The control will monitor all virtualisation activations and deactivations, to ensure they can be reconciled to an authorised and scheduled change. By doing this, information security can help ensure that virtualisation activity, which could result in information security risks, is adequately reviewed and mitigated, as well as helping control unauthorised 'virtualisation sprawl'.
Step Seven: Ensure virtualisation activities go through change management
Business value of good information security controls
Information security must work with change management and virtualisation managers to ensure that activating and deactivating a virtual computing environment is defined as a change. Consequently, these actions must be authorised, scheduled and audited by change management, just like any other change. If there is disagreement on whether virtualisation actions should be considered a type of change, consider the scenario of a
The 2006 and 2007 ITPI IT Controls Performance Studies were conducted to establish the link between controls and operational performance.5 From production system changes that fail half as often, to one quarter of the frequency of repeat audit findings and one half of the amount of unplanned work and fire fighting; highperforming organisations were more effective and efficient. The result is that server-to-system administrator ratios are two times higher.
Network Security
17
1H 2008 The studies also determined that the same high performers have superior information security effectiveness. The 2007 IT controls study found that when high performers had security breaches, the security breaches were far less likely to result in events that affected financial stability, reputation or customers. High performers are half as likely as medium performers and one-fifth as likely as low performers to experience security breaches that result in loss. Furthermore, the security breaches are far more likely to be detected using automated controls – rather than an external source such as newspaper headlines or a customer. High performers automatically detect security breaches 15% more often than medium performers and twice as often as low performers. In addition, security access breaches are detected far more quickly. High performers have a mean time to detect measured in minutes, compared with hours for medium performers and days for low performers. These high performing IT organisations have figured out how to build sustainable security controls that integrate into daily IT operational processes and
deliver value to other business stakeholders. In these organisations, information security simultaneously enables the business to respond more quickly to urgent business needs and helps provide stable, secure, and predictable IT services.
Conclusion Virtualisation is an important technology that has the potential to transform data centre costs. However, the business risks cannot be underestimated. The reality is that when information security controls are improperly implemented or overlooked in virtualised environments, real security risks and exposures are created, faster than ever. Taking the right steps towards securing the virtual infrastructure will not only reduce errors and drive up performance but also contribute to the incremental adoption of virtualisation across an organisation, adding stability to the entire infrastructure and building confidence in the virtual technology that will support on going deployment of increasingly mission-critical applications.
References 1. “Trip Report: Security and Risk Management Community.” Gartner Emerging Trends Symposium/ITxpo April 6-10, 2008. June 24, 2008. 2. Gene Kim, Paul Love and George Spafford.Visible Ops Security Handbook: Achieving Common Security and IT Operations Objectives in 4 Practical Steps. 3. The Center for Internet Security. “VMware ESX Server 3.x Benchmark, Version 1.0 October 2007.” Ed. Joel Kirch, WBB Consulting. June 24, 2008. http://www. cisecurity.org/tools2/vm/CIS_VMware_ ESX_Server_Benchmark_v1.0.pdf 4. “VMware Infrastructure 3, Security Hardening.” Latest Revision: Feb 21, 2007. VMware. June 24, 2008. 5. “IT Controls Performance Study”. 2007. IT Process Institute (ITPI). June 24, 2008. Extensive summary at June 24, 2008
2008: Six months of information security Dario Forte, CFE, CISM, founder and CEO of DFLabs (www.dflabs.com) As the first half of 2008 comes to a close, and we examine some of the latest events in information security, we find ourselves in an extremely delicate period. While companies have begun to understand the strategic importance of protecting their information, the economic crisis and consequent tightening of budgets have caused many people to revert to thinking about tactics and responses, thereby risking exposure to a number of potential problems. In this article we will review the most significant events and discuss their impact on business in general.
The United States: Information losses have increased by 69% This statistic is from the Washington Post, reporting on a recent investigation by the 18
Network Security
Identity Theft Resource Center (ITRC). The study, addressing a significant sample of US organisations, compared the first half of 2008 with the same period in 2007, noting that 342 incidents were reported from January 1 to June 27, 2008. Of these, more
Dario Forte
than 27% were recorded in the business world in general. While banks and healthcare organisations witnessed an increase in incidents of 15% and 10%, respectively, security incidents in education, government, and the military decreased.
August 2008