Posted on September 12, 2013 by Gento | Comment (0)
One of our roles in every data center installation is to bridge communication gaps between functional groups so goals can be met to everyone’s satisfaction.
That’s a consultative role, one that takes boots on the ground at first, and an ongoing relationship thereafter. At Raritan, we prefer to be on-site at deployments, working side-by-side with customers.Sometimes, the work we do isn’t particularly technical but rather logistical. Knowing, for example, that a customer is going to have 200 racks installed, we make sure that PDUs show up prior to installation so critical dates aren’t missed.
Often, PDU installation is only a small part of the process that gets us about 30% of the way there. Assuming the customer has the power strips, sensors and software running on Day 1, we can then help immediately start extracting value from their investment by Day 2. Now that they have all this great new power consumption and environmental data, what are they going to do differently? By the third day, we can start the iterative process of making those changes and measuring gains.
You’d be surprised to learn how many data centers are still plagued by server underutilization with one or very few applications running on a single server. It’s exacerbated by the fact that this particular server might be the least efficient machine to actually perform these tasks.
People still have a hard time unplugging stuff. Our system gives them the data to take an intelligent step forward. The conversation between Data Center Manager and User might go something like this: “You’re telling me you use this server?” And the user says “Yes, we use it.” With this new data, the manager can show the user that in the last 90 days the server hasn’t fluctuated at all from a power perspective and the network traffic has been essentially zero. The manager can now tell the user, “What you’re telling me doesn’t match the data we’re seeing. Let’s migrate this to a virtual appliance. We’ll decommission the other one, and if you have a service impact, we can discuss reprovisioning it.”
Hard data gives you a firm basis upon which to realize power and cooling efficiencies. It’s very important to Raritan to make sure our customers achieve the value that they are expecting from our product. The faster we can turn data into actionable information. The better the outcome will be.
Posted on September 6, 2013 by Gento | Comment (0)
A University of Michigan team has published their research on IPMI and BMC security issues. Entitled “Illuminating the Security Issues Surrounding Lights-Out Server Management,” this article follows up on the IPMI/BMC security issues highlighted by Dan Farmer and HD Moore as summarized in my previous blog on this subject. The Department of Homeland Security’s US-CERT team has posted an alert on the IPMI security risks.
The researchers provide an introduction to the issues, note the previous research, define IPMI/BMC security risks and analyze a particular implementation and describe their successful attack on this implementation. They found “blatant textbook vulnerabilities” and concluded that the implementations “suggest either incompetence or indifference towards customer security.” They then determine the number of publicly (Internet) accessible IPMI devices, which they determine to be more than 105,000. They provide some defenses and lessons and indicate areas for future work. The Washington Post has published an article on the paper and interviewed one of the authors, who criticizes the embedded device community for their security practices.
Customers who make use of BMC and IPMI based remote management cards and systems need to be aware of these issues and take the proper steps to safeguard their implementations. Given the severity of these issues, they should consider alternative remote management solutions such as KVM-over-IP switches, which can avoid most of these risks.
Posted on August 27, 2013 by Gento | Comment (0)
Thursday, September 12th, 2013
Hilton Bellevue, 300 – 112th Avenue SE, Bellevue, WA 98004
Check back here soon for our booth number.
Friday, July 12th
Be sure to stop by our booth to pick up our famous LED pens and durable Solo coffee cups.
For more information, please email Dorothy Ochs or call at 1.732.764.8886 ext.1220
Posted on August 12, 2013 by Gento | Comment (0)
The US Computer Emergency Readiness Team (US-CERT) issued an alert (TA13-207A) on IPMI usage on July 26th, with recommendations for IT departments. This is based on the work of Dan Farmer and HD Moore that I summarized in my previous blog on this topic.
The US-CERT alert summarizes many of the risks and issues, and provides recommended solutions for administrators. Solutions include: restrict IPMI to internal networks, utilize strong passwords, require authentication, sanitize flash memory at and of life, and identify affected products. A list of BMC’s are provided, although this is not a complete list of these devices and the servers utilizing them.
While these solutions are a step in the right direction, they are not sufficient to address all of the security issues listed by Farmer and Moore. For example, using IPMI on internal networks allows it to be attacked by viruses or worms that may exist on these networks. And if strong passwords are not enforced by systems, then some passwords may not be strong. Furthermore, authentication should be sufficiently strong, and for government and military organizations, FIPS 140-2 encryption is required along with two-factor authentication (e.g. CAC).
These solutions do not address many of the structural issues with BMC’s and IPMI including direct access to the server’s motherboard, storage of clear text passwords, virtually unlimited server control, and access to the BMC from a compromised server.
IT administrators and security officers should directly consult Farmer’s and Moore’s work to understand the specific dangers to their environment and take the appropriate actions. Administrators should follow the security best practices as defined by the server manufacturer and ensure that their servers have the latest BMC firmware, such that the latest security patches are applied. Its a good idea to make sure your security scanner audits these devices for vulnerabilities.
Posted on July 26, 2013 by Gento | Comment (0)
Recent articles in Network World, Dark Reading, Wired and Security Week have highlighted the shocking and widespread security vulnerabilities of Baseboard Management Controllers (BMC) (and the associated Intelligent Platform Management Interface (IPMI) protocol), used for remote server management by corporations, service providers and hosting companies.
BMC’s, available from all leading server manufacturers, have direct access to the server’s motherboard. This provides the ability to monitor, boot, and even reinstall the server. Many systems provide KVM-over-IP access and the connection of remote media. Access to the BMC provides virtually unlimited remote control of the server.
Two security researchers have identified these vulnerabilities: Dan Farmer, who originally discovered and documented the vulnerabilities, and HD Moore, who describes how to identify and test for these issues, using readily available security tools. Moore discovered over 300,000 IPMI-enabled vulnerable servers connected to the Internet, as well as additional vulnerabilities.
BMC/IPMI vulnerabilities include: Cyper 0 authentication allowing access with any password, BMC-provided password hashes which can be broken via brute force methods, BMC’s shipping with enabled “anonymous” access, a UPnP vulnerability that provides root access to the BMC, and storage of clear text passwords. Once the BMC is broken into there are multiple ways to infect, control and takeover the server. Conversely, for a compromised server, the BMC can be used to establish a backdoor user account.
All server administrators and security officers need to be aware of Farmer’s and Moore’s work and understand how it affects their servers. As IPMI and BMC implementations vary, consult your server manufacturer(s). Farmer provides IPMI security best practices and Moore provides a useful FAQ.
While this research is rather new and there is much to digest, Raritan’s experts do agree that there are indeed vulnerabilities that customers should take seriously. Given the power and opacity of the BMC, this is doubly true.
Moore: “In addition to vulnerabilities in the IPMI protocol itself, most BMCs seem to suffer from issues common across all embedded devices, namely default passwords, outdated open source software, and, in some cases, backdoor accounts and static encryption keys.”
Moore: “The world of BMCs is a mess that is not likely to get better anytime soon, and we need to be crystal clear about the risk these devices pose to our networks.”
Farmer: “Imagine trying to secure a computer with a small but powerful parasitic server on its motherboard; a bloodsucker that can’t be turned off and has no documentation; you can’t login, patch, or fix problems on it; server-based defensive, audit, or anti-malware software can’t be used. Its design is secret and implementation old.”
Farmer: “It’s also the perfect spy platform: nearly invisible to its host, it can fully control the computer’s hardware and software, and it was designed for remote control and monitoring.”