UF Health Shands, a premier healthcare provider, has leveraged DCIM (data center infrastructure management) to gain a better understanding and clearer picture of its data center resources — and is using that information to make better decisions for improving performance and capacity planning. “With dcTrack DCIM software we now have an up-to-minute picture of capacities in all our data centers in terms of power, space, networking and cooling,” says Joseph Keena, Manager of Data Center Operations at UF Health Shands in Gainesville, Fla. “ As a result, we are able to track our data center capacity more accurately and assess our infrastructure needs for the future.”
To provide quality patient care and services, Shands’ data centers support more than 200 applications, ranging from the typical ones found in most businesses – such as e-mail, online portals, back-office applications, HR, and financials — as well as applications supporting the work done in labs, ERs, hospital floors, clinics, radiology, and in just about every corner of the UF Health Shands network. Shands most important application is its hospital information system, called EPIC, that supports all patient caregivers. EPIC is a big data generator. Each day EPIC gathers information on caregiver-patient encounters, medical records, and every work order, such as lab work, in all of Shands’ locations. As a result of all that data collection, Shands recently passed the petabyte mark in data storage; and the healthcare provider expects to double its storage in the next 18 months.
Shands’ data centers have also grown — from one to four locations. Two serve as disaster recovery and one as second-tier storage. The main data center has 120 racks of equipment, two 500 kVA UPSs and a mainframe computer. The server racks — in the state-of-the-art 3,000-square-foot space — house a VMware farm with 20 physical servers and 185 virtual hosts, which have helped Shands save considerably on space.
In addition to better utilizing valuable data center space, Shands also was looking to better manage and use its other resources, such as equipment, power and cooling. Shands selected Raritan’s dcTrack solution because it maintains accurate, up-to-the-minute views and granu- lar details of every device, along with their end-to-end connections, across multiple data centers. It keeps track of servers, devices, and networking and facilities elements, and shows details on equipment, as well as raised floor space and rack elevations. Not only does dcTrack excel at asset management, but it also uniquely provides real-time views of a data center capacity in terms of power capacity, heat dissipation, and data network availability.
We’ve been growing the data center for quite a while, and with this growth we needed a better way to track the equipment we had in the data center and how that equipment was connected. We’ve done all the usual things, which is: you start off tracking your IT assets with paper, and then you go to placing inventory signs on the ends of every row, and then you start using spreadsheets to try to keep track of things. We had done all of those things in terms of trying to keep track of what’s where and what does our space and capacity look like.
Beyond trying to keep track of assets in a changing environment, we also were constantly doing a battle between power, space and cooling, just like any other data center. You solve one of those three issues, and then that drives one or both of the other items to be under capacity. So, we knew that a DCIM solution was going to be needed in order for us to have a chance of keeping up with capacity planning and growth; and, to track what’s where and what we are doing in the data center. It was a natural progression — we were at that point that we needed to invest in a DCIM solution.
We actually did a very extensive RFP process; and, through that process we narrowed it down to four different vendors. One of the reasons for selecting Raritan is that everyone at the company is willing to be a true partner with us in our DCIM journey. Raritan has been in the data center space for a very long time — so they weren’t new to this market space or it wasn’t an afterthought to other things that they’re doing. dcTrack has all the key elements — asset, capacity, and change management and works seamlessly with Raritan’s energy and power management solution (Power IQ®), so that all the key DCIM areas are covered. We purchased dcTrack virtual appliances for all of our racks.
We approached it in phases. The first phase focused on ‘let’s find all the devices and servers’, and the second phase is let’s find all the connections to those — whether it is KVM, copper, fiber or power connects.
We started with the Raritan training folks giving us hands-on training onsite with the dcTrack product. That was invaluable to us. After being trained, our implementation team took a couple of weeks to develop standards and implementation processes that we would use moving forward. They addressed items, such as how we should do a physical inventory. dcTrack lends itself to creating your own tailored processes, because the solution is flexible and you’re not tied to what comes in the package; even though dcTrack comes with an extensive device library, you can easily add your own elements. After establishing standards and processes, the actual deployment of the dcTrack virtual appliance was very straightforward and very easy to do. The server team responsible for that had no problems at all.
Because we are, obviously, a live 24/7 facility, you can’t drop everything and stop what you’re doing to do the implementation; so we divided the work. We started in our smaller sites, with asset inventory using a three-step process. We had somebody go out on the data center floor and do pre-inventory. They would go out and say, ‘Okay, this server, this make and model, named this, is in this rack, this U.’ The information was given to a second person to actually enter into dcTrack. And then, a third person would get the report generated from dcTrack saying, ‘Please go validate that this information is now correct compared to what’s actually in the rack.’ Having this process of checks and balances has proved critical to our success.
Once a rack or a site was completed, anything moving in or out of that location was required to go through dcTrack before physical work could be completed.
If there is a change, add or move, dcTrack’s auto- discovery engine updates the inventory database. This has reduced manual inputting of information and insures that information is accurate.
After completing the physical inventory for all our data center locations, we moved onto phase two, which focuses on all the connections — all of the KVM, copper connects, power connect and fiber connects.
Phase three for us was partnering with our network infrastructure group to do a pilot of one of their telecommunications rooms, which are located outside the data centers. So, we’ve provided onsite training to core members of their group, and we will be helping them to model one of their telecommunications rooms and all the connections in and out of that room in dcTrack.
In phase four, we’ll be starting the SNMP gathering and reporting on power utilization in the data centers. Shands plans to use dcTrack’s ability to track power usage to calculate the PUE (power utilization efficiency) of its data centers. dcTrack leverages Power IQ’s power and environmental data collection, and its ability to find unused capacity by determining in real time the draw of individual data center equipment.
Before we deployed dcTrack, it would mean a trip out onto the data center floor to confirm or deny that a server is in a certain location. Now, we can just pull up the asset information on the dcTrack screen and see that server X is located in this rack, in this U; or see that it is no longer a physical server, but is now a virtual server. Accurate asset records have given us a 50 percent gain in efficiency in terms of locating an asset’s physical location within the data centers.”
Early on, we had identified the need to integrate our service request process with our DCIM solution. So we had asked Raritan development for help. Raritan went off and wrote a Web services API that would not only work with our LANDesk, but with other service request systems out there.
We now have a seamless workflow process — from ticket creation to data center deployment. Requests come into the service desk, which then places the request into dcTrack. Based on space, power and network connectivity information, dcTrack’s search feature helps pick an optimal location for placing the server. dcTrack leads us through each step of the process to fulfill a new server request. Once the IT equipment gets delivered to us, we then go do the physical racking, connecting, labeling — and then come back and close out the request in dcTrack, which notifies LANDesk that the equipment is ready to go. This process makes it easier to make adds and changes, and improves the accuracy of our asset inventory.
You mentioned that there was another feature request.
One of the other big enhancement requests that we had asked Raritan was important for our network team — the ability to do end-to-end connection tracing across different media.
In today’s modern data center, the method of transmission changes along the way. It goes from copper to fiber in most data centers. And so, the ability to trace from a copper switch, back to fiber, back through a patch panel to this core router was important; and, that capability came out in dcTrack 2.6.
dcTrack’s intelligent connectivity management capabilities enable us to manage both network and power connections more efficiently. With better visibility of end-to-end connections, untapped capacity can be discovered and used – reducing capital and operational expenditures.
The entire data center team uses dcTrack. I use it as a monitoring tool to check our progress on the different phases of the project. So, I did get set up as a gatekeeper, which means I can go in there and look and see what’s going on, what stage are each of the requests at, things like that.
dcTrack’s workflow and change management capabilities support our daily moves, adds, and changes in our operations.
dcTrack’s “capacity search, reserve and place tool” will be used often. I see us using that tool for capacity planning and reserving much more, especially the reserving. I see us using the reserve function as an even better way of keeping an eye on capacity, so that we can put placeholders on resources in dcTrack as soon as somebody thinks that they’ve got a project lighting up. Being able to search and reserve that space for that potential project ahead of time is very helpful.
Some regulatory agencies do want to know where an application resides. So the ability to use dcTrack to tie an application to a physical or virtual server, which is phase five for us, will become critical moving forward. With dcTrack, we’ll be able to provide quicker responses, when needed, for those agencies that do want to see where you are holding information and the physical security surrounding it.
There are a number of goals that we are working towards. One big one that we see us working towards is PUE calculation, power utilization efficiency. dcTrack, and its ability to record what power is being used by what servers/application, definitely will help us.
And with capacity planning and reporting that dcTrack has in it, we’ll be able to project accurately when we will run out of capacity, so that we can start planning farther ahead, just like a server team or a storage team keeps an eye on their capacities. We’ll have a much more fluid and up-to-minute picture of our capacities in terms of power, space and cooling and be able to provide that information on a regular basis to management to say, ‘Here’s how we’re situated’ and ‘Here’s how we’ve been trending.’
dcTrack has made server moves, adds and changes more efficient because it tells us exactly where a server and its supporting infrastructure are located.
We also think that dcTrack will help us in troubleshooting and preventing problems. By providing physical trace information and knowing how it all physically connects, in terms of the power chain and data network chain, dcTrack helps speed repair time. Being able to see all the IT equipment supporting a specific application is also valuable. We also will be able to see if there are common elements from past events, such as a connectivity issue or certain applications not working right. By having a clear picture of all the connections and records of past events, we’re anticipating that we will be able to point people in the right direction for looking at possible causes from a physical standpoint.
We really value partnering with our vendors. We have a true partnership with Raritan. The partnership is two ways; it is about making sure that we’re both getting stronger moving forward.
UF Health Shands, affiliated with the University of Florida Health Science Center, is a premier health system that serves communities in the Southeast United States and employs 10,000-plus employees. Shands includes major teaching hospitals, specialty hospitals, a network of outpatient rehabilitation centers, home health agencies and emergency rooms (ER) — including one of the first pediatric ERs.
Also under the Shands umbrella are state-designated Level I Trauma Centers and emergency air and ground transport programs.
The Shands IT team not only wanted to deliver optimal services to users, it also wanted to efficiently use IT resources.
When the Shands IT team in Gainesville, Fla., became responsible for three additional data centers, the team began looking for new tools that would help answer what equipment was in each data center and exactly where every server, UPS, and device was located — and what was connected to what.
The old way of tracking IT assets, Excel spreadsheets and Visio floor plans, was not going to be flexible enough to support the added scope of multiple remote data centers – nor the ever-increasing demands on IT to support changes for a growing business. The IT team wanted a better and efficient way to track assets and capacity.
The IT team also was looking for solutions to help improve overall energy efficiency in the data centers. dcTrack DCIM software for managing data center resources — IT assets, rack space, and energy used to power servers and cool the environment.
dcTrack DCIM software for managing data center resources — IT assets, rack space, and energy used to power servers and cool the environment.