Software-defined data centers face constant challenges: IT departments need to shift their focus

Virtualization has revolutionized the data center, but IT departments must overcome several hurdles before the technology can reach the next level.

Innovation often comes at a cost. Companies introduce solutions to improve current operating models, and it’s entirely possible that something will go wrong in the process. For example, our transition from horse and buggy to mass-produced cars meant we could travel faster and farther. However, reliance on this new mode of transportation brought new challenges, including the availability and condition of roads, the availability of gas and mechanics, noise and air pollution. In many cases, this created new markets.

[[126207]]

Virtualization technology has also revolutionized today's data center. Virtualization has won our hearts by bringing many improvements and benefits, including significant cost reduction, significant improvement in flexibility, business efficiency and business continuity. However, it has also brought some new data center challenges, prompting IT departments to better align data center architecture with the requirements of virtualized workloads.

Virtualization technology abstracts the hardware components of a data center and then overlays a common software layer on top. Since the virtualization layer manages the underlying hardware, operations can be controlled through software. When data center services are virtualized, it eliminates silos such as processors, memory, storage, and network resources, which are usually located in single-purpose devices. Software-defined data centers enable service delivery and automated management through software.

So what's the problem? Many IT departments have only partially virtualized to achieve a software-defined data center. Siloed physical assets, including storage and data management equipment, have hampered progress in virtualization. To date, the limited range of infrastructure components that have been virtualized (in most cases, server virtualization still relies on specialized hardware systems that cannot scale out), which has brought greater complexity and higher costs, and scale will only exacerbate the problem.

Infrastructure Innovation

Over time, the proliferation of specialized equipment has created unnecessary complexity that has led to a data center in disarray. Innovation at different stages and from different vendors has resulted in a disparate layering of technologies that work together but are often extremely inefficient.

Backup to disk is one such example. Many companies invest a lot of money in backup hardware, including backup servers, disk storage systems, deduplication appliances, and WAN optimization systems, which are often deployed both in the primary data center and at remote disaster recovery sites. When backups are not running, the processors and memory on many of these specialized systems and devices are not fully utilized.

Another example of this problem is capacity efficiency. Over the past decade, IT departments have worked to address this problem by deploying a variety of technologies, such as WAN optimization systems and backup deduplication appliances. As a result, data efficiency technologies have become standard features of many different products.

If all of these products were put together in the data center, IT would end up processing the same data flowing through each device over and over again. This process is complex and expensive, and requires multiple management touch points. The need for a lot of resources defeats the purpose of virtualization in the first place.

Underutilized resources

Before virtualization became widely used, it was common for server utilization to average less than 10%. Virtualization has significantly increased average utilization. Now IT departments need to assign different work groups to manage different resources, such as servers, storage systems, networks, and end-user computing.

New workloads have created resource challenges, forcing IT departments to develop infrastructure environments for each service. Virtual desktop infrastructure (VDI) environments bring different resource usage patterns than server virtualization projects. With this in mind, IT professionals often deploy completely separate environments (from servers to storage systems) to meet user expectations.

Deployment Difficulty and Delays

Resource challenges are the number one reason many enterprises continue to have trouble deploying new applications and services, followed by management overhead. One example is allocating storage resources to run applications reliably. Many virtual machines run on a single logical unit number (LUN), which places a challenging I/O load on the storage system.

The term "IO blender" is used to describe the situation where the hypervisor manages multiple workloads with different input and output flows in a unified manner, resulting in random input and output flows competing for resources, thereby increasing the input and output operations per second (IOPS) required to serve the virtualized workloads. To overcome this challenge, IT departments often over-provision storage resources or use flash/SSD storage systems instead of spinning disks to improve performance, which results in a higher cost per GB of storage resources allocated to each virtual machine.

Mobility and management

Virtual machines can be migrated, but their migration range is often limited by the physical storage they are associated with. Virtual machines are bound to the data storage area of ​​the virtualization domain, and the data storage area is bound to the storage resource. Isolated physical storage resources are often managed at the unit level, including LUNs, volumes, RAID groups, and physical disks.

Policies are also configured at the cell level, which means that policies cannot be specified for individual VMs, but only for the storage cells that many VMs reside in. The mobility and management required in a software-defined data center requires a top-down approach: policy is established and managed at the VM and workload level.

Strategy Misalignment

In addition to the performance challenges of the post-virtualization world, enterprises that go virtualization also face challenges in both the physical and virtual worlds. Physical servers have a direct mapping from applications to servers, to storage arrays, to LUNs, and to storage policies. This approach makes storage upgrades very complex. For example, a data replication policy is applied to a LUN in storage array X at address Y, requiring that this LUN be replicated to storage array A at IP address B.

In the virtualized world, there are many applications on a single host, and many hosts use the same LUN. Applying policies to a single LUN is not efficient. Instead, applying backup and replication policies directly to individual applications (or virtual machines) facilitates better management of virtual environments. Replication policies specify a destination that is abstracted from the infrastructure - in this case, the data center. This allows administrators to upgrade the infrastructure in a data center without having to configure policies or migrate data, thereby increasing efficiency and reducing risk.

Organizational dislocation

IT departments often combine resource structures and skill sets. With a software-defined data center, it is expected that some manual work will not need to be performed by data center personnel. The abstraction layer should largely hide the complexity associated with hardware resources.

IT departments need to shift their focus. Rather than staffing silos with specialized expertise in hardware resources, they need broader knowledge to manage applications and virtualized environments.

Despite the challenges, IT professionals should not shy away from deploying a virtualized environment. However, they must fully consider the virtualization architecture in order to maintain efficiency and truly reap the benefits of virtualization.