Getting Started with Software Defined Storage (SDS)

With the Internet, the Internet of Things, and big data, data is growing exponentially. It is common to add terabytes of data every day, which will bring more storage needs and challenges. Traditional solutions are not enough to meet these storage needs, and an effective solution is urgently needed. Software-defined storage is a common solution to these problems.

[[236929]]

Origin of SDS

The earliest software definition in the field of IT infrastructure is software defined networking (SDN). SDN originated from the Clean Slate research project of Stanford University in 2006. In 2009, Professor McKeown formally proposed the concept of SDN. By separating the control plane and data plane of network devices and implementing programmable control, flexible control of network traffic is achieved, providing a good platform for innovation of core networks and applications.

In 2012, VMware first proposed the concept of software-defined data center (SDDC) at its vForum conference. As one of the five components of VMware software-defined data center (computing, storage, network, management and security), the concept of software-defined storage (SDS) was also first proposed.

EMC also released its SDS strategy at the EMC World conference that year, which triggered a great discussion on SDS in the industry, and SDS quickly became a research hotspot in the storage industry. The concept of SDS has emerged in recent years as the products of enterprises such as Huawei and Shanyan Data have gradually matured.

What is SDS?

In August 2012, VMware first proposed the concept of software-defined storage. VMware believes that software-defined storage products are a solution that abstracts hardware, allowing you to easily pool all resources and provide them to consumers through a friendly user interface (UI) or API. A software-defined storage solution allows you to scale up or scale out without increasing any workload.

In fact, the definition of SDS has been around for more than three years, but there is still no unified standard. SNIA, which has made the largest contribution, believes that SDS allows heterogeneous or proprietary platforms. What must be satisfied is that the platform can provide a self-service interface for deploying and managing its virtual storage space. In addition, SDS should include:

• Automation: Simplifies management and reduces the cost of maintaining storage architecture;
• Standard interface: Provides application programming interfaces for managing, deploying, and maintaining storage devices and storage services;
• Virtual data path: Provides block, file, and object interfaces, allowing applications to write data through these interfaces;
• Scalability: Provides seamless expansion of reliability and performance without interrupting applications;
• Transparency: Provides storage consumers with the ability to monitor and manage storage usage and costs.

SNIA believes that the storage service interface needs to allow data owners (storage users) to express their needs for both data and the required service level. The data needs are based on the virtualization of the data path (Data Path) established by SDS, and the control path (Control Path) also needs to be abstracted into a storage service. Clouds, data centers and storage systems, or data administrators can be used to deploy this service (referring to Control Path).

What should beginners pay attention to in SDS?

Although the SDS concept is simple, transitioning to the technology is both complex and challenging. With the combined lure of automation, flexibility, increased storage capacity and improved productivity, SDS has gone mainstream, with businesses such as banking, manufacturing, pharmaceuticals, healthcare, media and government rapidly turning to the technology.

1. Getting Started

Successful SDS users first choose discrete use cases as a starting point. Within the enterprise, tier 2 applications such as backup and archiving are often seen as the best way to store large-scale mission-critical data, which are the best way to demonstrate the scalability, availability and cost advantages of SDS. Over time, more use cases including big data and deep learning can come online, further improving the economic advantages of SDS.

For companies that have recently moved to hyperconverged infrastructure (HCI) and have begun using SDS, it is a good start to evaluate whether HCI is beneficial to the company. If HCI is beneficial to the company, then consider whether it is necessary to expand the SDS coverage of the data center. Even companies that have not adopted HCI usually use some type of virtualization technology in their environment.

VMware, HyperV are actually software-defined computing solutions, and software-defined storage products extend the benefits of virtualization to the data layer, but adopters also need to carefully examine whether the infrastructure is supported. Any enterprise, when they enter the next infrastructure refresh cycle, should need to evaluate the risks brought by the updated technology.

It is important to find an SDS product that can meet current and future storage requirements, especially in key areas such as compliance and security. Be discriminating and find a solution that reduces the complexity and tasks of the IT department, and then start migrating the workloads that are easiest to migrate, or data sets with special requirements that are not currently met, such as encryption, performance or accessibility. Enterprises should evaluate their technology roadmaps for the next few years and consider moving to SDS solutions, and if the existing environment requires new storage administrators, then perhaps consider hiring new system administrators who are proficient in software-defined storage.

2. Potential Pitfalls

One mistake enterprises often make when planning their transformation to SDS is to view the technology as a “point product.” Lee Caswell, vice president of VMware, said: “Software-defined solutions are ideal components of a larger stack that provides a common operating model for compute, storage, networking and cloud.”

The biggest problem for enterprises and technical staff working on SDS may be rushing into it too quickly. Some enterprises will regret their decisions because they were either too abrupt or did not plan properly beforehand. Those who do not have the right skills will be frustrated, perhaps thinking that any storage administrator or cloud practitioner can master this knowledge and training overnight.

Perhaps the biggest mistake that SDS novices make is believing that the technology is a panacea for all workloads. All businesses can benefit from a wide range of workloads, but what really matters is analyzing the business needs using available IT resources to come up with the best solution for the operation.

SDS Application Scenarios

Many years ago, McDonald's hoped to learn from consumers through questionnaires such as "How can the milkshake be improved so that you will buy more? Do you want this milkshake to be cheaper? How about more chocolate flavor?" So McDonald's invited Harvard Business School professor Clayton Christensen and his team to solve this problem. Later, it was discovered that all customers who bought milkshakes had to drive a long time to work early in the morning. The road was very boring, so they needed to do something to make the journey more interesting while driving; the second thing was to prevent their stomachs from growling before lunch. So, in this scenario, the role of milkshakes comes into play.

Similarly, when it comes to software-defined storage, we are also curious about the scenarios in which it is applicable.

• VDI scenario is the most accepted scenario at present. When expanding the shared storage of traditional virtual desktop environment (VDI), it is necessary to add servers and storage arrays. When SDS is used as VDI storage, it can be integrated with the virtualization platform for deployment. It only needs to expand the server and rely on the local storage in the server to increase the virtual shared storage capacity. It can be said that the storage of VDI is contained in a separate server, which can be expanded vertically by adding disks and horizontally by adding new server nodes.
• Private cloud scenario: In this scenario, software-defined storage can be deeply integrated with computing and networking that are also based on software definition. The large capacity, high security, high scalability, and linear performance growth of software-defined storage are exactly the storage features that best fit the needs of private clouds.
• Distributed storage technology is also useful for internal enterprise development and testing scenarios, enterprise development and testing environments, learning and experimental environments, and cloud computing data centers. For example, OLAP (online analytical processing) and OLTP (online transaction processing). In specific work, distributed client cache can reduce storage latency and meet OLTP requirements, as well as the requirements of large concurrent MBPS, high computing and storage bandwidth, and high IOPS concurrency required by OLAP.
• In scenarios where data high availability is required, just as some people are born to make a living by their looks, some businesses are born to be distributed. Software-defined storage can naturally split data and run it in a loosely coupled manner. It adopts a design without a central node and splits the data into multiple copies and stores them on different hard disks, nodes, and racks. Even if a node fails, the business can still run normally, thereby ensuring high data availability.
• Data backup scenarios can leverage the distributed characteristics of software-defined storage to provide more secure data backup services for the primary storage of production systems independently or in conjunction with backup software.

The above scenarios are just those that have been verified so far, but I believe that with the development of science and technology, the application scenarios of software-defined storage will not be limited to these.

Challenges of SDS

According to domestic and foreign forecasts, software-defined storage will also usher in a relatively large development period around 2020. Gartner, a company dedicated to information technology research and analysis, predicts that the adoption rate of enterprises will reach 35% in 2020, of which file storage applications will account for a large share, followed by block storage and object storage.

my country's software-defined storage market is also booming, and emerging markets are driving a large number of companies to further promote industrial development. "Although SDS has strong growth momentum, there are still some problems," said Li Wei.

As far as the Chinese market is concerned, the current challenges mainly lie in the issue of unified management and scheduling. The storage in many user data centers is a club of all nations. In response, the industry has launched storage virtualization solutions such as ViPR and SVC, which basically have the same purpose. In the era when there was no software-defined storage, storage virtualization was mentioned. Later, after the emergence of software-defined storage, many people thought that it was just a remake of storage virtualization, a new wine in an old bottle.

Another requirement is to separate storage software and hardware, from tight coupling to loose coupling, so that software and hardware are independent. The benefit is that it makes application integration easier. Taking SAN and NAS as an example, many users want a single device to provide both SAN and NAS needs, the so-called unified storage. If software and hardware are not separated, an additional NAS gateway is required. With the emergence of object storage application requirements, software-defined storage will make application support simple and flexible. It should be said that this is the main problem that software-defined storage solves.