How to address network automation risks and tasks

Many network engineers and network administrators are reluctant to deploy network automation. In practice, anyone who has run a network for a reasonable amount of time has likely experienced a major network outage, which is unpleasant and stressful, and network teams work hard to avoid. If simple changes can cause major outages, then we question why anyone would consider using automation, which could quickly propagate misconfigurations throughout the network.

[[265239]]

If automation is the cause of a network outage, the network team may not consider automation to fix the outage. And attack remediation tools are often command-line interfaces, and the network team can only configure one device at a time.

If a network team needs to update 100 devices, and each configuration takes one minute, the change will take an hour and a half. Considering how long the process actually takes and the number of devices involved, it’s easy to understand why network teams are reluctant to use automation.

But do the risks of network automation really outweigh the benefits? And can network teams mitigate these risks? First, let’s look at why organizations need to use network automation and the risks of not adopting it.

Why should you use network automation?

Standardized designs, not snowflake designs. Complex network designs (so-called snowflake designs) increase risk because each part of the network is configured differently. Lack of standards increases risk, and standardization is important because networks have few or no special cases. Network standardization allows for better identification of failure modes and development of standard procedures to handle network problems.

Using standardized building blocks in network design simplifies network automation. For building block design, enterprises may need to spend more on equipment, but the benefits are lower Opex and greater resilience. At the same time, by using standard operating procedures for troubleshooting and remediation, network teams can more easily understand and mitigate failures.

Furthermore, building block network designs are more easily automated. Automation can help with initial configuration, configuration updates, physical connectivity verification, and troubleshooting.

Network agility. Network automation has lagged behind computing and storage system automation, and it must catch up. Enterprises that delay deploying comprehensive IT automation may not be able to achieve the increased agility and thus lose competitiveness.

Automation means more efficient use of IT resources across the business. For the same number of employees, efficiency translates into productivity and higher profits. At the same time, a more stable IT environment means greater customer stability and higher customer satisfaction. In many cases, this can also lead to higher profits and a larger market share.

Flexible networks can also more easily adapt to new network technologies, with network teams only having to make incremental changes to a few building block designs and related automation tasks.

Network Automation Tasks

Automation is not without risk, however. Any poorly prepared and improperly implemented process can disrupt a network, and automation is no exception.

Network teams should consider the following points to reduce network automation risks:

• Start with small, simple tasks. When it comes to automation, it's best to start with simple tasks. In the beginning, build some simple scripts that do basic read-only troubleshooting or network analysis, such as tracking down media access control addresses or finding the root bridge in a spanning tree domain. You should automate the investigative or diagnostic tasks that are frequently used and take up the most time. Don't make any automated changes at this stage; instead, focus on learning the automation tools to provide real value to network operations.
• Testing. Network automation requires the same process as application development: extensive testing. Application developers can quickly spin up server VMs and client test VMs and automatically run a large number of tests. In contrast, network testing has historically been a problem because test labs were too expensive and time-consuming to set up.

Building block design reduces the number of changes that need to be tested. Vendors now also offer virtual instances of many device types, often at little or no charge, but with limited performance. Validating configuration changes to these devices is critical.

The network team may need to work with other IT departments to create a test environment that accurately reflects the operational network. Ideally, the test environment will include applications and test clients to generate network traffic.

• Network verification. Intent-based networking (IBN) is the hottest industry topic, and you can start implementing IBN by creating a set of basic network checks. Verifying the state of your network is a great way to reduce automation risk. Network verification is also the best tool to verify that your network is operating as expected, even before you deploy automated changes.

To verify that your network is connected and operating as expected, review the network status. This includes device interface status, address assignments, neighboring devices, and Layer 2 and Layer 3 network protocol information. You will not make any changes to the network at this stage. Intent-based validation scripts should create alerts when checks fail, which enables teams to take appropriate actions in a timely manner.

The network validation script can then become a tool you use during future changes to perform pre- and post-change network validation checks. If any pre-change validation check fails, abort the change. Likewise, if a post-validation check fails, alert network staff and potentially abort the change. Be sure to repeat the pre-change validation after the change to ensure the network is back to the state it was in before the change.

Make it feasible

The most important concept for any network change system is to have a process that reduces risk. Manual changes will use change control boards and review cycles, and these processes are still necessary. But automation will add other processes, such as automatic verification before and after the change.

If you are just getting started with automation, limit your work to read-only tasks that do not impact the network. Most importantly, you should start using network automation.