5G and IoT bring big data boom

When interest in Hadoop began a few years ago, WANDisco prepared for the expected surge in business with its big data mobile technology. However, the business never fully took off. When the Hadoop bubble finally burst, the company was left looking for new markets. Now that industrial IoT data collected over 5G networks is going mainstream, WANDisco has become part of the big data boom in the cloud.

Hadoop is a distributed system infrastructure developed by the Apache Foundation. Users can develop distributed programs without understanding the underlying details of the distributed system, making full use of the power of clusters for high-speed computing and storage.

Like many in the industry, WANdisco CEO David Richards was frustrated by the Hadoop era. Fortunately, he has successfully steered the company through the Hadoop storm and into more hospitable waters.

It was a blessing in disguise, Richards said of the whole Hadoop experience: “We built a product that solved a massive data problem and looked like a bunch of idiots because nobody had data at that scale.”

As migration to Hadoop wanes, WANDisco's core technology needs a new market, which can help users move large amounts of data across the network in a timely manner. The company's advantage is that it enables analysts or data scientists to start processing data the moment it arrives at its destination, rather than waiting days, weeks or months to move the entire data set.

Because we use consensus to ensure the data is accurate, we don’t have to scan the data, so we can start running machine learning within a second of turning on the product. We can also multiplex it over any available bandwidth and make the whole thing twice as efficient.

Despite the failure of Hadoop, cloud data platforms have risen, and the amount of object storage data flowing into the cloud is indeed huge. WANdisco was recently used to move 25 PB of data from a large telecommunications company to the cloud, where it could be analyzed using one of the popular cloud data warehouses. It was also involved in another deal to move 1 EB of data per year to the cloud for a large automotive company.

WANdisco became a 10-year success overnight. The technical theory never changed, that moving petabytes of data across the Internet is difficult, and ensuring that data arrives in the right order and in a timely manner is also a problem.

5G technology is initially promoted mainly as a consumer technology. The new wireless technology will allow us to download full movies in seconds and will open up a new era of interactive gaming. 5G will become the new network that will allow us to participate in augmented reality and enjoy the metaverse.

It turns out that’s not all 5G is for. “The use cases are not just video or mobile phones,” Richards said. “More industries are working on becoming IoT backbone companies with complete solutions from sensor edge cloud.”

The data collected from cars is particularly large and in need of time. Manufacturers are collecting reliability and performance data on electric vehicle components, while insurance companies are interested in driving data to help price premiums.

WANdisco, a U.K.-listed but U.S.-based company, saw revenue surge 967% in 2022, from about $12 million to $127 million. The company reported about $110 million in deferred revenue, up from less than $10 million last year.

Richards called the numbers "ridiculous," saying he was stunned by the volume of business WANDisco was suddenly generating. "It's crazy what's happening with the company right now. I've never seen anything like it. This is what we said was going to happen. But when it does happen, you have to have the capacity to support it."

Richards is optimistic that the big data spigot will only open wider. As companies realize the value of their data is far greater than the value of their traditional businesses, this will create demand for greater data movement to drive larger machine learning workloads.

WANdisco also released Data Activation Platform 2.0, which the company claims will lead to faster movement of larger data sets to the cloud. The new version provides better support for using multiple data transfer agents simultaneously; automatic cleansing of data source environments; enhanced data fidelity validation; greater control over data transfer priorities; and user interface enhancements.