Dan Yi from Liepin.com: Welcome to the heyday of machine learning

On November 25-26, 2016, the WOT 2016 Big Data Technology Summit hosted by 51CTO.com was held at the Beijing JW Marriott Hotel. Since 2012, the WOT brand conference has been successfully held for 12 sessions with the concept of "focusing on technology and serving technical personnel". It has not only accumulated a large number of expert resources, but also won the recognition and praise of IT practitioners and technology enthusiasts, and has become an important technology sharing and networking platform in the industry.

At the "Machine Learning" themed session of the WOT2016 Big Data Technology Summit, Shan Yi, Chief Data Officer of Liepin.com, gave a wonderful keynote speech on "Design Principles and Patterns for Efficient Machine Learning Systems". After the meeting, the reporter interviewed him and asked him to talk about Liepin.com's experience and insights in machine learning.

Shan Yi told reporters that he chose this topic because he found that in many machine learning conferences, people mainly discussed algorithms and models. In fact, in actual work, to apply machine learning technology, it is necessary to build a good system. "Unlike general information systems, machine learning is a data-driven system. In terms of output, performance testing must be done well, in terms of input, data quality must be guaranteed, and good design patterns must be used in data flow processing." Shan Yi summarized that a "global view" is required to develop an efficient, stable and reliable machine learning system. "

Big data + artificial intelligence = new ideas

According to Shan Yi, Liepin.com currently has more than 30 million registered users. Two years ago, CEO Dai Kebin realized that Liepin.com had accumulated a large amount of user data and corporate data, which had great social and commercial value. Shan Yi gave an example that Liepin.com's recruitment data can reflect the changes, rise and fall of various industries in China, and the recruitment needs of enterprises and the flow of talents are of great value not only to corporate HR, but also to the work of the government.

Moreover, Liepin.com is a website with recruitment as its core business, and the core issue in recruitment is "matching people with jobs". How to make the best use of talents? Obviously, manual labor is time-consuming and laborious and the effect is not good, but the use of big data + artificial intelligence technology has opened up new ideas for solving this problem.

In addition, Liepin.com also has a professional social networking business. How to accurately recommend content that tens of millions of users will be interested in and recommend workplace friends? This also requires the use of a big data platform to solve this problem.

"Based on the above three points, Liepin.com has formed a big data team to solve these problems with big data technology. At the same time, we also try to use data to drive the business, improve the operational efficiency of Liepin's various product lines and departments, and support innovation." Shan Yi said that he has joined Liepin for more than two years, and now the team has developed to a relatively mature level. In the past two years, they have gone through three stages from building a basic data platform to building a business analysis team and building machine learning application development. They have achieved the initial predetermined goals, and there are higher goals to be achieved in the future.

Use data analysis to support operational decisions

During the interview, the reporter mentioned that the position of National Data Officer is very rare in China. Shan Yi laughed and said that it is indeed the case, but he expressed confidence that data is being used more and more. He believes that as the scenarios for applying data and people's understanding of the value of data become clearer, there will be more and more such positions in China in the future.

Shan Yihe described in detail the work done by the chief data officer of liepin.com.

First, we need to build a big data infrastructure to collect and store data scattered around, and then use distributed computing to process it, so that high-level applications can be realized. On this basis, Shan Yi and his team will also develop various applications of machine learning, such as job recommendations, talent recommendations, and professional social recommendations.

In addition, big data technology is used to conduct industry research and analyze the development of mid- to high-end talents across the country. At the same time, all departments of Liepin.com, from sales to products and marketing, are gradually beginning to use data to support decision-making in daily operations to improve the reliability and effectiveness of decision-making.

"In the future, we will focus on two directions. One is the development of machine learning systems, mainly functions such as talent recommendation, job recommendation and professional social networking, which we need to continue to deepen. The other direction is to strengthen business analysis and industrial cooperation, and continue to promote the mining and sharing of talent data in various industries across the country." Shan Yi said.

Ensure high quality data input

Shan Yi has a lot of experience to share with the popular machine learning. He told the reporter, "There is a saying in the industry: if the input is garbage, the output of the system must also be garbage." He believes that the machine learning system is a data-driven system. In this system, the first thing to do is to ensure that the input data is high-quality data. At the same time, the system will encounter various problems during operation. It is necessary to grasp the changes in system performance in a timely manner and discover problems as soon as they occur. Therefore, output monitoring is also very important.

He also pointed out that the workload of data processing development in the middle is quite large, so it is very important to have an efficient system and tools for data processing, especially a streaming computing system.

He also revealed, "When doing model learning under big data, you need to choose a good computing mode for machine learning model training, or find the right tools. In many cases, if you can grasp these aspects, the system will be more likely to succeed."

Welcome to the heyday

Regarding the development of machine learning in 2017, Shan Yi believes that general intelligent algorithms, especially deep learning and reinforcement learning in natural language processing, reasoning, question answering, and image/speech recognition, will continue to be a hot topic. He said that in 2017, more open source machine learning tools will appear to help users develop a smart system more easily.

Shan Yi believes that machine learning has entered its heyday. More and more people are beginning to learn about machine learning and artificial intelligence, and its application in various industries is also blossoming. He told reporters that the heyday of machine learning will last for a long time, because the goal of machine learning is to create intelligent systems, and intelligent systems are a more long-term goal.