Things about WKWebView on iOS

[[413463]]

Background

Students who are familiar with iOS\macOS Hybrid development should have experienced that although WKWebView is a "new" component launched by Apple to replace UIWebView\WebView, most developers really "can't love" it. After all, for most domestic application developers, the so-called "advantages" of WKWebView may not be reflected in actual use, but the "pitfalls" it brings are indeed not shallow.

Most of the WKWebView-related materials that can be found in the community or online are old and mostly copy-pasted. The few developers who actually practice and explore may not elaborate on the problems and solutions in detail due to time or energy constraints. As a result, there are a lot of online WKWebView-related materials, but the quality is not high; and many articles have problems such as unclear explanation of the background of the problem and lack of effective verification of the solution.

I have been engaged in the development of terminal containers for many years, and have "confronted" WKWebView many times in the design of production environment solutions. At present, hybrid development has become the standard of modern apps. On the one hand, it is a summary of the usage experience over such a long time, and on the other hand, it also hopes to provide some new perspectives or solutions for students who are still struggling. Therefore, I plan to combine some source codes of WebKit to organize and share my understanding of this component and some solutions to problems. This article attempts to explain 3 things:

What are the typical problems in using WKWebView?

Why do these problems occur?

What are the solutions to these problems?

2. Basic Review

We can refer to the official documents for iOS network design and WKWebView design features. But in order to better explain the problem later, let's focus on reviewing two basic knowledge points related to the subsequent content of the article:

iOS network design and cookie management

WKWebView multi-process model

1 iOS Network Design and Cookie Management

Cookie management is a part that is often involved in hybrid development. In application development, we know that we can manage application cookies through NSHTTPCookie and NSHTTPCookieStorage. However, how cookies are managed at the system level and how they are linked with various modules at the network layer are crucial to our subsequent analysis of the cookie problem in WKWebView.

According to official data, we know that the relationship between network-related modules on the iOS platform is as follows:

The modules from top to bottom are:

WebKit: Application layer, client App and WKWebView are at this layer.
NSURL: It can be understood as an encapsulation extension layer for the underlying CF interface. NSURLConnection, NSURLSession, etc. are in this layer.
CFNetwork: The core implementation layer of the iOS network module is the most important part of the network layer design. It is responsible for the main tasks of assembling, sending and receiving network protocols, and is closely related to the CoreFoundation framework.
BSD socket: socket service based on the underlying hardware interface.
CFNetwork is the core module of the entire network system, responsible for assembling requests, processing responses, etc.:

The core content includes:

CFURLRequest: includes the request information such as URL/header/body. CFURLRequest will be further converted into CFHTTPMessage.
CFHTTPMessage: Mainly defines and converts the HTTP protocol, converting each request into a text in standard HTTP format.
CFURLConnection: Mainly handles request tasks, including pthread threads, CFRunloop, request queue management, etc. Provides APIs for operations such as start and cancel.
CFHost: Responsible for DNS, with functions such as CFHostStartInfoResolution, based on methods such as dns_async_start and getaddrinfo_async_start.
CFURLCache/CFURLCredential/CFHTTPCookie: handles cache/certificate/cookie related logic, and has corresponding NS classes.
From the above analysis, we can know the key information: the iOS Cookie management related module is in the CFNetwork layer. That is, the "set-cookie" field in the request Response is consumed and processed in CFNetwork.

2 WKWebView multi-process model

From official data, we know that a big change of WKWebView compared to UIWebView is the "multi-process model":

When WKWebView is running, the core module runs in an independent process, independent of the App process.

Many of the reasons why WKWebView causes various problems are closely related to the multi-process operation mode.

Detailed explanation of the multi-process model

But what exactly is multi-process? Let's use a simple diagram to illustrate:

WKWebView(WebKit) includes 3 processes: UI Process, Networking Process, WebContent Process.
UI Process: App process. Some modules in WKWebView (WebKit) run in this process, which is responsible for starting other processes.
Networking Process: Network module process, which is mainly responsible for network request related functions in WKWebView; this process will only be started once in the App and shared among multiple WKWebViews.
WebContent Process: Web module process, mainly responsible for the operation of WebCore, JSCore related modules, is the core process of WKWebView. This process will be started multiple times in the App, and each WKWebView will have its own independent WebContent process.
Each process communicates with each other through the CoreIPC process.
In general: in a client App, multiple WKWebViews share a UI process (shared with the App process), a Networking process, and each WKWebView instance has a dedicated WebContent process.

Example:

The official documentation does not explain the startup rules of WebContent Process and Networking Process very clearly, and due to version iteration and other reasons, the documentation is slightly different from the latest rules. To avoid confusion and ambiguity, the following is a brief analysis based on the WebKit source code.

WebContent process startup rules

According to the official documentation:

A WKProcessPool object represents a single process that WebKit uses to manage web content. To provide a more secure and stable experience, WebKit renders the content of web views in separate processes, rather than in your app's process space. By default, WebKit gives each web view its own process space until it reaches an implementation-defined process limit. After that, web views with the same WKProcessPool object share the same web content process.

The rule is to give priority to creating new processes. When the number of processes online exceeds a certain threshold, they will be shared. This is controlled by maximumProcessCount inside WebKit. However, this rule is only effective for systems before iOS13. For systems after iOS13, WKWebView will start a new WebContent Porcess each time it creates an instance. The relevant implementation is as follows.

Before iOS13:

iOS 13 and later:

Networking process startup rules

The Networking rule is relatively simple, ensuring that an instance is started during the App lifecycle (it will be recreated after a Crash). Related code:

Three major problems and solutions

When using WKWebView in a production environment, in addition to relatively simple usage and adaptation issues, there are four issues that are likely to cause trouble for developers and front-end colleagues:

Request agent problem
Cookie management issues Full screen adaptation issues
WebContent process crash problem The following explains the causes of these four problems, possible solutions, and the problems introduced by different solutions.

1 Request proxy problem

This should be the primary problem that hinders the deployment of WKWebView. The background of the problem is relatively simple. It is not difficult to implement technically, but Apple does not want WKWebView requests to be intercepted by applications, calling it "for security reasons". However, in actual usage scenarios, we need to proxy WebView requests to meet business and performance requirements, such as offline packages and traffic monitoring.

Since it is not officially supported and there are business use cases, we can only try to solve it through "black magic". Currently, there are two solutions with more applications:

Register the proxy through [WKBrowsingContextController registerSchemeForCustomProtocol:], which is referred to as proxy scheme 1 for convenience.
Register through [WKWebViewConfiguration setURLSchemeHandler:forURLScheme:], which is referred to as proxy scheme 2 for convenience.
Currently, there is a lot of information and instructions on how to implement the two solutions, so I will not go into details here.

Although these two solutions can "partially solve the problem" to some extent, they also bring relatively many side effects. How to choose between them in a production environment still needs to be decided by specific developers. The following is a brief explanation using "Proxy Solution 1" and "Proxy Solution 2" as references, which can be used as a reference for everyone when making a selection.

Agent Plan 1

This is the earliest WKWebView request proxy solution, which can be used by apps running iOS 9 and later (currently the latest is iOS 14). According to previous research and analysis, most apps in the industry that have proxy requirements use this solution or a variant of it.

1) Solution ideas

Register http(s) in the m_registeredSchemes array of Networking through WKBrowsingContextController. For the Scheme in the array, WebKit will send the request to the process where the App is located through WKCustomProtocol when initiating a request, and the App process will execute the sending.

When sending data from the Networking process to the App process, WebKit intentionally strips the Body part (see WebCoreArgumentCodersMac.mm):

Therefore, some special processing is needed for requests with body. The solution is to inject scripts into WKWebView and rewrite the request sending methods in WebView. Before the request is sent, the body part is serialized and passed to the App process through the bridge for temporary storage.

When the App process delegates the WKWebView request, it splices the cached body as needed according to the rules and sends the request after completion.

2) Disadvantages of the solution

Although this solution is widely applicable, it also has obvious drawbacks. There are two main aspects:

(1) Problem 1: It cannot be processed in a targeted manner, and can only be handled in a one-size-fits-all manner. If an App adopts this solution, all requests sent by its WKWebView instances need to be proxied. If a WKWebView instance does not have a script injected or a proxy executed, it may cause problems such as the request not being able to be sent or the request being sent with a missing body. This is common in WKWebView instances in some integrated second-party and third-party libraries.

(2) Problem 2: It is difficult to ensure the completeness of the rewritten script. Since the request sending logic needs to be rewritten at the JS layer, such as form submission, AJAX, Fetch and other interfaces, the quality of the rewritten interface directly determines the completeness of the solution. In addition, many capabilities of the original WKWebView design are implemented at the C++ level, and rewriting only in JS cannot guarantee alignment. Currently known issues are:

For synchronous requests, this scenario is currently not supported.
For streaming requests, such as upload scenarios, the support is currently poor. It can only be sent after the JS side has fully read the data.
Unable to process Fetch API Stream return value.
When using Form to submit content containing large amounts of data, loss or crash may occur.

Agent Plan 2

This solution is implemented based on the "extension" of the [WKWebViewConfiguration setURLSchemeHandler:forURLScheme:] interface opened by Apple on iOS 11. For devices after iOS 11.3, this solution has good practicality (WebKit handles some Body transmission issues).

1) Solution ideas

[WKWebViewConfiguration setURLSchemeHandler:forURLScheme:] can register a custom request scheme on the WKWebView instance. If the request sent by WKWebView matches the registered scheme, it will be delegated to the UI process (App process) to perform the sending action.
By default, WKWebView does not support registering standard schemes such as http(s), but there is a "black magic" method to bypass the restriction.
When AJAX sends BLOB data, the body may be lost. You can refer to the similar solution in Proxy Solution 1 to solve it.
2) Solution advantages

The two huge advantages of proxy solution 2 over solution 1 are:

Instead of a one-size-fits-all approach, configure the binding with the WKWebView instance: that is, we can process the WKWebView instance we need to process in a targeted manner, and have no impact on the objects in the third-party library, greatly improving security.
No need to rewrite all sent requests: In most cases, the body of the request can be carried to the App process, that is, we only need to handle some exceptions in a targeted manner, which greatly improves robustness.
3) Disadvantages of the solution

In addition to the system version limitation of iOS 11.3, this solution also has many difficult problems in actual operation, mainly as follows:

(1) Problem 1: When downloading multiple images in segments, there is a bug in the processing sequence inside WKWebView

Problem manifestation: When loading a large image in WKWebView and the large image data is returned in fragments, the abnormal timing processing inside WKWebView may cause the image to fail to display or be incomplete. The specific process of loading images in WebKit can be briefly explained as follows:

The problem lies in the execution order of step1, step2, and step3. Under abnormal circumstances, the execution order may occasionally be: step1 -> step3 -> step2, and step3 is no longer triggered (allDataReceived), which results in the final content of the image not being rendered on the screen.

Solution: There is no effective solution at present. Configuring suppressesIncrementalRendering to YES can alleviate the problem to some extent, but it cannot cure it and will slightly affect the experience.

(2) Problem 2: System synchronization AJAX causes crash on iOS 12 and below

Problem manifestation: If a web page sends a sync request in WKWebView, it may cause the WebContent process to crash, and WKWebView will call back webViewWebContentProcessDidTerminate, which will lead to problems such as a white screen. This problem is clearly a bug in WebKit, and there is a related Fix:

Bug1: WebURLSchemeHandlerProxy::loadSynchronously crashes with sync request (2018-08-06 14:14): https://bugs.webkit.org/show_bug.cgi?id=188358

Bug2: WKURLSchemeHandler crashes when sent errors with sync XHR (2019-06-20 01:20): https://bugs.webkit.org/show_bug.cgi?id=199063

Solution: For Bug 1, the solution is relatively simple, that is, to preferentially call back some empty data before calling back the error of the network request to avoid the problem; but for Bug 2, there is currently no effective solution.

(3) Problem 3: SWAP under 301 request causes page transition failure

Problem manifestation: If a page uses 301 for redirection, the redirected page may fail to load, leading to page abnormalities, white screen and other problems.

Solution: Turn off processSwapsOnNavigation and set it to NO (internal property).

In general, although Proxy Solution 2 has great advantages over Proxy Solution 1, the current usage of this solution is slightly lower than that of Proxy Solution 1 due to version restrictions and other reasons. Compared with Proxy Solution 1, this solution has obvious advantages and disadvantages. For example, in the multi-image sharding scenario, the problem of images not being displayed may occur, and no effective solution has been found so far. Whether Solution 2 can replace Solution 1 in the production environment is still up to the user to decide.

2. Cookie issues

According to official documents and information, we know that WKWebView has problems because of its "independent storage", which causes Cookies and Cache to be incompatible with App. However, this statement is rather vague, and in actual use, WKWebView and App Cookies are not completely isolated. This ambiguous performance makes it difficult for people to figure out where the boundary of "communication" or "incompatibility" is.

Below, based on my own understanding of this, I will try to explain what the problem of WKWebView using cookies is and the reasons behind it. Since Apple has not open-sourced all the code, a lot of the content below is my own understanding and inference, which cannot be guaranteed to be completely correct. I will only introduce some ideas and judgments for your reference when needed.

Cookie Management Policy

According to the background introduction in the previous section, we know that iOS Cookie-related content is managed by CFHTTPCookie, CFHTTPCookieStorage, etc. at the CFNetwork layer, which is part of the CFNetwork module. And for Session Cookies and Persistent Cookies, the system has different management strategies:

Session Cookie: Saved in memory and effective within the process cycle. On iOS mobile terminals, one App process corresponds to one Session, which means that Session Cookies can be shared within the process.
Persistent Cookies: In addition to being saved in memory, these cookies are also persisted to disk and can be used multiple times. Local files are stored in the sandbox folder/Library/Cookies/Cookies.binarycookies; it should be noted that persistent cookies are not synchronized to Cookies.binarycookies immediately after they are generated. According to experience, there will be a delay of 300ms ~ 3s.

WKWebView Cookie Issue

Based on the iOS Cookie management in the previous section and combined with the multi-process model, we can roughly infer the App and WKWebView Cookie management model, as shown in the following diagram:

Note: WKHTTPCookieStore is shown in the Networking process for illustration. In reality, this module is scattered in WebContent, Networking, and UI Process, and parts of each process are bridged through IPC.

According to the above figure, we can derive two core points related to WKWebView Cookie:

1) What exactly is the WKWebView Cookie problem?

For "Session Cookie": The App process and WKWebView process (WebContent + Networking) are completely isolated.
For "persistent cookies": there is a time difference in synchronization between the App process and the WKWebView process (WebContent + Networking).
2) The root cause of the WKWebView Cookie problem

The dual-process design of App process and Networking.

Core Goals

After understanding the WKWebView problem and the corresponding root cause, it is relatively clear how to deal with this problem: depending on whether the network request of WKWebView is proxied, we need different processing strategies.

Scenario 1 - WKWebView network request is not proxied: Cookies are completely managed by the Networking process, and WKWebView can be self-enclosed. In most cases, the App process does not need to be aware of it. If it is really necessary to be aware of it, you can choose JS bridging, forced persistence and other solutions according to the business scenario.
Scenario 2 - Proxied WKWebView network request: Most cookies are managed by the App process. What synchronization strategy should be adopted at this time?
Since we have not adopted scenario 1 in the production environment, this article does not intend to make a hasty analysis. The following mainly focuses on scenario 2 for further analysis. Our core goals in scenario 2 are:

Cookies generated in the App process can be synchronized to the Networking process in a timely manner: This mainly solves the problem of how the JS side can read related cookies in a timely manner when "Set-Cookie" exists in the Response.
For the cookies generated by JS in WebContent, they can be synchronized to the App process in time: This mainly solves the problem of how to ensure that after the cookies are generated on the JS side, they can be carried normally in the subsequent proxy network requests.

Synchronous means

Before confirming the solution, we must first clarify one question: What are the sources of cookies on the client side?

For App processes, there are two sources of cookies:

Written via NSHTTPCookieStorage.
Written in the network request Response Header via "Set-Cookie".
For the WebContent process, it is mainly written by JS through document.cookie (Set-Cookie will not take effect in the WKWebView process after the network proxy).

Secondly, we need to confirm what means are available for synchronization:

For systems after iOS 11, Apple has provided us with the WKHTTPCookieStore object for reading, writing, and monitoring the cookies corresponding to WKWebView, which can be used directly.

For systems before iOS 11, they need to be handled differently.

The simple process of synchronizing from the App process to the Networking process is as follows:

In the first step, you need to make the Session Cookie persistent and save it temporarily (note that it needs to be marked for recovery).
Step 2: Call NSHTTPCookieStorage internal interface _saveCookies to trigger forced synchronization.
Step 3: Restore the temporarily saved Session Cookie to avoid contamination.
Since the Networking process does not generate cookies, what we need to do next is to synchronize cookies from the WebContent process: the processing strategy is to rewrite the document.cookie method on the JS side, and when JS modifies cookies, pass the cookies to the App process through the bridge.

Solution

After clarifying the problems, goals, and available means, we can summarize the solutions to WKWebView Cookie-related issues:

For iOS 11 and later systems, we can hook the interface for reading and writing cookies in NSHTTPCookieStorage, listen to the "Set-Cookie" keyword in network requests, and synchronize to WKWebView when the App process cookies change; at the same time, WKHTTPCookieStore provides the cookiesDidChangeInCookieStore capability to monitor changes in cookies in WKWebView.
For systems before iOS 11, the processing strategy is similar. However, we need to use the NSHTTPCookieStorage interface to force synchronization, and pay attention to restoring the SessionOnly attribute of the cookie; at the same time, we need to rewrite document.cookie on the JS side to perceive changes in cookies in WKWebView.
Special attention:

When using the solution after iOS 11, you must pay attention that the operation of WKHTTPCookieStore will involve IPC communication. If the communication is too frequent and the amount of communication data is too large, it will cause obvious performance problems. Extreme cases may cause IPC module exceptions and all WKWebViews cannot be loaded. For example, in a typical scenario, if there are many requests for a page, each request carries "Set-Cookie", and for the sake of simplicity in business, all cookies of the App process are synchronized to WKWebView each time, then when there are too many cookies, there is a certain probability (brute force testing can be reproduced) that an IPC exception will be triggered, resulting in all subsequent WKWebView instances being unable to load normally, and only the App process can be restored after killing it. It is recommended to synchronize the changed parts as needed when synchronizing cookies.

3 Full screen adaptation issues

The full-screen adaptation problem is relatively uncomplicated, but due to the differences in performance between WKWebView and UIWebView, it can easily cause some troubles.

The problem is that UIWebView and WKWebView have slightly different support for front-end viewport-fit: UIWebView has better support for viewport-fit, and its performance is basically consistent with the official documentation. However, there is an unspoken rule in WKWebView that if the height of the body in the web page does not exceed the actual height of the WKWebView component, viewport-fit=cover may not take effect.

The solution is to avoid such situations in the page, such as configuring body height to 100vh (or other similar solutions).

4 WebContent process crash issue

This is a problem that has a low probability of occurrence, but lacks a general and effective solution. We know that in WKWebView multi-process mode, if the WebContent process crashes for various reasons, WKWebView will tell the developer through the webViewWebContentProcessDidTerminate callback. Generally, we will reload the page through the reload method. At the same time, if the user's device memory is tight, the system may actively KILL the WebContent process. That is, the App process (foreground) may be normal, but the WebContent crashes and the page is reloaded.

In most cases, entering this process will not necessarily cause trouble for user operations. However, if the memory shortage is caused by the front-end triggering business, such as calling the camera to upload pictures in the form, the impact of this process on the user may be fatal. Even if we restore the page through WebView reload, the user's upload action will be interrupted, causing abnormal submission process and affecting user operations. And if the user's device enters this state, in most cases the user's operation will trigger the same process again.

In this case, users cannot perceive the root cause of the problem in time, and most of their intuitive reactions are: "There is a bug in the App!" Therefore, from the user's perspective, there is a lack of automatic recovery and problem-solving methods.

Currently, there is no effective and unified solution to this problem. One solution is to configure the client and front-end, and design solutions for core processes that may have exceptions. Use the capabilities of the end side to persist data, and use persistent data to restore the scene after similar exceptions occur, so as to ensure that the user operation process is normal without the user noticing.

Conclusion

The above are some typical problems and corresponding solutions we encountered in the use of WKWebView during the design and development of the terminal container. In general, the current incoordination is mostly caused by the system platform's failure to fully consider the demands of developers and the poor compatibility of component design with historical businesses. Of course, this state is definitely not a reasonable state. In the future, whether it is the system platform, the business side, or the developer, when the contradiction cannot be reconciled, one party will always have to compromise. Before this time point comes, I hope that the above summary can provide some help for students who are troubled by such problems.