Building WebKit on Windows

Building a large and complex such as WebKit is anything but straightforward. WebKit is the core of multiple browsers, most notably Safari and Chrome. Even the official instructions are sometimes lacking. This is mostly because such large projects are moving targets and it’s hard to keep track of all the changes in the requirements. Another issue is that most developers don’t setup build systems from scratch regularly. Probably no more than once or twice during the lifetime of a project. Even blogs, articles, forums and mailing lists that discuss building WebKit are almost completely outdated.

At any rate, these notes are up-to-date as of September 2011, I hope you find them helpful.

Background

WebKit can be built in different flavors. The differences are important, because one must setup the build system slightly differently for each flavor. There are essentially 4 different flavors: Apple Windows, WinCairo, Qt and GTK. All four require external libraries to be present for the build to succeed. I have no experience building the Qt and GTK flavors. Here I’ll cover Apple Windows and WinCairo. I do not cover building Chromium.

Build Environment

There are some common requirements, so first and foremost make sure these are setup correctly. You may follow the instructions described here. Unless you want to build using VS2005, ignore the instructions on VS2005.

Here is a summary:

1. Install Cygwin. Use this downloader. Preferably install to C:\cygwin (I’ll assume that’s where it is). Never to a path with spaces! Make sure you install: svn, curl, unzip, make, perl, python, gcc, gcc-g++, gcc4, gcc4-g++, bison, flex, gperf.
2. Install QuickTime SDK in the default location.
3. Install DirectX SDK. However, if you’ll build with VS2005, don’t install anything newer than February 2010. It’s worth noting that the August 2007 version builds WebKit just fine (as of r95358 with default flags).
4. Install Visual Studio 2005 or 2008 with all updates and Service Packs. For VS2005 please refer to the updates listed here. Personally, I suggest sticking with 2008.
5. Install Windows SDK for Server 2003 R2 or newer. See below for more details.
6. Download the Support Libraries from Apple. Copy the file (don’t extract, just copy the zip) into the WebKit source folder.
7. Create WEBKITLIBRARIESDIR and WEBKITOUTPUTDIR environment variables. The values must be fully-qualified Windows paths and not Cygwin/unix paths. If you don’t want to change you system-wide path, skip this and see the script below.

Windows SDK

If you have multiple Windows SDKs (as is typically the case when installing multiple Visual Studio versions) then you will have to setup your target Visual Studio to use the SDK of choice. There are two ways to achieve this.

The first method is to change the BIN, INCLUDE and LIB directories from the Visual Studio Options dialog. From Tools > Options menu, from Projects and Solutions > VC++ Directories page, set the Executable files‘s top entry to %SDK%\bin folder, the %SDK%\lib at the top of the Library files entries and %SDK%\include to the top of the Include files‘s entries. Where %SDK% is the installation path of the SDK. Note: This will change the Visual Studio paths for all instances. So don’t make this change if other projects depend on your current settings, unless you care more about WebKit, of course.

The second method is much less intrusive. We can change the environment variables in a shell instance and pass them on to Visual Studio. This is relatively easy and convenient, but will require scripting. In the Tools/Scripts folder of WebKit there is a script called pdevenv . This Perl script is responsible for creating a Windows CMD file which ultimately runs Visual Studio. The reason for this is because Bash can’t execute the scripts necessary for setting up Visual Studio’s environment.

Here is an example of a modified pdevenv script that sets up the SDK:

Note: If you already have SDK 6.0, 6.0A, 7.0, 7.0A or 7.1, you should be fine, but be warned: DirectX conflicts with 7.x SDKs, so make sure DirectX headers come after Windows SDK. If you have 6.0 or 6.0A you should have no problems building.

Building with VS2008

1. Install VS2008 with all Service Packs and updates. (Express version needs a bit of a push to work. Please follow the these instructions.)
2. Run update-webkit and build-webkit at least once. Regardless of the outcome, delete the contents of WEBKITOUTPUTDIR folder, wherever it is.
3. Run VS2008 and open C:\cygwin\home\\WebKitTrunk\Source\WebKit\win\WebKit.vcproj\WebKit.sln . You’ll be prompted to convert the project files from VS2005 format to 2008. Convert and save-all.
4. WebKit projects have no tolerance for warnings and they are set to the highest level. Since VS2008 does emit some superfluous warnings (such as not inlining a force-inline function) the build will fail with these settings. As such, we need to change these setting.
   Select every project that doesn’t end in “generated”, right-click and select Properties. In the Properties window, from the top, change the Configuration to “All Configurations”. Navigate to Configuration Properties/C++/General in the left tree and on the right-side change the Warning Level to “Off: Turn Off All Warnings (/W0)” and Treat Warnings As Errors to “No”. Click OK to close this dialog.
   If you don’t see C++/General in the left tree it means you selected a non-C++ project. You may do this one project at a time.
5. The project files are now ready. Save all and exit Visual Studio.

Build Notes

1. Disable Anti-Virus scanners. (Can you spell incredible?) Cygwin needs to simulate Fork, which needs to clone process memory exactly. Many AV products add system hooks to scan processes in-memory. This disrupts memory layout and DLL relocation results in differences in the memory of forked processes resulting in memory corruption. This issue may never happen even with an AV running. However, if you see fork, out-of-memory or Virtual-Memory allocation failures, try disabling AV and try again. Some don’t remove their system hooks by temporarily disabling them, so you’ll need to uninstall them completely.
2. Don’t build the Trunk. WebKit is large, complex and fast moving. Unless you like testing the limits of your sanity, you may want to stay away from the Trunk, which can be broken and problem-laden. Instead, get a successfully-built and packaged source archive from the nightly builds. At least have one recent successful build from a nightly archive before attempting the Trunk.
3. Place Cygwin/bin before others in Path. In the system path, place CygWin binary folder after System folders but before anything else. Example: %SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;C:\cygwin\bin; . This will insure that if you have Perl, SVN and other similar utilities installed outside of Cygwin, the Cygwin version will take precendence. NOTE: This may break other products that rely on the non-cygwin variants, in that case, you’ll have to perform this change in a build-script (see below).
4. Place WebKit sources in your Cygwin home directory. If you install Cygwin in the root, your home should be here: C:\cygwin\home\ . WebKit should be in a folder directly within your home. Example: C:\cygwin\home\\webkit .
5. Add the scripts folder to the path. The webkit scripts must be reachable at all times, makes sure C:\cygwin\home\\webkit\Tools\Scripts is prepended to the path.
6. Don’t update the WinCairo code unless you want to build that flavor. It seems that running update-webkit —wincairo is changing enough in the code that it’s rendering the Apple Windows flavor broken. If this happens and you can’t build Apple Windows, then simply delete the WebKitLibraries folder, and run update-webkit (without —wincairo !).
7. Run update-webkit at least once. Even if you don’t need to update the code, you must run this script at least once. Note that if you’ve downloaded a nightly-build package this script won’t update the code to the Trunk! It’ll only update the support libraries and other scripts.
8. You may have to build twice before it works. The build almost always fails the first time and after deleting the WEBKITOUTPUTDIR folder, so the build script must be executed more than once to get a successful build.
9. Look for the first failed project to fix errors. Since the WebKit build copies many source files as well as output binaries from one project to another, it’s vital that all dependencies of a sub-project succeed before it is built. That is, if a project fails, typically due to missing headers, identifiers etc, before panicking look for the first failed project and try to fix it. Most probably the errors are due to a previously broken dependency. This can get tricky, as some projects depend on external scripts that may be partially completed and stop reporting errors on subsequent builds. In such a case you won’t see failed dependencies and the sub-project in question would be the first to fail. To make sure this is the case, delete the WEBKITOUTPUTDIR folder and build again. If you get errors, note the first error you got and build again. Repeat this until you get the same error. This is the error you need to fix.
10. Disable warning as error. If you get error C2220: warning treated as error — no ‘object’ file generated errors, then follow the instruction for disabling warning as error in the Building with VS2008 section above.

Here is a bash script to setup the environment, update and build. You must set your home folder name. Save this file in the webkit folder, name it build.

Apple Windows

This flavor is the one used by Safari on Windows. The external libraries are copyrighted by Apple and aren’t redistributable. As such, the build system will require one to manually download an archive holding them from Apple after accepting the license agreement. This build, therefore, cannot be distributed on the internet with the external libraries. Having said that, one should at least try to get this flavor successfully built before attempting others.

To build this flavor follow these steps:

1. Run the Cygwin shell.
2. cd webkit
3. ./build

WinCairo

To build the WinCairo flavor one had to manually download and install the various libraries, often having to build them manually or get a prebuilt version. Thanks to the efforts of volunteers now the update-webkit script automates all that. If you have built the Apple Windows flavor, you should be able to build WinCairo using the following:

1. Run the Cygwin shell.
2. cd webkit
3. ./build –wincairo

If the build fails, try running the build script again.

However if you get “CF” identifier errors, consider updating the CF library manually. All public CF library identifiers and even filenames either begin or end with “CF”, so they are easy to spot. WebKit WinCairo uses a modified version of the original Apple sources called OpenCFLite. Download a recent version from the project’s page or get the trunk. You’ll have to bite the bullet and build the library yourself, update the headers and libs and build WebKit again.

To build OpenCFLite, open the solution in VS2005 or 2008 (depending on which you’ll use for the WebKit build). Choose the Release configuration and build the CFLite project. Ignore the mountain of warning and see if the build succeeded or not. If all goes well, in the root of the OpenCFLite source folder you should see a dist folder. Now copy the contents of dist into %WEBKITLIBRARIESDIR%\win overwriting existing files.

Delete the %WEBKITLIBRARIESDIR% folder and build Webkit WinCairo again. You should get a clean build.

Debugging

Now that you have a successful build, you may want to build the Debug version by passing —debug switch to the build-webkit script. You can execute this script directly (without running the above build script or update-webkit ) or you can modify the above build script to the same effect.

With the debug build we can use the Visual Studio debugger or the ultra-powerful WinDbg. In Visual Studio simply open the solution file from C:\cygwin\home\\WebKitTrunk\Source\WebKit\win\WebKit.vcproj\WebKit.sln , select one of the runner test projects as the startup and hit F5.

Happy hacking and don’t forget, you can contribute by submitting bug reports and sending patches and fixes. Good luck.

References

I couldn’t have succeeded in building WebKit without the help of numerous developers who shared their experience on the web. Here is a list of the most helpful and relevant pages. Be forewarned however, they are mostly outdated!

WebKit on Windows

Contents

This guide provides instructions for building WebKit on Windows 8.1 without Cygwin (the AppleWin port of WebKit). You can optionally install Cygwin in order to run tests.

Installing Development Tools

Begin by installing the following programs and setting up your environment:

1. Visual Studio. (Make sure you include C++ support.) https://www.visualstudio.com/en-us/products/visual-studio-community-vs
2. ActiveState Perl. 64-bit is fine. http://www.activestate.com/activeperl/downloads
3. ActiveState Python. Make sure it’s version 2.x instead of 3.x. 64-bit is fine. http://www.activestate.com/activepython/downloads Make sure you don’t download it from python.org because we hardcode the executable name as “python2.7” and the python.org python doesn’t include an executable with that name.
4. Ruby. 64-bit is fine. http://rubyinstaller.org/
5. Bison. http://gnuwin32.sourceforge.net/packages/bison.htm Make sure that the installation path does not have spaces in it, because Bison does not correctly quote paths when passing them to m4.
6. Gperf. http://gnuwin32.sourceforge.net/packages/gperf.htm
7. Flex. http://gnuwin32.sourceforge.net/packages/flex.htm
8. CMake. https://cmake.org/download/
9. Git. http://www.git-scm.com/download/win Note: Make sure you select the option “Use Git from the Windows Command Prompt.” This is because it installs a sh.exe in your path, and if the location of that sh.exe has spaces in it, Make gets confused. When I installed this, I unchecked all the options.
10. iTunes. This is only needed for the AppleWin port, not for the WinCairo port. http://www.apple.com/itunes/download/ This is needed because it includes the .dlls which implement Apple frameworks like CoreGraphics, CoreAnimation, etc.
11. The “Ahem” font, used for testing. http://www.w3.org/Style/CSS/Test/Fonts/Ahem/
12. (Optional: If you intend to interact with Subversion patches) Subversion. http://sourceforge.net/projects/win32svn/
13. (Optional: To speed up the build process) Ninja Build. https://github.com/ninja-build/ninja/releases

Make sure all the programs you just installed (except Visual Studio and iTunes) are accessible through your $PATH http://www.itechtics.com/customize-windows-environment-variables/ You may be required to reboot after modifying $PATH before new shells will see the updated variable.

If you are behind a proxy, be sure to set the HTTP_PROXY and HTTPS_PROXY environment variables.

Setup the Git Repository

Tell git your name.

Tell git your email address.

Download a clone of the repository.

(Optional: If you intend to interact with Subversion patches)
Automatically configure the local clone to follow Subversion:

Setup Support Tools

Download the WebKit Support Libraries to the root of your source tree https://developer.apple.com/opensource/internet/webkit_sptlib_agree.html If the file is incorrectly named, rename it to WebKitSupportLibrary.zip. Do not extract its contents. This archive includes an SDK (headers only) for the Windows builds of the Apple stack of software (like CoreGraphics, for example).

If the above command asks you to reboot your machine, do so.

Note that this command will automatically download WebKitAuxiliaryLibrary.zip from Apple and extract it into your tree (inside the WebKitLibraries folder). This archive includes Windows builds of all the public libraries that WebKit depends on (like libxml2). Note also that this command might set environment variables.

Build WebKit

Note that this command will automatically extract WebKitSupportLibrary.zip (which you downloaded in step 6) into your tree (inside the WebKitLibraries folder).

Install Cygwin (optional)

You can build with either Visual Studio 2015 or Visual Studio 2015 Community Edition.

Use the default options for the installation.

Cygwin is a collection of utilities for Windows that includes not only a Subversion client, but also additional tools that are required to build the

WebKit source. We have made a downloader available that automatically collects all of the required packages.

Download cygwin-downloader.zip. Extract the content of the archive to some folder and start cygwin-downloader.exe from that folder. This will download all the Cygwin packages you need. When all the packages have finished downloading, the Cygwin installer will launch. Choose Install from Local Directory, then click Next until the install is complete. If you are running Vista, the installer won’t be able to launch automatically, so you will have to manually launch Cygwin’s Setup.exe .

In order to rebase all cygwin dlls a rebaseall script is provided in the cygwin installation’s bin directory (usually C:\cygwin\bin ). This script must be run from ash.exe (found in the same directory) while no other cygwin application is running.

ash.exe has very limited command line editing features so cutting and pasting to the commandline is recommended.
The PATH environment variable may not contain the /bin directory by default thus the script should be launched as /bin/rebaseall .
If rebaseall is complaining about failure to write to a Windows Temp directory (i.e. /cygdrive/c/Users/you/AppData/Local/Temp ), fix the permissions on that folder (i.e. /bin/chown you /cygdrive/c/Users/you/AppData/Local/Temp ) or change the TMP environment variable to point to a directory with the appropriate permission flags

(i.e.: export TMP=/tmp ). Note: The rebase process may need to be repeated to include any new libraries added later to cygwin by listing these libraries by full cygwin path in a file and passing the file to rebaseall via the -T fileName commandline argument. The dlls under LayoutTests/http/conf are a relevant use case.

Remove the following lines from /home/[username]/.bashrc (if present):

Those lines would prevent picking up the Windows TMP and TEMP environment variables.

If you are behind a proxy you need to do the following to allow cygwin svn to get through the proxy (similar steps are available for your Windows svn):

• In bash (Cygwin) run: svn help . This will create in your cygwin home directory the file .subversion/servers .
• Edit the file mentioned above and follow the instructions in the file to set up your proxy settings.

Install Perl
Download Perl 5.18 (or newer) from activestate.com and install it.

Install Python
Download Python 2.7.8 from activestate.com and install it.

Install Ruby
Download Ruby 2.0.0 from rubyinstaller.org and install it.

Install php, we use the following steps (you need this to run http layout tests):
1. cd /usr/local/bin
2. wget http://apt-cyg.googlecode.com/svn/trunk/apt-cyg
3. chmod a+x apt-cya
4. Change line 341 of the apt-cya script to wget -nc $mirror/../$install
5. apt-cyg -m ftp://sourceware.org/pub/cygwinports/x86 install php

Optional: Hotfix for 64-bit Windows (Windows 7 or earlier) to disable the user-mode callback filter for exceptions

Without the 976038 hotfix, exceptions may be thrown in callback routines that do not cause a crash but leave the application in an inconsistent state that might cause a crash later. You will need to click on “View and request hotfix downloads” and fill out the form. A link to download the hotfix and a password will be emailed to you.

Getting the Code for Cygwin

1. Open a Cygwin Shell Double-click the Cygwin icon on your Desktop to launch a new shell.
2. You can download a snapshot of the WebKit source tree. It is updated daily.

Type these commands to check out the WebKit source tree:

Note the archived checkout uses an HTTP connection. We recommend running the following command to use HTTPS instead:

Alternatively, type this command to check out the WebKit source tree:

Install the WebKit Support Libraries

Download the WebKit Support Libraries to the root of your source tree ( C:\cygwin\home\ \WebKit ).

If the file is incorrectly named, rename it to WebKitSupportLibrary.zip . Do not extract its contents.

Run the ./Tools/Scripts/update-webkit script to update your source tree.

If you downloaded the tarball, this will bring it up to date. Windows users must always execute this command after first obtaining the code, since it will download additional libraries that are needed to build.

Once you have a current tree, the next step is building WebKit.

Getting a Crash Log

Windows Error Reporting (WER) has been integrated into the operating system. By default, Windows uploads the crash logs to Microsoft, but does not save a local copy. This is configurable via the registry.

1. Save the following text to a file named wer.reg : Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting] «ForceQueue»=dword:00000001
2. Double-click the file from Windows Explorer and respond affirmatively to any prompts.
3. Reboot

The next time Safari (or any other application) crashes, the crash information will be written into a folder located inside %LOCALAPPDATA%\Microsoft\Windows\WER\ReportQueue . Check the modification date to make sure you are using the correct file. Be sure to include the following files in your bug report: