Wayland (display server protocol)

Wayland

Screenshot of a Wayland demonstration
Original author(s)	Kristian Høgsberg
Developer(s)	freedesktop.org et al.
Initial release	0.85 / 9 February 2012 (2012-02-09)[1]
Stable release	1.12 / 21 September 2016 (2016-09-21)[2]
Repository	cgit.freedesktop.org/wayland/wayland/
Development status	Active
Written in	C
Operating system	Linux, DragonflyBSD
Size	c. 100 KB
Type	Windowing system Display server
License	MIT License[3][4][5]
Website	wayland.freedesktop.org

Wayland is a computer protocol that specifies the communication between a display server (called Wayland compositor) and its clients, as well as a reference implementation of the protocol in the C programming language.^[6]

Wayland is developed by a group of volunteers initially led by Kristian Høgsberg as a free and open community-driven project with the aim of replacing the X Window System with a modern, simpler windowing system in Linux and other Unix-like operating systems.^[6] The project's source code is published under the terms of the MIT License, a permissive free software licence.^[7][3]

As part of its efforts, the Wayland project also develops a reference implementation of a Wayland compositor called Weston.^[6]

Overview

① The evdev module of the Linux kernel gets an event and sends it to the Wayland compositor.
② The Wayland compositor looks through its scenegraph to determine which window should receive the event. The scenegraph corresponds to what's on screen and the Wayland compositor understands the transformations that it may have applied to the elements in the scenegraph. Thus, the Wayland compositor can pick the right window and transform the screen coordinates to window local coordinates, by applying the inverse transformations. The types of transformation that can be applied to a window is only restricted to what the compositor can do, as long as it can compute the inverse transformation for the input events.
③ As in the X case, when the client receives the event, it updates the UI in response. But in the Wayland case, the rendering happens by the client via EGL, and the client just sends a request to the compositor to indicate the region that was updated.
④ The Wayland compositor collects damage requests from its clients and then re-composites the screen. The compositor can then directly issue an ioctl to schedule a pageflip with KMS.

In recent years, Linux desktop graphics has moved from having "a pile of rendering interfaces... all talking to the X server, which is at the center of the universe" towards putting the Linux kernel and its components (i.e. DRI, DRM) "in the middle", with "window systems like X and Wayland ... off in the corner". This will be "a much-simplified graphics system offering more flexibility and better performance".^[8]

Kristian Høgsberg could have added an extension to X as many recent projects have done, but preferred to "[push] X out of the hotpath between clients and the hardware" for reasons explained in the project's FAQ:^[7]

What’s different now is that a lot of infrastructure has moved from the X server into the kernel (memory management, command scheduling, mode setting) or libraries (cairo, pixman, freetype, fontconfig, pango, etc.), and there is very little left that has to happen in a central server process. ... [An X server has] a tremendous amount of functionality that you must support to claim to speak the X protocol, yet nobody will ever use this. ... This includes code tables, glyph rasterization and caching, XLFDs (seriously, XLFDs!), and the entire core rendering API that lets you draw stippled lines, polygons, wide arcs and many more state-of-the-1980s style graphics primitives. For many things we've been able to keep the X.org server modern by adding extension such as XRandR, XRender and COMPOSITE ... With Wayland we can move the X server and all its legacy technology to an optional code path. Getting to a point where the X server is a compatibility option instead of the core rendering system will take a while, but we'll never get there if [we] don’t plan for it.

Wayland consists of a protocol and a reference implementation named Weston. The project is also developing versions of GTK+ and Qt that render to Wayland instead of to X. Most applications are expected to gain support for Wayland through one of these libraries without modification to the application.

Wayland does not currently provide network transparency, but it may in the future.^[9] It was attempted as a Google Summer of Code project in 2011, but was not successful.^[10] Adam Jackson has envisioned providing remote access to a Wayland application by either "pixel-scraping" (like VNC) or getting it to send a "rendering command stream" across the network (as in RDP, SPICE or X11).^[11] As of early 2013, Høgsberg is experimenting with network transparency using a proxy Wayland server which sends compressed images to the real compositor.^[12][13]

Software architecture

Protocol architecture

In the Wayland protocol architecture, a client and a compositor communicate through the Wayland protocol using the reference implementation libraries.

Wayland protocol follows a client–server model in which clients are the graphical applications requesting the display of pixel buffers on the screen, and the server (compositor) is the service provider controlling the display of these buffers.

The Wayland reference implementation has been designed as a two-layer protocol:^[14]

• A low-level layer or wire protocol that handles the inter-process communication between the two involved processes—​​client and compositor—​​and the marshalling of the data that they interchange. This layer is message-based and usually implemented using the kernel IPC services, specifically Unix domain sockets in the case of Linux and Unix-like operating systems.^[15]:9
• A high-level layer built upon it, that handles the information that client and compositor need to exchange to implement the basic features of a window system. This layer is implemented as "an asynchronous object-oriented protocol".^[15]:9

While the low-level layer was written manually in C language, the high-level layer is automatically generated from a description of the elements of the protocol stored in XML format.^[16] Every time the protocol description of this XML file changes, the C source code that implements such protocol can be regenerated to include the new changes, allowing a very flexible, extensible and error-proof protocol.

The reference implementation of Wayland protocol is split in two libraries: a library to be used by Wayland clients called libwayland-client and a library to be used by Wayland compositors called libwayland-server.^[15]:57

Protocol overview

The Wayland protocol is described as an "asynchronous object-oriented protocol."^[15]:9 Object-oriented means that the services offered by the compositor are presented as a series of objects living on the same compositor. Each object implements an interface which has a name, a number of methods (called requests) as well as several associated events. Every request and event has zero or more arguments, each one with a name and a data type. The protocol is asynchronous in the sense that requests do not have to wait for synchronized replies or ACKs, avoiding round-trip delay time and achieving improved performance.

The Wayland clients can make a request (a method invocation) on some object if the object's interface supports that request. The client must also supply the required data for the arguments of such request. This is the way the clients request services from the compositor. The compositor in turn sends information back to the client by causing the object to emit events (probably with arguments too). These events can be emitted by the compositor as a response to a certain request, or asynchronously, subject to the occurrence of internal events (such as one from an input device) or state changes. The error conditions are also signaled as events by the compositor.^[15]:9

For a client to be able to make a request to an object, it first needs to tell the server the ID number it will use to identify that object.^[15]:9 There are two types of objects in the compositor: global objects and non-global objects. Global objects are advertised by the compositor to the clients when they are created (and also when they are destroyed), while non-global objects are usually created by other objects that already exist as part of their functionality.^[17]

The interfaces and their requests and events are the core elements that define the Wayland protocol. Each version of the protocol includes a set of interfaces, along with their requests and events, which are expected to be in any Wayland compositor. Optionally, a Wayland compositor may define and implement its own interfaces that support new requests and events, thereby extending functionality beyond the core protocol.^[15]:10 To facilitate changes to the protocol, each interface contains a "version number" attribute in addition to its name; this attribute allows for distinguishing variants of the same interface. Each Wayland compositor exposes not only what interfaces are available, but also the supported versions of those interfaces.^[15]:12

Wayland core interfaces

The interfaces of the current version of Wayland protocol are defined in the file protocol/wayland.xml of the Wayland source code.^[16] This is an XML file that lists the existing interfaces in the current version, along with their requests, events and other attributes. This set of interfaces is the minimum required to be implemented by any Wayland compositor.

Some of the most basic interfaces of the Wayland protocol are:^[15]:10-12

• wl_display – the core global object, a special object to encapsulate the Wayland protocol itself
• wl_registry – the global registry object, in which the compositor registers all the global objects that it wants to be available to all clients
• wl_compositor – an object that represents the compositor, and is in charge of combining the different surfaces into one output
• wl_surface – an object representing a rectangular area on the screen, defined by a location, size and pixel content
• wl_buffer – an object that, when attached to a wl_surface object, provides its displayable content
• wl_output – an object representing the displayable area of a screen
• wl_pointer, wl_keyboard, wl_touch – objects representing different input devices like pointers or keyboards
• wl_seat – an object representing a seat (a set of input/output devices) in multiseat configurations

A typical Wayland client session starts by opening a connection to the compositor using the wl_display object. This is a special local object that represents the connection and does not live within the server. By using its interface the client can request the wl_registry global object from the compositor, where all the global object names live, and bind those that the client is interested in. Usually the client binds at least a wl_compositor object from where it will request one or more wl_surface objects to show the application output on the display.^[17]

Wayland extension interfaces

A Wayland compositor can define and export its own additional interfaces.^[15]:10 This feature is used to extend the protocol beyond the basic functionality provided by the core interfaces, and has become the standard way to implement Wayland protocol extensions. Certain compositors can choose to add custom interfaces to provide specialized or unique features. The Wayland reference compositor, Weston, used them to implement new experimental interfaces as a testbed for new concepts and ideas, some of which later became part of the core protocol (such as wl_subsurface interface added in Wayland 1.4^[18]).

Extension protocols to the core protocol

XDG-Shell protocol

XDG-Shell protocol (see freedesktop.org for XDG) is an extended way to manage surfaces under Wayland compositors (not only Weston). The traditional way to manipulate (maximize, minimize, fullscreen, etc.) surfaces is to use the wl_shell_*() functions, which are part of the core Wayland protocol and live in libwayland-client. An implementation of the xdg-shell protocol, on the contrary, is supposed to be provided by the Wayland compositor. So you will find the xdg-shell-client-protocol.h header in the Weston source tree. Each Wayland compositor is supposed to provide its own implementation.

As of June 2014, XDG-Shell protocol was not versioned and still prone to changes.

xdg_shell is a protocol aimed to substitute wl_shell in the long term, but will not be part of the Wayland core protocol. It starts as a non-stable API, aimed to be used as a development place at first, and once features are defined as required by several desktop shells, it can be finally made stable. It provides mainly two new interfaces: xdg_surface and xdg_popup. The xdg_surface interface implements a desktop-style window that can be moved, resized, maximized, etc.; it provides a request for creating child/parent relationship. The xdg_popup interface implements a desktop-style popup/menu; an xdg_popup is always transient for another surface, and also has implicit grab.^[19]

IVI-Shell protocol

IVI-Shell is an extension to the Wayland core protocol, targeting in-vehicle infotainment (IVI) devices.^[20]

Rendering model

Wayland compositor and its clients use EGL to draw directly into the framebuffer; X.Org Server with XWayland and Glamor.

The Wayland protocol does not include a rendering API.^{[15]:7[7][21][22]:2} Instead, Wayland follows a direct rendering model, in which the client must render the window contents to a buffer shareable with the compositor.^[15]:7 For that purpose, the client can choose to do all the rendering by itself, use a rendering library like Cairo or OpenGL, or rely on the rendering engine of high-level widget libraries with Wayland support, such as Qt or GTK+. The client can also optionally use other specialized libraries to perform specific tasks, such as Freetype for font rendering.

The resulting buffer with the rendered window contents are stored in a wl_buffer object. The internal type of this object is implementation dependent. The only requirement is that the content data must be shareable between the client and the compositor. If the client uses a software (CPU) renderer and the result is stored in the system memory, then client and compositor can use shared memory to implement the buffer communication without extra copies. The Wayland protocol already provides natively this kind of shared memory buffers through wl_shm and wl_shm_pool interfaces .^{[15]:11, 20-21} The drawback of this method is that the compositor may need to do additional work (usually to copy the shared data to the GPU) to display it, which leads to slower graphics performance.

The most typical case is for the client to render directly into a video memory buffer using a hardware (GPU) accelerated API such as OpenGL, OpenGL ES or Vulkan. Client and compositor can share this GPU-space buffer using a special handler to reference it.^[23] This method allows the compositor to avoid additional copies of data to the GPU, resulting in faster graphics performance than using shm buffers, and therefore the preferred one. The compositor can further optimize the composition of the final scene to show on the display by using the same hardware acceleration API as the client.

When the rendering is done and the buffer shared, the Wayland client should instruct the compositor to present the rendered contents of the buffer on the display. For this purpose, the client binds the buffer object that stores the rendered contents to the surface object, and sends a "commit" request to the surface, transferring the effective control of the buffer to the compositor.^[14] Then, the client waits for the compositor to release the buffer (signaled by an event) if it wants to reuse the buffer to render another frame, or it can use another buffer to render the new frame and, when the rendering is finished, to bind this new buffer to the surface and commit its contents.^[15]:7 The procedure used for rendering, including the number of buffers involved and their management, is entirely under the client control.^[15]:7

Comparison with other window systems

Differences between Wayland and X

There are several differences between Wayland and X in regards to performance, code maintainability, and security:^[24]

• Architecture: the composition manager is a separate, additional feature in X, while Wayland merges display server and compositor as a single function.^[25][21] Also, it incorporates some of the tasks of the window manager, which in X is a separate client-side process.^[26]
• Composition: compositing is optional in X, but mandatory in Wayland. Compositing in X is "active"; that is, the compositor must fetch all pixel data, which introduces latency. In Wayland, compositing is "passive", which means the compositor receives pixel data directly from clients.^[27]:8-11
• Rendering: the X server itself is able to perform rendering, although it can also be instructed to display a rendered window sent by a client. In contrast, Wayland does not expose any API for rendering, but delegates to clients such tasks (including the rendering of fonts, widgets, etc.).^[25][21] Window decorations can be rendered on the client side (e.g., by a graphics toolkit) or on the server side (by the compositor).^[28]
• Security: Wayland isolates the input and output of every window, achieving confidentiality, integrity and availability in both cases; X lacks these important security features.^[29][30][31] Also, with the vast majority of the code running in the client, less code needs to run with root privileges, improving security.^[29]
• Inter-process communication: the X server provides a basic communication method between X clients, later extended by ICCCM conventions. This X client-to-client communication is used by window managers and also to implement X sessions, selections and drag-and-drop, and other features. Wayland core protocol does not support communication between wayland clients at all, and the corresponding functionality (if needed) should be implemented by the desktop environments (like KDE or GNOME), or by a third party (for example, by using native IPC of the underlying operating system).
• Networking: The X Window System is an architecture that was designed at its core to run over a network. Wayland does not offer network transparency by itself;^[7] however, a compositor can implement any remote desktop protocol to achieve remote displaying. In addition, there is research into Wayland image streaming and compression that would provide remote frame buffer access similar to that of VNC.^[13]

Compatibility with X

A screenshot showing xwayland

XWayland is an X Server running as a Wayland client, thus capable of displaying native X11 client applications in a Wayland compositor environment.^[32] This is similar to the way XQuartz runs X applications in OS X’s native windowing system. The goal of XWayland is to facilitate the transition from X Window System to Wayland environments, providing a way to run unported applications in the meantime. XWayland was mainlined into X.Org Server version 1.16.^[33]

Widget toolkits such as Qt 5 and GTK+ 3 can switch their graphical back-end at run time,^[34] allowing users to choose at load time whether they want to run the application over X or over Wayland. Qt 5 provides the -platform command-line option^[35] to that effect, whereas GTK+ 3 lets users select the desired GDK back-end by setting the GDK_BACKEND Unix environment variable.^[34][36]

Wayland compositors

Typical elements of a window. Neither Wayland nor X11 specifies what software is responsible for rendering the window decoration. Weston requires that they are drawn by the client, but KWin will implement server-side decoration.^[28]

Display servers that implement the Wayland display server protocol are also called Wayland compositors because they additionally perform the task of a compositing window manager.

• Weston – the reference implementation of a Wayland compositor; Weston implements client-side decoration
• Lipstick – mobile graphical shell framework which implements Wayland compositor. It is used in Sailfish OS and Nemo Mobile.^[37]
• Enlightenment has full Wayland support since version 0.20.^[38]
• KWin had incomplete Wayland support in April 2013.^[39]
• Mutter maintains a separate branch for the integration of Wayland for GNOME 3.9 (in September 2013).^[40]
• Clayland is a simple example Wayland compositor using Clutter.
• Westeros is a Wayland compositor library that allows applications to create their own Wayland displays, which allows nesting and embedding of third party applications.^[41]

Weston

Weston is the reference implementation of a Wayland compositor also developed by the Wayland project. It is written in C and published under the MIT License. Weston only has official support for the Linux operating system due to its dependence on certain features of the Linux kernel, such as kernel mode-setting, Graphics Execution Manager (GEM), and udev, which have not yet been implemented in other Unix-like operating systems.^[42] When running on Linux, handling of the input hardware relies on evdev, while the handling of buffers relies on Generic Buffer Management (GBM). However, in 2013 a prototype port of Weston to FreeBSD was announced.^[43]

Weston relies on GEM to share application buffers between the compositor and applications. It contains a plugin system of "shells" for common desktop features like docks and panels.^[13] Clients are responsible for the drawing of their window borders and their decorations. For rendering, Weston can use OpenGL ES^[44] or the pixman library to do software rendering.^[45] The full OpenGL implementation is not used, because on most current systems, installing the full OpenGL libraries would also install GLX and other X Window System support libraries as dependencies.^[46]

Maynard is a graphical shell and has been written as a plugin for Weston, similar as the GNOME Shell has been written as a plugin to Mutter.^[47]

A remote access interface for Weston was proposed in October 2013 by a RealVNC employee.^[48]

libinput

libinput was created to consolidate the input stack across multiple Wayland compositors.

Libinput handles input devices for multiple Wayland compositors and also provides a generic X.Org Server input driver. It aims to provide one implementation for multiple Wayland compositors with a common way to handle input events while minimizing the amount of custom input code compositors need to include. libinput provides device detection (via udev), device handling, input device event processing and abstraction.^[49][50]

Version 1.0 of libinput followed version 0.21, and included support for tablets, button sets and touchpad gestures. This version will maintain stable API/ABI.^[51]

As GNOME/GTK+ and KDE Frameworks 5^[52] have mainlined the required changes, Fedora 22 will replace X.Org's evdev and Synaptics drivers with libinput.^[53]

The Weston code for handling input devices (keyboards, pointers, touch screens, etc.) was split into its own separate library, called libinput, for which support was first merged in Weston 1.5.^[54][55]

With version 1.16, the X.Org Server obtained support for the libinput library in form of a wrapper called xf86-input-libinput.^[56][57]

Wayland Security Module

Wayland Security Module is a proposition that resembles the Linux Security Module interface found in the Linux kernel.^[58]

Some applications (especially the ones related to accessibility) require privileged capabilities that should work across different Wayland compositors. Currently, applications under Wayland are generally unable to perform any sensitive tasks such as taking screenshots or injecting input events. Wayland developers are actively looking for feasible ways to handle privileged clients securely and then designing privileged interfaces for them.

Wayland Security Module is a way to delegate security decisions within the compositor to a centralized security decision engine.^[58]

Adoption

As explained in the "Software architecture" section above, the Wayland protocol is designed to be simple so that additional protocols and interfaces need to be defined and implemented to achieve a holistic windowing system. As of July 2014, these additional interfaces are actively being worked on. So, while the toolkits already fully support Wayland, the developers of the graphical shells are cooperating with the Wayland developers creating the necessary additional interfaces.

Desktop Linux distributions

In general, out of the box support for a full desktop running Wayland in major Linux distributions is still in early stages. However, for many distributions, switching from X.org to Wayland eventually, is a high-priority goal, or have done so already.

• Arch Linux has Wayland as the default, depending on the desktop, e.g. when using GNOME. It is also straightforward to launch a Wayland desktop session using the command line.^[59]

• RebeccaBlackOS is a live USB Wayland-based Linux distribution that allows a convenient way to try out a real Wayland desktop without having to make any modifications to the main operating system of the computer.^[60]

• Fedora starting with version 25 uses Wayland for the default GNOME desktop session, with X.Org as a fallback; if the graphics driver cannot support Wayland.^[61]

Toolkit support

Toolkits supporting Wayland include the following:

• Clutter has complete Wayland support.^[62][63][64]
• EFL has complete Wayland support, except for selection.^[65]
• GTK+ 3.20 has complete Wayland support.^[66]
• Qt 5 has complete Wayland support.
• SDL support for Wayland debuted with the 2.0.2 release^[67] and was enabled by default since version 2.0.4.^[68]
• GLFW 3.2 has Wayland support.^[69]
• FreeGLUT has initial Wayland support.^[70]

Desktop environments

Desktop environments in process of being ported from X to Wayland include GNOME,^[71] KDE Plasma 5^[72] and Enlightenment^[73]. The Hawaii desktop environment is a desktop environment that exclusively supports Wayland.

In November 2015, Enlightenment e20 was announced with "full Wayland support".^[74][38][75] GNOME plans that 3.20 will be the first version "to have a full Wayland session".^[76] Wayland support for KDE was delayed until the release of Plasma 5^[77], though previously KWin 4.11 got an experimental Wayland support.^[78] The version 5.4 of Plasma was the first with a Wayland session.^[79]

Other software

Other software supporting Wayland includes the following:

• Intelligent Input Bus is working on Wayland support, it could be ready for Fedora 22.^[80]
• RealVNC published a Wayland developer preview in July 2014.^[48][81][82]
• Maliit is an input method framework that runs under Wayland.^[83][84][85]
• kmscon supports Wayland with wlterm.^[86]
• Mesa has Wayland support integrated.^[87]
• Eclipse was made to run on Wayland during a GSoC-Project in 2014.^[88]
• The Vulkan WSI (Window System Interface) is a set of API calls serve a similar purpose as EGL does for OpenGL ES or GLX for OpenGL. Vulkan WSI includes support for Wayland from day one: VK_USE_PLATFORM_WAYLAND_KHR. Vulkan clients can run on unmodified Wayland servers, including Weston, GENIVI LayerManager, Mutter / GNOME Shell, Enlightenment, and more. The WSI allows applications to discover the different GPUs on the system, and display the results of GPU rendering to a window system.^[89]

Mobile and embedded hardware

Mobile and embedded hardware supporting Wayland includes the following:

• GENIVI Alliance: The GENIVI automotive industry consortium for in-vehicle infotainment (IVI) supports Wayland.^[90]
• Raspberry Pi: The Raspberry Pi Foundation in collaboration with Collabora released Maynard and work on improving performance and memory consumption, but do not expect to be able to replace X11 as the default display server until later in 2013.^[91][92]
• Jolla: Smartphones from Jolla use Wayland. It is also used as standard when Linux Sailfish OS is used with hardware from other vendors or when it is installed into Android devices by users.^[93][94][95]
• Tizen: Tizen up to 2.x supports Wayland in in-vehicle infotainment (IVI) setups^[96] and from 3.0 onward defaults to Wayland.^[97]

History

Wayland uses direct rendering over EGL.

Kristian Høgsberg, a Linux graphics and X.Org developer who previously worked on AIGLX and DRI2, started Wayland as a spare-time project in 2008 while working for Red Hat.^{[98][99][100][101]} His stated goal was a system in which "every frame is perfect, by which I mean that applications will be able to control the rendering enough that we'll never see tearing, lag, redrawing or flicker." Høgsberg was driving through the town of Wayland, Massachusetts when the underlying concepts "crystallized", hence the name.^[100][102]

In October 2010, Wayland became a freedesktop.org project.^[103][104] As part of the migration the prior Google Group was replaced by the wayland-devel mailing list as the project's central point of discussion and development.

The Wayland client and server libraries were initially released under the MIT License,^[105] while the reference compositor Weston and some example clients used the GNU General Public License version 2.^[106] Later all the GPL code was relicensed under the MIT license "to make it easier to move code between the reference implementation and the actual libraries".^[107] In 2015 it was discovered that the license text used by Wayland was a slightly different and older version of the MIT license, and the license text was updated to the current version used by the X.Org project (known as MIT Expat License).^[3]

Wayland works with all Mesa-compatible drivers with DRI2 support^[87] as well as Android drivers via the Hybris project.^{[108][109][110]}

The developers of Wayland are largely current X.Org Server developers.^[12]

Releases

See also

• Mir (software)
• X Window System

References

External links

• Official website
• Wayland at Open Hub
• Høgsberg, Kristian. "Wayland at FOSDEM 2012". 
• Brockmeier, Joe 'Zonker' (17 November 2010). "The way to Wayland: Preparing for life After X". LWN. 
• Hillesley, Richard (13 February 2012). "Wayland – Beyond X". The H. 
• Griffith, Eric (7 June 2013). "The Wayland Situation: Facts About X vs. Wayland". Phoronix. 
• Wayland on DragonflyBSD

Free and open-source software projects hosted by freedesktop.org
OS components	Avahi Compiz D-Bus DRI Enchant flatpak Kmscon luit nouveau PackageKit Plymouth PulseAudio systemd Wayland X.Org Server Glamor Xephyr
Libraries	AppStream cairo Fontconfig GStreamer Mesa 3D fprint HarfBuzz Pango Poppler Swfdec Video Acceleration API XCB Xft
Frameworks	CppUnit Galago GeoClue Open Collaboration Services Telepathy
Meetings	Create Project Portland Project