This is excellent. Too often people guess at things when they could be more empirical about them. Ever since I learned the scientific method (I think 3rd or 4th grade) I was all about the 'let's design an experiment' :-).
Let me state up front that I have no idea why Wayland would have this additional latency. That said, having been active in the computer community at the 'birth' of X11 (yes I'm that old) I can tell you that there was, especially early on, a constant whine about screen latency. Whether it was cursor response or xterm scrolling. When "workstations" became a thing, they sometimes had explicit display hardware for just the mouse because that would cut out the latency of rendering the mouse in the frame. (not to mention the infamous XOR patent[1])
As a result of all this whinging, the code paths that were between keyboard/mouse input and their effect on the screen, were constantly being evaluated for ways to "speed them up and reduce latency." Wayland, being something relatively "new" compared to X11, has not had this level of scrutiny for as long. I'm looking forward to folks fixing it though.
Display devices (usually part of your GPU) still have "explicit display hardware just for the mouse" in form of cursor planes. Later this has been generalized as overlay planes.
Planes can be updated and repositioned without redrawing the rest of the screen (the regular screen image is on the primary plane), so moving the cursor is just a case of committing the new plane position.
The input latency introduced by GNOME's Mutter (the Wayland server used here) is likely simply a matter of their input sampling and commit timing strategy. Different servers have different strategies and priorities there, which can be good and bad.
Wayland, which is a protocol, is not involved in the process of positioning regular cursors, so this is entirely display server internals and optimization. What happens on the protocol level is allowing clients to set the cursor image, and telling clients where the cursor is.
Protocols can bake in unfortunate performance implications simply by virtue of defining an interface that doesn't fit the shape needed for good performance. Furthermore, this tends to happen "by default" unless there is a strong voice for performance in the design process.
Hopefully this general concern doesn't apply to Wayland and the "shape" you have described doesn't sound bad, but the devil is in the details.
I don't think the Wayland protocol is actually involved in this. Wayland describes how clients communicate with the compositor. Neither the cursor, nor the mouse are a client, so no where in the path between moving the mouse and the cursor moving on screen is Wayland actually involved.
The story is different for applications like games that hide the system cursor to display their own. In those cases, the client needs to receive mouse events from the compositor, then redraw the surface appropriately, all of which does go through Wayland.
According to Asahi Lina, X does async ioctl that can update the cursor even during the scanout of the current frame, while Wayland does atomic, synced updates on everything, cursor involved, which has the benefit of no tearing and the cursor's state is always in sync with the content, but it does add an average of 1 more frame latency (either updates just in time for the next frame), or it will go to the next frame.
This is not what Wayland does, it is what a particular display server with Wayland support decided to do.
Second, just to be clear, this only discusses mouse cursors on the desktop - not the content of windows, and in particular not games even if they have cursors. Just the white cursor you browse the Web with.
Anyway, what you refer to is the legacy drm interface that was replaced by the atomic one. The legacy interface is very broken and does not expose new hardware features, but it did indeed handle cursors as its own magical entity.
The atomic API does support tearing updates, but cursor updates are currently rejected in that path as drivers are not ready for that, and at the same time, current consensus is that tearing is toggled on when a particular fullscreen game demands it, and games composite any cursors in their own render pass so they're unaffected. Drivers will probably support this eventually, but it's not meant to be a general solution.
The legacy API could let some hardware swap the cursor position mid-scanout, possibly tearing the cursor, but just because the call is made mid-scanout does not mean that the driver or hardware would do it.
> but it does add an average of 1 more frame latency
If you commit just in time (display servers aim to commit as late as possible), then the delay between the commit and a tearing update made just before the pixels were pushed is dependent on the cursor position - if the cursor is at the first line shown, it makes no difference, if on the last shown, it'll be almost a frame newer.
Averaging cursor positions mean half a frame of extra latency, but with a steady sampling rate instead of rolling shutter.
Proper commit timing is usually the proper solution, and more importantly helps every other aspect of content delivery as well.
> This is not what Wayland does, it is what a particular display server with Wayland support decided to do.
To the user that's an irrelevant distinction.
I also don't think this matters that much - with X11 this was optimized in one place by people that care about such details while with Wayland now every compositor developer (who in general are much more interested in window management policty) needs to become a low leve performance expert.
> Second, just to be clear, this only discusses mouse cursors on the desktop - not the content of windows, and in particular not games even if they have cursors.
Games can and sometimes do use "hardware" cursors as well - after all, they also care about latency.
Sure, it's what Gnome Wayland does, but the Wayland protocol does sort of mandate that every frame should be perfect, and the cursor has to match the underlying content, e.g. if it moves over a text it has to change to denote that it is selectable.
> Anyway, what you refer to is the legacy drm interface that was replaced by the atomic one. The legacy interface is very broken and does not expose new hardware features, but it did indeed handle cursors as its own magical entity.
Isn't it what many people refer to as "hardware cursor"? Is it possible for Wayland to rely on such a feature?
Wayland display servers will already be using what is commonly referred to as hardware cursors.
They just use the atomic API to move a cursor or overlay plane, which reflect how the hardware handles things. That the legacy API exposed a specialized cursor API was just a quirk of the design.
Note that planes are a power optimization more than anything else, as it allows e.g. the cursor to move or for decoded video frames to be displayed while GPU's render-related units are powered down. Drawing the cursor move, even though the render task is a rounding error, would require the render-related units to be on.
Thank you. So, if I get this right, the cursor position, which is what the video card needs to position the mouse pointer picture on the screen as an overlay to the actual framebuffer, isn't updated asynchronously to the screen update (ie. whenever the mouse is moved), but instead each time a frame is being rendered, and thus the pointer is only moved at these times, which may avoid tearing (though I don't see why) and other nasty effects, yet introduces a small rendering lag.
I don't know however if the mouse pointer picture is still handled the VESA way, or if GPUs video cards nowadays have a more generic API, or what.
There really isn't such a thing as "the actual framebuffer". Instead the display hardware can do composition during scanout from a set of buffers at a set of positions with varying capabilities. These buffers then just being arbitrary dmabufs.
It doesn't give a damn if you give it 2 buffers and one contains a mouse cursor and the other everything else or if you give it 2 buffers and one is everything including the mouse and the other is a video, allowing complete power collapse of the GPU rendering units.
Often they support more than 2 of these as well, and with color conversions, 1D & 3D LUTs, and a handful of other useful properties. Mobile SoCs in particular, like your typical mid/high end snapdragon, actually have upwards of a dozen overlay planes. This is how Android manages to almost never hit GPU composition at all.
On desktop linux all of these go through the drm/kms APIs.
Well, GPUs from the big players do give a damn as they tend to have surprisingly limited plane count and capabilities. It is often just a single primary, cursor and overlay plane, sometimes the latter is shared across all outputs, and sometimes what the plane can do depends on what the plane it overlaps with is doing.
Mobile chips are as you mention far ahead in this space, with some having outright arbitrary plane counts.
Even though it's only 3 planes, they are relatively feature-rich still. In a typical desktop UI that would indeed be primary, cursor, and video planes. But if the system cursor is hidden, such as in a game, that frees up a plane that can be used for something else - such as the aforementioned game.
What you are showing is just a standard color pipeline, which is the bare minimum for color management.
On AMD in particular, the cursor plane must match several aspects of any plane it overlaps with, including transform and color pipeline IIRC.
The AMD SoC in my laptop (much newer than the steam deck) only exposes two overlay planes to share among all 4 display controllers. Intel used to have a single overlay plane per display.
The Raspberry Pi 5 on the other hand intentionally limited the exposed overlay planes to "just" 48, as it can be as many as you have memory for.
> which may avoid tearing (though I don't see why)
What I meant here is that I didn't see why asynchronous updates may introduce tearing; but my tired brain couldn't quite formulate it properly. And to answer that, it's clear to me know that an update of the pointer position while the pointer sprite is being drawn would introduce a shift somewhere within that sprite, which is, I suppose, the tearing discussed (and not a whole frame tearing).
> I don't know however if the mouse pointer picture is still handled the VESA way, or if GPUs video cards nowadays have a more generic API, or what.
Also, the VESA interface doesn't seem to handle mouse pointers, it's something that was available in the VGA BIOS, to provide a uniform support for this feature, as each vendor most likely did it their own way.
It seems like it should be possible to do the X async method without tearing.
When updating the cursor position, check if line being output overlaps with the cursor. If it isn't, it's safe to update the hardware cursor immediately, without tearing. Otherwise, defer updating the cursor until later (vblank would work) to avoid tearing.
Of course, this assumes it's possible to read what row of the frame buffer is being displayed. I think most hardware would support it, but I could see driver support being poorly tested, or possibly even missing entirely from Linux's video APIs.
This would have to be done by the kernel driver for you GPU. I kind of doubt that it's possible (you're not really scanning out lines anymore with things like Display Stream Compression, partial panel self refresh and weird buffer formats), and doubt even more that kernel devs would consider it worth the maintenance burden...
I mean at some point it's a fundamental choice though, right? You can either have sync problems or lag problems and there's a threshold past which improving one makes the other worse. (This is true in audio, at least, and while I don't know video that well I can't see why it would be different.)
Well there are opportunities to do the wrong thing though, like sending an event to the client every time it get an update. Which means that high poll rate mice would DDOS less efficient clients. This used to be a problem in Mutter, but that particular issue was fixed.
Yeah, Wayland isn't designed in such a way that would require any additional latency on cursor updates. The Wayland protocols almost entirely regard how applications talk to the compositor, and don't really specify how the compositor handles input or output directly. So the pipeline from mouse input coming from evdev devices and then eventually going to DRM planes doesn't actually involve Wayland.
You're kind of getting tripped up on terminology. The OP didn't measure Wayland; they measured GNOME Shell which does take responsibility for its performance. Also, I'm not aware of any latency-related mistakes in Wayland/Weston (given its goal of tear-free compositing).
> You're kind of getting tripped up on terminology.
I'm not. My comment doesn't address the latency of gnome shell. I understand the boring technical distinctions between wayland and wayland client libraries and wayland display servers and gnome shell and mutter and sway, blah blah blah. Much like I understand that Linux is a kernel. That it is inspired by UNIX, but it is technically not a UNIX. I also understand that if someone describes themselves as a Linux user they probably don't just mean that they have a Android phone or that the display controller in their dishwasher or wireless access point happens to include Linux the kernel.
The "well acksually wayland is just the name of the protocol" that emerges whenever a problem is brought up is a symptom of the underlying problem with wayland the system. The confusion that gives rise to these deflections is also a symptom of that problem.
By Conway's law systems end up resembling the organisations that produce them. In this way the design of wayland the system seems to be designed by people who don't want to work together. I can see a parallel with microservice architecture.
The distinction between protocol and implementation IS significant here.
Imagine comparing HTTP1.1 vs HTTP3. These are protocols, but in practice one compares implementations. I can pick curl for http1.1, but Python for http3, and http3 would very likely measures as slower.
Which brings up the other problem that Wayland introduced, that instead of one incredibly old inscrutable software stack doing these things there are now five (and counting!) new and insufficiently tested software stacks doing these things in slightly different ways.
It would have been nice if KDE and Valve could (would?) work together to reimplement KWin's features on top of wlroots. That would have basically made Gnome the sole holdout, and I imagine they'd eventually have switched to the extended wlroots as well, or at least forked it.
All this proves is that it's possible for a protocol to not be the determining factor; which says nothing about whether it's possible that it _is_ a determining factor.
You’re quite right. We’d need similar benchmarks done with other compositors.
I very much doubt that Wayland makes a difference for this test; Wayland is for IPC between the client and server. Moving the cursor around is done by the server, without needing to talk to the client.
> I understand the boring technical distinctions between wayland and wayland client libraries and wayland display servers and gnome shell and mutter and sway
I do not. Does anyone know a good but quick introduction into these concepts and the problems they cause or fix?
All I know is that Wayland was supposed to be faster by taking some slow parts out of the loop, but that doesn't seem to be working, according to these figures.
I would just rather pay someone 100 bucks and not care whose fault it is. (And yes, that's why I don't use Linux on the desktop anymore, as much as I loved i3 and tiling environments.)
It's probably in a hundred places that all add up, all of which are no particular person's responsibility. So, that probably means that 10 people from 10 different projects need to get on a call or mailing list together and find a plan of attack.
But it won't happen. And people will keep wondering why Linux can't ever get a foothold on the desktop.
Your biggest strength is also your biggest weakness.
The organisation of Wayland sounds great, but it is very hard to share optimised code between compositors since key parts that affect performance (in this case latency) are largely developed outside of any shared library code.
The "organisation" of Wayland reminds me of the UNIX wars; this is going to get worse before it gets better.
SVR4 Wayland anyone?
xref the time it has taken the Rust rewrite of the GNU coreutils and arguably coreutils is a much easier problem.
The difference is that x11 was SK bag that you required a tons of plugins to have a working system(xorg) and no one could reliable maintain that mess of codes and plugins where every change could break another
The extension situation is already much worse in Wayland than in X11. In my X11 server there are about 25 extensions and only a handful are required for modern desktop functionality. Wayland already has well over a hundred extensions (protocols) and dozens of them have to be implemented by each compositor just to get basic stuff working.
Could've fooled me! I've only ever seen Linux devices run Xorg. Maybe some embedded hardware runs an alternative, but Xorg seems to be the de facto standard for desktop Linux.
This isn't a big issue because Xorg splits a basic GUI into dozens of different daemons and services (compositors, window managers, input daemons) so it seems like there are different X11 implementations even when there's usually only Xorg.
Give Wayland a bit more time and it will also ossify to the point where it becomes unviable to create competing implementations. Most of the smaller compositors are already there.
Is it a weakness of the web that HTTP is just a protocol specification?
The "problem" with this in Wayland is that before people 'ran Xorg with GNOME on top", now they just run GNOME the same way they run Chrome or Firefox to use HTTP - it will take time for people to get used to this.
I think the answer is also Wayland. It's very confusing because there is wayland the protocol and wayland the system, where the protocol is a part of the system - but there is no separate name for it. When people discuss wayland they are very rarely talking about wayland the protocol, and are much more likely talking about wayland the system.
It's not an "argument", it's just a description of what Wayland is. But no, the correct protocol has had copy-paste since day one, and I dont remember there being issues with screensavers.
In the metaphor of a web server and a web browser, Wayland would be the HTTP specification. What you're usually interested in is what server you're running, e.g. GNOME's Mutter, KDE's Kwin, sway, niri, or what client you're running, e.g. Gtk4, Qt6, etc.
Wayland don't support screensaver and clipboard is managed by portals no?, idk why the complain tho, no one uses screensaver and clipboard being managed by a portal dsounds more logical than creating a protocol for that
Once upon a time XFree86 and Xorg updated the pointer directly in a SIGIO handler. But that's ancient history at this point, and nowadays I wouldn't expect Wayland and Xorg to have a hugely different situation in this area.
IIRC it all started going downhill in Xorg when glamour appeared. After the cursor rendering path wasn't async-safe for execution from the signal handler (which something opengl-backed certainly wouldn't be), the latency was worse.
I remember when even if your Linux box was thrashing your mouse pointer would stay responsive, and that was a reliable indicator of if the kernel was hung or not. If the pointer prematurely became unresponsive, it was because you were on an IDE/PATA host and needed to enable unmask irq w/hdparm. An unresponsive pointer in XFree86 was that useful of a signal that something was wrong or misconfigured... ah, the good old days.
> they sometimes had explicit display hardware for just the mouse because that would cut out the latency
That's still the case. The mouse cursor is rendered with a sprite/overlay independent from the window composer swapchain, otherwise the lag would be very noticeable (at 60Hz at least). The trickery starts when dragging windows or icons around, because that makes the swapchain lag visible. Some window composers don't care any longer because a high refresh rate makes the lag not as visible (eg macOS) others (eg Windows I think) switch to a software mouse cursor while dragging.
I'll be reading a dream of spring in my grave at this rate.
I understand I'm complaining about free things, but this is a forced change for the worse for so long. Wayland adoption should have been predicated on a near universal superiority in all input and display requirements.
Intel and AMD and Nvidia and Arm makers should be all in on a viable desktop Linux as a consortium. Governments should be doing the same because a secure Linux desktop is actually possible. It is the fastest path to showcasing their CPUs and 3d bling, advanced vector /computer hardware.
Wayland simply came at a time to further the delay of the Linux desktop, at a time when Windows was attempting to kill Windows with its horrid tiles and Apple staunchly refused a half billion in extra market cap by offering osx on general x86.
Wayland is a protocol. The problems people complain about are generally implementation details specific to GNOME or KDE or (in general) one particular implementation.
There's rarely any such thing as "universal superiority", usually you're making a tradeoff. In the case of X vs Wayland it's usually latency vs. tearing. Personally I'm happy with Wayland because there was a time when watching certain videos with certain media players on Linux was incredibly painful because of how blatant and obtrusive the tearing was. Watching the same video under Wayland worked fine.
Early automobiles didn't have "universal superiority" to horses, but that wasn't an inhibitor to adoption.
"Wayland is a protocol" is exactly the problem. Protocols suck; they just mean multiplying the possibility of bugs. A standard implementation is far more valuable any day.
With X11, it was simple: everybody used Xfree86 (or eventually the Xorg fork, but forks are not reimplementations) and libX11 (later libxcb was shimmed underneath with careful planning). The WM was bespoke, but it was small, nonintrusive, and out of the critical path, so bugs in it were neither numerous nor disastrous.
But today, with Wayland, there is no plan. And there is no limit to the bugs, which must get patched time and time again every time they are implemented.
X had garbage handling of multiple monitors and especially multiple monitors with different DPIs, and there was "no plan" to deal with that either. Nobody wanted to work on the X codebase anymore. The architecture bore no resemblance to the way any other part of the desktop stack (or hardware) works.
Most of the garbage aspect is because toolkits refuse to support multiple monitors on DPI with X11 with the argument that "Wayland is just around the corner", for decades now.
For example Qt does per-monitor DPI just fine on X11; it's just that the way to specify/override the DPI values just sucks (an environment variable).
This stupid decision is going to chase us until the end of times since Xwayland will have no standardized way to tell its clients about per-display DPI.
X could do seveal different screens I did have this working once. However then moving an application to a different display was impossible (an app could do it but it was a lot of work so nobody bothered). I few cad programs supported two streens but they were seperate and the two didn't meet.
Most people want to drag windown between screens and sometimes even split down the middle. One large display supports that much easier so that is what everyone switched to in the late 1990
I was using it that way until about 2020. (Mint 13 MATE supported, but it seems that capability was lost somewhere along the line. Shame, because I have a dual monitor setup where the second monitor is often displaying the picture from a different device, so in that situation I absolutely cannot have applications deciding to open on the busy-elsewhere screen. I miss being able to set a movie running on one monitor and have it not disappear if I flipped virtual desktops on the other!)
Yes, separate screens would be a much better model for me as well. Much better than KDE randomly deciding to show KRunner on the turned off TV for some reason unless I manually disable the output.
X11 does a lot of things that are outdated now, and multiple independent screens is one of them. Ideally, you'd be able to select either independent screens or one big virtual screen, and still have a window manager be able to move windows between independent screens. I don't know how that would be achieved though.
X's "mechanism, not policy" has proven to be a failure. Without a consistent policy you end up with a pile of things that don't work together. "One big virtual screen" is a policy and it's one that works well enough.
Deciding what needs to be in the protocol and what should be in the application is never easy. It's important to be able to iterate quickly and avoid locking in bad practices. I think that's what Wayland tried to do by making everything a fine-grained extension, but it doesn't really work like that as the extensions become mandatory in practice.
For the record: you can specify a different DPI _for each monitor_ for Qt X11. You just cannot change it after the program has started, which is exactly the limitation I was referring to.
But you can definitely move windows to another monitor and Qt will use the right DPI for it. It is the same behavior as Wayland. "One large wide screen display" is exactly how Wayland works...
X11 used to provide separate displays, but at some point due to hardware changes (and quite probably due to prominence of intel hardware, actually) it was changed to merged framebuffer with virtual cut out displays.
In a way, Wayland in this case developed a solution for issue its creators brought into this world first
It can still provide seperate displays. The problem is you couldn't do something like drag a window from display 1 to 2°. IIRC it's also annoying to launch two instances of a program on both displays. The hacky merged framebuffer thing is a workaround to these problems. But you can have independent DPIs on each display.
Yeah there were certainly tradeoffs. It's much harder to use separate displays now, though - last time I tried, I could address the two displays individually (":0.0" and ":0.1") if I launched X on its own, but something (maybe the display manager?) was merging them into a single virtual display (":0") as soon as I tried to use an actual desktop environment. (This was was Mint 20, MATE edition, a few years ago - I gave up and reverted to a single-monitor setup at that point.)
Yes. It just proves that all you needed is a better way to specify the per-monitor DPI, one that can be updated afterwards, or even set by the WM on windows.
That, i think, is the main issue. Nobody wants to work with GTK1, or GTK2, or GTK3 anymore. Nobody wants to work with QT1, or QT2, or QT3 or QT4 anymore. Everybody wants the new shiny toy. Over and over again.
It is CADT all over.
Earlier X was developed by an industry consortium. Now Wayland is a monopoly pushed by RedHat.
Exactly, same situation as with audio. Instead of improving the OSS API we got ALSA, PulseAudio, and now PipeWire (and a bunch of less mainstream alternatives) -- all with their own weird issues. And of course it's application (or at least toolkit) developers that bear the cost of this churn, not the developers that get to work on their fancy new NIH tech that they'll then abandon before being 100% stable for yet another reimplementation.
Software has bugs and water is wet. Wait til you hear about HTTP, TCP, UDP, IP torrents, etc... and "simple" is not really a term I would designate to X11. I mean, its fine, but even just the ecosystem surrounding X is convoluted, outdated and absurd. Things like xinit, startx, .Xauthority, xresources, xhost etc... are all a mess.
> Wayland is a protocol. The problems people complain about are generally implementation details specific to GNOME or KDE or (in general) one particular implementation.
I feel like at some point this is a cop-out. Wayland is a protocol but its also a "system" involving many components. If the product as a whole doesn't work well then its still a failure regardless of which component's fault it is.
Its a little like responding to someone saying we haven't reached the year of linux on the desktop by saying: well actually linux is just the kernel and its been ready for the desktop for ages. Technically true but also missing the point.
X is fine, most of the problems people bring up are niche and minor. Meanwhile, Wayland induces problems of its own, like breaking all sorts of accessibility systems and multi-window X applications with no solution in sight.
To be fair this wayland issue is also niche. Author of the linked article wrote that this is reported by really small percentage of users most of the people do not notice it.
I believe you. Can you share an example? To be clear, I'm pretty sure this can be done with Gtk and Qt, but maybe you are talking about older apps written directly in Xlib or Xt?
It's not the toolkit (as they work on X11 and other OSes with the existing toolkits), it's Wayland that's the issue (there are a series of Wayland protocols to implement this, but they are being blocked).
The key detail in the “Wayland is a protocol” is that there are several other implementations, some of them extremely mature. The implementation being tested here isn’t exactly know to be a good one.
If there were a single Wayland implementation in existence, I’d agree with your sentiment.
What components? Wayland is literally an XML protocol that turns an XML file into a method of communication. libwayland-server and libwayland-client only handle communication and the internal event loop. Its completely up to the developer to write the implementations of these functions and register them in the server.
Then a client is going to query the server and ask request to do stuff via a unix socket. In fact, you don't even need libwayland, you can raw dog it over sockets manually. The idea is that there are standard protocols that can be queried and used,and you can implement this in any environment you want to. You could write the "frontend" in html and JS and run a wayland compositor on the web (which has been done [1]), you could do it with text or anything really, most people use some graphics stack.
- the compositor, of which there are multiple implementations (gnome, kde, all the wlroots compositors)
- the client, which often uses one of several toolkits (gtk, qt, several smaller frameworks, or even directly using the protocol)
- the wayland protocol (or rather protocols, because there are several extensions) itself
- other specifications and side channels for communication, in particular dbus.
Many issues (although I don't think the one in OP) are due to the protocol being underspecified, and so the client and compositor disagree about some semantics, or doesn't even have a standard way to accomplish something across all compositors.
Nit: Wayland isn't an XML protocol. The "calls" and their arguments are described in XML, but the data is transmitted in a fairly simple binary encoding.
I mean most of the things are the fault of a badly designed or non existent protocols:
-Problems with non western input systems
-Accessibility
-Remote control(took around 2 years to be stable I think?)
-Bad color management
Then there's the things that did work in x11 but not in wayland:
-Bad support for keymapping(the input library says keymapping should be implemented by the compositor, gnome says not in scope, so we have a regression)
-bad nvidia support for the first two years? three years?
While these things are compositor/hw vendor faults, the rush to use wayland and nearly every distro making it as default, forced major regressions and wayland kinda promised to improve the x11 based experience.
Yes, and to the parents point it was a CHANGE in protocol.
I get there was cruft in x. But the moment selected was a barrier to Linux desktop adoption precisely when the greatest opportunity in decades was present.
And the desktop was reimplemented.
Now in this period kde and gnome both decided to do rewrites, Ubuntu did their own desktop, got it up to snuff, and abandoned it. The lunacy wasn't just Wayland.
If we are complaining the gnome compositor sucks... I mean , should that be the goddamn reference implementation? What percent of desktops are gnome, 80% at least? If the gnome composting ready for primetime, then Wayland isn't ready for primetime.
>If the gnome composting ready for primetime, then Wayland isn't ready for primetime.
I use Sway, which uses a different compos[i]tor than Gnome. I would like to see similar results for wlroots, Sway's compositor, though I'm not actually interested enough to do the experiment (I guess that would be comparing Sway with i3). Cursor lag in Sway is not enough to bother me. I have on occasion used Gnome on the same machine(s), and never been bothered by lag.
As others have pointed out, Wayland is a protocol, not a compositor.
I have no idea of the difference(s) in performance between wlroots and Gnome's compositor. Protocols do not have performance, implementations do. If someone can prove that the Wayland protocol, on a certain set of tasks, and in a certain environment, has a better or worse performance than the X11 protocol, then it might be possible to make abstract comparisons on protocol.
Given that Gnome runs on both X and Wayland, it might be interesting to hear from the Gnome authors on the performance differences.
Nvidia support is still poor (at least on latest cards), I'm forced to use X or I get tons of glitches.
I need the proprietary drivers for machine learning.
Not that I mind particularly, X is fine and everything works as expected.
I can even have the integrated (amd gpu) take care of the desktop while the Nvidia gpu does machine learning (I need all the vram I can get, the desktop alone would be 100-150mb of vram) - then I start a game on steam and the nvidia gpu gets used.
Funnily enough I had wayland enabled by default when I installed the system and I didn't understand why I was getting random freeze and artifacts for weeks. Then I realized I was not using X11
I've used Wayland on Fedora for 16 years. Had to disable Nvidia GPU on laptop And use Intel GPU for first 11 years. Now have been using AMD GPU smoothly for 5 years.
But I hear Nvidia is slowly getting back on the game. Let's hope they get stable support less than 20 years of wayland being in use on major desktop Linux distros.
> the rush to use wayland and nearly every distro making it as default,
Which rush? It has been done by only a small fraction of distros like Fedora, after years of development of the first wayland compositors. Fedora main purpose has always been to implement bleeding edge tech stuff early so that bugs get found and fixed before people using more stable distros have to suffer from it.
Nobody has been forced in any regression and x11 has continued to be available until now and there is no sign that the most conservative distros will drop x11 support anytime soon.
I think that gnome has had built-in IME, but at least for a long time, it wasn't possible to use a third party system with gnome, or use gnome's with other compositors. And I'm pretty sure the situation was the same for sreen readers and on-screen keyboards. The wlroots project created their own protocols to support external applications to provide such features, since that is out of scope for a compositor like sway, but there are still missing pieces.
> there was a time when watching certain videos with certain media players on Linux was incredibly painful because of how blatant and obtrusive the tearing was
It was because of the crap LCD monitors (5 to 20 ms GtG) and how they are driven. The problem persists today. The (Wayland) solution was to render and display a complete frame at a time without taking into account the timings involved in hardware (you always have a good static image, but you have to wait).
I tried Tails (comes with some Wayland compositor) on a laptop. The GUI performance was terrible with only a Tor browser open and one tab.
If you do not care about hardware, you will, sooner or later, run into problems. Not everybody has your shiny 240 Hz monitor.
> Wayland adoption should have been predicated on a near universal superiority in all input and display requirements.
Totally agree. The people saying "Wayland is a protocol" miss the point. Wayland is a protocol, but Wayland adoption means implementing stuff that uses that protocol, and then pushing it onto users.
Measure twice, cut once. Look before you leap. All that kind of thing. Get it working FIRST, then release it.
You have to releaseethings like this in parts because it needs too many external people to do things to make it useful. Managing those parts is something nobody has figured out and so people live you end up using it before it is ready for your use and then complaining.
When I say "release" I mean "release to users". You can release stuff to other developers, no problem. But it should all come with large warnings saying "This is not for daily use". The failure is when distros like Fedora start not only shipping it, but then saying they're going to drop the working alternative before Wayland is actually ready to do everything that X does.
(Also, I don't use Wayland. I mean I tried it out but don't see any real benefit so I don't use it regularly.)
This would be best solved by maintaining support for old protocols instead of reinventing the wheel. There is no chicken/egg problem when you have proper backwards compatibility.
I don’t really agree, because I think there’s something of a chicken-and-egg effect here.
With complex projects like these that have to work well on such a wide array of hardware and configurations, lots of real world usage is required to achieve any level of refinement. Without widespread adoption, Wayland likely would have been stuck in experimental/toy status for much longer than it will as things are going currently.
This argument don't make sense because Wayland started as a hobby and not to replace x11, was after it got traction, other people/companies started contributing that it matter
You're arguing against Wayland, but for a more secure Linux desktop? I recommend you spend more time getting to know the X11 protocol then, because it has plenty of design decisions that simply cannot be secured. The same people who used to develop XFree86 designed Wayland to fix things that could not be fixed in the scope of X11.
> the X11 protocol [...] has plenty of design decisions that simply cannot be secured.
I've been hearing this for over a decade now. I don't get it. Just because xorg currently makes different clients aware of each other and broadcasts keypresses and mouse movements to all clients and allows screen capturing doesn't mean it has to. You could essentially give every application the impression that they are the only thing running.
It might seem difficult to implement, but compare it to the effort that has gone into wayland across the whole ecosystem. Maybe that was the point - motivating people to work on X was too difficult, and the wayland approach manages to diffuse the work out to more people.
I was really bullish on Wayland 10 years ago. Not so much any more. In retrospect it seems like a failure in technical leadership.
It'll be challenging to even figure out which one of the things connecting to $DISPLAY is the real window manager. Good luck on your lonely[1] journey!
[1]: The people who actually developed Xorg are now working on various Wayland-related things.
> It'll be challenging to even figure out which one of the things connecting to $DISPLAY is the real window manager.
I suspect it would be less challenging than writing a whole new wayland server.
Off the top of my head, I'd use a separate abstract domain socket for the window manager including some UUID, and then pass that to the window manager when launching it.
You could create these sockets on demand - one for each security context. On linux typically a different security contexts will either have different UIDs - in which case filesystem permissions would be sufficient - or they have different mount namespaces - in which case you make different sockets visible in different namespaces.
For SSH forwarding you could have SSH ask the X server for a new socket for forwarding purposes - so remote clients can't snoop on local clients.
> Good luck on your lonely[1] journey!
>
> [1]: The people who actually developed Xorg are now working on various Wayland-related things.
This is what I mean by a failure of technical leadership.
"You could create these sockets on demand - one for each security context. On linux typically a different security contexts will either have different UIDs - in which case filesystem permissions would be sufficient - or they have different mount namespaces - in which case you make different sockets visible in different namespaces."
This is reminiscent of how Trusted Solaris[0] implements Mandatory Access Control (MAC) a la Orange Book[1].
> For SSH forwarding you could have SSH ask the X server for a new socket for forwarding purposes - so remote clients can't snoop on local clients.
SSH pretty much already does this. Per default (using -X) X11 forwarding is in untrusted mode, which makes certain unsafe X11 extensions unavailable.
So remote clients already cannot snoop the whole keyboard input.
> Off the top of my head, I'd use a separate abstract domain socket for the window manager including some UUID, and then pass that to the window manager when launching it.
And then the window manager spawns whatever programs the user wants, and they end up sharing that $DISPLAY.
I'm not trying to claim that X is secure as is. The claim I'm making is:
1. There is nothing fundamentally insecure about the design of X that couldn't be fixed with a bit of effort.
2. The effort required would be significantly less than what has gone into Wayland over the last 16 years.
Fixing the security of the system certainly would involve changes to more than just xorg. Anywhere there is a security boundary you'd need to make modifications - but that's OK, most of these security boundaries are younger than wayland anyway.
This discussion of $DISPLAY seems like distracting minutiae. Sure it's a medium size problem that would need solving, but that's all it is.
To address your specific point: it could be the responsibility of the window manager to set $DISPLAY to a less powerful socket when starting processes. Ultimately it doesn't matter though, because if the display server isn't doing some sort of sandboxing then the spawned process can just ptrace xorg and do whatever it wants. X being secure only matters in the presence of a security boundary and in that case it would be the responsibility of whatever is setting up that boundary to create a less privileged socket. Whether that be ssh or flatpak or whatever.
So now we've reached the point of "you could secure X11 if you changed everything". Sure, fine.
Once again, for emphasis: nobody has stepped up to do that, in the history of X (outside of proprietary products, of unknown quality).
Wayland is not some outsider project competing against X11. It's the people who were developing Xorg, saying they're fed up with X11 and need a clean slate to fix its problems.
Here's my unpriviledged user trying to ptrace my Wayland compositor running as me:
Nearby, I have a Chromebook where a potentially hostile virtual machine can display Wayland windows seamlessly on the host. The worst the VM can do is fill bitmaps with offensive imagery.
In a world where the people doing the work switched to developing the Wayland ecosystem, arguing about a "could" that requires changing every window manager seems like a guarantee X11 will not get improved. Feel free to put effort into it...
> I understand I'm complaining about free things, but this is a forced change for the worse for so long.
Then write code.
Asahi Lina has demonstrated that a single person can write the appropriate shims to make things work.
Vulkan being effectively universally available on all the Linux graphics cards means that you have the hardest layer of abstraction to the GPU taken care of.
A single or small number of people could write a layer that sits above Wayland and X11 and does it right. However, no one has.
This article only tests one compositor for Wayland and one for X11. It also doesn't specify if vsync is enabled for either one. Some X11 compositors default to vsync off, whereas every Wayland compositor I've seen defaults to on. This alone would be enough to explain the latency differences measured here.
I like what's being done here but we need a more thorough experiment before we can start drawing conclusions.
On sway, if you use the proprietary Nvidia drivers, it is often necessary to disable hardware cursors. I wonder if there is something similar happening here. Maybe wayland gnome doesn't use hardware cursors?
I'm not sure where this absolute "tearing is bad" and "correctness" mentality comes from. To me, the "correct" thing specifically for the mouse cursor is to minimize input lag. Avoiding tearing is for content. A moving cursor is in various places along the screen anyway, which will include biological "afterglow" in one's eyes and therefore there's going to be some ghosted perception of the cursor anyway. Tearing the cursor just adds another ghost. And at the same time, this ghosting is the reason keeping cursor input lag to a minimum is so important, to keep the brain's hand-eye coordination in sync with the laggy display.
I guess you're asking what happens if the UI element can't be updated in the same cycle?
I'd say: it really doesn't matter at all, in fact it's even more argument to make the pointer as low latency as possible. The mouse pointer is your control element as the controller of the interface. The UI element (and everything else onscreen) is the computer's element. It is an interaction between two entities and there is no need to introduce an expectation that the computer be "zero frame delay" on UI elements. It certainly sucks if it's more than a few frames for it to react, but so be it. However for the mouse cursor itself, it should absolutely be immediate because it is the user's representation on this virtual surface, and as such should follow inputs without being unnecessarily impeded. (Case in point: some UIs intentionally add delay or effects [like a fade-in] to UI element response. It's a… "choice"… but sometimes it works.)
Ultimately, the mouse pointer is an extension of hand-eye coordination; the only thing worse than making it lag (which you can learn to adapt to, albeit it will be very annoying) is to make it lag inconsistently (which you can't adapt to and will grow frustrated with extremely quickly.)
And, unfortunately, from personal experience, the mouse pointer on Wayland gets choppy if system load is high. X11 was significantly better with this; with Wayland the compositor [kwin_wayland to be specific, might be a KDE issue] doesn't even seem to be "unnice" or have some kind of scheduling priority over other tasks — I'm not sure if I'm missing something there but that seems to be a glaring oversight on its own.
We don't even know if it's actually a 1.5 frame latency for real because the test didn't try to establish that. The author said it looked like it was 1.5 frames, but that could be a coincidence. It just looks like a 1.5 frame overhead. It could, in reality, be a constant time overhead rather than a function of frames and so would be fine with a lower frame rate.
Tearing would affect everyone that uses a computer with X11 but your proposed example of a TV with 30i refresh rate would only affect the tiny subset of users that use a CRT television as a monitor, right?
I don't think there was ever a 30i screen. GP probably meant 60 interlaced, which needs two full scans for a change, so an effective 30 full frames per second, but enough for 60 half images. For a Wayland compositor and the entire hardware stack would treat it as 30hz only if it really wanted to avoid interlace artifacts, but I don't recall that being a configuration that existed in my lifetime.
> Let me state up front that I have no idea why Wayland would have this additional latency
It's not really hard to guess at - it's probably caused by abstraction layers resulting in multiple layers of buffering. Of course, it's all just speculation until proven by experiment, so good on TFA for doing some of that.
Retro hardware from the C64 or NES era was single-tasking with such predictable timing you could change the video resolution partway through the screen refresh and have two resolutions on screen at once. If you want, you can check the player input and update Mario's position right before he gets scanned out - the minimum possible latency by design is zero frames. (Of course, in practice the whole screen data is buffered up during vblank, which is the same latency as using a framebuffer. Also the NES didn't allow changing video registers outside of blanking periods. But the C64 did.)
X11 isn't that close to the metal, but it's designed with a single level of buffering (assuming a compositing window manager isn't used). There is a single framebuffer covering the whole screen. Framebuffer updates are completely asynchronous to scanout, so there is around 0.5 frames of latency on average between when a pixel is changed and when it's scanned out. Apps borrow pixels from this big global framebuffer to draw into. An app sends a "draw rectangle" command relative to its own borrowed space, and the server calculates where the rectangle should go on the screen, and draws it into the framebuffer, and then with an average latency of half a frame, that part of the framebuffer is scanned out to the screen.
On Wayland, there are more layers. The app has to draw into its own pixel buffer (noting that it is probably double- or triple-buffered and has to redraw the entire window rather than relying on non-changing stuff to still be there) and then once the compositor receives the buffer from the app, it has to do the same thing again, copying all the per-app buffers into a big screen-size buffer, which is probably also double- or triple-buffered, and once the vblank hits, the most recent full frame is drawn to the screen. It's just more work with more layers, and there's no surprise that latency is higher. It's hardly the first or 100th time that an elegant software architecture resulted in the computer having to do more work for the sake of keeping the architecture pure.
Something important to note is that tearing reduces latency. If new data is available while the scanout is partway through a frame, you have the choice to draw it immediately, causing tearing, or wait until the next frame, increasing latency. Always doing the high-latency thing is an explicit design goal of Wayland. (A third theoretical choice is to always time things precisely so that the new frame is calculated right before the end of vblank, but... in a modular multitasking architecture with apps written by different organizations and unpredictable CPU load, good luck with that.)
Now the really big caveat: I'd be surprised if GNOME's X11 WM wasn't compositing. If it's compositing, then X11 behaves similarly to Wayland but with even more IPC and what I just said is no reason it should have one frame less latency on average. Still something to think about though.
> Wayland, being something relatively "new" compared to X11, has not had this level of scrutiny for as long. I'm looking forward to folks fixing it though.
Part of the problem will doubtless be the USB (and Bluetooth) stacks including the device hardware and firmware. When keyboards and mice were serial devices with their own interrupt making the code path fast was achievable. I'm not so confident that modern peripheral stacks can be made to run with the same priority. It becomes even more challenging for devices sitting on a bus with multiple device classes or multiple protocols between the device and driver (USB -> Bluetooth -> Mouse).
I hope devices can be sped up but we're a long way from a keypress triggering an interrupt and being handled in tens of milliseconds[0].
The article you're commenting on is about someone running X11 and Wayland on the same computer with the same mouse and experiencing higher input latency on Wayland. I don't think differences between serial and USB mice are relevant here.
As a systems I guy I completely agree with this. One of the things that was interesting about X11 is that it draws an API "wall" around the stuff involved in presentation. Many folks don't remember this but we had X "terminals" which was a computer system that did nothing else except render the screen and handle the input.
In that compartmentalization there was nothing else competing for attention. You didn't get USB bus contention because there was just the one mouse, ever, on the line between the user and the X11 server in the X terminal.
For anyone who uses ffmpeg for this type of per frame analysis, `ffmpeg -skip_frame nokey -i file -vsync 0 -frame_pts true out%d.png` will get the "presentation time" of each frame in the video. That's more precise than just dumping frames and calculating timestamps. You can also do something similar in a web browser by playing a <video> and using `requestVideoFrameCallback()`. Though, you might need to set `.playbackRate` to a low value if the computer can't decode all the frames fast enough.
> With my 144Hz screen,....Wayland, on average, has roughly 6.5ms more cursor latency than X11 on my system...Interestingly, the difference is very close to 1 full screen refresh. I don't know whether or not that's a coincidence.
The fact that the latency is almost 1/144th of a second means that it might become 1/60th of a second on standard 60Hz monitors. This is hard to notice consciously without training, but most people can "feel" the difference even if they can't explain it.
> The fact that the latency is almost 1/144th of a second means that it might become 1/60th of a second on standard 60Hz monitors.
My guess: the "true" numbers are close to 2.5 (half a frame of random phase of when the mouse is touched vs. refresh, plus 2 frames to move cursor) and 3.5. If you throw out the low outlier from each set you get pretty close to that.
(of course, the 125Hz mouse poll rate is another confound for many users, but this guy used a 1KHz mouse).
> This is hard to notice consciously without training, but most people can "feel" the difference even if they can't explain it.
Yah. 7ms difference is not bad vs 16.6ms is starting to be a lot.
IMO, we should be putting in effort on computers to reach 1.6 frames of latency -- half a frame of random phase, plus one frame, plus a little bit of processing time.
To have a compositor not introduce a frame of latency more or less requires it to race the beam, which has definitely been suggested[1], but you can see how it’d be difficult, and so far no operating systems have tried as far as I know. And as for good frame pacing support in UI toolkits (and not just game engines), well, one can dream. Until both of these are in place, 2.5±0.5 seems to be the hard limit, the question here is more where the Mutter is losing another frame (which even the greats tell us[2] is not hard to do by accident).
I’ve read these arguments quite a few times and always found them a bit questionable. Sure, if everything is driven by the vblank time (or any other clock that counts in frames), it makes sense. But that’s a silly approach! There is nothing whatsoever special about allocating one full frame interval to the compositor to composite a frame — if it takes 16ms to composite reliably, it will take 16ms to composite reliably at 30Hz or 60Hz or 144Hz. So shouldn’t the system clock itself on a time basis, not a frame basis?
Put another way, a system fast enough to composite at 144Hz should be able to composite at 60Hz while only allocating 1/144 seconds to the compositor, which would require offsetting the presentation times as seen by the compositor’s clients by some fraction of a frame time, which doesn’t actually seem that bad.
It gets even better if variable refresh rate / frame timing works well, because then frames don’t drop even if some fraction of compositing operations are a bit too slow.
I assume I’m missing some reason why this isn’t done.
We have multiple processes on a time-share system. This mouse is updating at 1kHz, it just doesn't make sense consistently rerendering everything when the screen aint updating.
We need to cut some deadline and doing it at vsync is the easiest way
I was pretty shocked by the eyestrain difference I felt going from 30hz to 60hz with a 4K monitor while only doing coding tasks (i.e. text and mouse, no real graphics or animations).
I recently went from 60 to 144, also on a 4k (27") monitor, and to me too it has been a very noticeable upgrade. No tearing of any kind when dragging windows around, and scrolling feels buttery smooth.
It's almost certainly because of one extra frame of buffering between the mouse move and the screen. Vsync can cause this, but it should be possible to get vsync with just double buffering.
Note that the results will differ between compositors, different GPUs, and different configurations. This is somewhat less the case with X11 since there is only one X server implementation (that Linux desktop systems use, anyhow.)
I think there may still be the issue that many compositor/GPU combinations don't get hardware cursor planes, which would definitely cause a latency discrepancy like this.
TIL Wayland is 16 years old already; in a few years (I can't math) it'll be as old as X was when Wayland came out, but it seems it's still considered mediocre or not as good as X was.
Initial release of X was June 1984, and Wayland was first released in 2008 -- so it won't be until 2032 that Wayland is the same age. When people complain about Wayland being mediocre or "as good as X was" what they often mean is "Wayland doesn't support $very_specific_feature" or "my video card vendor's proprietary drivers aren't well-tested with Wayland and it makes me sad".
Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
I don't really have a stake in this argument, except I'm bone-tired of seeing people whine about Wayland when they're using software somebody else gave them for free. There are enough Wayland whiners that, by now, they could've banded together and started maintaining / improving X and (if their complaints and theories were correct) left Wayland behind.
Strangely -- even though X is open source and eminently forkable (we know this, because XFree86 -> X.org) it gathers dust and none of its proponents are doing anything towards its upkeep.
When someone shows up with "Wayland isn't as good as X, so here is my modernized fork of X anyone can use" -- I'll be quite interested.
> Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and
Are you saying that Wayland is only for developers? Are people not allowed to complain when The New Thing turns out to be less good than the supposedly Obsolete Thing?
> 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
I'm fully aware of that. I understand X11 has its limitations, and some of the goals of Wayland sound very appropriate for the modern PC environment, but if after 16 years, there are still many situations where Wayland does worse than X, that's not a great sign, and it will make people continue to use X.
I enjoy reading the whinging around Wayland by people who feel so entitled and so outraged. Every time they never fail to remind me of a previous coworker who was always using the same tactics: « this breaks my existing workflow, I will not adapt, I am all for trying new things but only if they are very exactly equal to what I already know and then what's the point, this is not how I have always worked the past ten years and I will not bulge ».
Not talking about the technical points of the whole X/Wayland considerations here, but a group of people in such debate is always as vocal as they'll do nothing and put unreasonably high expectations on other people to listen to them and have no choice but to agree. The fact that X.org's core team initiated Wayland and stopped X.org development is disregarded: these people Will Be Right against literally everything, they can't be reasoned.
My previous coworker was fired for gross incompetence in the end yet he never acknowledged anything wrong. He always knew better than literally the whole world that bash scripts run manually where superior to Terraform for the whole company's infrastructure. Yes, he was that stupid. I guess this is why he ended lying like hell on his LinkedIn profile, concealing he had had four employers in four years.
Seeing technical debates with such poor quality is worrying. If people with quantitative minds fail to entertain a fact-based discussion, how can we expect our political life to acknowledge there is only one reality and stop lying?
Before you voice your outrage, here are some facts that won't dis-exist just because you disagree with them:
> what they often mean is "Wayland doesn't support $very_specific_feature"
My primary complaint with Wayland is that it is a textbook example of Brooks’s second-system effect. It forsook backwards compatibility (yes, there’s an X11 server but to my knowledge there is no way to just run Wayland and one’s X11 desktop environment).
> Strangely -- even though X is open source and eminently forkable (we know this, because XFree86 -> X.org) it gathers dust and none of its proponents are doing anything towards its upkeep.
I suspect that is because the X/Wayland guys have sucked all the oxygen out of that particular room. A newbie shows up and is told that X.org is legacy and he shouldn’t work on it, so … he doesn’t.
And of course X.org really is a bit of a disaster due to being written in C.
The core Wayland and X.org don't exist in a vacuum. I have written a lot of code over the years that works only on X, as have many others. I have not directly contributed to the X.org server, but have to the "wider X ecosystem". I will have to rewrite some of that. This is the case for many people.
"I have not directly contributed to the X.org server, but have to the "wider X ecosystem". I will have to rewrite some of that. This is the case for many people."
This is a fair point, but the quality of Wayland isn't really at issue for this -- Wayland could be much better than X by all accounts and it would still require you to rewrite software for it. (Assuming XWayland doesn't suit your needs, anyway.)
Strongly disagree. For operating system components backwards compatibility should always be a primary and non-negotiable goal. If the backwards compatibility isn't 100% transparent that's a quality issue.
> it would still require you to rewrite software for it
As I understand, most complex apps in the 2020s are using something like Gtk or Qt. These already have Wayland rendering backends. (I assume other widget toolkits have done the same.) Unless your GUI directly uses Xt or Xlib, I don't think that any rewrite is required. Do I misunderstand your point?
A good part of why so may of us are cautious about Wayland is precisely that one or more of the things we rely on doesn’t support it. For example, my window manager doesn’t. I use that window manager every single day, and there is nothing out there which works the way it does. It’s a vital part of my workflow.
Does Wayland support the entire Xcompose system or an equally powerful alternative?
Wayland does genuinely sound interesting, if flaky and half-baked (it reminds me of systemd in that way). Certainly, the world doesn’t owe me a port of my window manager. But if the Wayland transition forces me to use GNOME then my life will be worse.
> if flaky and half-baked (it reminds me of systemd in that way)
In 2025, is systemd still considered "flaky and half-baked"? As I understand, the "init.d war" was lost; systemd is the clear winner on Linux and continues to improve. That said, lots of good developers died on that hill.
> There are enough Wayland whiners that, by now, they could've banded together and started maintaining / improving X and (if their complaints and theories were correct) left Wayland behind.
They already have. X is already more full-featured and stable than Wayland (yes it is missing certain niche features that Wayland has). Sometimes the most important thing you can do with a piece of software is not screw around with it.
And yet, X is as flawed as it can be. There's a problem with global shortcuts, that Xorg only fires an event on keyup/key release, rather than on first match on keypress. That is a protocol limitation, and fixing it means breaking a bunch of stuff. The complains about wayland are of that nature, but at least they are fixable.
Are they fixable? One of my beefs with Wayland is that every compositor handles things differently. How GNOME's does some setting is different from how KDE's is different from Sway's. There's no guarantee they've even implemented the setting either. In X11 you can always globally configure something for any WM the same way. Wayland by design you can't. It's not fixable. I think it was a really poor decision to do a protocol and just made an OS where things are often too fragmented even more fragmented.
> The complains about wayland are of that nature, but at least they are fixable.
Eh maybe. Ultimately you can never be sure something is possible without doing it.
I'm pragmatic here, if and when Wayland offers me a better experience than X then I'll use it. I just resent distros etc. pushing me towards it when it's currently a downgrade.
> When people complain about Wayland being mediocre or "as good as X was" what they often mean is "Wayland doesn't support $very_specific_feature" or "my video card vendor's proprietary drivers aren't well-tested with Wayland and it makes me sad".
...Yes? Wayland being a regression in terms of features and bugginess is kinda a sticking point.
> Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
If X works for someone and Wayland doesn't, none of that matters. It doesn't matter how much you insult X users, it won't make their usecases invalid or make Wayland good enough.
> I don't really have a stake in this argument, except I'm bone-tired of seeing people whine about Wayland when they're using software somebody else gave them for free.
It cuts both ways: Users aren't entitled to free work, and developers aren't entitled to their work being well-regarded. Giving software away for free has never meant that people can't point out its problems.
From my standpoint as a software packager I have trouble believing that upending the entire video ecosystem on Linux was worth fixing this alleged "tearing" problem which is the main thing people bring up as the reason we all spent a decade doing this. (Along with some honestly fantastical worries about screen scraping -- the answer for that is open source.)
Honestly Wim did this right on the audio side. If it was so important to bring what Wayland has brought someone should have written a display server that talked to display clients just like the old server did and upgraded gracefully if the client had the new capabilities. You didn't even have to recompile software to use pipewire, let alone rewrite it.
More precisely, to put proprietary "apps" on an appliance. Windows desktop works fine for proprietary programs without any of the "security" features that Wayland-proponents want to push.
How so? Proprietary apps work fine on X. Or do you mean the security angle where flatpak wants to be able to isolate untrusted (including proprietary) apps?
I generally trust FOSS to not be malicious. I don't trust it to be bug free when processing untrusted input. So no, I really do want to sandbox plenty of apps regardless.
Sandboxing is still a solution in search of a problem. The only actual valuable data on my computer is the personal stuff that even sandboxed apps are meant to manipulate anyways. Everything else in the system I can download from the Internet for free, but my web browser still has to interact with my passwords and my spreadsheet with my finances, and sandboxing can't fix that.
> my web browser still has to interact with my passwords and my spreadsheet with my finances, and sandboxing can't fix that
True, but sandboxing can fix your browser being able to read your spreadsheets without asking, or your spreadsheet having network access. Just because you have to work with programs and data doesn't mean they're all in the same security domain.
I haven't seen any tearing in X in at least 5 years - with nvidia, nouveau and intel drivers. I don't think I'm doing anything special to achieve this, so your comment seems like a strange assertion to me?
I agree that X11 has some faults, and I really want Wayland to work, but everytime I try it, something ends up being broken.
I do see heavy tearing when watching videos under i3 on Ubuntu LTS and a 60 Hz monitor. So what? I just don't care. Tearing in videos does not hinder me from getting my work done. Watching videos is just wasting time most of the cases anyway.
> So what? I just don't care. Tearing in videos does not hinder me from getting my work done. Watching videos is just wasting time most of the cases anyway.
It's fine for you to not care about that usecase, but other people do care and that's also valid. There even exist devices that mostly exist to watch video and we'd really like Linux and some standard graphics stack to work on them.
True; also various scaling problems, and its security situation. We're in an unfortunate situation where no option has everything. If you want to say "Wayland is better for what I need", or "Xorg is better for what I need", or even "Arcan is better for what I need", then that is 100% legitimate. I'm also cool with "the current problems with (X|Wayland|Arcan) are..." because that's constructive and useful for trying to improve them. It's only pretending that one doesn't have faults or insulting its users/proponents that I mind.
Tiling window managers tend to tear more than any other desktop out there. Especially on x11, where you're expected to composite your own desktop without any (!!!) synchronization primitives, it's extremely common to get halfscan output from your windows or status bar. dwm, awesome and i3wm will all tear by default because there is no dispatcher, internal clock or even adaptive sync support.
Now, you might not notice the tearing if you've got V-sync properly configured. But V-Sync doesn't stop screen tearing, it just drops torn frames and decreases your refresh rate until your system can keep up with demand. It's an ugly hack on laptop hardware and an even uglier one on the desktop.
Well, I guess ‘window tearing’ doesn’t mean what I thought it did — or rather that it’s more than I thought. I always thought that it meant the artifact when dragging a changing window, which of course doesn’t happen if one doesn’t drag.
I don’t believe that I’ve ever noticed or been bothered by static tearing, but maybe I’m just used to it because I’m old and displays have always had it?
> It cuts both ways: Users aren't entitled to free work, and developers aren't entitled to their work being well-regarded. Giving software away for free has never meant that people can't point out its problems.
While true, this statement is also useless. On a meta level, what you're essentially saying is that users' rights to feel resentful, is more valuable than achieving an outcome.
> While true, this statement is also useless. On a meta level, what you're essentially saying is that users' rights to feel resentful, is more valuable than achieving an outcome.
It's more useful than calling X users "whiners" while ignoring their complaints. And no, I'm pointing out that the users are resentful because they're being pushed to an outcome that's worse and therefore not desirable to achieve.
Sometimes that outcome is not the outcome that those users want. For example I'd stay on X11 forever because none of its problems are relevant to me.
The only relevant problem with X11 to me is that all of its developers are stopping to work on it and it won't run on any new hardware soon (and probably on old hardware too.)
You both miss the point completely. What I mean is that just by being resentful, you don't achieve the outcome that you want to achieve (not the outcome that they want to achieve).
The only way to achieve the outcome you want, is for someone to do the work. They're not going to do it. If you'd rather keep being resentful rather than do the work towards your own claimed desired outcome, then apparently you don't really value the outcome you so claim to desire; you value being resentful more.
> Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
Wayland made some huge design changes, and those changes all have pros and cons. How do we tend to communicate about pros and cons? By complaining.
It sucks that Xorg's accessibility features are so baked-in that you must be compatible with them; because that demand can become a wall of assumptions. It sucks that Wayland leaves 100% of accessibility to the compositor; because that creates a lot of redundant work, and the best implementations get siloed into plasma or gnome, creating multiple smaller instances of the tight integration that made UX less flexible to begin with.
Using Xorg, you can run an incredibly lightweight window manager that never even thought about managing keymaps, adjusting mouse acceleration, setting the display resolution, etc., because that's not it's job. I can get absolutely everything I want from i3wm on Xorg... except fractional scaling. So far, it looks like plasma is the only one that has really figured that one out, and I still get a tiny pointer sometimes.
The best thing about free software is that we can choose how it works. The more tightly integrated the pieces are, the fewer choices are available. Fewer choices results in more complaining. Wayland may be the most significant instance of this pattern in Linux history. The second most significant instance was already present: Desktop Environments.
The biggest source of complaints I have is that the best aspects of plasma are trapped inside KDE. If you want the best Wayland compositor out there, you have to have it wrapped in an entire full-featured Desktop Environment.
---
Complaining is the best thing about Linux, so long as it's able to drive real change. The more stubborn and inflexible a system (and whoever designs it) is, the more disconnected complaints become. That's the real problem that deserves our attention.
If X is 40 and Wayland is 16,that means a difference of 24 years. Hence Wayland compositors have 8 years to work out the kinks. I am currently using Wayland via Plasma 6 and it works well enough but I don't have special needs, so I don't know how well... say... screen readers work.
Wayland came out at the same time as the Windows compositor in Vista. Let's be generous and consider the next Windows version, 7, as having a "good/stable" compositor. So Wayland is 13 years behind Windows.
16 years old and there's still video tearing in the browsers. I try it with every Ubuntu LTS release, and then switch back to X in a few days. It solves problems I just don't have.
Another way to look at these figures is that in a few years a Wayland successor is due, born out of the same motivation as Wayland ie. lack of people with a desire to maintain legacy software. My expectation is that browser frontend stacks will eat the desktop completely at that point; it's not like there weren't many new desktop apps on Linux anyway.
I'm getting much higher framerates in Civ VI on linux since I switched to Wayland, so there is that. For the rest it just works for me, use it both professionally and privately.
> but it seems it's still considered mediocre or not as good as X was.
Because there was a big fracture and doubt in the community support for Wayland, not to mention alternatives like mir popping up and diverting resources.
A lot of things liked mir happened over the years and didn't go anywhere. Most tried to fix problems with x that the x developers said were not the real problems. When wayland came it was supported by people who knew the real problems with x. They have also proven to keep working - no surprise at they had a history of working on x.
Based on what exactly? On a single frame difference, which is easily explained by the fundamental point of Wayland: no tearing? It's simply that X puts the cursor asynchronously, potentially tearing it, while Wayland always renders a correct frame, but the mouse movements' result has to wait for the very next frame.
Why this does not surprise me? every attempt to rewrite a working solution to make it more "modern, easy to maintain and future proof" rarely do so. It always end up slower, with more latency and lifted by faster hardware, not by faster software.
Every 20 years a new generation comes weaker, pampered with the abtsractions of the previous generations who did the grunt work for them. After 3 generations of software developers all we have is library/framework callers and nobody really knows about performance and optimization.
This is not my experience. When I upgraded several of my computers to KDE 6 switching to Wayland, overall responsiveness and snappiness of the system increased very visibly. There are still features that Wayland lacks compared to X11, but I am willing to compromise in favor of its other benefits.
Same here, switching from i3 to sway resulted in a noticeably more responsive experience on my aging hardware. Of course, this is just an anecdote, and I could probably get the same results on X with some fiddling, but I do think the value of simpler and more modern systems is demonstrated here.
On the other hand, I just did a fresh install of Artix Linux. Installed KDE just to have something functional while I get my tiling setup working. Boot into Plasma(Wayland) and it utterly shits itself, turning my main monitor on and off repeatedly until it finally crashes. So I pick Plasma(X11) instead and that just works.
In fact, in almost 2 decades of using Linux every day, I can't remember X doing anything similar. It's always just worked.
> In fact, in almost 2 decades of using Linux every day, I can't remember X doing anything similar. It's always just worked.
Well, we have very different memories then. Sure, X worked reliably once configured. But configuring it was a marathon in hell, as per by my memory, and all the litany of forum posts crying out for help all across the internet. Like, I have at one point had to rescue an install by changing back the config file without a screen at all!
I mean, I've certainly had issues with X before, back when you had to configure it. But they were usually initial issues while figuring out the right configuration. And if you had a GPU where the choice was between shitty proprietary blob drivers and woefully incomplete open ones, you might have a bad experience regardless of what you put in the config. But that's not really Xorg's fault. It can't magically make shitty drivers good. With decent drivers, once configured, it was rock solid, and has been for as long as I've been using Linux. And for a long time the configuration has been addressed; it now figures that stuff out on its own the vast majority of the time, unless you have really specific needs, in which case I bet Wayland compositors don't fare any better.
I also fundamentally disagree with the idea that X "does too much", which is often cited in favour of Wayland. The fact that X encompasses everything about how I interact with my computer is one of the things I love about it. I might switch WMs and DEs, but my xmodmap, xresources etc remain the same.
I used X11 in the early 90s (HP-UX) and since 2009 (Ubuntu until 22.04 made me switch to Debian.) I never had to configure anything. Maybe some xmodmap in the 90s. I did everything with dialog windows in this century, like I would do it in Windows.
Yes. Grandparent talked about 20 years ago. 15 years ago Ubuntu required no configuration at all if you had a graphics card that worked (most newer ones did, Nvidia was a nightmare).
Old computers had less latency, but otoh on many OSes a single app crashing meant the whole OS was irresponsive and you had to reboot the whole system.
The various wayland compositors all have huge problems with keyboards, mice, and other input devices because the wayland spec and reference implementation(s) chose not to support such things. So each wayland compositor picks a different solution. Popular ones are libei and libinput but often there's just no support for "advanced" mouse/keyboard inputs at all (looking at you, weston). And you can never tell if a particular piece of linux software will work on any given wayland. The fragmentation is terrible. On X with the strong X11 reference that actually implements everything you can be sure something that works on one will work on another.
Hell, it's been 12 years and not a single wayland compositor supports screen readers for the visually disabled yet. They are toys.
I'm told they left out color calibration support, too (as I read it, there's no way for calibration apps to control, or even know, what compensation is applied when they're trying to measure things). Oh, and client side window decorations still around, giving up on the chance to fix an ancient security hole.
So no good for games, no good for professional graphics, no good if you don't see well... basically no good if you're any different from the people who hacked it together.
But, hey, with any luck they cut down on the screen tearing that I've never noticed.
Color management support, though taking a long time, is moving along[1], and compositor implementations have recently started appearing. Device calibration support is a different question, but a much more specialized one.
Wayland is an extensible protocol. The core is literally just "here a buffer", but it has several extension in various phases that can greatly extend its capabilities. This way, a Wayland compositor can be used as a kiosk screen for a single program, or be a full-blown compositor+window manager, all relying on standard Linux interfaces for the display part.
Color calibration can absolutely be retrofitted into this (versioned) protocol, and there is work ongoing.
It's unfortunate that colour calibration took a while to be fully implemented (xx-color-management-v4 landed on most compositors half a year ago I think, though LTS distros may need to wait longer to get it).
That being said, thanks to Nvidia I never got colour calibration to work right in X11 either, so I have no horse in this race. Would be cool to finally get HDR working for the first time, but I don't know if that'll ever happen on Linux with Nvidia hardware. Guess I should keep dual booting for HDR content until either Windows 10 dies off or I get new hardware.
I do actually notice the lack of tearing in Wayland, especially under heavy load. Used to annoy me to no end to see my windows tear when just dragging them across a 1080p screen. I don't know if it was an Intel driver bug (I tried all the config settings) or something X11 specific, but Wayland did finally fix the tearing issues I had.
I haven't noticed any problems with colours in either Gnome or Gamescope (except for the lack of HDR, of course, but that's also true on X11) so whatever is causing issues for you seems to be DE specific. Looks like we both have issues other people never encountered, that's what makes graphics stacks so impossible to debug and code for.
If you control what goes into a distribution, you can not drop support for stuff that's working and already implements those needs in favor of whatever you pull out of your butt.
If not now, soon. Gtk itself is entirely dropping X support. And it's not their fault of wayland is so incomplete but I do wish people like them, and distro managers, would realize that the waylands are not actually replacements for X yet. And they might not ever be if you're visually impaired.
This is why I ultimately ended up on Windows after switching from macOS and focusing my free time development work on improving the power user/developer experience on Windows with a tiling window manager ecosystem.[1]
One of the really cool things that has come out of this is seeing how people are adding accessibility workflows on top of the ecosystem. There is one person who controls their entire desktop via the tiling window manager using voice control![2]
Windows has added several things with the newer WDDMs to help alleviate compositor problems: flip model, overlays, etc. They've definitely identified the problem areas and are trying to correct them. It's been quite the turnaround since Vista.
> there's just no support for "advanced" mouse/keyboard inputs at all
It feels like you're probably blaming the wrong people here. You should look at the companies that make this peripherals that don't also offer you linux drivers.
You say this, but have you seen the quality of the drivers they produce on platforms they ostensibly support? You do not want vendor mouse/keyboard software on any platform. Instead, improve the USB class drivers so that you don’t need third-party drivers.
I agree with you on general drivers, but what generic devices does Linux still lack? Controller HID support seems pretty universal, as do all manners of touch screens/touch pads, drawing tablets, and just about anything resembling a keyboard.
The only hardware I know for sure is lacking is stuff like Stream Decks, but that kind of hardware is difficult to generalise for.
Plus, most working hardware that doesn't fit a standard HID protocol already has drivers in the Linux kernel written by manufacturers.
There are definitely a lot of missing protocols when it comes to input though - wl_seat has pointer, keyboard and touch input support, but where are gamepads and wheels?
Nope. Because I don't mean special hardware. I mean using normal hardware in advanced ways: like sharing mouse and keyboards between computers. Something that's worked perfectly in X for 20+ years but still doesn't really work with the waylands even if the wayland compositor in question supports libei (which most do not).
I get major lag spikes when the gpu is under heavy load (like doing Stable Diffusion inference or something). TBF I haven't A/B tested with X11, but I don't ever remember it being like that. An extra frame of latency isn't great on it's own, but the occasional spikes in lag are really irritating.
May still happen especially if it is thrashing vram in and out of system memory or something, but have you tried lowering priority of the stable diffusion process?
I can also attest to horrific lagspikes on an Optimus laptop even if Intel is driving the desktop. Memory pressure is definitely the problem here. Lagspikes actually lessened when I switched to Wayland Gnome. I think they lessened further with PREEMPT_RT on kernel 6.12. Nvidia requires an environment variable to build on real time kernels but it plays surprisingly nice as of driver 570. But if you have this config, you need at least 11th gen Intel iGPU or AMD APU, because i915 does not build for real-time kernels. Only the Xe driver works and only if you force_probe the ID if it's Tiger Lake.
...Which I don't get because the Xe driver is said to explicitly support, at minimum, Tiger Lake. I played Minecraft on the iGPU with Xe and it was perfectly fine. It... drew 3D graphics at expected framerates.
One of the places that macOS has always excelled, all the way back to classic Mac OS.
Even System 1 on almost infinitely less powerful gear 40+ years ago super-prioritized cursor responsiveness, outshining all competitors. The OS and UI underpinnings are now 100% different, but the priority remains.
Most of this is offset by the godawful mouse acceleration profile MacOS uses by default. Many external mice, even with the sensitivity turned up to max, will not respond to small wrist movements at all and instead require disabled acceleration curves or more exaggerated mouse movements.
Those curves work fine for the Magic Trackpad, but they should not come default for any desktop Macs that use mice as their primary input.
Yeah, I'm baffled this is so bad. Especially in Linux. Surely moving the mouse should take precedence over everything else? Even highlighting the thing you move the mouse over shouldn't slow down the mouse cursor itself.
I recently had a problem with extreme mouse slowdowns on KDE/Plasma 6 on X11 with NVidia. I noticed the slowdown was particularly extreme over the tabs of my browser.
The fix, in case anyone else also has this problem, is to disable OpenGL flipping. I have no idea what OpenGL flipping does, but disabling it (in nvidia-settings and in /etc/X11/xorg.conf) fixed the problem.
I'd love it if someone could explain why. Or why this isn't the default.
OpenGL flipping is iirc the weird name nvidia gives for vsync. Disabling it means you're going to have tearing artifacts, hence why it's not the default.
But why does it slow down the mouse? Is there no way to get both? Maybe I should try if Wayland gets me both, although at the moment it just gives me a black screen.
I have another somewhat unrelated pet peeve - when I started using Linux, the desktop was software composited, essentially what I assume was going on was that the UI was drawn by the CPU writing into the RAM which got blitted into the GPU's framebuffer, as it was done forever.
When compositors (even with X11 composite) and GPU renderers becamse fashionable, the latency became awful, literally 100s of msecs. Which was weird considering, I grew up playing DOS games that didn't have this issue. Things improved over time but it's not clear to me if we ever got back to the golden days of CPU rendering.
Beautiful work. Could it be worth repeating the experiment with the monitor running at a very low refresh rate, i.e. 30hz? If Wayland is always a frame slower than X11, it should be much easier to observe.
That there's a probability of 4.20e-05 that the observed difference of that large would happen by chance due to observation noise if there would be no real difference (given your assumptions about the data generating process holds).
The normality assumption is a large stretch, especially since there an absolute lower limit near the observation and a somewhat discrete distribution, so a t-test isn't appropriate. But then again it looks significant, so there's no real need for a test
I used to be a NetHack speedrunner. Pressing the keys real fast. All of it in terminals. I had the fastest run for almost 10 years. Input lag was a thing I thought sometimes. Everything below is based on pure gut feelings because I never measured any of it.
I've long wondered why some terminals and operating systems felt like they were pretty laggy. In the sense that if I press my character to move, how long until they actually move?
iTerm2 on MacOS is worst offender. But I can't tell if it's iTerm2 or MacOS itself. I remember trying other terminals on Mac, and it was mostly the same. iTerm2 itself also had bunch of rendering options, but I couldn't get them to do anything I could actually feel affected input lag. One thought I had was: maybe iTerm2 added 1 frame lateness, and MacOS itself added another? But that would be ~30ms on a 60fps monitor which I can easily already tell on a server-connection NetHack.
I also have no idea if measuring "N frames late" is actually a sensible way to think about this. I assume computers know when a key is pressed at a higher frequency?
Linux xterm and rxvt were the fastest that I remember, on X11 (I've never used Wayland like in the article, well not for NetHack anyway).
I have no idea how compositors work on any of these operating systems or X11/Wayland to say anything smart about them or why they would be slower or faster.
Reading the article...I realized I have that same iPhone 15 Pro with 240fps recording. I could replicate that experiment, but testing out terminals on various operating systems instead I used to play on. So I could now test if my gut feeling was right all along or not. or maybe it lied to me all these years.
I wrote the idea in the article down on my notes, maybe when I'm bored enough I'll try it :) and I can stop saying propaganda about iTerm2 being slow if it turns out I was completely wrong. Maybe I have easier time because all my NetHack days were 60fps, so I have more leeway in measurements.
I'm not active in NetHack playing anymore, although I sometimes play it as a comfort game.
Ooh that is interesting. Also noticed this passage:
"When people measure actual end-to-end latency for games on normal computer setups, they usually find latencies in the 100ms range."
That is really surprising to me. 100ms is HUGE. On NetHack, sshing into servers the difference between 50ms and 150ms is enormous. If do my experiment and I'm not too lazy I want to check on that 100ms figure, your link points to more sources which I didn't check on right now.
I don't know if I'm blind but I can't tell when that article was written exactly. But it mentioned MacBook 2014 and Lubuntu 16.04 so maybe it's around mid 2010, 2016 possibly? (I should check properly if I do my experiments)
The author in the Wayland vs X11 used mouse flicks on the video. Just now while writing I kind of had a thought if this needs some more deep thinking: how do I accurately measure "my finger touched this button and X milliseconds later character moved". Wondering what to consider as the "finger touched this button" event in a 240fps video. Maybe I can do what the author did and map a key to a mouse instead, because maybe there's a much-easier-to-see physical movement with a mouse. But then, am I really measuring faithfully my old NetHack-like environment? Or maybe this isn't really an issue and it'll be easy to tell.
Too much for my small brain to think in the moment :) that link gave me at least some validation that iTerm2 maybe really is a bit slow, or at least was at that time. There's also bunch of terminals on that page I've never heard of.
Today iTerm2 is my daily driver, but xterm is still really fast on my Fedora Linux system I have on the side.
Wayland has just been a disaster for the open source desktop. It's absorbed all of the effort into something horrifically subpar while breaking compatibility with the past.
Accessibly is far worse to the point of embarrassment. Latency is worse. Compatibility is gone. Everyone in the ecosystem needs to do more work to end up with fewer features for end users.
This is the opposite of systemd in every way. Systemd upset a lot of people because it was entirely user focused and didn't care about the beauty of the architecture. Wayland is only obsessed with how nice their apis are, not what the end user experience is.
Let's end this disaster. It's been 16 years. It's almost old enough to vote. When will we accept it has failed?
I mean X11 was a pile of hacks, it became an unmaintainable pile of hacks.
I think reduced scope was a good call, writing a display server is a hard problem. A lot projects started and failed to replace X11.
If you feel this way I recommend watching this talk by Daniel Stone
from linux.conf.au 2013,
The Real Story Behind Wayland and X
https://www.youtube.com/watch?v=RIctzAQOe44
Just because you don't agree with it (which is weird given the guy is a big X/Wayland maintainer), doesn't make it propaganda.
X sucks for how graphics work in the 21st century, there's a big reason Google didn't use it for Android and instead made something that's actually more akin to the Wayland model .
I've used https://isitsnappy.com/ on the iPhone to do these kinds of measurements pretty quickly, to substantiate why I sucked at Steam Deck rhythm games when connected to a TV.
Cool site! I didn't know this was an established way of measuring input lag but it's how I've been doing it since I found out my Samsung phone came with a 960 fps camera. Manually counting frames with ffmpeg is a pain, but I never even thought to look for an app that does this. Thanks for sharing, I'll take a look around :)
Never got to the bottom of it. TV was set to game mode, I tried various scaler settings on the deck itself, couldn't get it below about 90-100ms. I've tried on two TVs.
You can learn to play Space Channel 5 by tapping 100ms ahead of the beat. But I worried that if I succeeded too much, I'd end up doing everything that way: jumping red lights, taking espresso shots while they were still being poured etc.
Note that Wayland is a protocol, not an implementation. This testing was done with Gnome's implementation of Wayland, which doesn't mean much for any other Wayland desktop.
> Was Wayland not architected to fix latency issues?
It was designed to fix tearing issues, not latency issues.
But then the Wayland designers found out that players often prefer tearing to minimize latency, so the tearing protocol was implemented.
When a committee of perfectionists, rather than real people or companies, design something, you often get something that is completely unusable by anyone but that committee.
And that's exactly how it's played out so far. Wayland is still largely incomplete, not to mention it doesn't even have a reference implementation [1], and still doesn't include/describe several essential desktop APIs and features, which results in this protocol not having a ton of universal tools and applications that work across all of its implementations, including but not limited to:
* Keyboard layout switching and input management
* Display configuration
* Clipboard management
* Tools to manage windows/automate tasks
* Lots more
It's a damn ugly mess that has led to a lot of fragmentation with no obvious benefit. Currently, only KDE and Gnome have somewhat usable Wayland implementations. If you use anything else? You're SoL.
It's almost like you don't want tearing when browsing the web and your documents, but games can take over the precise control of rendering.. because they are games that want to do so?
The numbers presented in the article (~1 additional frame of latency on Wayland) are from someone using GNOME with the Mutter compositor. This means that neither the X11 or Wayland session has screen tearing. I imagine the X11 input latency numbers would go down by around half a frame on average if the author was using a different window manager without a compositor.
Of the things you list, IMO, maybe the display configuration should be part of the Wayland protocol.
It might be a better technical design to have the other stuff outside of the display protocol. Just because Xorg implemented something does not mean you have to put it in the Wayland protocol.
Routing of input events to windows has turned out to also be extremely important for a display subsystem in practice. It wasn’t just X11 that did this, it was also the NeXT window server, and Sun’s NeWS, and SGI’s pre-X window system, and the original VAXstation window system on VMS, as well as how both modern macOS and modern iOS and iOS-derived platforms work.
In any of these cases there may be one or more daemons behind the scenes handling the “raw” input—possibly even in cooperation with kernel-level pre-processing code, to ensure low latencey—but most event delivery to applications is associated with windows, with options to get lower-level access if needed.
One of the things that helps many of the systems described above with latency is kernel participation, whether by pushing much of the preprocessing of events down to the drivers so there’s little for userspace to do, or by implementing kernel-level zero-copy IPC (e.g. use of Mach messages by NeXT and Apple).
If human interface IPC happens entirely in userspace and requires multiple context switches to get an event from device to a display change, you’ll wind up with hitches and delays unless there’s some sort of scheduler hinting that ensures each stage in the pipeline runs immediately after the last.
This is, of course, why there was a lot of desire by Wayland advocates for kernel dbus support, but they went at the problem backwards: “Let’s take DBus, and make it fast by putting it in-kernel,” *without* first trying to make it as fast as possible without kernel support, *and* without trying to figure out the minimal feature set for kernel-level IPC that would be needed to support it (which may not look like DBus).
As I understand it, the tearing thing (no vsync) was only implemented after Valve pleaded, begged, and sacrificed a small animal to the Wayland developers.
Wayland is just a protocol that is probably implemented over and over and each implementation might have issues. It's mostly likely the different implementations having issues.
Was Wayland not architected to be simpler to implement than the (antique) X11 protocol?
Not arguing though, you are right it is just issues: Drivers, the Wayland implementations, how some plethora of apps and libraries have been battle tested then optimized, years over years, for X11. Not as much for Wayland display yet.
Wayland is simpler because it does less. And among the things Wayland doesn't do and X does are things that people need. It means these things have to be implemented elsewhere.
It is a common problem with these "simple" things. The problem is complex, and if you simplify one part, you are just pushing complexity elsewhere, you are not making it disappear. It is sometimes a good thing, but in the case of Wayland, it didn't go well.
There is also the option to make complexity disappear by just convincing people that they should not want to do what they want to do. This was the kind of response to what I needed to do. Then I had to switch back to x11.
In any case wayland is not bad if you only have pretty basic needs I guess, some basic things look easier to me there from a user perspective, and troubleshooting x11-related issues for a non-technical person is no fun either.
> Was Wayland not architected to be simpler to implement than the (antique) X11 protocol?
I think it's safe to assume that it is actually simpler, given that we already have multiple Wayland implementations, but still basically just the one X11 server implementation. Can one or more of those implementations shave off one or two milliseconds overs the next 40 years... Probably yes.
Sure, on linux all you really have is xorg, what used to be xfree86, but I have used xsgi, and there are X servers for windows, heck, I have even seen this amazing demo Xserver that ran on the web.
footnote: there is something wrong with the commonly used web search engines and I am unable to find that X11 web demo, I think they are prioritizing recent content over good content. you would think with how interesting that web demo was it was it would show up right away. but I got nothing, so no links. Anyway it was a deep dive into the intricacies of X11 window painting and the author had ported a limited subset of an X server to javascript in order to provide a live demonstration of the topic. I will keep looking.
Architecture contributes, but all it takes to blow input latency is some inefficient code almost anywhere in the input or rendering pipeline. The Apple IIe didn’t really have very many layers between the keyboard and the CRT. Wayland might have an optimal architecture in theory for latency — I don’t know — and latency could still be ruined by not fully optimizing the other components in the stack.
Gnome was fixed a few years ago to remove significant latency it was adding. May have been a factor if you used that as well, but it wasn't wayland specific to my knowledge.
I don't like the design of Wayland at all, so it'll take a lot more than performance analytics to get me to switch. I think that one of the core issues in Linux is nothing is ever consolidated, only moved toward more fragmentation. Wayland had the opportunity to consolidate, but instead to not fix fragmentation (each app is responsible for drawing its own window borders) and introduce more (multiple compositors each with their own quirks and non-standard APIs)
For those of you who are application developers on Linux/Wayland, do you use any other methods for benchmarking true latencies between input events and screen? Less heroic than the high-speed camera method. I'm embarrassed I've still never figured this out properly.
A photocell is I think the standard (or at least "the standard" in psychophysics applications I have also used). You need, though, to make sure your event causes a change that a simple photocell would detect (eg you have a corner of your screen or application window having a small square that turns from black to white and vice versa). You can connect that to an arduino or whatever you can use to get both signals in some low latency way (one from user interaction or whatever causes the event, and one from the event itself with that light intensity change). There are also photocells with a higher granularity for luminosity changes than just light/dark, so maybe you do not need to have the black/white square thing, just something that causes enough luminosity change to be detected. This should, in principle, give you very accurate timings if you can also get the event initiator somehow with low latency (or you can do stuff like simulate an event periodically and look for skipped frames/variance in the screen presentation rather than latency).
You might like Typometer. I believe it was created by a JetBrains developer to improve input latency in IDEA.
> Typometer works by generating OS input events and using screen capture to measure the delay between a keystroke and a corresponding screen update. Hence, the measurement encompasses all the constituents of processing latency (i. e. OS queue, VM, editor, GPU pipeline, buffering, window manager and possible V-Sync).
I wouldn't use that. Try KDE if you want a decent gaming experience. Gnome was very slow in even implementing something like adaptive sync, so they hardly are prioritizing latency issues for the end user.
I.e. Gnome's problems ≠ Wayland problems in the broader sense. Conflating the two is wrong.
Wayland lacks a strong reference implementation. The resulting fragmentation in the ecosystem turns the most popular compositors into de-facto standards every application developer will have to use as a reference.
This means Gnome's problems = Wayland problems.
Furthermore, the latency issues with Wayland are inherent. Compositors will always have worse latency than writing directly to the front buffer.
The kind of issues referenced in the article are not related to compositors being compositors though, but to bad implementations.
Gnome is big, but it's not the most popular DE out there, so it can't be treated even as a de-facto standard in the Wayland world. Some things might be treated that way, but they are lower level (let's say Pipewire, libinput etc.).
And regardless, it can't be used an example that blames Wayland for Gnome's issues, unless literally every single compositor has this problem and the real reason is Wayland's inherent limitations. There are such limitations in general indeed due to Wayland still evolving, but this isn't one of them.
> Compositors will always have worse latency than writing directly to the front buffer.
This is not really true, if a full screen application is detected, the compositor can tell the GPU to instead just display the contents of the applications buffer, skipping the compositor.
What do you mean? Most games run on XWayland when you're in a Wayland session, wouldn't input latency caused by XWayland therefore affect gamers who use Wayland?
I’ll look into it, but it used to be a choice in the game settings that bypassed the drawing display server entirely, and caused that full screen flicker you might recall when tabbing back to your desktop (and the switchover of rendering control, and subsequent huge performance loss of the running game).
I wasn’t aware that this changed, but you could be right. Its definitely the same on Windows as it always was, which is the platform I most developed games for.
> When using Fullscreen Optimizations, your game believes that it is running in Fullscreen Exclusive, but behind the scenes, Windows has the game running in borderless windowed mode. When a game is run in borderless windowed mode, the game does not have full control of the display– that overarching control is given back to the Desktop Window Manager (DWM).
I think Windows now also supports hardware overlays, so games running in fullscreen Windows can skip the compositor, which reduces the latency in no longer allowing games to directly write to the FB.
The DWM can do this even for windowed applications if they use flip model -- it will dynamically transition individual windows between compositing and independent flip (overlay).
Most compositors can do direct scanout when they detect a full screen surface. (This is broken right now with Gnome on X11 because of the way window frames are drawn: https://gitlab.gnome.org/GNOME/mutter/-/issues/2794 ) If the game uses the platform cursor then that will reflect the additional latency, but the actual input signals won't be delayed.
If an application framebuffer is full screen and in a compatible pixel format the compositor can do "direct scan out" where the compositor sends the framebuffer directly to the crtc instead of compositing first. I know that wlroots supports that. I'm not sure how much performance it saves to be honest.
And I'll second that most commercial games do go through XWayland, though it depends on what games you like!
Reposting my analysis I performed with a Rpi pico, oscilloscope and a photo diode from another site:
I created a test setup using a Raspberry Pi Pico running CircuitPython and a USB HiD library to emulate mouse clicks. I connected button switch to a GPIO pin and polled the state of the pin to send mouse click events over usb to my laptop. I connected a photo diode to the power rail of the pico and pointed it at the screen. Finally I connected the push button to channel 1 of my oscilloscope and the photo diode to channel 2.
I created small application with raylib which would toggle the screen from black to white when clicked. When the screen changes from white to black, the voltage across of the photo diode would drop, which would create a falling edge I could trigger on using the oscilloscope.
Test Setup:
I ran the following tests on my AMD Framework 13, under CachyOS with all packages up to date. For each test I set the Raylib application to run at 800x600@60Hz and left the photo diode in place near the bottom right corner of the screen. For each test, the display output scaling was set to 1x (2256x1504), and variable refresh rate was disabled. Using the oscilloscope I measured from the first trigger buffer sample that the voltage of the button reached steady state to the first sample where the voltage on the photo diode dropped below steady state to 1.5v
Caveats:
The times seem high, but they encompass the entire chain of Push button -> App rendering next frame.
There is some loss in time resolution because of the abysmally small size of my cheap oscilloscope’s trace buffer.
I only collected 9 samples for each configuration because I’m clumsy on the controls of the oscilloscope and this took a little bit.
The raylib application was running under Xwayland for the Wayland tests which somewhat skews the results.
Its very interesting, but I would love to see other Wayland implementations other than GNOME's. Its kind of hard to tell if its the GNOME specific implementation or if this is going to be a trend across KDE X vs KDE wayland and specific window managers like sway -> i3 and hyprland etc... But still a really cool blog post.
i run nvidia/intel/optimus/wayland in optimus/integrated mode on my laptop. i have anecdotally noticed perceptually salient increased response times when:
1) dpi scaling is in use
2) external displays are used via usb-c/displayport
3) the nvidia memory clock has been scaled down by power management
that said there have been fantastic improvements overall to both stability and response times since kde6 and nvidia-open have stabilized and i haven't noticed any of it in a long time now. it really is pretty great these days.
I also have no ideas how it works, but my guess would be that is actually translates exactly like that into games. Why would the mouse on the desktop have a latency that games do not have. Linux does not have a real fullscreen mode like Windows has (that I do not use anymore anyway for faster alt-tabbing). So my guess is, the cursor or mouse input gets into games very much the same way as it gets in the desktop.
Wayland has support for games completely taking over the GPU; notoriously critical for VR support. This is basically the same as Windows exclusive full screen.
Wine/proton would need to support it, XWayland would need to support it (Wine/Proton are one major step away from native Wayland support: Vulkan), and finally the compositor would need to support it. Gnome is about the worst compositor that you could be testing any of this stuff on, they are comically hostile towards ideas not their own. The chances of this ever working on Gnome are near zero. KDE is pretty good for support, Hyprland seems to be trying to support every under the sun.
> notoriously critical for VR support. This is basically the same as Windows exclusive full screen.
D3D12 dropped support for exclusive fullscreen, and I don't think headsets even go through DXGI but their own swap chain APIs. Why do VR games on Linux need the equivalent from the Wayland compositor?
Can I have the source for this? DX12 dropped exclujsive FS? Why does every game still has this as a option then? Or are you talking about VR specifically?
Games may show it as an option if the engine itself can support other GPU APIs and the devs working on the options menu don't know or care that this particular settings combination is misleading.
> Gnome is about the worst compositor that you could be testing any of this stuff on, they are comically hostile towards ideas not their own. The chances of this ever working on Gnome are near zero.
GNOME has supported direct scanout for fullscreen apps for a while and drm-lease was implemented not too long ago either.
I doubt that Wine would actually request it. Some compositors also have "tearing mode," which is basically as good: the game gets full control of the presentation timing, so vsync/tearing would be determined by what the game wants to do.
Funny, I didn't even know this was a debate. I didn't even notice when Fedora switched from X to Wayland by default - maybe because I'm still a 60hz plebian?
I have experienced issues with my mouse input locking up/jumping - especially noticeable when I'm playing a shooter - but I'm pretty sure that's just my poor ancient mouse giving out.
Since this is all mouse pointer relates and there is hardware for that, I can only assume it's a fixable problem.
OTOH the gnome developers have been talking about implementing triple buffering in the compositor. Tripple buffering is only needed when your renderer can't complete a frame faster than the scanout of a frame. Given that any modern GPU should be able to composite hundreds of frames per second this make me think something about Wayland or the gnome compositor in particular isn't designed as well as it could be.
Dynamic triple buffering is basically just giving the hardware time to clock itself up when there is a sudden change in workload, e.g. idling then opening the overview (drawing basically nothing -> redrawing basically the whole desktop).
If you just want to measure user space software input lag, shouldn't it be more accurate if you print out the time when the event arrives in the kernel, and again when the mouse moves within a application window?
Depends what you want to measure. But generally, people are more sensitive to latencies in movement of the visual cursor than they are to applications responding to it.
Hm. I am currently using Ubuntu 24.10 and it also has Wayland and gnome as a default, with the nvidia 565 driver. And it is really, really bad. For example, i use chrome and the tabs, man, it takes ages to switch from one tab to another and to get focus in the location bar again.
So much for the whining. I do not know what to do. I could record this behaviour, but it somehow needs to show the timestamp of when i intend a behaviour and the time, that behaviour becomes actualized.
I used to be quick with all the hotkeys but i do make so many mistakes now, because the system is simply not responsive and all my instincts await instant action.
Won’t get you end to end latency, but should be able to trace the input events through chrome, and show the context swaps of all the Wayland/dbus stuff.
Has implications for competitive FPS gaming on modern Linux — one more problem to fix. For example "The Finals" allows Linux players, but imagine having this much input delay and having to revert to classic x11 to play games, lol.
It's actually just cursor latency specific to the windowed environment you're running. From what I've experienced (with a 4060 Ti) it doesn't seem to impact FPS games at all.
I haven't tried any games that use a cursor with Wayland yet so I don't know if it would have an impact there.
I think it has to do with whether or not the game in question is reading the mouse device directly (e.g. through SSL) or via the compositor. If it's reading the device directly it stands to reason that there would be less latency.
> having to revert to classic x11 to play games, lol.
It would be more of a problem the other way around, if we had to resort to Wayland to get low latency. I think most of us using Linux for gaming and casual stuff are happy to stick to X11 for now and the foreseeable future. It has good support in software, its failure modes are well-documented, and it doesn't add one more layer to the pile of complexity that desktop linux already is; at least, not one us users have to consciously keep in mind as a common source of issues.
As I type this, I'm playing "the finals" on Linux via Proton on Wayland. I won't pretend I'm any kind of competitive gamer type, but it's perfectly fine and I don't feel like I'm being held back by input lag. So this is very much a niche issue to have.
From outside it's hard to tell if it's truly protocol differences or just the age of the implementations on X11, but when Wayland came out every project has claimed improvements over the old X11 stack. Also, from the early Wayland days presentations bashed the protocol as something that couldn't be fixed without a rework that was not going to happen due to the dead weight of backwards compatibility and awful older hardware.
As a user applications running on Wayland have consistently improved on how nice things feel if you don't miss your latency deadlines. It's easy to perceive on apps, and straight out obvious in games.
I got to diamond in Apex Legends on Linux using Wayland on KDE Plasma. Didn't feel any noticeable difference between wayland, x and windows in my Apex playing.
I have trouble believing that 6.5ms in increased latency would be perceptible to any more than a fraction of a percent of the most elite gamers. Most the people claiming that this level of difference is impacting their gameplay are victims of confirmation bias.
David Eagleman has done some work with drummers. Granted the audio system might be a bit more accurate than the visual, or maybe drummers are just weird. On the other hand, vim taking 30 milliseconds to start (ugh) and having sluggish cursor motions is why I'm on vi now. Haven't tried Wayland. Maybe in some number of years once it's more portable and more developed? (And how many years has it already been out?)
> “I was working with Larry Mullen, Jr., on one of the U2 albums,” Eno told me. “ ‘All That You Don’t Leave Behind,’ or whatever it’s called.” Mullen was playing drums over a recording of the band and a click track—a computer-generated beat that was meant to keep all the overdubbed parts in synch. In this case, however, Mullen thought that the click track was slightly off: it was a fraction of a beat behind the rest of the band. “I said, ‘No, that can’t be so, Larry,’ ” Eno recalled. “ ‘We’ve all worked to that track, so it must be right.’ But he said, ‘Sorry, I just can’t play to it.’ ”
> Eno eventually adjusted the click to Mullen’s satisfaction, but he was just humoring him. It was only later, after the drummer had left, that Eno checked the original track again and realized that Mullen was right: the click was off by six milliseconds. “The thing is,” Eno told me, “when we were adjusting it I once had it two milliseconds to the wrong side of the beat, and he said, ‘No, you’ve got to come back a bit.’ Which I think is absolutely staggering.”
It doesn't need to be perceptible to cause a difference in a game.
Suppose two players notice each other at the same time (e.g. as would naturally happen when walking around a corner in a shooter), first to shoot wins, and their total latencies are identical Gaussians with a standard deviation of 100ms. Then a 6.5ms reduction in latency is worth an additional 2.5% chance of winning the trade. Maybe you won't notice this on a moment by moment basis, but take statistics and its impact should be measurable.
In ELO terms a 2.5% gain in win rate is around a 10 point increase (simplifying by assuming that single Gaussian is the entire game). That's small, but if you were a hardcore player and all it took to raise your ELO by 10 points was using a better monitor/mouse/OS... why not? Doing that is cheap compared to the time investment required to improve your ELO another 10 points with practice (unless you're just starting).
Also, I think you'd be surprised what people can perceive in a context where they are practiced. Speed runners hit frame perfect tricks in 60FPS games. That's not reaction time but it does intimately involve consistent control latency between practice and execution.
> Suppose two players notice each other at the same time (e.g. as would naturally happen when walking around a corner in a shooter)
This is not true for third person games. Depending on a left sided or right sided peek and your angle or approach, players see asymmetrically.
For example, Fortnite is a right side peek game. Peeking right is safer than peeking left as less of your body is exposed before your camera turns the corner.
I believe distance also plays a part in the angles.
Yeah, network latency and client side prediction and accuracy will also play huge roles. The actual distributions will be very complex, but in general reacting faster is going to be better.
Do people not play deathmatches on LAN parties anymore these days? 2.5 is huge if the game lasts long enough that someone would be leading with 200. ;)
I would postulate that 100% of professional (i.e. elite) competitive gamers would be able to tell the difference. See this old touchscreen demonstration: https://www.youtube.com/watch?v=vOvQCPLkPt4
Those sorts of latencies actually are noticeable! As an example, 6.5ms latency between a virtual instrument and its UI is definitely noticeable.
I didn’t think it was. But it is. I promise!
It’s not necessarily a reaction-time game-winning thing. It’s a feel.
With virtual instruments, my experience is that when you get down to ~3ms you don’t notice the latency anymore… but!, when you go below 3ms, it starts feeling more physically real.
About the difference between 60hz and 120hz monitor, instantly noticeable just by moving the mouse in windows (just by looking
at the distance cursor updates as it moves). Would you argue that all gaming monitors are placebo?
I actually would. Gaming monitors are the equivalent of fancy audiophile gear. It's a way to fleece people by making them think they can perceive a difference that isn't really there.
You may think 6.5 ms of input latency is imperceptible. But combine it with the rest of the stack (monitor refresh rate, local network latency, RTT between client and server, time for server to register input from client and calculate “winner”), and it becomes the diff between an L and W. In the case of pros, the diff between a multimillion dollar cash prize and nil.
There are noticability thresholds where this could push it over. For fighting games if you have the reactions to whiff punish N frame recovery moves this may push you to only being able to punish N+1 recovery moves and really impact your ranking. This is a little over 1/3rd of a 60hz frame.
"Competitive FPS gaming" stops allowing anything besides fresh Windows installs after a certain point. It's a diminutive distinction, like pointing out that emulating GameCube games won't let Smash Melee players fight competitively; nobody playing on either side actually cares.
Composited desktops have their drawbacks. With a display system like X11 or old-school GDI-based Windows, draws to the display are next to immediate, whereas with composited desktops, everybody has to draw to a back buffer, then the back buffers have to be composited together on the final display, adding at least one frame of latency. Mouse cursors have been a hardware sprite since the 90s, but Wayland compositors may favor using the GPU to draw a soft cursor.
It'd be interesting to see a compositor that can speak Wayland, yet performs immediate, region-clipped draws of all windows like X11 conventionally does.
I did some work to create a simple device to measure this some yeas ago. The intention was for developers to use it to regularly benchmark their software. It costs like 20 bucks and takes 10 minutes to build.
I might get to run some benchmarks myself next week.
Somebody should point LDAT at that [1] or the custom setup with LED triggered by the mouseclick + a high FPS camera that Battlenonsense used [2] before LDAT released.
Ported to Linux? It runs on Linux. What exactly would need porting?
What would be required to make surfaceflinger a practical Linux desktop compositor is merely some translation layer, equivalent to XWayland, that supports hardware acceleration. Such things have been written, but not open source and they never got traction.
> Ported to Linux? It runs on Linux. What exactly would need porting?
It runs on Android/Linux, not GNU/Linux. I would be surprised if there wasn't at least a bit of work needed to handle things like bionic vs glibc, and you'd really want it to act "normal" by being installed to FHS paths instead of Android's weird filesystem layout. All doable, I expect, but it would be a port.
Agreed. That was just the latest related article I saw. I was bring it up to support my claim of "high framerate support is still new/developing in Gnome"... embarrassingly my old man brain compressed time... it's been 6 years[1], oh well.
Why is there latency beyond a single frame at all?? I did a very quick replica of the camera frame counting on Windows with a 144Hz display and it's definitely below the camera's 60FPS, i.e. the cursor moves on the same frame that the mouse moves. Moving a window also has 0 lag in that test.
Huh, that is one of the things why I never liked Linux Desktop at all. Switching from Windows to Linux Desktop feels... weird, laggy. People told me I was the problem, but it seems there are more people who noticed this lag. Maybe someone can fix this :)
This reminds me of the early work of the structuralists in psychology. Measuring the time it takes for a person (or in this case Wayland) to react to a sensation.
Yes, I was expecting some software that listens directly to the mouse input, and watches a pixel on the screen. Messing around with cameras and counting frames introduces a whole bunch of other variables like the quality of the monitor and mouse and phone used for recording.
The camera, mouse, and monitor all stayed the same for the tests, but there was a significant difference in latency. Out of the 16 times the experiment was run, only once did Wayland have lower latency. It would be an amazing coincidence if the monitor, mouse, and/or camera were the reason for this.
Figuring out why there's increased latency is a job for software tooling, but I think this guy's experiment is one of the best ways to measure what users care about: The time it takes for an input (such as mouse movement) to result in a change (such as cursor movement).
Note that this doesn't mean that the Wayland protocol itself is the reason for the higher latency. It may be Gnome's implementation (testing with a wlroots compositor might shed light on this). It may be differences in default configuration options. It may be that Wayland and X11 start up different services, and the Wayland helper processes increase load on the machine. But I seriously doubt the reason for the difference in latency was because the same hardware was used throughout the experiment.
I got results with a P-value of under 0.001. That should be enough to demonstrate that there's a real difference.
Using a camera allows the methodology to he identical between Wayland and X, while I don't know how to listen for mouse movements from software in a way that wouldn't introduce its own problems. What if the photons are emitted from the screen after the same number of milliseconds across X and Wayland, but Mutter is a few milliseconds slower about notifying my measurement application? Conversely, what if Mutter has more real latency than X, but the latency is introduced after the stage where my measurement application sits?
The variables you mention are identical between the Wayland test and the X test. It is admittedly a challenge for reproducibility, but doesn't affect these results.
The mouse and monitor don't matter here. Unless their delay completely dominates the timing (it doesn't here) they can be ignored because the setup is constant been the tests. We're interested in the difference, not the absolute numbers.
Looked up the mouse to see how fast it updates and read that it is wireless. Kind of disappointing and might explain some of the variance. Although if X11 is consistently faster the test should still have some validity.
Would like to see a test with gnome and the wireless mouse removed from the equation.
It depends on the model. Wireless mice don't have to be slower than wired. But even then, for this test it's irrelevant. It's the same mouse in both scenarios, and we only care about the difference. Even if a wired mouse stripped X ms on both sides, the result would be the same.
That's only a problem if the noise of wireless is higher than the signal (measured delay difference). And we can see from the data it isn't. Unless you're claiming it can be consistently different by ~5ms between those two tests... but that's very unlikely. Like, "you should demonstrate it" unlikely.
The title and conclusion are disingenuous - they've proven that Gnome Wayland has higher latency than Gnome Xorg. This could very well have nothing to do with Wayland at all - we need more testing.
I'm pretty sure this is incorrect? I half-remember having cursor rendering bugs on GNOME Wayland with NVidia ages ago which I had to disable something h hardware cursor related to fix. But I don't know if that's what you're talking about or not. Could you link to a source which says that Mutter doesn't make use of hardware cursor?
(Hi, author here by the way! ... Don't worry, that disabling hardware cursors thing was at least one OS re-install ago)
Yes, it should use hardware cursor by default on most hardware. But I don't think that RPi's VideoCore (in the linked article) supports hardware cursor or multiple planes. The difference between X11 and Wayland might then be that the cursor isn't vsynced as part of the composite process on X11, where it is on Wayland.
Mutter definitely throttles cursor updates on Wayland, too, which will contribute slightly to latency even with a hardware cursor. In general, with Wayland, the compositor is responsible for cursor updates, and I'm not sure which other ones throttle. But that would be where the difference comes from when using hardware cursors.
I think the difference mort96 is seeing is the cursor update throttling. If it updates at the same rate of the refresh rate then it's a crapshoot where in that interval it hits relative to vsync, with a worst case additional latency of the frame rate. X11 updates the cursor whenever it moves, so on scanout it's always where you expect it to be, even if the motion was almost immediately before vsync.
I should mention that in the past there's been problematic interactions on amdgpu with atomic updates of the cursor and display planes. This resulted in either one being starved from changing, causing stuttering in one or the other depending how it was handled. IIRC, that's why the throttle is there. You could try MUTTER_DEBUG_ENABLE_ATOMIC_KMS=0 to see if they only added the throttle to the atomic code path.
I wonder if the latency is due to the compositor receiving a damage event for the area of the cursor, thus being a frame late to render. But probably that wouldn't be an issue with hardware cursors.
It should be easy to create something better than X11. And if not pour more resources into X11 developement. Wayland on the other hand is a complete failure on all fronts. It should be abandoned ASAP.
What exactly is better? I can't even think of a single thing.
It has worse performance on almost all metrics. Heavy fragmentation. Very limited and incomplete API measured in functionality but still extremely complicated to work with as developer. Extremely slow development. The only thing that it has over X11 is better funding (for whatever reason).
Are we really pretending multi-monitor on X is not an absolutely shit show? Try mixed DPI and VRR in one of the monitors and see how you get that working well.
Now drag a window from one monitor to the other. The choice on X is to have both monitors run with the same specs (well, not for everything but yeah) or to have two completely separate screens at the X level which means they're isolated from each other and you can't move thing between them.
Thank you! X just plain doesn't work for that usecase, you'll either get blurry text on one of the monitors or a tiny window with mixed DPI. It fundamentally doesn't work.
Reading some of this thread has made me feel like I'm going insane. I have a wildly different experience of both X and Wayland from a lot of people in this thread it seems.
The VRR issue can be solved with a simple patch that never got merged into mainline (no idea why). Hence it's not inherently a X11 problem and honestly also a rather niche problem compared to the humongous deficits Wayland brings.
Mixed DPI works perfectly fine on X11 if the toolkit supports it. The xrandr protocol provides all the necessary dpi information. GNOME simply chose not to support it on X11.
This is excellent. Too often people guess at things when they could be more empirical about them. Ever since I learned the scientific method (I think 3rd or 4th grade) I was all about the 'let's design an experiment' :-).
Let me state up front that I have no idea why Wayland would have this additional latency. That said, having been active in the computer community at the 'birth' of X11 (yes I'm that old) I can tell you that there was, especially early on, a constant whine about screen latency. Whether it was cursor response or xterm scrolling. When "workstations" became a thing, they sometimes had explicit display hardware for just the mouse because that would cut out the latency of rendering the mouse in the frame. (not to mention the infamous XOR patent[1])
As a result of all this whinging, the code paths that were between keyboard/mouse input and their effect on the screen, were constantly being evaluated for ways to "speed them up and reduce latency." Wayland, being something relatively "new" compared to X11, has not had this level of scrutiny for as long. I'm looking forward to folks fixing it though.
[1] https://patents.google.com/patent/US4197590
Display devices (usually part of your GPU) still have "explicit display hardware just for the mouse" in form of cursor planes. Later this has been generalized as overlay planes.
Planes can be updated and repositioned without redrawing the rest of the screen (the regular screen image is on the primary plane), so moving the cursor is just a case of committing the new plane position.
The input latency introduced by GNOME's Mutter (the Wayland server used here) is likely simply a matter of their input sampling and commit timing strategy. Different servers have different strategies and priorities there, which can be good and bad.
Wayland, which is a protocol, is not involved in the process of positioning regular cursors, so this is entirely display server internals and optimization. What happens on the protocol level is allowing clients to set the cursor image, and telling clients where the cursor is.
Protocols can bake in unfortunate performance implications simply by virtue of defining an interface that doesn't fit the shape needed for good performance. Furthermore, this tends to happen "by default" unless there is a strong voice for performance in the design process.
Hopefully this general concern doesn't apply to Wayland and the "shape" you have described doesn't sound bad, but the devil is in the details.
I don't think the Wayland protocol is actually involved in this. Wayland describes how clients communicate with the compositor. Neither the cursor, nor the mouse are a client, so no where in the path between moving the mouse and the cursor moving on screen is Wayland actually involved.
The story is different for applications like games that hide the system cursor to display their own. In those cases, the client needs to receive mouse events from the compositor, then redraw the surface appropriately, all of which does go through Wayland.
According to Asahi Lina, X does async ioctl that can update the cursor even during the scanout of the current frame, while Wayland does atomic, synced updates on everything, cursor involved, which has the benefit of no tearing and the cursor's state is always in sync with the content, but it does add an average of 1 more frame latency (either updates just in time for the next frame), or it will go to the next frame.
This is not what Wayland does, it is what a particular display server with Wayland support decided to do.
Second, just to be clear, this only discusses mouse cursors on the desktop - not the content of windows, and in particular not games even if they have cursors. Just the white cursor you browse the Web with.
Anyway, what you refer to is the legacy drm interface that was replaced by the atomic one. The legacy interface is very broken and does not expose new hardware features, but it did indeed handle cursors as its own magical entity.
The atomic API does support tearing updates, but cursor updates are currently rejected in that path as drivers are not ready for that, and at the same time, current consensus is that tearing is toggled on when a particular fullscreen game demands it, and games composite any cursors in their own render pass so they're unaffected. Drivers will probably support this eventually, but it's not meant to be a general solution.
The legacy API could let some hardware swap the cursor position mid-scanout, possibly tearing the cursor, but just because the call is made mid-scanout does not mean that the driver or hardware would do it.
> but it does add an average of 1 more frame latency
If you commit just in time (display servers aim to commit as late as possible), then the delay between the commit and a tearing update made just before the pixels were pushed is dependent on the cursor position - if the cursor is at the first line shown, it makes no difference, if on the last shown, it'll be almost a frame newer.
Averaging cursor positions mean half a frame of extra latency, but with a steady sampling rate instead of rolling shutter.
Proper commit timing is usually the proper solution, and more importantly helps every other aspect of content delivery as well.
> This is not what Wayland does, it is what a particular display server with Wayland support decided to do.
To the user that's an irrelevant distinction.
I also don't think this matters that much - with X11 this was optimized in one place by people that care about such details while with Wayland now every compositor developer (who in general are much more interested in window management policty) needs to become a low leve performance expert.
> Second, just to be clear, this only discusses mouse cursors on the desktop - not the content of windows, and in particular not games even if they have cursors.
Games can and sometimes do use "hardware" cursors as well - after all, they also care about latency.
Sure, it's what Gnome Wayland does, but the Wayland protocol does sort of mandate that every frame should be perfect, and the cursor has to match the underlying content, e.g. if it moves over a text it has to change to denote that it is selectable.
I do believe it is a useful tradeoff, though.
> Anyway, what you refer to is the legacy drm interface that was replaced by the atomic one. The legacy interface is very broken and does not expose new hardware features, but it did indeed handle cursors as its own magical entity.
Isn't it what many people refer to as "hardware cursor"? Is it possible for Wayland to rely on such a feature?
Wayland display servers will already be using what is commonly referred to as hardware cursors.
They just use the atomic API to move a cursor or overlay plane, which reflect how the hardware handles things. That the legacy API exposed a specialized cursor API was just a quirk of the design.
Note that planes are a power optimization more than anything else, as it allows e.g. the cursor to move or for decoded video frames to be displayed while GPU's render-related units are powered down. Drawing the cursor move, even though the render task is a rounding error, would require the render-related units to be on.
Thank you. So, if I get this right, the cursor position, which is what the video card needs to position the mouse pointer picture on the screen as an overlay to the actual framebuffer, isn't updated asynchronously to the screen update (ie. whenever the mouse is moved), but instead each time a frame is being rendered, and thus the pointer is only moved at these times, which may avoid tearing (though I don't see why) and other nasty effects, yet introduces a small rendering lag.
I don't know however if the mouse pointer picture is still handled the VESA way, or if GPUs video cards nowadays have a more generic API, or what.
There really isn't such a thing as "the actual framebuffer". Instead the display hardware can do composition during scanout from a set of buffers at a set of positions with varying capabilities. These buffers then just being arbitrary dmabufs.
It doesn't give a damn if you give it 2 buffers and one contains a mouse cursor and the other everything else or if you give it 2 buffers and one is everything including the mouse and the other is a video, allowing complete power collapse of the GPU rendering units.
Often they support more than 2 of these as well, and with color conversions, 1D & 3D LUTs, and a handful of other useful properties. Mobile SoCs in particular, like your typical mid/high end snapdragon, actually have upwards of a dozen overlay planes. This is how Android manages to almost never hit GPU composition at all.
On desktop linux all of these go through the drm/kms APIs.
Well, GPUs from the big players do give a damn as they tend to have surprisingly limited plane count and capabilities. It is often just a single primary, cursor and overlay plane, sometimes the latter is shared across all outputs, and sometimes what the plane can do depends on what the plane it overlaps with is doing.
Mobile chips are as you mention far ahead in this space, with some having outright arbitrary plane counts.
Although desktop GPU plane counts are much more limited, it's not as restricted as you're portraying. Here's the AMD SoC in the Steam Deck, for example: https://github.com/ValveSoftware/gamescope/blob/master/src/d...
Even though it's only 3 planes, they are relatively feature-rich still. In a typical desktop UI that would indeed be primary, cursor, and video planes. But if the system cursor is hidden, such as in a game, that frees up a plane that can be used for something else - such as the aforementioned game.
What you are showing is just a standard color pipeline, which is the bare minimum for color management.
On AMD in particular, the cursor plane must match several aspects of any plane it overlaps with, including transform and color pipeline IIRC.
The AMD SoC in my laptop (much newer than the steam deck) only exposes two overlay planes to share among all 4 display controllers. Intel used to have a single overlay plane per display.
The Raspberry Pi 5 on the other hand intentionally limited the exposed overlay planes to "just" 48, as it can be as many as you have memory for.
You can peek at the reported capabilities of various devices here: https://drmdb.emersion.fr/
> which may avoid tearing (though I don't see why)
What I meant here is that I didn't see why asynchronous updates may introduce tearing; but my tired brain couldn't quite formulate it properly. And to answer that, it's clear to me know that an update of the pointer position while the pointer sprite is being drawn would introduce a shift somewhere within that sprite, which is, I suppose, the tearing discussed (and not a whole frame tearing).
> I don't know however if the mouse pointer picture is still handled the VESA way, or if GPUs video cards nowadays have a more generic API, or what.
Also, the VESA interface doesn't seem to handle mouse pointers, it's something that was available in the VGA BIOS, to provide a uniform support for this feature, as each vendor most likely did it their own way.
It seems like it should be possible to do the X async method without tearing.
When updating the cursor position, check if line being output overlaps with the cursor. If it isn't, it's safe to update the hardware cursor immediately, without tearing. Otherwise, defer updating the cursor until later (vblank would work) to avoid tearing.
Of course, this assumes it's possible to read what row of the frame buffer is being displayed. I think most hardware would support it, but I could see driver support being poorly tested, or possibly even missing entirely from Linux's video APIs.
This would have to be done by the kernel driver for you GPU. I kind of doubt that it's possible (you're not really scanning out lines anymore with things like Display Stream Compression, partial panel self refresh and weird buffer formats), and doubt even more that kernel devs would consider it worth the maintenance burden...
I believe this is called "racing the beam".
But given that different displays work differently, I'm not sure it would worth the hassle.
I mean at some point it's a fundamental choice though, right? You can either have sync problems or lag problems and there's a threshold past which improving one makes the other worse. (This is true in audio, at least, and while I don't know video that well I can't see why it would be different.)
Wayland support tearing too, not sure if gnome do but KDE if the application supports, it can draw with tearing for less latency in full screen
Well there are opportunities to do the wrong thing though, like sending an event to the client every time it get an update. Which means that high poll rate mice would DDOS less efficient clients. This used to be a problem in Mutter, but that particular issue was fixed.
Yeah, Wayland isn't designed in such a way that would require any additional latency on cursor updates. The Wayland protocols almost entirely regard how applications talk to the compositor, and don't really specify how the compositor handles input or output directly. So the pipeline from mouse input coming from evdev devices and then eventually going to DRM planes doesn't actually involve Wayland.
Software works best when the developers take responsibility for solving user's problems.
> Wayland, which is a protocol
This is wayland's biggest weakness. The effect is diffusion of responsibility.
You're kind of getting tripped up on terminology. The OP didn't measure Wayland; they measured GNOME Shell which does take responsibility for its performance. Also, I'm not aware of any latency-related mistakes in Wayland/Weston (given its goal of tear-free compositing).
> You're kind of getting tripped up on terminology.
I'm not. My comment doesn't address the latency of gnome shell. I understand the boring technical distinctions between wayland and wayland client libraries and wayland display servers and gnome shell and mutter and sway, blah blah blah. Much like I understand that Linux is a kernel. That it is inspired by UNIX, but it is technically not a UNIX. I also understand that if someone describes themselves as a Linux user they probably don't just mean that they have a Android phone or that the display controller in their dishwasher or wireless access point happens to include Linux the kernel.
The "well acksually wayland is just the name of the protocol" that emerges whenever a problem is brought up is a symptom of the underlying problem with wayland the system. The confusion that gives rise to these deflections is also a symptom of that problem.
By Conway's law systems end up resembling the organisations that produce them. In this way the design of wayland the system seems to be designed by people who don't want to work together. I can see a parallel with microservice architecture.
The distinction between protocol and implementation IS significant here.
Imagine comparing HTTP1.1 vs HTTP3. These are protocols, but in practice one compares implementations. I can pick curl for http1.1, but Python for http3, and http3 would very likely measures as slower.
Is that the protocols fault?
>> ”well acksually wayland is just the name of the protocol" […] is a symptom of the underlying problem
> well acksually
It’s not the protocol’s fault, but the system and organisation that brought it.
Which brings up the other problem that Wayland introduced, that instead of one incredibly old inscrutable software stack doing these things there are now five (and counting!) new and insufficiently tested software stacks doing these things in slightly different ways.
It would have been nice if KDE and Valve could (would?) work together to reimplement KWin's features on top of wlroots. That would have basically made Gnome the sole holdout, and I imagine they'd eventually have switched to the extended wlroots as well, or at least forked it.
There's also EFL and one other I'm forgetting at the moment, but yeah.
All this proves is that it's possible for a protocol to not be the determining factor; which says nothing about whether it's possible that it _is_ a determining factor.
You’re quite right. We’d need similar benchmarks done with other compositors.
I very much doubt that Wayland makes a difference for this test; Wayland is for IPC between the client and server. Moving the cursor around is done by the server, without needing to talk to the client.
> I understand the boring technical distinctions between wayland and wayland client libraries and wayland display servers and gnome shell and mutter and sway
I do not. Does anyone know a good but quick introduction into these concepts and the problems they cause or fix?
All I know is that Wayland was supposed to be faster by taking some slow parts out of the loop, but that doesn't seem to be working, according to these figures.
As a user, why should I care?
I would just rather pay someone 100 bucks and not care whose fault it is. (And yes, that's why I don't use Linux on the desktop anymore, as much as I loved i3 and tiling environments.)
It's probably in a hundred places that all add up, all of which are no particular person's responsibility. So, that probably means that 10 people from 10 different projects need to get on a call or mailing list together and find a plan of attack.
But it won't happen. And people will keep wondering why Linux can't ever get a foothold on the desktop.
It's the biggest strength!
Every time someone complains about wayland there's someone informing you how it achtually it isn't.
Your biggest strength is also your biggest weakness.
The organisation of Wayland sounds great, but it is very hard to share optimised code between compositors since key parts that affect performance (in this case latency) are largely developed outside of any shared library code.
The "organisation" of Wayland reminds me of the UNIX wars; this is going to get worse before it gets better.
SVR4 Wayland anyone?
xref the time it has taken the Rust rewrite of the GNU coreutils and arguably coreutils is a much easier problem.
In the end, Poettering is going to swoop in and turn the graphics stack into another part of systemd.
See also AI discussions and holding a telephone incorrectly, also do we not own phones
It's no different from X11, which is also a protocol/spec with many implementations.
If there is no difference then how does the official reference implementation of Wayland, that nearly everyone uses daily, handle it? /s
The difference is that x11 was SK bag that you required a tons of plugins to have a working system(xorg) and no one could reliable maintain that mess of codes and plugins where every change could break another
The extension situation is already much worse in Wayland than in X11. In my X11 server there are about 25 extensions and only a handful are required for modern desktop functionality. Wayland already has well over a hundred extensions (protocols) and dozens of them have to be implemented by each compositor just to get basic stuff working.
https://wayland.app/protocols/
Funny seing as X.Org is still being maintained and still works to this day.
Works: Yes
Maintained: Only for bugfixes, except by metux, who is working on starting a fork.
Could've fooled me! I've only ever seen Linux devices run Xorg. Maybe some embedded hardware runs an alternative, but Xorg seems to be the de facto standard for desktop Linux.
This isn't a big issue because Xorg splits a basic GUI into dozens of different daemons and services (compositors, window managers, input daemons) so it seems like there are different X11 implementations even when there's usually only Xorg.
Give Wayland a bit more time and it will also ossify to the point where it becomes unviable to create competing implementations. Most of the smaller compositors are already there.
wlroots appears to be the single ossified implementation of Wayland.
https://en.wikipedia.org/wiki/Category:X_servers
Only linux. I have run several different X servers on windows (3.1), OS/2, and such. But xorg is free and so that is what you use on linux.
Is it a weakness of the web that HTTP is just a protocol specification?
The "problem" with this in Wayland is that before people 'ran Xorg with GNOME on top", now they just run GNOME the same way they run Chrome or Firefox to use HTTP - it will take time for people to get used to this.
HTTP is to Wayland as the web is to what???
I think the answer is also Wayland. It's very confusing because there is wayland the protocol and wayland the system, where the protocol is a part of the system - but there is no separate name for it. When people discuss wayland they are very rarely talking about wayland the protocol, and are much more likely talking about wayland the system.
There are problems on the web, which originate from aspects of HTTP. It's an okay protocol, but clearly not intended for interactive applications.
Wayland as a just protocol... Isn't that the same argument they used when it shipped without copy/paste or a screensaver?
It's not an "argument", it's just a description of what Wayland is. But no, the correct protocol has had copy-paste since day one, and I dont remember there being issues with screensavers.
In the metaphor of a web server and a web browser, Wayland would be the HTTP specification. What you're usually interested in is what server you're running, e.g. GNOME's Mutter, KDE's Kwin, sway, niri, or what client you're running, e.g. Gtk4, Qt6, etc.
Wayland don't support screensaver and clipboard is managed by portals no?, idk why the complain tho, no one uses screensaver and clipboard being managed by a portal dsounds more logical than creating a protocol for that
Once upon a time XFree86 and Xorg updated the pointer directly in a SIGIO handler. But that's ancient history at this point, and nowadays I wouldn't expect Wayland and Xorg to have a hugely different situation in this area.
IIRC it all started going downhill in Xorg when glamour appeared. After the cursor rendering path wasn't async-safe for execution from the signal handler (which something opengl-backed certainly wouldn't be), the latency was worse.
I remember when even if your Linux box was thrashing your mouse pointer would stay responsive, and that was a reliable indicator of if the kernel was hung or not. If the pointer prematurely became unresponsive, it was because you were on an IDE/PATA host and needed to enable unmask irq w/hdparm. An unresponsive pointer in XFree86 was that useful of a signal that something was wrong or misconfigured... ah, the good old days.
> they sometimes had explicit display hardware for just the mouse because that would cut out the latency
That's still the case. The mouse cursor is rendered with a sprite/overlay independent from the window composer swapchain, otherwise the lag would be very noticeable (at 60Hz at least). The trickery starts when dragging windows or icons around, because that makes the swapchain lag visible. Some window composers don't care any longer because a high refresh rate makes the lag not as visible (eg macOS) others (eg Windows I think) switch to a software mouse cursor while dragging.
> The trickery starts when dragging windows or icons around
Cool. Never really thought about why they don't exactly keep up with the cursor.
How old is Wayland?
I'll be reading a dream of spring in my grave at this rate.
I understand I'm complaining about free things, but this is a forced change for the worse for so long. Wayland adoption should have been predicated on a near universal superiority in all input and display requirements.
Intel and AMD and Nvidia and Arm makers should be all in on a viable desktop Linux as a consortium. Governments should be doing the same because a secure Linux desktop is actually possible. It is the fastest path to showcasing their CPUs and 3d bling, advanced vector /computer hardware.
Wayland simply came at a time to further the delay of the Linux desktop, at a time when Windows was attempting to kill Windows with its horrid tiles and Apple staunchly refused a half billion in extra market cap by offering osx on general x86.
Wayland is a protocol. The problems people complain about are generally implementation details specific to GNOME or KDE or (in general) one particular implementation.
There's rarely any such thing as "universal superiority", usually you're making a tradeoff. In the case of X vs Wayland it's usually latency vs. tearing. Personally I'm happy with Wayland because there was a time when watching certain videos with certain media players on Linux was incredibly painful because of how blatant and obtrusive the tearing was. Watching the same video under Wayland worked fine.
Early automobiles didn't have "universal superiority" to horses, but that wasn't an inhibitor to adoption.
"Wayland is a protocol" is exactly the problem. Protocols suck; they just mean multiplying the possibility of bugs. A standard implementation is far more valuable any day.
With X11, it was simple: everybody used Xfree86 (or eventually the Xorg fork, but forks are not reimplementations) and libX11 (later libxcb was shimmed underneath with careful planning). The WM was bespoke, but it was small, nonintrusive, and out of the critical path, so bugs in it were neither numerous nor disastrous.
But today, with Wayland, there is no plan. And there is no limit to the bugs, which must get patched time and time again every time they are implemented.
X had garbage handling of multiple monitors and especially multiple monitors with different DPIs, and there was "no plan" to deal with that either. Nobody wanted to work on the X codebase anymore. The architecture bore no resemblance to the way any other part of the desktop stack (or hardware) works.
Most of the garbage aspect is because toolkits refuse to support multiple monitors on DPI with X11 with the argument that "Wayland is just around the corner", for decades now.
For example Qt does per-monitor DPI just fine on X11; it's just that the way to specify/override the DPI values just sucks (an environment variable).
This stupid decision is going to chase us until the end of times since Xwayland will have no standardized way to tell its clients about per-display DPI.
It's not useful if you have to specify a scaling factor before the application has started, when the application can move monitors.
This is something feasible on Wayland, X draws one large wide screen display.
X could do seveal different screens I did have this working once. However then moving an application to a different display was impossible (an app could do it but it was a lot of work so nobody bothered). I few cad programs supported two streens but they were seperate and the two didn't meet.
Most people want to drag windown between screens and sometimes even split down the middle. One large display supports that much easier so that is what everyone switched to in the late 1990
I was using it that way until about 2020. (Mint 13 MATE supported, but it seems that capability was lost somewhere along the line. Shame, because I have a dual monitor setup where the second monitor is often displaying the picture from a different device, so in that situation I absolutely cannot have applications deciding to open on the busy-elsewhere screen. I miss being able to set a movie running on one monitor and have it not disappear if I flipped virtual desktops on the other!)
Yes, separate screens would be a much better model for me as well. Much better than KDE randomly deciding to show KRunner on the turned off TV for some reason unless I manually disable the output.
X11 does a lot of things that are outdated now, and multiple independent screens is one of them. Ideally, you'd be able to select either independent screens or one big virtual screen, and still have a window manager be able to move windows between independent screens. I don't know how that would be achieved though.
X's "mechanism, not policy" has proven to be a failure. Without a consistent policy you end up with a pile of things that don't work together. "One big virtual screen" is a policy and it's one that works well enough.
Deciding what needs to be in the protocol and what should be in the application is never easy. It's important to be able to iterate quickly and avoid locking in bad practices. I think that's what Wayland tried to do by making everything a fine-grained extension, but it doesn't really work like that as the extensions become mandatory in practice.
For the record: you can specify a different DPI _for each monitor_ for Qt X11. You just cannot change it after the program has started, which is exactly the limitation I was referring to.
But you can definitely move windows to another monitor and Qt will use the right DPI for it. It is the same behavior as Wayland. "One large wide screen display" is exactly how Wayland works...
X11 used to provide separate displays, but at some point due to hardware changes (and quite probably due to prominence of intel hardware, actually) it was changed to merged framebuffer with virtual cut out displays.
In a way, Wayland in this case developed a solution for issue its creators brought into this world first
It can still provide seperate displays. The problem is you couldn't do something like drag a window from display 1 to 2°. IIRC it's also annoying to launch two instances of a program on both displays. The hacky merged framebuffer thing is a workaround to these problems. But you can have independent DPIs on each display.
° For most programs.
Yeah there were certainly tradeoffs. It's much harder to use separate displays now, though - last time I tried, I could address the two displays individually (":0.0" and ":0.1") if I launched X on its own, but something (maybe the display manager?) was merging them into a single virtual display (":0") as soon as I tried to use an actual desktop environment. (This was was Mint 20, MATE edition, a few years ago - I gave up and reverted to a single-monitor setup at that point.)
> It's not useful if you have to specify a scaling factor before the application has started, when the application can move monitors.
Windows does this. Try to use in Windows 2 monitors with 2 different scalling factors. It is hit or miss. 100 and 150 works. 100 and 125 doesn't.
Yes. It just proves that all you needed is a better way to specify the per-monitor DPI, one that can be updated afterwards, or even set by the WM on windows.
This is not "just fine" in my book: https://lobste.rs/s/oxtwre/hard_numbers_wayland_vs_x11_input...
>Nobody wanted to work on the X codebase anymore.
That, i think, is the main issue. Nobody wants to work with GTK1, or GTK2, or GTK3 anymore. Nobody wants to work with QT1, or QT2, or QT3 or QT4 anymore. Everybody wants the new shiny toy. Over and over again.
It is CADT all over. Earlier X was developed by an industry consortium. Now Wayland is a monopoly pushed by RedHat.
Exactly, same situation as with audio. Instead of improving the OSS API we got ALSA, PulseAudio, and now PipeWire (and a bunch of less mainstream alternatives) -- all with their own weird issues. And of course it's application (or at least toolkit) developers that bear the cost of this churn, not the developers that get to work on their fancy new NIH tech that they'll then abandon before being 100% stable for yet another reimplementation.
> Now Wayland is a monopoly pushed by RedHat.
Pushed but still it seems it flails and takes too long to do the basic stuff
RedHat is still mostly a server side company isn't it?
Can we trust them to "care" about desktops?
Oh wait. There is no more RedHat. There is IBM.
They care about applicances and phone-likes. Which is what Wayland is geared towards. Desktop is just there to do the free testing for them.
Software has bugs and water is wet. Wait til you hear about HTTP, TCP, UDP, IP torrents, etc... and "simple" is not really a term I would designate to X11. I mean, its fine, but even just the ecosystem surrounding X is convoluted, outdated and absurd. Things like xinit, startx, .Xauthority, xresources, xhost etc... are all a mess.
> Wayland is a protocol. The problems people complain about are generally implementation details specific to GNOME or KDE or (in general) one particular implementation.
I feel like at some point this is a cop-out. Wayland is a protocol but its also a "system" involving many components. If the product as a whole doesn't work well then its still a failure regardless of which component's fault it is.
Its a little like responding to someone saying we haven't reached the year of linux on the desktop by saying: well actually linux is just the kernel and its been ready for the desktop for ages. Technically true but also missing the point.
Wayland. Solving yesterday's problems, tomorrow.
unlike x which couldn't solve yesterdays problems.
X is fine, most of the problems people bring up are niche and minor. Meanwhile, Wayland induces problems of its own, like breaking all sorts of accessibility systems and multi-window X applications with no solution in sight.
To be fair this wayland issue is also niche. Author of the linked article wrote that this is reported by really small percentage of users most of the people do not notice it.
It's not the toolkit (as they work on X11 and other OSes with the existing toolkits), it's Wayland that's the issue (there are a series of Wayland protocols to implement this, but they are being blocked).
The key detail in the “Wayland is a protocol” is that there are several other implementations, some of them extremely mature. The implementation being tested here isn’t exactly know to be a good one.
If there were a single Wayland implementation in existence, I’d agree with your sentiment.
It's the implementation backed by Red Hat, which is probably the largest company in the Linux graphics space and the one employing the most experts.
It's also the implementation most unsuspecting users will end up with.
What components? Wayland is literally an XML protocol that turns an XML file into a method of communication. libwayland-server and libwayland-client only handle communication and the internal event loop. Its completely up to the developer to write the implementations of these functions and register them in the server.
Then a client is going to query the server and ask request to do stuff via a unix socket. In fact, you don't even need libwayland, you can raw dog it over sockets manually. The idea is that there are standard protocols that can be queried and used,and you can implement this in any environment you want to. You could write the "frontend" in html and JS and run a wayland compositor on the web (which has been done [1]), you could do it with text or anything really, most people use some graphics stack.
[1] https://github.com/udevbe/greenfield
The components are:
- the compositor, of which there are multiple implementations (gnome, kde, all the wlroots compositors)
- the client, which often uses one of several toolkits (gtk, qt, several smaller frameworks, or even directly using the protocol)
- the wayland protocol (or rather protocols, because there are several extensions) itself
- other specifications and side channels for communication, in particular dbus.
Many issues (although I don't think the one in OP) are due to the protocol being underspecified, and so the client and compositor disagree about some semantics, or doesn't even have a standard way to accomplish something across all compositors.
Nit: Wayland isn't an XML protocol. The "calls" and their arguments are described in XML, but the data is transmitted in a fairly simple binary encoding.
I mean most of the things are the fault of a badly designed or non existent protocols:
-Problems with non western input systems
-Accessibility
-Remote control(took around 2 years to be stable I think?)
-Bad color management
Then there's the things that did work in x11 but not in wayland:
-Bad support for keymapping(the input library says keymapping should be implemented by the compositor, gnome says not in scope, so we have a regression)
-bad nvidia support for the first two years? three years?
While these things are compositor/hw vendor faults, the rush to use wayland and nearly every distro making it as default, forced major regressions and wayland kinda promised to improve the x11 based experience.
Yes, and to the parents point it was a CHANGE in protocol.
I get there was cruft in x. But the moment selected was a barrier to Linux desktop adoption precisely when the greatest opportunity in decades was present.
And the desktop was reimplemented.
Now in this period kde and gnome both decided to do rewrites, Ubuntu did their own desktop, got it up to snuff, and abandoned it. The lunacy wasn't just Wayland.
If we are complaining the gnome compositor sucks... I mean , should that be the goddamn reference implementation? What percent of desktops are gnome, 80% at least? If the gnome composting ready for primetime, then Wayland isn't ready for primetime.
>If the gnome composting ready for primetime, then Wayland isn't ready for primetime.
I use Sway, which uses a different compos[i]tor than Gnome. I would like to see similar results for wlroots, Sway's compositor, though I'm not actually interested enough to do the experiment (I guess that would be comparing Sway with i3). Cursor lag in Sway is not enough to bother me. I have on occasion used Gnome on the same machine(s), and never been bothered by lag.
As others have pointed out, Wayland is a protocol, not a compositor.
"As others have pointed out, Wayland is a protocol, not a compositor."
But the Wayland protocol requires a compositor, so here we are.
I have no idea of the difference(s) in performance between wlroots and Gnome's compositor. Protocols do not have performance, implementations do. If someone can prove that the Wayland protocol, on a certain set of tasks, and in a certain environment, has a better or worse performance than the X11 protocol, then it might be possible to make abstract comparisons on protocol.
Given that Gnome runs on both X and Wayland, it might be interesting to hear from the Gnome authors on the performance differences.
Nvidia support is still poor (at least on latest cards), I'm forced to use X or I get tons of glitches. I need the proprietary drivers for machine learning.
Not that I mind particularly, X is fine and everything works as expected. I can even have the integrated (amd gpu) take care of the desktop while the Nvidia gpu does machine learning (I need all the vram I can get, the desktop alone would be 100-150mb of vram) - then I start a game on steam and the nvidia gpu gets used.
Funnily enough I had wayland enabled by default when I installed the system and I didn't understand why I was getting random freeze and artifacts for weeks. Then I realized I was not using X11
Isn't this an issue with Nvidia not releasing open source drivers ?
I've used Wayland on Fedora for 16 years. Had to disable Nvidia GPU on laptop And use Intel GPU for first 11 years. Now have been using AMD GPU smoothly for 5 years. But I hear Nvidia is slowly getting back on the game. Let's hope they get stable support less than 20 years of wayland being in use on major desktop Linux distros.
> the rush to use wayland and nearly every distro making it as default,
Which rush? It has been done by only a small fraction of distros like Fedora, after years of development of the first wayland compositors. Fedora main purpose has always been to implement bleeding edge tech stuff early so that bugs get found and fixed before people using more stable distros have to suffer from it.
Nobody has been forced in any regression and x11 has continued to be available until now and there is no sign that the most conservative distros will drop x11 support anytime soon.
I have run Wayland since it was available for testing on Fedora Workstation and I have had zero problems inputting Japanese and Chinese.
With regards to accessibility, what problems have you had exactly?
> With regards to accessibility, what problems have you had exactly?
Fedora shipped a broken screen reader for 8 years.
Is there a bug report for this anywhere? Does everything work now?
I vaguely remember someone made a blog post about it and got piled-on. I think it's now fixed or being fixed.
Edit: Found it https://ar.al/2024/06/23/fedora-has-been-shipping-with-a-bro...
> I have had zero problems inputting Japanese and Chinese.
That may be fine.
Neo2 does not work. Neo has 3 modifier keys, Gnome/Mutter/Wayland/Whatever does only support two. Neo2 has a compose key, Wayland does not honor it.
https://neo-layout.org/
I use mod4 for navigation (arrow keys, site up) and compose for Slavish (read Polish) input (źżąę)
2 finger scroll doesn't work on my thinkpad model.
Not a bug, apparently. https://gitlab.freedesktop.org/libinput/libinput/-/issues/10...
I think that gnome has had built-in IME, but at least for a long time, it wasn't possible to use a third party system with gnome, or use gnome's with other compositors. And I'm pretty sure the situation was the same for sreen readers and on-screen keyboards. The wlroots project created their own protocols to support external applications to provide such features, since that is out of scope for a compositor like sway, but there are still missing pieces.
> there was a time when watching certain videos with certain media players on Linux was incredibly painful because of how blatant and obtrusive the tearing was
It was because of the crap LCD monitors (5 to 20 ms GtG) and how they are driven. The problem persists today. The (Wayland) solution was to render and display a complete frame at a time without taking into account the timings involved in hardware (you always have a good static image, but you have to wait).
I tried Tails (comes with some Wayland compositor) on a laptop. The GUI performance was terrible with only a Tor browser open and one tab.
If you do not care about hardware, you will, sooner or later, run into problems. Not everybody has your shiny 240 Hz monitor.
Tearing hasn't been a thing under X.Org for me since the introduction of Option "TearFree".
> How old is Wayland?
About 16 years old, for comparison, X is 40.
Hm, X works fine since 20 years. Wayland is still a protocol after 16. /s
> Wayland adoption should have been predicated on a near universal superiority in all input and display requirements.
Totally agree. The people saying "Wayland is a protocol" miss the point. Wayland is a protocol, but Wayland adoption means implementing stuff that uses that protocol, and then pushing it onto users.
Measure twice, cut once. Look before you leap. All that kind of thing. Get it working FIRST, then release it.
You have to releaseethings like this in parts because it needs too many external people to do things to make it useful. Managing those parts is something nobody has figured out and so people live you end up using it before it is ready for your use and then complaining.
When I say "release" I mean "release to users". You can release stuff to other developers, no problem. But it should all come with large warnings saying "This is not for daily use". The failure is when distros like Fedora start not only shipping it, but then saying they're going to drop the working alternative before Wayland is actually ready to do everything that X does.
(Also, I don't use Wayland. I mean I tried it out but don't see any real benefit so I don't use it regularly.)
> You have to releaseethings like this in parts because it needs too many external people to do things to make it useful.
30 years ago X came with a server and some clients. Why it is so hard to do this, for wayland, today ?
This would be best solved by maintaining support for old protocols instead of reinventing the wheel. There is no chicken/egg problem when you have proper backwards compatibility.
> Wayland simply came at a time to further the delay of the Linux desktop
I can't tell if you are serious or not.
Serious
I don’t really agree, because I think there’s something of a chicken-and-egg effect here.
With complex projects like these that have to work well on such a wide array of hardware and configurations, lots of real world usage is required to achieve any level of refinement. Without widespread adoption, Wayland likely would have been stuck in experimental/toy status for much longer than it will as things are going currently.
>How old is Wayland?
This argument don't make sense because Wayland started as a hobby and not to replace x11, was after it got traction, other people/companies started contributing that it matter
>but this is a forced change for the worse for so long
Can you explain who is forced to do what in that context?
AMD states that bugs only get fixed for Wayland.
Coincidentally I have got a graphics driver that likes crashing on OpenGL (AMD Ryzen 7 7840U w/ Radeon 780M Graphics)
You're arguing against Wayland, but for a more secure Linux desktop? I recommend you spend more time getting to know the X11 protocol then, because it has plenty of design decisions that simply cannot be secured. The same people who used to develop XFree86 designed Wayland to fix things that could not be fixed in the scope of X11.
> the X11 protocol [...] has plenty of design decisions that simply cannot be secured.
I've been hearing this for over a decade now. I don't get it. Just because xorg currently makes different clients aware of each other and broadcasts keypresses and mouse movements to all clients and allows screen capturing doesn't mean it has to. You could essentially give every application the impression that they are the only thing running.
It might seem difficult to implement, but compare it to the effort that has gone into wayland across the whole ecosystem. Maybe that was the point - motivating people to work on X was too difficult, and the wayland approach manages to diffuse the work out to more people.
I was really bullish on Wayland 10 years ago. Not so much any more. In retrospect it seems like a failure in technical leadership.
X always had the capability to isolate clients, but it is not used it would need some work which nobody does because of Wayland.
Some aspects of the client isolation are used by default when doing X11 forwarding via SSH. A remote keylogger will not work for instance.
> You could essentially give every application the impression that they are the only thing running.
> It might seem difficult to implement
This is exactly what OS did (my daily driver): https://forum.qubes-os.org/t/inter-vm-keyboard-isolation/315...
It'll be challenging to even figure out which one of the things connecting to $DISPLAY is the real window manager. Good luck on your lonely[1] journey!
[1]: The people who actually developed Xorg are now working on various Wayland-related things.
> It'll be challenging to even figure out which one of the things connecting to $DISPLAY is the real window manager.
I suspect it would be less challenging than writing a whole new wayland server.
Off the top of my head, I'd use a separate abstract domain socket for the window manager including some UUID, and then pass that to the window manager when launching it.
You could create these sockets on demand - one for each security context. On linux typically a different security contexts will either have different UIDs - in which case filesystem permissions would be sufficient - or they have different mount namespaces - in which case you make different sockets visible in different namespaces.
For SSH forwarding you could have SSH ask the X server for a new socket for forwarding purposes - so remote clients can't snoop on local clients.
> Good luck on your lonely[1] journey! > > [1]: The people who actually developed Xorg are now working on various Wayland-related things.
This is what I mean by a failure of technical leadership.
"You could create these sockets on demand - one for each security context. On linux typically a different security contexts will either have different UIDs - in which case filesystem permissions would be sufficient - or they have different mount namespaces - in which case you make different sockets visible in different namespaces."
This is reminiscent of how Trusted Solaris[0] implements Mandatory Access Control (MAC) a la Orange Book[1].
[0] https://www.oracle.com/technetwork/server-storage/solaris10/... [1] https://public.milcyber.org/activities/magazine/articles/202...
> For SSH forwarding you could have SSH ask the X server for a new socket for forwarding purposes - so remote clients can't snoop on local clients.
SSH pretty much already does this. Per default (using -X) X11 forwarding is in untrusted mode, which makes certain unsafe X11 extensions unavailable. So remote clients already cannot snoop the whole keyboard input.
> Off the top of my head, I'd use a separate abstract domain socket for the window manager including some UUID, and then pass that to the window manager when launching it.
And then the window manager spawns whatever programs the user wants, and they end up sharing that $DISPLAY.
I'm not trying to claim that X is secure as is. The claim I'm making is:
1. There is nothing fundamentally insecure about the design of X that couldn't be fixed with a bit of effort.
2. The effort required would be significantly less than what has gone into Wayland over the last 16 years.
Fixing the security of the system certainly would involve changes to more than just xorg. Anywhere there is a security boundary you'd need to make modifications - but that's OK, most of these security boundaries are younger than wayland anyway.
This discussion of $DISPLAY seems like distracting minutiae. Sure it's a medium size problem that would need solving, but that's all it is.
To address your specific point: it could be the responsibility of the window manager to set $DISPLAY to a less powerful socket when starting processes. Ultimately it doesn't matter though, because if the display server isn't doing some sort of sandboxing then the spawned process can just ptrace xorg and do whatever it wants. X being secure only matters in the presence of a security boundary and in that case it would be the responsibility of whatever is setting up that boundary to create a less privileged socket. Whether that be ssh or flatpak or whatever.
So now we've reached the point of "you could secure X11 if you changed everything". Sure, fine.
Once again, for emphasis: nobody has stepped up to do that, in the history of X (outside of proprietary products, of unknown quality).
Wayland is not some outsider project competing against X11. It's the people who were developing Xorg, saying they're fed up with X11 and need a clean slate to fix its problems.
Here's my unpriviledged user trying to ptrace my Wayland compositor running as me:
Nearby, I have a Chromebook where a potentially hostile virtual machine can display Wayland windows seamlessly on the host. The worst the VM can do is fill bitmaps with offensive imagery.In a world where the people doing the work switched to developing the Wayland ecosystem, arguing about a "could" that requires changing every window manager seems like a guarantee X11 will not get improved. Feel free to put effort into it...
> I understand I'm complaining about free things, but this is a forced change for the worse for so long.
Then write code.
Asahi Lina has demonstrated that a single person can write the appropriate shims to make things work.
Vulkan being effectively universally available on all the Linux graphics cards means that you have the hardest layer of abstraction to the GPU taken care of.
A single or small number of people could write a layer that sits above Wayland and X11 and does it right. However, no one has.
GNOME has unredirection support, so I don't think this test is applicable to actual in-game performance.
A fullscreen app ought to be taking a fast path through / around the compositor.
This isn't about in-game performance, this is about the desktop feeling sluggish.
This article only tests one compositor for Wayland and one for X11. It also doesn't specify if vsync is enabled for either one. Some X11 compositors default to vsync off, whereas every Wayland compositor I've seen defaults to on. This alone would be enough to explain the latency differences measured here.
I like what's being done here but we need a more thorough experiment before we can start drawing conclusions.
Hardware cursor is still a thing to this day on pretty much all platforms.
On sway, if you use the proprietary Nvidia drivers, it is often necessary to disable hardware cursors. I wonder if there is something similar happening here. Maybe wayland gnome doesn't use hardware cursors?
Asahi Lina's comment on the topic: https://lobste.rs/s/oxtwre/hard_numbers_wayland_vs_x11_input...
I'm not sure where this absolute "tearing is bad" and "correctness" mentality comes from. To me, the "correct" thing specifically for the mouse cursor is to minimize input lag. Avoiding tearing is for content. A moving cursor is in various places along the screen anyway, which will include biological "afterglow" in one's eyes and therefore there's going to be some ghosted perception of the cursor anyway. Tearing the cursor just adds another ghost. And at the same time, this ghosting is the reason keeping cursor input lag to a minimum is so important, to keep the brain's hand-eye coordination in sync with the laggy display.
What about a cursor moving over an element that requires changing its look? (E.g. you go over a link?)
I guess you're asking what happens if the UI element can't be updated in the same cycle?
I'd say: it really doesn't matter at all, in fact it's even more argument to make the pointer as low latency as possible. The mouse pointer is your control element as the controller of the interface. The UI element (and everything else onscreen) is the computer's element. It is an interaction between two entities and there is no need to introduce an expectation that the computer be "zero frame delay" on UI elements. It certainly sucks if it's more than a few frames for it to react, but so be it. However for the mouse cursor itself, it should absolutely be immediate because it is the user's representation on this virtual surface, and as such should follow inputs without being unnecessarily impeded. (Case in point: some UIs intentionally add delay or effects [like a fade-in] to UI element response. It's a… "choice"… but sometimes it works.)
Ultimately, the mouse pointer is an extension of hand-eye coordination; the only thing worse than making it lag (which you can learn to adapt to, albeit it will be very annoying) is to make it lag inconsistently (which you can't adapt to and will grow frustrated with extremely quickly.)
And, unfortunately, from personal experience, the mouse pointer on Wayland gets choppy if system load is high. X11 was significantly better with this; with Wayland the compositor [kwin_wayland to be specific, might be a KDE issue] doesn't even seem to be "unnice" or have some kind of scheduling priority over other tasks — I'm not sure if I'm missing something there but that seems to be a glaring oversight on its own.
That is a great comment and everyone should read it. It also demonstrates a common truism that system goals dictate performance.
Tl;dr: X bad and tears but Wayland with 1.5 frame latency (or more) good. Now, if you have a monitor (or TV) with 30i refresh rate, you're screwed.
We don't even know if it's actually a 1.5 frame latency for real because the test didn't try to establish that. The author said it looked like it was 1.5 frames, but that could be a coincidence. It just looks like a 1.5 frame overhead. It could, in reality, be a constant time overhead rather than a function of frames and so would be fine with a lower frame rate.
Tearing would affect everyone that uses a computer with X11 but your proposed example of a TV with 30i refresh rate would only affect the tiny subset of users that use a CRT television as a monitor, right?
I don't think there was ever a 30i screen. GP probably meant 60 interlaced, which needs two full scans for a change, so an effective 30 full frames per second, but enough for 60 half images. For a Wayland compositor and the entire hardware stack would treat it as 30hz only if it really wanted to avoid interlace artifacts, but I don't recall that being a configuration that existed in my lifetime.
1.5 is the worse case, and implementation specific
Reminds me of how we used 'dir' command to test computer speed by watching how fast it would scroll by.
Or later, seeing how long it took to load /usr/bin in a filemanager.
> Let me state up front that I have no idea why Wayland would have this additional latency
It's not really hard to guess at - it's probably caused by abstraction layers resulting in multiple layers of buffering. Of course, it's all just speculation until proven by experiment, so good on TFA for doing some of that.
Retro hardware from the C64 or NES era was single-tasking with such predictable timing you could change the video resolution partway through the screen refresh and have two resolutions on screen at once. If you want, you can check the player input and update Mario's position right before he gets scanned out - the minimum possible latency by design is zero frames. (Of course, in practice the whole screen data is buffered up during vblank, which is the same latency as using a framebuffer. Also the NES didn't allow changing video registers outside of blanking periods. But the C64 did.)
X11 isn't that close to the metal, but it's designed with a single level of buffering (assuming a compositing window manager isn't used). There is a single framebuffer covering the whole screen. Framebuffer updates are completely asynchronous to scanout, so there is around 0.5 frames of latency on average between when a pixel is changed and when it's scanned out. Apps borrow pixels from this big global framebuffer to draw into. An app sends a "draw rectangle" command relative to its own borrowed space, and the server calculates where the rectangle should go on the screen, and draws it into the framebuffer, and then with an average latency of half a frame, that part of the framebuffer is scanned out to the screen.
On Wayland, there are more layers. The app has to draw into its own pixel buffer (noting that it is probably double- or triple-buffered and has to redraw the entire window rather than relying on non-changing stuff to still be there) and then once the compositor receives the buffer from the app, it has to do the same thing again, copying all the per-app buffers into a big screen-size buffer, which is probably also double- or triple-buffered, and once the vblank hits, the most recent full frame is drawn to the screen. It's just more work with more layers, and there's no surprise that latency is higher. It's hardly the first or 100th time that an elegant software architecture resulted in the computer having to do more work for the sake of keeping the architecture pure.
Something important to note is that tearing reduces latency. If new data is available while the scanout is partway through a frame, you have the choice to draw it immediately, causing tearing, or wait until the next frame, increasing latency. Always doing the high-latency thing is an explicit design goal of Wayland. (A third theoretical choice is to always time things precisely so that the new frame is calculated right before the end of vblank, but... in a modular multitasking architecture with apps written by different organizations and unpredictable CPU load, good luck with that.)
Now the really big caveat: I'd be surprised if GNOME's X11 WM wasn't compositing. If it's compositing, then X11 behaves similarly to Wayland but with even more IPC and what I just said is no reason it should have one frame less latency on average. Still something to think about though.
> Wayland, being something relatively "new" compared to X11, has not had this level of scrutiny for as long. I'm looking forward to folks fixing it though.
Part of the problem will doubtless be the USB (and Bluetooth) stacks including the device hardware and firmware. When keyboards and mice were serial devices with their own interrupt making the code path fast was achievable. I'm not so confident that modern peripheral stacks can be made to run with the same priority. It becomes even more challenging for devices sitting on a bus with multiple device classes or multiple protocols between the device and driver (USB -> Bluetooth -> Mouse).
I hope devices can be sped up but we're a long way from a keypress triggering an interrupt and being handled in tens of milliseconds[0].
[0] https://danluu.com/input-lag/
The article you're commenting on is about someone running X11 and Wayland on the same computer with the same mouse and experiencing higher input latency on Wayland. I don't think differences between serial and USB mice are relevant here.
As a systems I guy I completely agree with this. One of the things that was interesting about X11 is that it draws an API "wall" around the stuff involved in presentation. Many folks don't remember this but we had X "terminals" which was a computer system that did nothing else except render the screen and handle the input.
In that compartmentalization there was nothing else competing for attention. You didn't get USB bus contention because there was just the one mouse, ever, on the line between the user and the X11 server in the X terminal.
USB devices today are dummy fast, USB 3.1 signals at 10 GHz. You just need to check the USB queue more often :D
Alternatively you move straight up to USB 4, where we get pci-e interrupts again! :)
> USB devices today are dummy fast,
In theory, yes. Not so, in practice.
> USB 3.1 signals at 10 GHz.
Yes but a mouse works at 12 MHz.
For anyone who uses ffmpeg for this type of per frame analysis, `ffmpeg -skip_frame nokey -i file -vsync 0 -frame_pts true out%d.png` will get the "presentation time" of each frame in the video. That's more precise than just dumping frames and calculating timestamps. You can also do something similar in a web browser by playing a <video> and using `requestVideoFrameCallback()`. Though, you might need to set `.playbackRate` to a low value if the computer can't decode all the frames fast enough.
> With my 144Hz screen,....Wayland, on average, has roughly 6.5ms more cursor latency than X11 on my system...Interestingly, the difference is very close to 1 full screen refresh. I don't know whether or not that's a coincidence.
The fact that the latency is almost 1/144th of a second means that it might become 1/60th of a second on standard 60Hz monitors. This is hard to notice consciously without training, but most people can "feel" the difference even if they can't explain it.
> The fact that the latency is almost 1/144th of a second means that it might become 1/60th of a second on standard 60Hz monitors.
My guess: the "true" numbers are close to 2.5 (half a frame of random phase of when the mouse is touched vs. refresh, plus 2 frames to move cursor) and 3.5. If you throw out the low outlier from each set you get pretty close to that.
(of course, the 125Hz mouse poll rate is another confound for many users, but this guy used a 1KHz mouse).
> This is hard to notice consciously without training, but most people can "feel" the difference even if they can't explain it.
Yah. 7ms difference is not bad vs 16.6ms is starting to be a lot.
IMO, we should be putting in effort on computers to reach 1.6 frames of latency -- half a frame of random phase, plus one frame, plus a little bit of processing time.
To have a compositor not introduce a frame of latency more or less requires it to race the beam, which has definitely been suggested[1], but you can see how it’d be difficult, and so far no operating systems have tried as far as I know. And as for good frame pacing support in UI toolkits (and not just game engines), well, one can dream. Until both of these are in place, 2.5±0.5 seems to be the hard limit, the question here is more where the Mutter is losing another frame (which even the greats tell us[2] is not hard to do by accident).
[1] https://raphlinus.github.io/ui/graphics/2020/09/13/composito...
[2] http://number-none.com/blow/john_carmack_on_inlined_code.htm...
I’ve read these arguments quite a few times and always found them a bit questionable. Sure, if everything is driven by the vblank time (or any other clock that counts in frames), it makes sense. But that’s a silly approach! There is nothing whatsoever special about allocating one full frame interval to the compositor to composite a frame — if it takes 16ms to composite reliably, it will take 16ms to composite reliably at 30Hz or 60Hz or 144Hz. So shouldn’t the system clock itself on a time basis, not a frame basis?
Put another way, a system fast enough to composite at 144Hz should be able to composite at 60Hz while only allocating 1/144 seconds to the compositor, which would require offsetting the presentation times as seen by the compositor’s clients by some fraction of a frame time, which doesn’t actually seem that bad.
It gets even better if variable refresh rate / frame timing works well, because then frames don’t drop even if some fraction of compositing operations are a bit too slow.
I assume I’m missing some reason why this isn’t done.
Actually, this is sometimes done exactly as you describe. See "AMD Anti-Lag" or "Nvidia Ultra Low Latency"/"Nvidia Reflex Low Latency".
We have multiple processes on a time-share system. This mouse is updating at 1kHz, it just doesn't make sense consistently rerendering everything when the screen aint updating.
We need to cut some deadline and doing it at vsync is the easiest way
https://news.ycombinator.com/item?id=42835019
A strong argument that it's never more than one frame's worth of latency.
I found low latency terminals make a big improvement to even simple tasks like typing.
I was pretty shocked by the eyestrain difference I felt going from 30hz to 60hz with a 4K monitor while only doing coding tasks (i.e. text and mouse, no real graphics or animations).
I recently went from 60 to 144, also on a 4k (27") monitor, and to me too it has been a very noticeable upgrade. No tearing of any kind when dragging windows around, and scrolling feels buttery smooth.
what would you count as low latency terminal?
See https://beuke.org/terminal-latency/. Single-digit milliseconds I’d say. These numbers are minus the keyboard and display latency.
Some of those include display latency (e.g. Konsole) because you can't throw lazy rendering out of some terminals.
Has anyone run this test on Ghostty?
I get a "Cannot detect the reference pattern" error when I try with Ghostty with Typometer.
thanks!
It's almost certainly because of one extra frame of buffering between the mouse move and the screen. Vsync can cause this, but it should be possible to get vsync with just double buffering.
Note that the results will differ between compositors, different GPUs, and different configurations. This is somewhat less the case with X11 since there is only one X server implementation (that Linux desktop systems use, anyhow.)
I think there may still be the issue that many compositor/GPU combinations don't get hardware cursor planes, which would definitely cause a latency discrepancy like this.
TIL Wayland is 16 years old already; in a few years (I can't math) it'll be as old as X was when Wayland came out, but it seems it's still considered mediocre or not as good as X was.
Initial release of X was June 1984, and Wayland was first released in 2008 -- so it won't be until 2032 that Wayland is the same age. When people complain about Wayland being mediocre or "as good as X was" what they often mean is "Wayland doesn't support $very_specific_feature" or "my video card vendor's proprietary drivers aren't well-tested with Wayland and it makes me sad".
Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
I don't really have a stake in this argument, except I'm bone-tired of seeing people whine about Wayland when they're using software somebody else gave them for free. There are enough Wayland whiners that, by now, they could've banded together and started maintaining / improving X and (if their complaints and theories were correct) left Wayland behind.
Strangely -- even though X is open source and eminently forkable (we know this, because XFree86 -> X.org) it gathers dust and none of its proponents are doing anything towards its upkeep.
When someone shows up with "Wayland isn't as good as X, so here is my modernized fork of X anyone can use" -- I'll be quite interested.
> Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and
Are you saying that Wayland is only for developers? Are people not allowed to complain when The New Thing turns out to be less good than the supposedly Obsolete Thing?
> 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
I'm fully aware of that. I understand X11 has its limitations, and some of the goals of Wayland sound very appropriate for the modern PC environment, but if after 16 years, there are still many situations where Wayland does worse than X, that's not a great sign, and it will make people continue to use X.
I enjoy reading the whinging around Wayland by people who feel so entitled and so outraged. Every time they never fail to remind me of a previous coworker who was always using the same tactics: « this breaks my existing workflow, I will not adapt, I am all for trying new things but only if they are very exactly equal to what I already know and then what's the point, this is not how I have always worked the past ten years and I will not bulge ».
Not talking about the technical points of the whole X/Wayland considerations here, but a group of people in such debate is always as vocal as they'll do nothing and put unreasonably high expectations on other people to listen to them and have no choice but to agree. The fact that X.org's core team initiated Wayland and stopped X.org development is disregarded: these people Will Be Right against literally everything, they can't be reasoned.
My previous coworker was fired for gross incompetence in the end yet he never acknowledged anything wrong. He always knew better than literally the whole world that bash scripts run manually where superior to Terraform for the whole company's infrastructure. Yes, he was that stupid. I guess this is why he ended lying like hell on his LinkedIn profile, concealing he had had four employers in four years.
Seeing technical debates with such poor quality is worrying. If people with quantitative minds fail to entertain a fact-based discussion, how can we expect our political life to acknowledge there is only one reality and stop lying?
Before you voice your outrage, here are some facts that won't dis-exist just because you disagree with them:
- https://www.phoronix.com/review/x_wayland_situation
- https://ajaxnwnk.blogspot.com/2020/10/on-abandoning-x-server...
> what they often mean is "Wayland doesn't support $very_specific_feature"
My primary complaint with Wayland is that it is a textbook example of Brooks’s second-system effect. It forsook backwards compatibility (yes, there’s an X11 server but to my knowledge there is no way to just run Wayland and one’s X11 desktop environment).
> Strangely -- even though X is open source and eminently forkable (we know this, because XFree86 -> X.org) it gathers dust and none of its proponents are doing anything towards its upkeep.
I suspect that is because the X/Wayland guys have sucked all the oxygen out of that particular room. A newbie shows up and is told that X.org is legacy and he shouldn’t work on it, so … he doesn’t.
And of course X.org really is a bit of a disaster due to being written in C.
The core Wayland and X.org don't exist in a vacuum. I have written a lot of code over the years that works only on X, as have many others. I have not directly contributed to the X.org server, but have to the "wider X ecosystem". I will have to rewrite some of that. This is the case for many people.
"I have not directly contributed to the X.org server, but have to the "wider X ecosystem". I will have to rewrite some of that. This is the case for many people."
This is a fair point, but the quality of Wayland isn't really at issue for this -- Wayland could be much better than X by all accounts and it would still require you to rewrite software for it. (Assuming XWayland doesn't suit your needs, anyway.)
Yes, but motivation and enthusiasm are important when it comes to spare time projects done by hobbyists.
You're far more likely to get people to expend the requisite time and effort if it's indisputable that the end result will be worth it.
Strongly disagree. For operating system components backwards compatibility should always be a primary and non-negotiable goal. If the backwards compatibility isn't 100% transparent that's a quality issue.
A good part of why so may of us are cautious about Wayland is precisely that one or more of the things we rely on doesn’t support it. For example, my window manager doesn’t. I use that window manager every single day, and there is nothing out there which works the way it does. It’s a vital part of my workflow.
Does Wayland support the entire Xcompose system or an equally powerful alternative?
Wayland does genuinely sound interesting, if flaky and half-baked (it reminds me of systemd in that way). Certainly, the world doesn’t owe me a port of my window manager. But if the Wayland transition forces me to use GNOME then my life will be worse.
> There are enough Wayland whiners that, by now, they could've banded together and started maintaining / improving X and (if their complaints and theories were correct) left Wayland behind.
They already have. X is already more full-featured and stable than Wayland (yes it is missing certain niche features that Wayland has). Sometimes the most important thing you can do with a piece of software is not screw around with it.
And yet, X is as flawed as it can be. There's a problem with global shortcuts, that Xorg only fires an event on keyup/key release, rather than on first match on keypress. That is a protocol limitation, and fixing it means breaking a bunch of stuff. The complains about wayland are of that nature, but at least they are fixable.
Are they fixable? One of my beefs with Wayland is that every compositor handles things differently. How GNOME's does some setting is different from how KDE's is different from Sway's. There's no guarantee they've even implemented the setting either. In X11 you can always globally configure something for any WM the same way. Wayland by design you can't. It's not fixable. I think it was a really poor decision to do a protocol and just made an OS where things are often too fragmented even more fragmented.
> The complains about wayland are of that nature, but at least they are fixable.
Eh maybe. Ultimately you can never be sure something is possible without doing it.
I'm pragmatic here, if and when Wayland offers me a better experience than X then I'll use it. I just resent distros etc. pushing me towards it when it's currently a downgrade.
> When people complain about Wayland being mediocre or "as good as X was" what they often mean is "Wayland doesn't support $very_specific_feature" or "my video card vendor's proprietary drivers aren't well-tested with Wayland and it makes me sad".
...Yes? Wayland being a regression in terms of features and bugginess is kinda a sticking point.
> Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
If X works for someone and Wayland doesn't, none of that matters. It doesn't matter how much you insult X users, it won't make their usecases invalid or make Wayland good enough.
> I don't really have a stake in this argument, except I'm bone-tired of seeing people whine about Wayland when they're using software somebody else gave them for free.
It cuts both ways: Users aren't entitled to free work, and developers aren't entitled to their work being well-regarded. Giving software away for free has never meant that people can't point out its problems.
X also isn't wrking for some. If you care about tearing for instance x doesn't work.
From my standpoint as a software packager I have trouble believing that upending the entire video ecosystem on Linux was worth fixing this alleged "tearing" problem which is the main thing people bring up as the reason we all spent a decade doing this. (Along with some honestly fantastical worries about screen scraping -- the answer for that is open source.)
Honestly Wim did this right on the audio side. If it was so important to bring what Wayland has brought someone should have written a display server that talked to display clients just like the old server did and upgraded gracefully if the client had the new capabilities. You didn't even have to recompile software to use pipewire, let alone rewrite it.
Tearing is the easiest issue to demonstrate to people. Security is something you don't realize you care about until it is too late.
The real reason wayland was done is to put proprietary applications on the desktop. That's it.
Which is ironic since it's the proprietary stuff I have to use that is least likely to work on Wayland at this point...
More precisely, to put proprietary "apps" on an appliance. Windows desktop works fine for proprietary programs without any of the "security" features that Wayland-proponents want to push.
How so? Proprietary apps work fine on X. Or do you mean the security angle where flatpak wants to be able to isolate untrusted (including proprietary) apps?
untrusted are only the proprietary apps… if you can review the code they aren't untrusted :)
I generally trust FOSS to not be malicious. I don't trust it to be bug free when processing untrusted input. So no, I really do want to sandbox plenty of apps regardless.
Sandboxing is still a solution in search of a problem. The only actual valuable data on my computer is the personal stuff that even sandboxed apps are meant to manipulate anyways. Everything else in the system I can download from the Internet for free, but my web browser still has to interact with my passwords and my spreadsheet with my finances, and sandboxing can't fix that.
> my web browser still has to interact with my passwords and my spreadsheet with my finances, and sandboxing can't fix that
True, but sandboxing can fix your browser being able to read your spreadsheets without asking, or your spreadsheet having network access. Just because you have to work with programs and data doesn't mean they're all in the same security domain.
I haven't seen any tearing in X in at least 5 years - with nvidia, nouveau and intel drivers. I don't think I'm doing anything special to achieve this, so your comment seems like a strange assertion to me?
I agree that X11 has some faults, and I really want Wayland to work, but everytime I try it, something ends up being broken.
I do see heavy tearing when watching videos under i3 on Ubuntu LTS and a 60 Hz monitor. So what? I just don't care. Tearing in videos does not hinder me from getting my work done. Watching videos is just wasting time most of the cases anyway.
> So what? I just don't care. Tearing in videos does not hinder me from getting my work done. Watching videos is just wasting time most of the cases anyway.
It's fine for you to not care about that usecase, but other people do care and that's also valid. There even exist devices that mostly exist to watch video and we'd really like Linux and some standard graphics stack to work on them.
Sure, I have no problem with people wanting to fix it.
My datapoint was just in reply to someone writing that tearing under X has not occurred for 5 years.
True; also various scaling problems, and its security situation. We're in an unfortunate situation where no option has everything. If you want to say "Wayland is better for what I need", or "Xorg is better for what I need", or even "Arcan is better for what I need", then that is 100% legitimate. I'm also cool with "the current problems with (X|Wayland|Arcan) are..." because that's constructive and useful for trying to improve them. It's only pretending that one doesn't have faults or insulting its users/proponents that I mind.
No tearing on my X.Org-based desktop for literally a decade.
Tiling window managers don’t tear.
Tiling window managers tend to tear more than any other desktop out there. Especially on x11, where you're expected to composite your own desktop without any (!!!) synchronization primitives, it's extremely common to get halfscan output from your windows or status bar. dwm, awesome and i3wm will all tear by default because there is no dispatcher, internal clock or even adaptive sync support.
Now, you might not notice the tearing if you've got V-sync properly configured. But V-Sync doesn't stop screen tearing, it just drops torn frames and decreases your refresh rate until your system can keep up with demand. It's an ugly hack on laptop hardware and an even uglier one on the desktop.
Well, I guess ‘window tearing’ doesn’t mean what I thought it did — or rather that it’s more than I thought. I always thought that it meant the artifact when dragging a changing window, which of course doesn’t happen if one doesn’t drag.
I don’t believe that I’ve ever noticed or been bothered by static tearing, but maybe I’m just used to it because I’m old and displays have always had it?
> It cuts both ways: Users aren't entitled to free work, and developers aren't entitled to their work being well-regarded. Giving software away for free has never meant that people can't point out its problems.
While true, this statement is also useless. On a meta level, what you're essentially saying is that users' rights to feel resentful, is more valuable than achieving an outcome.
> While true, this statement is also useless. On a meta level, what you're essentially saying is that users' rights to feel resentful, is more valuable than achieving an outcome.
It's more useful than calling X users "whiners" while ignoring their complaints. And no, I'm pointing out that the users are resentful because they're being pushed to an outcome that's worse and therefore not desirable to achieve.
You miss the point. See my reply here https://news.ycombinator.com/item?id=42870194
Sometimes that outcome is not the outcome that those users want. For example I'd stay on X11 forever because none of its problems are relevant to me.
The only relevant problem with X11 to me is that all of its developers are stopping to work on it and it won't run on any new hardware soon (and probably on old hardware too.)
You both miss the point completely. What I mean is that just by being resentful, you don't achieve the outcome that you want to achieve (not the outcome that they want to achieve).
The only way to achieve the outcome you want, is for someone to do the work. They're not going to do it. If you'd rather keep being resentful rather than do the work towards your own claimed desired outcome, then apparently you don't really value the outcome you so claim to desire; you value being resentful more.
> Approximately 99.999% of those complaints are uttered by people who 1) are not doing the work, 2) are not interested or capable of doing the work, and 3) do not understand or refuse to acknowledge that the Wayland and X developers are _mostly the same folks_ and they _do not want to work on X11 anymore_.
It's clarifying if you simply say "users."
People complain about the things they care about.
Wayland made some huge design changes, and those changes all have pros and cons. How do we tend to communicate about pros and cons? By complaining.
It sucks that Xorg's accessibility features are so baked-in that you must be compatible with them; because that demand can become a wall of assumptions. It sucks that Wayland leaves 100% of accessibility to the compositor; because that creates a lot of redundant work, and the best implementations get siloed into plasma or gnome, creating multiple smaller instances of the tight integration that made UX less flexible to begin with.
Using Xorg, you can run an incredibly lightweight window manager that never even thought about managing keymaps, adjusting mouse acceleration, setting the display resolution, etc., because that's not it's job. I can get absolutely everything I want from i3wm on Xorg... except fractional scaling. So far, it looks like plasma is the only one that has really figured that one out, and I still get a tiny pointer sometimes.
The best thing about free software is that we can choose how it works. The more tightly integrated the pieces are, the fewer choices are available. Fewer choices results in more complaining. Wayland may be the most significant instance of this pattern in Linux history. The second most significant instance was already present: Desktop Environments.
The biggest source of complaints I have is that the best aspects of plasma are trapped inside KDE. If you want the best Wayland compositor out there, you have to have it wrapped in an entire full-featured Desktop Environment.
---
Complaining is the best thing about Linux, so long as it's able to drive real change. The more stubborn and inflexible a system (and whoever designs it) is, the more disconnected complaints become. That's the real problem that deserves our attention.
> 16 years old [...] in a few years [...] it'll be as old as X was when Wayland came out
Lol no. X is from 1984: https://www.talisman.org/x-debut.shtml
That means Wayland is only 66% the age of X when Wayland came out, or you'd need 50% more of Wayland's life before its as old as X was.
If X is 40 and Wayland is 16,that means a difference of 24 years. Hence Wayland compositors have 8 years to work out the kinks. I am currently using Wayland via Plasma 6 and it works well enough but I don't have special needs, so I don't know how well... say... screen readers work.
That means in two more years it will be legal for X11 and Wayland to have a baby!
We can call it V to signify another step backwards and make it the default, people can just use the terminal until the first implementation crops up.
Maybe like we did we pulseaudio (throw it away and make pipewire) needs to be done with wayland.
I'd rather get something that actually works reliably this time.
What consenting adults do is their business, but still such an age gap feels a bit icky to me.
Wayland came out at the same time as the Windows compositor in Vista. Let's be generous and consider the next Windows version, 7, as having a "good/stable" compositor. So Wayland is 13 years behind Windows.
16 years old and there's still video tearing in the browsers. I try it with every Ubuntu LTS release, and then switch back to X in a few days. It solves problems I just don't have.
Another way to look at these figures is that in a few years a Wayland successor is due, born out of the same motivation as Wayland ie. lack of people with a desire to maintain legacy software. My expectation is that browser frontend stacks will eat the desktop completely at that point; it's not like there weren't many new desktop apps on Linux anyway.
I'm getting much higher framerates in Civ VI on linux since I switched to Wayland, so there is that. For the rest it just works for me, use it both professionally and privately.
> but it seems it's still considered mediocre or not as good as X was.
Because there was a big fracture and doubt in the community support for Wayland, not to mention alternatives like mir popping up and diverting resources.
A lot of things liked mir happened over the years and didn't go anywhere. Most tried to fix problems with x that the x developers said were not the real problems. When wayland came it was supported by people who knew the real problems with x. They have also proven to keep working - no surprise at they had a history of working on x.
They "claimed" to know the real problems with X. But the fact that wayland is still lacking features shows otherwise.
X is also lacking features. I use wayland and find it works great.
The fact that it works for you doesn't invalidate the fact that it doesn't work for a lot of people.
That goes both ways. X worked for me before I switched to wayland. However there are plenty of people who X didn't work for and they were complaining.
Yes and 16 years of work later we're worst off than we were 16 years ago.
Do you see how some might think it wasn't the best strategy?
Based on what exactly? On a single frame difference, which is easily explained by the fundamental point of Wayland: no tearing? It's simply that X puts the cursor asynchronously, potentially tearing it, while Wayland always renders a correct frame, but the mouse movements' result has to wait for the very next frame.
That's literally it.
Why this does not surprise me? every attempt to rewrite a working solution to make it more "modern, easy to maintain and future proof" rarely do so. It always end up slower, with more latency and lifted by faster hardware, not by faster software. Every 20 years a new generation comes weaker, pampered with the abtsractions of the previous generations who did the grunt work for them. After 3 generations of software developers all we have is library/framework callers and nobody really knows about performance and optimization.
This is not my experience. When I upgraded several of my computers to KDE 6 switching to Wayland, overall responsiveness and snappiness of the system increased very visibly. There are still features that Wayland lacks compared to X11, but I am willing to compromise in favor of its other benefits.
Same here, switching from i3 to sway resulted in a noticeably more responsive experience on my aging hardware. Of course, this is just an anecdote, and I could probably get the same results on X with some fiddling, but I do think the value of simpler and more modern systems is demonstrated here.
I’d be interested in seeing similar benchmarks done on x11+i3 vs sway.
There’s nothing Wayland-specific that would introduce this latency, so I wonder if wlroots/sway have any additions lag.
????
After upgrading to plasma6 from 5, all the desktop animations have started stuttering. Probably your hardware is too new.
On the other hand, I just did a fresh install of Artix Linux. Installed KDE just to have something functional while I get my tiling setup working. Boot into Plasma(Wayland) and it utterly shits itself, turning my main monitor on and off repeatedly until it finally crashes. So I pick Plasma(X11) instead and that just works.
In fact, in almost 2 decades of using Linux every day, I can't remember X doing anything similar. It's always just worked.
> In fact, in almost 2 decades of using Linux every day, I can't remember X doing anything similar. It's always just worked.
Well, we have very different memories then. Sure, X worked reliably once configured. But configuring it was a marathon in hell, as per by my memory, and all the litany of forum posts crying out for help all across the internet. Like, I have at one point had to rescue an install by changing back the config file without a screen at all!
I remember this too, but it was a really long time ago.
I mean, I've certainly had issues with X before, back when you had to configure it. But they were usually initial issues while figuring out the right configuration. And if you had a GPU where the choice was between shitty proprietary blob drivers and woefully incomplete open ones, you might have a bad experience regardless of what you put in the config. But that's not really Xorg's fault. It can't magically make shitty drivers good. With decent drivers, once configured, it was rock solid, and has been for as long as I've been using Linux. And for a long time the configuration has been addressed; it now figures that stuff out on its own the vast majority of the time, unless you have really specific needs, in which case I bet Wayland compositors don't fare any better.
I also fundamentally disagree with the idea that X "does too much", which is often cited in favour of Wayland. The fact that X encompasses everything about how I interact with my computer is one of the things I love about it. I might switch WMs and DEs, but my xmodmap, xresources etc remain the same.
I used X11 in the early 90s (HP-UX) and since 2009 (Ubuntu until 22.04 made me switch to Debian.) I never had to configure anything. Maybe some xmodmap in the 90s. I did everything with dialog windows in this century, like I would do it in Windows.
That was like over 10 years ago… It has been a long time since Xorg required configuring.
Yes. Grandparent talked about 20 years ago. 15 years ago Ubuntu required no configuration at all if you had a graphics card that worked (most newer ones did, Nvidia was a nightmare).
X was hard to configure ... back when Wayland didn't even exist.
It hasn't needed any manual configuration for a normal setup in ages.
Old computers had less latency, but otoh on many OSes a single app crashing meant the whole OS was irresponsive and you had to reboot the whole system.
Less latency matters little if it's just to wait behind some other operation like disk IO or network.
The various wayland compositors all have huge problems with keyboards, mice, and other input devices because the wayland spec and reference implementation(s) chose not to support such things. So each wayland compositor picks a different solution. Popular ones are libei and libinput but often there's just no support for "advanced" mouse/keyboard inputs at all (looking at you, weston). And you can never tell if a particular piece of linux software will work on any given wayland. The fragmentation is terrible. On X with the strong X11 reference that actually implements everything you can be sure something that works on one will work on another.
Hell, it's been 12 years and not a single wayland compositor supports screen readers for the visually disabled yet. They are toys.
I'm told they left out color calibration support, too (as I read it, there's no way for calibration apps to control, or even know, what compensation is applied when they're trying to measure things). Oh, and client side window decorations still around, giving up on the chance to fix an ancient security hole.
So no good for games, no good for professional graphics, no good if you don't see well... basically no good if you're any different from the people who hacked it together.
But, hey, with any luck they cut down on the screen tearing that I've never noticed.
Color management support, though taking a long time, is moving along[1], and compositor implementations have recently started appearing. Device calibration support is a different question, but a much more specialized one.
[1] https://gitlab.freedesktop.org/wayland/wayland-protocols/-/m...
Wayland is an extensible protocol. The core is literally just "here a buffer", but it has several extension in various phases that can greatly extend its capabilities. This way, a Wayland compositor can be used as a kiosk screen for a single program, or be a full-blown compositor+window manager, all relying on standard Linux interfaces for the display part.
Color calibration can absolutely be retrofitted into this (versioned) protocol, and there is work ongoing.
It's unfortunate that colour calibration took a while to be fully implemented (xx-color-management-v4 landed on most compositors half a year ago I think, though LTS distros may need to wait longer to get it).
That being said, thanks to Nvidia I never got colour calibration to work right in X11 either, so I have no horse in this race. Would be cool to finally get HDR working for the first time, but I don't know if that'll ever happen on Linux with Nvidia hardware. Guess I should keep dual booting for HDR content until either Windows 10 dies off or I get new hardware.
I do actually notice the lack of tearing in Wayland, especially under heavy load. Used to annoy me to no end to see my windows tear when just dragging them across a 1080p screen. I don't know if it was an Intel driver bug (I tried all the config settings) or something X11 specific, but Wayland did finally fix the tearing issues I had.
I haven't noticed any problems with colours in either Gnome or Gamescope (except for the lack of HDR, of course, but that's also true on X11) so whatever is causing issues for you seems to be DE specific. Looks like we both have issues other people never encountered, that's what makes graphics stacks so impossible to debug and code for.
FWIW, I don’t think they left it out so much as haven’t spent the time to build it. It’s not like they’re pulling implementations off the shelf.
> basically no good if you're any different from the people who hacked it together.
Why would you expect it any different? How can one implement things that they have no need or no hardware for? The entitlement is a bit jarring.
Also I think they merged something last year: https://github.com/swaywm/sway/pull/7681
If you control what goes into a distribution, you can not drop support for stuff that's working and already implements those needs in favor of whatever you pull out of your butt.
What was the "stuff that's already implemented"? Xorg? Which distribution has dropped Xorg?
If not now, soon. Gtk itself is entirely dropping X support. And it's not their fault of wayland is so incomplete but I do wish people like them, and distro managers, would realize that the waylands are not actually replacements for X yet. And they might not ever be if you're visually impaired.
You can do whatever you want if you control a distribution. Use something else if you don't like it.
This is why I ultimately ended up on Windows after switching from macOS and focusing my free time development work on improving the power user/developer experience on Windows with a tiling window manager ecosystem.[1]
One of the really cool things that has come out of this is seeing how people are adding accessibility workflows on top of the ecosystem. There is one person who controls their entire desktop via the tiling window manager using voice control![2]
[1]: https://github.com/LGUG2Z/awesome-komorebi
[2]: https://youtu.be/fiPJLmhnnXM
Windows has added several things with the newer WDDMs to help alleviate compositor problems: flip model, overlays, etc. They've definitely identified the problem areas and are trying to correct them. It's been quite the turnaround since Vista.
> there's just no support for "advanced" mouse/keyboard inputs at all
It feels like you're probably blaming the wrong people here. You should look at the companies that make this peripherals that don't also offer you linux drivers.
You say this, but have you seen the quality of the drivers they produce on platforms they ostensibly support? You do not want vendor mouse/keyboard software on any platform. Instead, improve the USB class drivers so that you don’t need third-party drivers.
I agree with you on general drivers, but what generic devices does Linux still lack? Controller HID support seems pretty universal, as do all manners of touch screens/touch pads, drawing tablets, and just about anything resembling a keyboard.
The only hardware I know for sure is lacking is stuff like Stream Decks, but that kind of hardware is difficult to generalise for.
Plus, most working hardware that doesn't fit a standard HID protocol already has drivers in the Linux kernel written by manufacturers.
> The only hardware I know for sure is lacking is stuff like Stream Decks, but that kind of hardware is difficult to generalise for.
I know that Georges Stavracas has been working on a controller for the Elgato devices for a while (https://flathub.org/apps/com.feaneron.Boatswain) though it needs direct access to the devices instead of through some generic driver (https://gitlab.gnome.org/World/boatswain#udev-rules).
There are definitely a lot of missing protocols when it comes to input though - wl_seat has pointer, keyboard and touch input support, but where are gamepads and wheels?
Nope. Because I don't mean special hardware. I mean using normal hardware in advanced ways: like sharing mouse and keyboards between computers. Something that's worked perfectly in X for 20+ years but still doesn't really work with the waylands even if the wayland compositor in question supports libei (which most do not).
What is lacking when you are using Orca on Gnome with wayland?
I get major lag spikes when the gpu is under heavy load (like doing Stable Diffusion inference or something). TBF I haven't A/B tested with X11, but I don't ever remember it being like that. An extra frame of latency isn't great on it's own, but the occasional spikes in lag are really irritating.
May still happen especially if it is thrashing vram in and out of system memory or something, but have you tried lowering priority of the stable diffusion process?
I can also attest to horrific lagspikes on an Optimus laptop even if Intel is driving the desktop. Memory pressure is definitely the problem here. Lagspikes actually lessened when I switched to Wayland Gnome. I think they lessened further with PREEMPT_RT on kernel 6.12. Nvidia requires an environment variable to build on real time kernels but it plays surprisingly nice as of driver 570. But if you have this config, you need at least 11th gen Intel iGPU or AMD APU, because i915 does not build for real-time kernels. Only the Xe driver works and only if you force_probe the ID if it's Tiger Lake.
...Which I don't get because the Xe driver is said to explicitly support, at minimum, Tiger Lake. I played Minecraft on the iGPU with Xe and it was perfectly fine. It... drew 3D graphics at expected framerates.
Cursor input latency is the continuous galling impediment to my satisfaction with an application or operating system.
IDE text cursor delay? Unacceptable. Shell text cursor delay? Unacceptable. GUI mouse cursor delay? Unacceptable.
All are immediate deal-breakers for me.
One of the places that macOS has always excelled, all the way back to classic Mac OS.
Even System 1 on almost infinitely less powerful gear 40+ years ago super-prioritized cursor responsiveness, outshining all competitors. The OS and UI underpinnings are now 100% different, but the priority remains.
Most of this is offset by the godawful mouse acceleration profile MacOS uses by default. Many external mice, even with the sensitivity turned up to max, will not respond to small wrist movements at all and instead require disabled acceleration curves or more exaggerated mouse movements.
Those curves work fine for the Magic Trackpad, but they should not come default for any desktop Macs that use mice as their primary input.
Yeah, I'm baffled this is so bad. Especially in Linux. Surely moving the mouse should take precedence over everything else? Even highlighting the thing you move the mouse over shouldn't slow down the mouse cursor itself.
I recently had a problem with extreme mouse slowdowns on KDE/Plasma 6 on X11 with NVidia. I noticed the slowdown was particularly extreme over the tabs of my browser.
The fix, in case anyone else also has this problem, is to disable OpenGL flipping. I have no idea what OpenGL flipping does, but disabling it (in nvidia-settings and in /etc/X11/xorg.conf) fixed the problem.
I'd love it if someone could explain why. Or why this isn't the default.
OpenGL flipping is iirc the weird name nvidia gives for vsync. Disabling it means you're going to have tearing artifacts, hence why it's not the default.
But why does it slow down the mouse? Is there no way to get both? Maybe I should try if Wayland gets me both, although at the moment it just gives me a black screen.
I have another somewhat unrelated pet peeve - when I started using Linux, the desktop was software composited, essentially what I assume was going on was that the UI was drawn by the CPU writing into the RAM which got blitted into the GPU's framebuffer, as it was done forever.
When compositors (even with X11 composite) and GPU renderers becamse fashionable, the latency became awful, literally 100s of msecs. Which was weird considering, I grew up playing DOS games that didn't have this issue. Things improved over time but it's not clear to me if we ever got back to the golden days of CPU rendering.
Beautiful work. Could it be worth repeating the experiment with the monitor running at a very low refresh rate, i.e. 30hz? If Wayland is always a frame slower than X11, it should be much easier to observe.
> I don't have the statistics expertise necessary to properly analyze whether this difference is statistically significant or not,
The two sample t-test statistic is -4.74 with a p-value of 4.20e-05.
and that means?
That there's a probability of 4.20e-05 that the observed difference of that large would happen by chance due to observation noise if there would be no real difference (given your assumptions about the data generating process holds).
The normality assumption is a large stretch, especially since there an absolute lower limit near the observation and a somewhat discrete distribution, so a t-test isn't appropriate. But then again it looks significant, so there's no real need for a test
Something like Mann-Whitney U would be safer. But it's indeed quite obvious from the samples that the difference is robust.
In a lot of fields, p=0.05 is good enough to publish. p=0.0001 means it's really f*ing unlikely that the difference in means is due to random chance.
Evidence more than strong enough to be published in most fields. Not quite strong enough to qualify as "discovery" in high-energy physics.
Depends on your alpha ;) . But it's significant.
That really does not answer the question.
This is probably compositor specific, so I wonder if this is a mutter issue or if it can be replicated in kwin.
Mutter has one of the laggiest Wayland implementations imho.
I used to be a NetHack speedrunner. Pressing the keys real fast. All of it in terminals. I had the fastest run for almost 10 years. Input lag was a thing I thought sometimes. Everything below is based on pure gut feelings because I never measured any of it.
I've long wondered why some terminals and operating systems felt like they were pretty laggy. In the sense that if I press my character to move, how long until they actually move?
iTerm2 on MacOS is worst offender. But I can't tell if it's iTerm2 or MacOS itself. I remember trying other terminals on Mac, and it was mostly the same. iTerm2 itself also had bunch of rendering options, but I couldn't get them to do anything I could actually feel affected input lag. One thought I had was: maybe iTerm2 added 1 frame lateness, and MacOS itself added another? But that would be ~30ms on a 60fps monitor which I can easily already tell on a server-connection NetHack.
I also have no idea if measuring "N frames late" is actually a sensible way to think about this. I assume computers know when a key is pressed at a higher frequency?
Linux xterm and rxvt were the fastest that I remember, on X11 (I've never used Wayland like in the article, well not for NetHack anyway).
I have no idea how compositors work on any of these operating systems or X11/Wayland to say anything smart about them or why they would be slower or faster.
Reading the article...I realized I have that same iPhone 15 Pro with 240fps recording. I could replicate that experiment, but testing out terminals on various operating systems instead I used to play on. So I could now test if my gut feeling was right all along or not. or maybe it lied to me all these years.
I wrote the idea in the article down on my notes, maybe when I'm bored enough I'll try it :) and I can stop saying propaganda about iTerm2 being slow if it turns out I was completely wrong. Maybe I have easier time because all my NetHack days were 60fps, so I have more leeway in measurements.
I'm not active in NetHack playing anymore, although I sometimes play it as a comfort game.
iTerm2 is on 45ms and more in this measuring: https://danluu.com/term-latency/
There, it's distinct in that other terminals have lower latency on the same system.
Ooh that is interesting. Also noticed this passage:
"When people measure actual end-to-end latency for games on normal computer setups, they usually find latencies in the 100ms range."
That is really surprising to me. 100ms is HUGE. On NetHack, sshing into servers the difference between 50ms and 150ms is enormous. If do my experiment and I'm not too lazy I want to check on that 100ms figure, your link points to more sources which I didn't check on right now.
I don't know if I'm blind but I can't tell when that article was written exactly. But it mentioned MacBook 2014 and Lubuntu 16.04 so maybe it's around mid 2010, 2016 possibly? (I should check properly if I do my experiments)
The author in the Wayland vs X11 used mouse flicks on the video. Just now while writing I kind of had a thought if this needs some more deep thinking: how do I accurately measure "my finger touched this button and X milliseconds later character moved". Wondering what to consider as the "finger touched this button" event in a 240fps video. Maybe I can do what the author did and map a key to a mouse instead, because maybe there's a much-easier-to-see physical movement with a mouse. But then, am I really measuring faithfully my old NetHack-like environment? Or maybe this isn't really an issue and it'll be easy to tell.
Too much for my small brain to think in the moment :) that link gave me at least some validation that iTerm2 maybe really is a bit slow, or at least was at that time. There's also bunch of terminals on that page I've never heard of.
Today iTerm2 is my daily driver, but xterm is still really fast on my Fedora Linux system I have on the side.
Wayland has just been a disaster for the open source desktop. It's absorbed all of the effort into something horrifically subpar while breaking compatibility with the past.
Accessibly is far worse to the point of embarrassment. Latency is worse. Compatibility is gone. Everyone in the ecosystem needs to do more work to end up with fewer features for end users.
This is the opposite of systemd in every way. Systemd upset a lot of people because it was entirely user focused and didn't care about the beauty of the architecture. Wayland is only obsessed with how nice their apis are, not what the end user experience is.
Let's end this disaster. It's been 16 years. It's almost old enough to vote. When will we accept it has failed?
"Let's end this disaster. It's been 16 years. It's almost old enough to vote. When will we accept it has failed?"
Use what you think is better then.
I mean X11 was a pile of hacks, it became an unmaintainable pile of hacks. I think reduced scope was a good call, writing a display server is a hard problem. A lot projects started and failed to replace X11.
If you feel this way I recommend watching this talk by Daniel Stone from linux.conf.au 2013, The Real Story Behind Wayland and X https://www.youtube.com/watch?v=RIctzAQOe44
People believing this old propaganda is what brought us into this mess.
Just because you don't agree with it (which is weird given the guy is a big X/Wayland maintainer), doesn't make it propaganda.
X sucks for how graphics work in the 21st century, there's a big reason Google didn't use it for Android and instead made something that's actually more akin to the Wayland model .
x11 failed in other areas, and earlier.
So what is your proposal, starting again from a blank sheet?
I don't think it will help.
> x11 failed in other areas, and earlier.
Still works great for me!
as do wayland for the majority of its users.
Fine, we are all happy then.
Exactly -- so nothing has failed at all!
I've used https://isitsnappy.com/ on the iPhone to do these kinds of measurements pretty quickly, to substantiate why I sucked at Steam Deck rhythm games when connected to a TV.
(I do not suck, my setup had 100ms lag)
Cool site! I didn't know this was an established way of measuring input lag but it's how I've been doing it since I found out my Samsung phone came with a 960 fps camera. Manually counting frames with ffmpeg is a pain, but I never even thought to look for an app that does this. Thanks for sharing, I'll take a look around :)
Great app. I hope next iPhones have improved slow-motion camera modes. 1000fps will be a long shot but maybe 480/500fps.
What was the issue with your setup? TV not in game mode?
Never got to the bottom of it. TV was set to game mode, I tried various scaler settings on the deck itself, couldn't get it below about 90-100ms. I've tried on two TVs.
You can learn to play Space Channel 5 by tapping 100ms ahead of the beat. But I worried that if I succeeded too much, I'd end up doing everything that way: jumping red lights, taking espresso shots while they were still being poured etc.
Living With Lag: https://www.fastcompany.com/3029918/what-its-like-to-experie...
...what it's like to experience video game lag in real life.
Note that Wayland is a protocol, not an implementation. This testing was done with Gnome's implementation of Wayland, which doesn't mean much for any other Wayland desktop.
I'm glad more people are looking into this. Maybe one day we'll discover the cause of the Laggy Mouse bug which has been a problem since 2015
https://bugzilla.gnome.org/show_bug.cgi?id=745032 https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/749
Was Wayland not architected to fix latency issues?
I also find Wayland more laggy and more buggy without concrete evidence of course.
It has significantly improved lately though, far more stable than it used to be.
> Was Wayland not architected to fix latency issues?
It was designed to fix tearing issues, not latency issues.
But then the Wayland designers found out that players often prefer tearing to minimize latency, so the tearing protocol was implemented.
When a committee of perfectionists, rather than real people or companies, design something, you often get something that is completely unusable by anyone but that committee.
And that's exactly how it's played out so far. Wayland is still largely incomplete, not to mention it doesn't even have a reference implementation [1], and still doesn't include/describe several essential desktop APIs and features, which results in this protocol not having a ton of universal tools and applications that work across all of its implementations, including but not limited to:
It's a damn ugly mess that has led to a lot of fragmentation with no obvious benefit. Currently, only KDE and Gnome have somewhat usable Wayland implementations. If you use anything else? You're SoL.1. https://gitlab.freedesktop.org/wayland/wayland/-/issues/233
It's almost like you don't want tearing when browsing the web and your documents, but games can take over the precise control of rendering.. because they are games that want to do so?
How is that bad design?
The numbers presented in the article (~1 additional frame of latency on Wayland) are from someone using GNOME with the Mutter compositor. This means that neither the X11 or Wayland session has screen tearing. I imagine the X11 input latency numbers would go down by around half a frame on average if the author was using a different window manager without a compositor.
Of the things you list, IMO, maybe the display configuration should be part of the Wayland protocol.
It might be a better technical design to have the other stuff outside of the display protocol. Just because Xorg implemented something does not mean you have to put it in the Wayland protocol.
Routing of input events to windows has turned out to also be extremely important for a display subsystem in practice. It wasn’t just X11 that did this, it was also the NeXT window server, and Sun’s NeWS, and SGI’s pre-X window system, and the original VAXstation window system on VMS, as well as how both modern macOS and modern iOS and iOS-derived platforms work.
In any of these cases there may be one or more daemons behind the scenes handling the “raw” input—possibly even in cooperation with kernel-level pre-processing code, to ensure low latencey—but most event delivery to applications is associated with windows, with options to get lower-level access if needed.
One of the things that helps many of the systems described above with latency is kernel participation, whether by pushing much of the preprocessing of events down to the drivers so there’s little for userspace to do, or by implementing kernel-level zero-copy IPC (e.g. use of Mach messages by NeXT and Apple).
If human interface IPC happens entirely in userspace and requires multiple context switches to get an event from device to a display change, you’ll wind up with hitches and delays unless there’s some sort of scheduler hinting that ensures each stage in the pipeline runs immediately after the last.
This is, of course, why there was a lot of desire by Wayland advocates for kernel dbus support, but they went at the problem backwards: “Let’s take DBus, and make it fast by putting it in-kernel,” *without* first trying to make it as fast as possible without kernel support, *and* without trying to figure out the minimal feature set for kernel-level IPC that would be needed to support it (which may not look like DBus).
> It was designed to fix tearing issues, not latency issues.
Actually is was designed because the X11 codebase was bad and nobody wanted to work on it.
> Actually is was designed because the X11 codebase was bad and nobody wanted to work on it.
This sounds like "Wayland codebase is good and everybody wants to work on it".
As I understand it, the tearing thing (no vsync) was only implemented after Valve pleaded, begged, and sacrificed a small animal to the Wayland developers.
> Currently, only KDE and Gnome have somewhat usable Wayland implementations. If you use anything else? You're SoL.
Wlroots exists.
A library exists, correct.
Good luck using it as your graphics subsystem.
Wayland is just a protocol that is probably implemented over and over and each implementation might have issues. It's mostly likely the different implementations having issues.
Was Wayland not architected to be simpler to implement than the (antique) X11 protocol?
Not arguing though, you are right it is just issues: Drivers, the Wayland implementations, how some plethora of apps and libraries have been battle tested then optimized, years over years, for X11. Not as much for Wayland display yet.
Wayland is simpler because it does less. And among the things Wayland doesn't do and X does are things that people need. It means these things have to be implemented elsewhere.
It is a common problem with these "simple" things. The problem is complex, and if you simplify one part, you are just pushing complexity elsewhere, you are not making it disappear. It is sometimes a good thing, but in the case of Wayland, it didn't go well.
There is also the option to make complexity disappear by just convincing people that they should not want to do what they want to do. This was the kind of response to what I needed to do. Then I had to switch back to x11.
In any case wayland is not bad if you only have pretty basic needs I guess, some basic things look easier to me there from a user perspective, and troubleshooting x11-related issues for a non-technical person is no fun either.
> Was Wayland not architected to be simpler to implement than the (antique) X11 protocol?
I think it's safe to assume that it is actually simpler, given that we already have multiple Wayland implementations, but still basically just the one X11 server implementation. Can one or more of those implementations shave off one or two milliseconds overs the next 40 years... Probably yes.
Sure, on linux all you really have is xorg, what used to be xfree86, but I have used xsgi, and there are X servers for windows, heck, I have even seen this amazing demo Xserver that ran on the web.
footnote: there is something wrong with the commonly used web search engines and I am unable to find that X11 web demo, I think they are prioritizing recent content over good content. you would think with how interesting that web demo was it was it would show up right away. but I got nothing, so no links. Anyway it was a deep dive into the intricacies of X11 window painting and the author had ported a limited subset of an X server to javascript in order to provide a live demonstration of the topic. I will keep looking.
Found it. https://magcius.github.io/xplain/article/x-basics.html
It's called a SEO driven content vs quality drive content. Unfortunately the engine has has poor sense of taste.
Architecture contributes, but all it takes to blow input latency is some inefficient code almost anywhere in the input or rendering pipeline. The Apple IIe didn’t really have very many layers between the keyboard and the CRT. Wayland might have an optimal architecture in theory for latency — I don’t know — and latency could still be ruined by not fully optimizing the other components in the stack.
I'm impressed how wlroots compiles in under 3 seconds for me
No, to fix tearing issues. Tearing and latency generally exist in a tradeoff relationship.
Gnome was fixed a few years ago to remove significant latency it was adding. May have been a factor if you used that as well, but it wasn't wayland specific to my knowledge.
I don't like the design of Wayland at all, so it'll take a lot more than performance analytics to get me to switch. I think that one of the core issues in Linux is nothing is ever consolidated, only moved toward more fragmentation. Wayland had the opportunity to consolidate, but instead to not fix fragmentation (each app is responsible for drawing its own window borders) and introduce more (multiple compositors each with their own quirks and non-standard APIs)
For those of you who are application developers on Linux/Wayland, do you use any other methods for benchmarking true latencies between input events and screen? Less heroic than the high-speed camera method. I'm embarrassed I've still never figured this out properly.
A photocell is I think the standard (or at least "the standard" in psychophysics applications I have also used). You need, though, to make sure your event causes a change that a simple photocell would detect (eg you have a corner of your screen or application window having a small square that turns from black to white and vice versa). You can connect that to an arduino or whatever you can use to get both signals in some low latency way (one from user interaction or whatever causes the event, and one from the event itself with that light intensity change). There are also photocells with a higher granularity for luminosity changes than just light/dark, so maybe you do not need to have the black/white square thing, just something that causes enough luminosity change to be detected. This should, in principle, give you very accurate timings if you can also get the event initiator somehow with low latency (or you can do stuff like simulate an event periodically and look for skipped frames/variance in the screen presentation rather than latency).
You might like Typometer. I believe it was created by a JetBrains developer to improve input latency in IDEA.
> Typometer works by generating OS input events and using screen capture to measure the delay between a keystroke and a corresponding screen update. Hence, the measurement encompasses all the constituents of processing latency (i. e. OS queue, VM, editor, GPU pipeline, buffering, window manager and possible V-Sync).
https://pavelfatin.com/typing-with-pleasure/
Would have been nice to see if the issue is present on wlroots compositors, GNOME rolled their own.
> GNOME Wayland session
I wouldn't use that. Try KDE if you want a decent gaming experience. Gnome was very slow in even implementing something like adaptive sync, so they hardly are prioritizing latency issues for the end user.
I.e. Gnome's problems ≠ Wayland problems in the broader sense. Conflating the two is wrong.
Wayland lacks a strong reference implementation. The resulting fragmentation in the ecosystem turns the most popular compositors into de-facto standards every application developer will have to use as a reference.
This means Gnome's problems = Wayland problems.
Furthermore, the latency issues with Wayland are inherent. Compositors will always have worse latency than writing directly to the front buffer.
The kind of issues referenced in the article are not related to compositors being compositors though, but to bad implementations.
Gnome is big, but it's not the most popular DE out there, so it can't be treated even as a de-facto standard in the Wayland world. Some things might be treated that way, but they are lower level (let's say Pipewire, libinput etc.).
And regardless, it can't be used an example that blames Wayland for Gnome's issues, unless literally every single compositor has this problem and the real reason is Wayland's inherent limitations. There are such limitations in general indeed due to Wayland still evolving, but this isn't one of them.
> Compositors will always have worse latency than writing directly to the front buffer.
This is not really true, if a full screen application is detected, the compositor can tell the GPU to instead just display the contents of the applications buffer, skipping the compositor.
IIRC this is broken on Xwayland GNOME.
> Compositors will always have worse latency than writing directly to the front buffer.
I mean... in the sense that "ALSA has to be faster than Jack ultimately", I guess; that fact may not be useful for a given setup
It would be interesting to have numbers under gamescope instead, Gnome doesn't care much about gamers & power users that mesure latencies.
Games don’t depend on a compositor for anything, they draw direct to the framebuffer.
Anything related to the compositor won’t affect the game, unless it’s in windowed mode, then there can be some strange interactions.
Valve's gamescope addresses something that you don't seem to believe exist :
https://github.com/ValveSoftware/gamescope
That will indeed help with Windowed mode
Games don't, but xWayland does.
I thought we were talking about gamers, xWayland doesn’t affect gamers at all either.
What do you mean? Most games run on XWayland when you're in a Wayland session, wouldn't input latency caused by XWayland therefore affect gamers who use Wayland?
Games don’t run via X11, they write direct to the framebuffer using a framework like Vulkan.
In windowed mode they can be captured, then things get tricky, but in full screen they draw direct to the gpu, vulkan itself is completely headless.
I don't think that's true on modern platforms where the compositor (Mutter in the case of Gnome) is always responsible for presenting the frame.
Gamescope's direct scanout bypasses this.
I’ll look into it, but it used to be a choice in the game settings that bypassed the drawing display server entirely, and caused that full screen flicker you might recall when tabbing back to your desktop (and the switchover of rendering control, and subsequent huge performance loss of the running game).
I wasn’t aware that this changed, but you could be right. Its definitely the same on Windows as it always was, which is the platform I most developed games for.
> Its definitely the same on Windows as it always was, which is the platform I most developed games for.
Not really. Most games use "borderless windowed" mode instead of fullscreen exclusive, and even FSE is not true exclusive mode anymore in most cases.
https://devblogs.microsoft.com/directx/demystifying-full-scr...
> When using Fullscreen Optimizations, your game believes that it is running in Fullscreen Exclusive, but behind the scenes, Windows has the game running in borderless windowed mode. When a game is run in borderless windowed mode, the game does not have full control of the display– that overarching control is given back to the Desktop Window Manager (DWM).
see also https://learn.microsoft.com/en-us/windows/win32/direct3ddxgi...
I think Windows now also supports hardware overlays, so games running in fullscreen Windows can skip the compositor, which reduces the latency in no longer allowing games to directly write to the FB.
The DWM can do this even for windowed applications if they use flip model -- it will dynamically transition individual windows between compositing and independent flip (overlay).
Most compositors can do direct scanout when they detect a full screen surface. (This is broken right now with Gnome on X11 because of the way window frames are drawn: https://gitlab.gnome.org/GNOME/mutter/-/issues/2794 ) If the game uses the platform cursor then that will reflect the additional latency, but the actual input signals won't be delayed.
Doesn't really work like that in Wayland.
If an application framebuffer is full screen and in a compatible pixel format the compositor can do "direct scan out" where the compositor sends the framebuffer directly to the crtc instead of compositing first. I know that wlroots supports that. I'm not sure how much performance it saves to be honest.
And I'll second that most commercial games do go through XWayland, though it depends on what games you like!
https://wayland.app/protocols/fullscreen-shell-unstable-v1
It's much worse with Nvidia 30+ series on Wayland, feels worse than a Bluetooth mouse, using Nouveau eliminates lag for the most part.
Reposting my analysis I performed with a Rpi pico, oscilloscope and a photo diode from another site: I created a test setup using a Raspberry Pi Pico running CircuitPython and a USB HiD library to emulate mouse clicks. I connected button switch to a GPIO pin and polled the state of the pin to send mouse click events over usb to my laptop. I connected a photo diode to the power rail of the pico and pointed it at the screen. Finally I connected the push button to channel 1 of my oscilloscope and the photo diode to channel 2.
I created small application with raylib which would toggle the screen from black to white when clicked. When the screen changes from white to black, the voltage across of the photo diode would drop, which would create a falling edge I could trigger on using the oscilloscope.
Test Setup:
I ran the following tests on my AMD Framework 13, under CachyOS with all packages up to date. For each test I set the Raylib application to run at 800x600@60Hz and left the photo diode in place near the bottom right corner of the screen. For each test, the display output scaling was set to 1x (2256x1504), and variable refresh rate was disabled. Using the oscilloscope I measured from the first trigger buffer sample that the voltage of the button reached steady state to the first sample where the voltage on the photo diode dropped below steady state to 1.5v
Caveats:
Results: KDE Plasma (X11): 154ms, 134ms, 136ms, 137ms, 126ms, 144ms, 135ms, 142ms, 150ms => Avg 139msKDE Plasma (Wayland): 181ms, 188ms, 139ms, 148m, 137ms, 156ms, 163ms, 164ms, 151ms => Avg 158ms
Sway (Wayland): 172ms, 175ms, 168ms, 153ms, 144ms, 152ms, 161ms, 172ms, 156ms => Avg 161ms
Its very interesting, but I would love to see other Wayland implementations other than GNOME's. Its kind of hard to tell if its the GNOME specific implementation or if this is going to be a trend across KDE X vs KDE wayland and specific window managers like sway -> i3 and hyprland etc... But still a really cool blog post.
i run nvidia/intel/optimus/wayland in optimus/integrated mode on my laptop. i have anecdotally noticed perceptually salient increased response times when:
1) dpi scaling is in use 2) external displays are used via usb-c/displayport 3) the nvidia memory clock has been scaled down by power management
that said there have been fantastic improvements overall to both stability and response times since kde6 and nvidia-open have stabilized and i haven't noticed any of it in a long time now. it really is pretty great these days.
would love to see this test repeated with a non-compositing WM on X11
Isn’t 3 frames of latency already way too much? I would say they are both too slow!
I also have no ideas how it works, but my guess would be that is actually translates exactly like that into games. Why would the mouse on the desktop have a latency that games do not have. Linux does not have a real fullscreen mode like Windows has (that I do not use anymore anyway for faster alt-tabbing). So my guess is, the cursor or mouse input gets into games very much the same way as it gets in the desktop.
Wayland has support for games completely taking over the GPU; notoriously critical for VR support. This is basically the same as Windows exclusive full screen.
Wine/proton would need to support it, XWayland would need to support it (Wine/Proton are one major step away from native Wayland support: Vulkan), and finally the compositor would need to support it. Gnome is about the worst compositor that you could be testing any of this stuff on, they are comically hostile towards ideas not their own. The chances of this ever working on Gnome are near zero. KDE is pretty good for support, Hyprland seems to be trying to support every under the sun.
> notoriously critical for VR support. This is basically the same as Windows exclusive full screen.
D3D12 dropped support for exclusive fullscreen, and I don't think headsets even go through DXGI but their own swap chain APIs. Why do VR games on Linux need the equivalent from the Wayland compositor?
Can I have the source for this? DX12 dropped exclujsive FS? Why does every game still has this as a option then? Or are you talking about VR specifically?
Sure, I took it from the docs [1].
Games may show it as an option if the engine itself can support other GPU APIs and the devs working on the options menu don't know or care that this particular settings combination is misleading.
[1]: https://learn.microsoft.com/en-us/windows/win32/direct3d12/s...
VR on X.Org also already bypasses the X server for the VR display.
> Gnome is about the worst compositor that you could be testing any of this stuff on, they are comically hostile towards ideas not their own. The chances of this ever working on Gnome are near zero.
GNOME has supported direct scanout for fullscreen apps for a while and drm-lease was implemented not too long ago either.
Did not know Wayland has exclusive FS.
I doubt that Wine would actually request it. Some compositors also have "tearing mode," which is basically as good: the game gets full control of the presentation timing, so vsync/tearing would be determined by what the game wants to do.
Hardware cursor might be used in games. You notice the difference on Windows if a GPU does not support it, even on the desktop.
Funny, I didn't even know this was a debate. I didn't even notice when Fedora switched from X to Wayland by default - maybe because I'm still a 60hz plebian?
I have experienced issues with my mouse input locking up/jumping - especially noticeable when I'm playing a shooter - but I'm pretty sure that's just my poor ancient mouse giving out.
Mouse is moved. Preemption happens. Hardware cursor is repositioned.
The Amiga could do it in 1985. Linux has realtime scheduler classes (i.e. SCHED_FIFO and SCHED_RR). These days, even PREEMPT_RT.
Some Wayland composers are already leveraging hardware cursors when possible.
Why is it so hard? What's stopping them from accomplishing this?
Does the Nvidia FrameView app works under Linux (or more like under X11 and Wayland)? You can measure general system latency with that https://www.nvidia.com/en-us/geforce/technologies/frameview/
Since this is all mouse pointer relates and there is hardware for that, I can only assume it's a fixable problem.
OTOH the gnome developers have been talking about implementing triple buffering in the compositor. Tripple buffering is only needed when your renderer can't complete a frame faster than the scanout of a frame. Given that any modern GPU should be able to composite hundreds of frames per second this make me think something about Wayland or the gnome compositor in particular isn't designed as well as it could be.
Dynamic triple buffering is basically just giving the hardware time to clock itself up when there is a sudden change in workload, e.g. idling then opening the overview (drawing basically nothing -> redrawing basically the whole desktop).
If you just want to measure user space software input lag, shouldn't it be more accurate if you print out the time when the event arrives in the kernel, and again when the mouse moves within a application window?
Depends what you want to measure. But generally, people are more sensitive to latencies in movement of the visual cursor than they are to applications responding to it.
Hm. I am currently using Ubuntu 24.10 and it also has Wayland and gnome as a default, with the nvidia 565 driver. And it is really, really bad. For example, i use chrome and the tabs, man, it takes ages to switch from one tab to another and to get focus in the location bar again.
So much for the whining. I do not know what to do. I could record this behaviour, but it somehow needs to show the timestamp of when i intend a behaviour and the time, that behaviour becomes actualized.
I used to be quick with all the hotkeys but i do make so many mistakes now, because the system is simply not responsive and all my instincts await instant action.
https://ui.perfetto.dev/
Won’t get you end to end latency, but should be able to trace the input events through chrome, and show the context swaps of all the Wayland/dbus stuff.
EDIT: looks like Linux system tracking is more work, but possible https://perfetto.dev/docs/quickstart/linux-tracing
Thank you!
> Interestingly, the difference is very close to 1 full screen refresh. I don't know whether or not that's a coincidence.
Some sort of double buffering? To ensure it's consistently one frame, it would make sense to test on a slower computer via the same monitor.
There was some discussion on HN in 2017 regarding latency differences: https://news.ycombinator.com/item?id=15748146
As mentioned in those comments, it could be from the compositor
Has implications for competitive FPS gaming on modern Linux — one more problem to fix. For example "The Finals" allows Linux players, but imagine having this much input delay and having to revert to classic x11 to play games, lol.
It's actually just cursor latency specific to the windowed environment you're running. From what I've experienced (with a 4060 Ti) it doesn't seem to impact FPS games at all.
I haven't tried any games that use a cursor with Wayland yet so I don't know if it would have an impact there.
I think it has to do with whether or not the game in question is reading the mouse device directly (e.g. through SSL) or via the compositor. If it's reading the device directly it stands to reason that there would be less latency.
> Has implications for competitive FPS gaming
That has not been proven in the article. Input handling follows different paths for full screen games.
> having to revert to classic x11 to play games, lol.
It would be more of a problem the other way around, if we had to resort to Wayland to get low latency. I think most of us using Linux for gaming and casual stuff are happy to stick to X11 for now and the foreseeable future. It has good support in software, its failure modes are well-documented, and it doesn't add one more layer to the pile of complexity that desktop linux already is; at least, not one us users have to consciously keep in mind as a common source of issues.
If anything, Wayland removes a layer. Replaces the insanely huge monolith with binary blobs with bog-standard Linux kernel APIs.
As I type this, I'm playing "the finals" on Linux via Proton on Wayland. I won't pretend I'm any kind of competitive gamer type, but it's perfectly fine and I don't feel like I'm being held back by input lag. So this is very much a niche issue to have.
On CS2 Wayland gave a major performance boost, but it's being held back by a regression since a change in device input layer.
https://github.com/ValveSoftware/csgo-osx-linux/issues/3856
From outside it's hard to tell if it's truly protocol differences or just the age of the implementations on X11, but when Wayland came out every project has claimed improvements over the old X11 stack. Also, from the early Wayland days presentations bashed the protocol as something that couldn't be fixed without a rework that was not going to happen due to the dead weight of backwards compatibility and awful older hardware.
As a user applications running on Wayland have consistently improved on how nice things feel if you don't miss your latency deadlines. It's easy to perceive on apps, and straight out obvious in games.
I got to diamond in Apex Legends on Linux using Wayland on KDE Plasma. Didn't feel any noticeable difference between wayland, x and windows in my Apex playing.
It doesn't. This article doesn't measure that, and full screen works differently. This article also only measures gnome.
I have trouble believing that 6.5ms in increased latency would be perceptible to any more than a fraction of a percent of the most elite gamers. Most the people claiming that this level of difference is impacting their gameplay are victims of confirmation bias.
David Eagleman has done some work with drummers. Granted the audio system might be a bit more accurate than the visual, or maybe drummers are just weird. On the other hand, vim taking 30 milliseconds to start (ugh) and having sluggish cursor motions is why I'm on vi now. Haven't tried Wayland. Maybe in some number of years once it's more portable and more developed? (And how many years has it already been out?)
> “I was working with Larry Mullen, Jr., on one of the U2 albums,” Eno told me. “ ‘All That You Don’t Leave Behind,’ or whatever it’s called.” Mullen was playing drums over a recording of the band and a click track—a computer-generated beat that was meant to keep all the overdubbed parts in synch. In this case, however, Mullen thought that the click track was slightly off: it was a fraction of a beat behind the rest of the band. “I said, ‘No, that can’t be so, Larry,’ ” Eno recalled. “ ‘We’ve all worked to that track, so it must be right.’ But he said, ‘Sorry, I just can’t play to it.’ ”
> Eno eventually adjusted the click to Mullen’s satisfaction, but he was just humoring him. It was only later, after the drummer had left, that Eno checked the original track again and realized that Mullen was right: the click was off by six milliseconds. “The thing is,” Eno told me, “when we were adjusting it I once had it two milliseconds to the wrong side of the beat, and he said, ‘No, you’ve got to come back a bit.’ Which I think is absolutely staggering.”
It doesn't need to be perceptible to cause a difference in a game.
Suppose two players notice each other at the same time (e.g. as would naturally happen when walking around a corner in a shooter), first to shoot wins, and their total latencies are identical Gaussians with a standard deviation of 100ms. Then a 6.5ms reduction in latency is worth an additional 2.5% chance of winning the trade. Maybe you won't notice this on a moment by moment basis, but take statistics and its impact should be measurable.
In ELO terms a 2.5% gain in win rate is around a 10 point increase (simplifying by assuming that single Gaussian is the entire game). That's small, but if you were a hardcore player and all it took to raise your ELO by 10 points was using a better monitor/mouse/OS... why not? Doing that is cheap compared to the time investment required to improve your ELO another 10 points with practice (unless you're just starting).
Also, I think you'd be surprised what people can perceive in a context where they are practiced. Speed runners hit frame perfect tricks in 60FPS games. That's not reaction time but it does intimately involve consistent control latency between practice and execution.
Slightly off topic…
> Suppose two players notice each other at the same time (e.g. as would naturally happen when walking around a corner in a shooter)
This is not true for third person games. Depending on a left sided or right sided peek and your angle or approach, players see asymmetrically.
For example, Fortnite is a right side peek game. Peeking right is safer than peeking left as less of your body is exposed before your camera turns the corner.
I believe distance also plays a part in the angles.
Yeah, network latency and client side prediction and accuracy will also play huge roles. The actual distributions will be very complex, but in general reacting faster is going to be better.
Do people not play deathmatches on LAN parties anymore these days? 2.5 is huge if the game lasts long enough that someone would be leading with 200. ;)
I would postulate that 100% of professional (i.e. elite) competitive gamers would be able to tell the difference. See this old touchscreen demonstration: https://www.youtube.com/watch?v=vOvQCPLkPt4
Those sorts of latencies actually are noticeable! As an example, 6.5ms latency between a virtual instrument and its UI is definitely noticeable.
I didn’t think it was. But it is. I promise!
It’s not necessarily a reaction-time game-winning thing. It’s a feel.
With virtual instruments, my experience is that when you get down to ~3ms you don’t notice the latency anymore… but!, when you go below 3ms, it starts feeling more physically real.
About the difference between 60hz and 120hz monitor, instantly noticeable just by moving the mouse in windows (just by looking at the distance cursor updates as it moves). Would you argue that all gaming monitors are placebo?
Just to nitpick, that difference is still above 6.5 ms.
I actually would. Gaming monitors are the equivalent of fancy audiophile gear. It's a way to fleece people by making them think they can perceive a difference that isn't really there.
But the difference between 60hz and 120hz is instantly noticeable in a blind test, without even opening a game. That's by definition not placebo.
There have been actual tests showing players have better accuracy up to 360 fps displays.
https://www.youtube.com/watch?v=OX31kZbAXsA
Guess you think all speakers are the same.
You may think 6.5 ms of input latency is imperceptible. But combine it with the rest of the stack (monitor refresh rate, local network latency, RTT between client and server, time for server to register input from client and calculate “winner”), and it becomes the diff between an L and W. In the case of pros, the diff between a multimillion dollar cash prize and nil.
There are noticability thresholds where this could push it over. For fighting games if you have the reactions to whiff punish N frame recovery moves this may push you to only being able to punish N+1 recovery moves and really impact your ranking. This is a little over 1/3rd of a 60hz frame.
Now download Mixxx and try DJing and using the waveforms for cueing
"Competitive FPS gaming" stops allowing anything besides fresh Windows installs after a certain point. It's a diminutive distinction, like pointing out that emulating GameCube games won't let Smash Melee players fight competitively; nobody playing on either side actually cares.
Composited desktops have their drawbacks. With a display system like X11 or old-school GDI-based Windows, draws to the display are next to immediate, whereas with composited desktops, everybody has to draw to a back buffer, then the back buffers have to be composited together on the final display, adding at least one frame of latency. Mouse cursors have been a hardware sprite since the 90s, but Wayland compositors may favor using the GPU to draw a soft cursor.
It'd be interesting to see a compositor that can speak Wayland, yet performs immediate, region-clipped draws of all windows like X11 conventionally does.
I did some work to create a simple device to measure this some yeas ago. The intention was for developers to use it to regularly benchmark their software. It costs like 20 bucks and takes 10 minutes to build.
I might get to run some benchmarks myself next week.
https://jnsn.dev/posts/frametime/ with a follow-up in https://jnsn.dev/posts/fastisslow/ or if you just want to see the code without my jabbering: https://github.com/DelusionalLogic/Frametime
Somebody should point LDAT at that [1] or the custom setup with LED triggered by the mouseclick + a high FPS camera that Battlenonsense used [2] before LDAT released.
[1] https://developer.nvidia.com/nvidia-latency-display-analysis...
[2] https://www.youtube.com/watch?v=_MaeJbd1xaM
Can’t we port whatever android uses? Clearly the community cannot execute GUI related things at the level that a focused company can.
Possibly because the best contributors don’t care about GUI anyway.
Surface Flinger currently doesn't support well or at all multi-monitor configurations which are essential for the PC user.
Otherwise, it's actually far more advanced than Wayland, and it had essential modern features implemented years ago, including HDR, VRR, and others.
Why hasn't it been ported to Linux? Probably NiH, probably it's Google's child.
Ported to Linux? It runs on Linux. What exactly would need porting?
What would be required to make surfaceflinger a practical Linux desktop compositor is merely some translation layer, equivalent to XWayland, that supports hardware acceleration. Such things have been written, but not open source and they never got traction.
The idea has been toyed with for over a decade: https://news.ycombinator.com/item?id=5317638
> Ported to Linux? It runs on Linux. What exactly would need porting?
It runs on Android/Linux, not GNU/Linux. I would be surprised if there wasn't at least a bit of work needed to handle things like bionic vs glibc, and you'd really want it to act "normal" by being installed to FHS paths instead of Android's weird filesystem layout. All doable, I expect, but it would be a port.
Just do it!
I knew it! And when it does stall it takes way longer than X11.
Great work, I hope the gnome/wayland devs appreciate it.
I wonder if there is an issue with high frame rate support in Gnome (or possibly wayland?). It seems like they are actively still working on it:
https://9to5linux.com/gnome-47-3-improves-frame-rate-for-mon...
No, that article is about copying frames between gpus
Agreed. That was just the latest related article I saw. I was bring it up to support my claim of "high framerate support is still new/developing in Gnome"... embarrassingly my old man brain compressed time... it's been 6 years[1], oh well.
[1]:https://www.phoronix.com/news/GNOME-Still-Dropping-Latency
The latency issue isn't blocking me from trying Wayland. Wayland seems fine on my Raspberry Pi.
The issue is screen-sharing in MS Teams. Does anyone know if that works now?
Since I use Artix, I expect a potentially painful switch and don't really want to have to then do it all again to switch back immediately.
Few options. Use a browser that has supported screen sharing on Wayland for many years: Firefox.
Or petition Microsoft to enable the Wayland-enable bits in their electron build. Electron has supported screen sharing on Wayland for soon 5 years.
Why is there latency beyond a single frame at all?? I did a very quick replica of the camera frame counting on Windows with a 144Hz display and it's definitely below the camera's 60FPS, i.e. the cursor moves on the same frame that the mouse moves. Moving a window also has 0 lag in that test.
Huh, that is one of the things why I never liked Linux Desktop at all. Switching from Windows to Linux Desktop feels... weird, laggy. People told me I was the problem, but it seems there are more people who noticed this lag. Maybe someone can fix this :)
This reminds me of the early work of the structuralists in psychology. Measuring the time it takes for a person (or in this case Wayland) to react to a sensation.
The unfortunate thing about Wayland, imo, is that most people think it’s a single implementation like x11
This is pretty bad methodology for someone to call it "Hard numbers"
If you're unhappy about their methodology, you're welcome to write productively about your specific gripes.
Yes, I was expecting some software that listens directly to the mouse input, and watches a pixel on the screen. Messing around with cameras and counting frames introduces a whole bunch of other variables like the quality of the monitor and mouse and phone used for recording.
The camera, mouse, and monitor all stayed the same for the tests, but there was a significant difference in latency. Out of the 16 times the experiment was run, only once did Wayland have lower latency. It would be an amazing coincidence if the monitor, mouse, and/or camera were the reason for this.
Figuring out why there's increased latency is a job for software tooling, but I think this guy's experiment is one of the best ways to measure what users care about: The time it takes for an input (such as mouse movement) to result in a change (such as cursor movement).
Note that this doesn't mean that the Wayland protocol itself is the reason for the higher latency. It may be Gnome's implementation (testing with a wlroots compositor might shed light on this). It may be differences in default configuration options. It may be that Wayland and X11 start up different services, and the Wayland helper processes increase load on the machine. But I seriously doubt the reason for the difference in latency was because the same hardware was used throughout the experiment.
I got results with a P-value of under 0.001. That should be enough to demonstrate that there's a real difference.
Using a camera allows the methodology to he identical between Wayland and X, while I don't know how to listen for mouse movements from software in a way that wouldn't introduce its own problems. What if the photons are emitted from the screen after the same number of milliseconds across X and Wayland, but Mutter is a few milliseconds slower about notifying my measurement application? Conversely, what if Mutter has more real latency than X, but the latency is introduced after the stage where my measurement application sits?
The variables you mention are identical between the Wayland test and the X test. It is admittedly a challenge for reproducibility, but doesn't affect these results.
The mouse and monitor don't matter here. Unless their delay completely dominates the timing (it doesn't here) they can be ignored because the setup is constant been the tests. We're interested in the difference, not the absolute numbers.
How would you "watch a pixel on the screen" in pure software. You need a camera and an input device in the loop.
Is it not possible to simply query the frame buffer right before its sent down the HDMI cable. What color is pixel 0,0?
Maybe you could do that by hacking the driver in a Linux system? I don't know actually.
The chosen protocol works fine to answer the question. Overall latency is what counts.
Only one wayland compositor and then it is all wayland compositors???
Looked up the mouse to see how fast it updates and read that it is wireless. Kind of disappointing and might explain some of the variance. Although if X11 is consistently faster the test should still have some validity.
Would like to see a test with gnome and the wireless mouse removed from the equation.
You need a setup like this guy: https://www.youtube.com/watch?v=nbu3ySrRNVc
Yeah testing this with a wireless mouse is dumb. The proper way is to use an Arduino or similar as a USB mouse, and connect a photodiode to it too.
USB mice update at 1 kHz.
It depends on the model. Wireless mice don't have to be slower than wired. But even then, for this test it's irrelevant. It's the same mouse in both scenarios, and we only care about the difference. Even if a wired mouse stripped X ms on both sides, the result would be the same.
It does matter because the latency of a wireless mouse isn't necessarily consistent, so it adds noise into an already very high noise set-up.
That's only a problem if the noise of wireless is higher than the signal (measured delay difference). And we can see from the data it isn't. Unless you're claiming it can be consistently different by ~5ms between those two tests... but that's very unlikely. Like, "you should demonstrate it" unlikely.
Asahi Linas explanation:
https://lobste.rs/s/oxtwre/hard_numbers_wayland_vs_x11_input...
An excerpt from the linked comment's own TL;DR summary:
> X11 has no cursor vsync and tears. Wayland compositors use atomic cursor updates synchronized with vsync.
The title and conclusion are disingenuous - they've proven that Gnome Wayland has higher latency than Gnome Xorg. This could very well have nothing to do with Wayland at all - we need more testing.
This boils down to software cursor vs hardware cursor.
I'm pretty sure this is incorrect? I half-remember having cursor rendering bugs on GNOME Wayland with NVidia ages ago which I had to disable something h hardware cursor related to fix. But I don't know if that's what you're talking about or not. Could you link to a source which says that Mutter doesn't make use of hardware cursor?
(Hi, author here by the way! ... Don't worry, that disabling hardware cursors thing was at least one OS re-install ago)
Yes, it should use hardware cursor by default on most hardware. But I don't think that RPi's VideoCore (in the linked article) supports hardware cursor or multiple planes. The difference between X11 and Wayland might then be that the cursor isn't vsynced as part of the composite process on X11, where it is on Wayland.
Mutter definitely throttles cursor updates on Wayland, too, which will contribute slightly to latency even with a hardware cursor. In general, with Wayland, the compositor is responsible for cursor updates, and I'm not sure which other ones throttle. But that would be where the difference comes from when using hardware cursors.
Pis VideoCore has like 48 planes split up between writeback and HDMI output CRTCs. There’s one explicit Cursor plane per CRTC.
Thanks, I didn't know this.
I think the difference mort96 is seeing is the cursor update throttling. If it updates at the same rate of the refresh rate then it's a crapshoot where in that interval it hits relative to vsync, with a worst case additional latency of the frame rate. X11 updates the cursor whenever it moves, so on scanout it's always where you expect it to be, even if the motion was almost immediately before vsync.
I should mention that in the past there's been problematic interactions on amdgpu with atomic updates of the cursor and display planes. This resulted in either one being starved from changing, causing stuttering in one or the other depending how it was handled. IIRC, that's why the throttle is there. You could try MUTTER_DEBUG_ENABLE_ATOMIC_KMS=0 to see if they only added the throttle to the atomic code path.
I wonder if the latency is due to the compositor receiving a damage event for the area of the cursor, thus being a frame late to render. But probably that wouldn't be an issue with hardware cursors.
This assumption is incorrect.
Thanks, I think you're right.
I think you mean, The display server cursor (we can draw directly to the frame buffer) vs The Compositor Cursor (this involves and extra copy)
It's all software.
honestly, I still don't get the point of wayland.
Kill all the fun in writing window managers?
It should be easy to create something better than X11. And if not pour more resources into X11 developement. Wayland on the other hand is a complete failure on all fronts. It should be abandoned ASAP.
Well it's already better than X11, so task accomplished.
What exactly is better? I can't even think of a single thing.
It has worse performance on almost all metrics. Heavy fragmentation. Very limited and incomplete API measured in functionality but still extremely complicated to work with as developer. Extremely slow development. The only thing that it has over X11 is better funding (for whatever reason).
Are we really pretending multi-monitor on X is not an absolutely shit show? Try mixed DPI and VRR in one of the monitors and see how you get that working well.
I have zero issues with multiple monitors, with mixed DPIs and refresh rates, and different scaling factors on each display, using XFCE under Xorg.
Now drag a window from one monitor to the other. The choice on X is to have both monitors run with the same specs (well, not for everything but yeah) or to have two completely separate screens at the X level which means they're isolated from each other and you can't move thing between them.
No, I'm not having any of the problems you're describing. I'm moving windows between monitors with no visible issues.
Thank you! X just plain doesn't work for that usecase, you'll either get blurry text on one of the monitors or a tiny window with mixed DPI. It fundamentally doesn't work.
Reading some of this thread has made me feel like I'm going insane. I have a wildly different experience of both X and Wayland from a lot of people in this thread it seems.
The VRR issue can be solved with a simple patch that never got merged into mainline (no idea why). Hence it's not inherently a X11 problem and honestly also a rather niche problem compared to the humongous deficits Wayland brings.
Mixed DPI works perfectly fine on X11 if the toolkit supports it. The xrandr protocol provides all the necessary dpi information. GNOME simply chose not to support it on X11.