When the user chooses to create a new XDM virtual desktop with option "Query an X desktop manager" set in the connection GUI, the client shows the "Creating a new XDM virtual desktop" message for a while and immediately after the session fails to start with:
The session negotiation failed. Error: Failed to connect to nxclient --monitor.
The same happens also if option "Let the NX server decide" is used.
This problem has been reproduced with NoMachine server v. 5.1.44 on Debian 8.0 Jessie.
Analysis and workarounds
If possible, switch to a different display manager (GDM, LightDM).
Otherwise, follow the steps below.
Check the XDMCP configuration
Be sure the configuration is correct. All you need to change are two files.
1. /etc/kde4/kdm/Xaccess: Remove heading # from lines:
#* #any host can get a login window
#* CHOOSER BROADCAST #any indirect host can get a chooser
2. /etc/kde4/kdm/kdmrc: In section [Xdmcp] set Enable as true:
Display agent and KDM greeter (kdm_greet) are run by different users. This causes
X Error of failed request: BadAccess (attempt to access private resource denied)
Major opcode of failed request: 130 (MIT-SHM)
Minor opcode of failed request: 1 (X_ShmAttach)
and it looks that kdm_greet provides no fallback if attach fails.
Workaround: wrap /usr/lib/kde4/libexec/kdm_greet to set QT_X11_NO_MITSHM=1
in the environment. As root, run commands:
systemctl stop kdm
mv kdm_greet kdm_greet.bin
echo -ne '#!/bin/bashnnexec env QT_X11_NO_MITSHM=1 $0.bin "$@"' > kdm_greet
chmod +x kdm_greet
systemctl start kdm
The KDM greeter kills all X clients connected to the display before creating its own windows.
This makes nxclient --monitor terminate prematurely and session fails.
Workaround: add a delay to nxclient startup. Edit /usr/NX/bin/nxclient
to add "sleep 5" before the exec command:
exec "$NXPath/bin/nxclient.bin" "$@"
Further information are available here:
Cannot create a NoMachine XDM virtual desktop (solving MIT-SHM issues).