These are the most common cases and their possible solutions.
a) ERROR: Could not open display :0
This can happen because the X server is turned off. In general the display is :0, but it can occur also on a different display.
Check if the X server is up and running by executing in a console:
ps -f -C Xorg
If the X server is not running, start it:
a.1) Enable the display manager, if not already enabled. For example, if you want to enable GDM (GNOME Display Manager) on RHEL 7/CentOS 7:
systemctl enable gdm
a.2) Start the dsiplay manager. E.g on RHEL 7/CentOS 7:
systemctl start display-manager
b) Invalid MIT-MAGIC-COOKIE-1 key[VGL] ERROR: Could not open display :0.
This happens when the X server doesn't have the authorization open to local connections. In order to debug the problem, check if the display is open to local connections by executing in a console:
env DISPLAY=:0 xhost
This should return "LOCAL:" among the results. If not, please ensure that:
b.1) the VirtualGL environment has been correctly set-up. See https://www.nomachine.com/AR05P00982.
b.2) the display manager has been restarted. For example on RHEL 7/CentOS 7 and the other OS based on systemd:
systemctl restart display-manager
c) gnome-session-binary: WARNING: software acceleration check failed: Child process exited with code
The message above is related to a GNOME desktop environment, but it can occur also with other desktop environments using 3D effects and requiring 3D acceleration like KDE Plasma, Cinnamon.
To solve the problem, try to launch the desktop environment by setting the XDG_SESSION_TYPE=x11 environment variable. Add it to the DefaultDesktopCommand key in /usr/NX/etc/node.cfg.
For example, for GNOME:
DefaultDesktopCommand "/etc/gdm/Xsession 'env XDG_SESSION_TYPE=x11 gnome-session --session=gnome'"
Otherwise, for GNOME you can add the --disable-acceleration-check to the gnome-session command:
DefaultDesktopCommand "/etc/gdm/Xsession 'gnome-session --session=gnome --disable-acceleration-check'"
d) IMPORTANT: AMD and NVIDIA GPUs require to use proprietary drivers for VirtualGL.
From https://cdn.rawgit.com/VirtualGL/virtualgl/2.6.1/doc/index.html#hd004001, par. 4.1:
Install the AMD or nVidia proprietary drivers. Open source drivers for these GPUs do not generally provide full 3D acceleration, and some of those drivers do not provide Pbuffer support.
General steps to debug VirtualGL issues
The following steps are helpful for a general debug and self-troubleshooting. Provide the logs and any additional information you collected to the Support Team to receive technical assistance.
The first section, basic debug, is preliminary to start a specific investigation on your case.
1) Check the .xsession-errors file in the home of the user who experienced the problem. It may report why the desktop environment failed to start in the NoMachine virtual desktop.
2) If the OS of the remote machine where the problem occurs is systemd-based, run in a console:
journalctl -b -0 > journalctl.txt
In case of problems related to GNOME desktop, you can make a specific 'grep' to gather only log messages related to GNOME:
journalctl -b -0 | grep -i gnome > journalctl-gnome.txt
If you know the time range when the problem occurred, you may filter data by time, e.g.:
journalctl --since "2019-04-10 17:15:00"
journalctl --since "
2019-04-10" --until "2019-04-11 03:00"
3) Further information about the problem can be in the NoMachine log files ('session' and 'clients') in the session directory on the remote computer. Follow steps below to retrieve them.
3.1) Execute on the remote host where the problem occurs:
sudo /etc/NX/nxserver --debug --enable agent
3.2) Reproduce the problem by creating a new session.
sudo /etc/NX/nxserver --debug --collect
The log archive is created in /usr/NX/var/log/archives.
3.4) Restore normal log level:
sudo /etc/NX/nxserver --debug --disable agent
If you're running an old NoMachine server version, up to v. 6.3, collect logs as explained here: https://www.nomachine.com/DT10O00163#1.1
4) Running glxinfo and/or glxgears under the provision of VirtualGL can also provide information about the problem. Execute the following test:
4.1) turn VirtualGL off by setting in /usr/NX/etc/node.cfg:
4.2) Connect by NoMachine and choose to create a new custom session with command xterm. In the terminal, execute:
/usr/NX/scripts/vgl/vglrun +tr +v glxgears
/usr/NX/scripts/vgl/vglrun +tr +v glxinfo
Possible problems should be displayed in the console output and in the system error files, see point 1) and 2) above.
4.3) Or, if glxinfo and glxgears run without errors, you can try to execute the whole desktop by adding vglrun to the command in the
DefaultDesktopCommand key in /usr/NX/etc/node.cfg.
For example: DefaultDesktopCommand "/usr/NX/scripts/vgl/vlgrun dbus-launch --exit-with-session gnome-session --session=ubuntu"