Session logs on server side should report which kind of H.264 encoder is used also when the sessions is running in X11 vector graphics mode ('AgentX11VectorGraphics 1' in node.cfg):
Info: Using H.264 hardware encoder
or:
Info: Using H.264 software encoder
Note that in the current version, only H.264 software encoding is used and only for streaming multimedia content. Future versions will use H.264 hardware encoding, when available, for streaming multimedia content in case of sessions in X11 vector graphics mode, see: https://www.nomachine.com/FR11O03550.