Payara 5.2020.4 didn't start

This is due to a change in Payara Server configuration that's not compatible with how the older version of the Netbeans Payara (GlassFish) plugin launches Payara Server. The plugin uses a hacky mechanism that reads the domain configuration and launches the Java process of Payara Server directly instead of using the asadmin launcher.

You can easily fix this by editing the domain.xml file in glassfish/domains/domain1/config/domain.xml. Just remove all jvm-option elements where you find [ and ] brackets. These define the Java version for which the JVM option is applicable. Usually this is for running on JDK 9+, so it's safe to remove those options if you run on Java 8.

So, remove this option and all similar options:

<jvm-option>[9|]--add-opens=java.logging.java.util.logging=ALL-UNNAMED</jvm-option>

Upgrading Netbeans to the latest version 12.1 also fixes this problem as Netbeans has been updated to understand this change in the configuration.

I have got a similar problem with jdk1.8.0_311 , payara-5.2021.10 on netbeans 8.2 . payara start successfully in a terminal with command :

glassfish/bin/asadmin start-domain

, but when I start the server inside netbeans, i have some errors :

Unrecognized VM option 'UseOpenJSSE'
Unrecognized VM option 'HotswapAgent=core'
Unrecognized option: -Xlog:redefine+class*=info

The solution by OndroMih works fine. I have removed this lines in domain.xml. "glassfish/domains/domain1/config/domain.xml"

 <jvm-options>[Azul-1.8.0u222|1.8.0u260]-XX:+UseOpenJSSE</jvm-options>
 <jvm-options>[Dynamic Code Evolution-11.0.10|]-XX:HotswapAgent=core</jvm-options>
 <jvm-options>[Dynamic Code Evolution-11.0.10|]-Xlog:redefine+class*=info</jvm-options>

now its ok