Tracking clustering issues - Alfresco Content Services - 23.4 - 23.4 - Ready - Alfresco - external

Alfresco Content Services

Platform
Alfresco
Product
Alfresco Content Services
Release
23.4
License

Use this information to track clustering issues.

  • The main clustering debug information can be customized using the following log4j setting (default value is INFO):

      logger.alfresco-enterprise-repo-cluster.name=org.alfresco.enterprise.repo.cluster
      logger.alfresco-enterprise-repo-cluster.level=info
    
  • For a better control and more detailed clustering debug information, the following category can be configured:

      org.alfresco.enterprise.repo.cluster.core.ClusteringBootstrap
    

    This controls clustering initialization and shutdown. It provides INFO level startup and shutdown messages. It also provides WARN level messages, if clustering is disabled or an invalid license is installed.

    Here is an example output:

      12:38:38,769 INFO [org.alfresco.enterprise.repo.cluster.core.ClusteringBootstrap] Cluster started, name:
            MainRepository-35ee3b27-0276-4224-9613-3fd8089c6e11
      12:38:38,776 INFO [org.alfresco.enterprise.repo.cluster.core.ClusteringBootstrap] Current cluster
            members:
          10.248.10.205:5701 (hostname: node1.alf.example.com)
          10.208.63.40:5701 (hostname: node2.alf.example.com)
    
  • When a cluster member leaves or joins, the following class generates an informative INFO level message:

      org.alfresco.enterprise.repo.cluster.core.MembershipChangeLogger
    

    Here is an example output:

      12:38:47,560 INFO [org.alfresco.enterprise.repo.cluster.core.MembershipChangeLogger] Member joined:
            10.65.41.64:5701 (hostname: node1.alf.example.com)
      12:38:47,569 INFO [org.alfresco.enterprise.repo.cluster.core.MembershipChangeLogger] Current cluster
            members:
          10.208.63.40:5701 (hostname: solr.alf.example.com)
          10.248.10.205:5701 (hostname: node2.alf.example.com)
          10.65.41.64:5701 (hostname: node1.alf.example.com)
    
  • An important aspect of clustering is caching. To log cache creation (for example, increase the cache related logging to DEBUG level), enable the following log categories:

      logger.alfresco-enterprise-repo-cluster-cache.name=org.alfresco.enterprise.repo.cluster.cache
      logger.alfresco-enterprise-repo-cluster-cache.level=info
      logger.alfresco-repo-cache.name=org.alfresco.repo.cache
      logger.alfresco-repo-cache.level=info
    
  • The underlying clustering technology, Hazelcast, is configured to use log4j for logging. Therefore, you can configure logging for the whole Hazelcast top-level package, as shown:

      logger.hazelcast.name=com.hazelcast
      logger.hazelcast.level=info
    

    To increase logging from Hazelcast’s member joining mechanism, enable the following log category:

      logger.hazelcast-impl-TcpIpJoiner.name=com.hazelcast.impl.TcpIpJoiner
      logger.hazelcast-impl-TcpIpJoiner.level=debug
    
  • Alfresco uses the Hazelcast library internally for clustering synchronization of data. With the addition of Java 11 modules, there is a warning in the application startup log for the repository app, as shown:

      WARNING [localhost-startStop-1] com.hazelcast.instance.HazelcastInstanceFactory.null Hazelcast is starting
          in a Java modular environment (Java 9 and newer) but without proper access to required Java packages. Use
          additional Java arguments to provide Hazelcast access to Java internal API. The internal API access is used
          to get the best performance results. Arguments to be used:
      --add-modules java.se
      --add-exports java.base/jdk.internal.ref=ALL-UNNAMED
      --add-opens java.base/java.lang=ALL-UNNAMED
      --add-opens java.base/java.nio=ALL-UNNAMED
      --add-opens java.base/sun.nio.ch=ALL-UNNAMED
      --add-opens java.management/sun.management=ALL-UNNAMED
      --add-opens jdk.management/com.sun.management.internal=ALL-UNNAMED
    

    See the Hazelcast documentation for more details.

    For more details about the danger of using those parameters, see https://openjdk.java.net/jeps/261. Note the warning on that page:

    Note: The --add-exports and --add-opens options must be used with great care. You can use them to gain access to an internal API of a library module, or even of the JDK itself, but you do so at your own risk: If that internal API is changed or removed, then your library or application will fail.

    We decided not to hide this warning, since that may hide other problems in other areas of the code. This decision won’t impact the performance or security of the repository app. Any required modifications will be reviewed and addressed in the next available Java releases. It is assumed that Hazelcast will probably do the same.