Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent possible deadlock #5830

Merged
merged 1 commit into from
Apr 14, 2022
Merged

Conversation

trask
Copy link
Member

@trask trask commented Apr 14, 2022

I ran into the deadlock below in our distro, but it could potentially affect other people.

I found various discussions of the underlying Java 8 bug, but this is probably the best reference: https://bugs.openjdk.java.net/browse/JDK-8030092, and a good note at the bottom why it's no longer an issue in Java 9+.

Found one Java-level deadlock:
=============================
"sender":
  waiting to lock monitor 0x00007f8c38005b18 (object 0x000000073e4a2f00, a org.apache.juli.ClassLoaderLogManager),
  which is held by "main"
"main":
  waiting to lock monitor 0x00007f8c38003c28 (object 0x000000073e986518, a java.lang.Class),
  which is held by "sender"

Java stack information for the threads listed above:
===================================================
"sender":
        at java.util.logging.LogManager.ensureLogManagerInitialized(LogManager.java:317)
        - waiting to lock <0x000000073e4a2f00> (a org.apache.juli.ClassLoaderLogManager)
        at java.util.logging.LogManager.getLogManager(LogManager.java:378)
        at java.util.logging.Logger.getPlatformLogger(Logger.java:572)
        at java.util.logging.LoggingProxyImpl.getLogger(LoggingProxyImpl.java:41)
        at sun.util.logging.LoggingSupport.getLogger(LoggingSupport.java:100)
        at sun.util.logging.PlatformLogger$JavaLoggerProxy.<init>(PlatformLogger.java:602)
        at sun.util.logging.PlatformLogger$JavaLoggerProxy.<init>(PlatformLogger.java:597)
        at sun.util.logging.PlatformLogger.<init>(PlatformLogger.java:239)
        at sun.util.logging.PlatformLogger.getLogger(PlatformLogger.java:198)
        - locked <0x000000073e986518> (a java.lang.Class for sun.util.logging.PlatformLogger)
        at jdk.internal.event.EventHelper.isLoggingSecurity(EventHelper.java:138)
        at sun.security.provider.X509Factory.commitEvent(X509Factory.java:766)
        at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:108)
        at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339)
        at sun.security.provider.JavaKeyStore.engineLoad(JavaKeyStore.java:763)
        - locked <0x000000073e8af628> (a java.util.Hashtable)
        at sun.security.provider.JavaKeyStore$JKS.engineLoad(JavaKeyStore.java:57)
        at sun.security.provider.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:224)
        at sun.security.provider.JavaKeyStore$DualFormatJKS.engineLoad(JavaKeyStore.java:71)
        at java.security.KeyStore.load(KeyStore.java:1445)
        at sun.security.ssl.TrustStoreManager$TrustAnchorManager.loadKeyStore(TrustStoreManager.java:365)
        at sun.security.ssl.TrustStoreManager$TrustAnchorManager.getTrustedCerts(TrustStoreManager.java:313)
        - locked <0x000000073e8832e8> (a sun.security.ssl.TrustStoreManager$TrustAnchorManager)
        at sun.security.ssl.TrustStoreManager.getTrustedCerts(TrustStoreManager.java:55)
        at sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:49)
        at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:250)
        at sun.security.ssl.SSLContextImpl.engineInit(SSLContextImpl.java:88)
        at javax.net.ssl.SSLContext.init(SSLContext.java:282)
        at io.netty.handler.ssl.JdkSslContext.<clinit>(JdkSslContext.java:75)
        at io.netty.handler.ssl.SslContext.newClientContextInternal(SslContext.java:824)
        at io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:611)
        ...

"main":
        at sun.util.logging.PlatformLogger.redirectPlatformLoggers(PlatformLogger.java:209)
        - waiting to lock <0x000000073e986518> (a java.lang.Class for sun.util.logging.PlatformLogger)
        at java.util.logging.LogManager$3.run(LogManager.java:402)
        at java.util.logging.LogManager$3.run(LogManager.java:396)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.util.logging.LogManager.readPrimordialConfiguration(LogManager.java:396)
        - locked <0x000000073e4a2f00> (a org.apache.juli.ClassLoaderLogManager)
        at java.util.logging.LogManager.access$800(LogManager.java:145)
        at java.util.logging.LogManager$2.run(LogManager.java:345)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.util.logging.LogManager.ensureLogManagerInitialized(LogManager.java:338)
        - locked <0x000000073e4a2f00> (a org.apache.juli.ClassLoaderLogManager)
        at java.util.logging.LogManager.getLogManager(LogManager.java:378)
        at java.util.logging.Logger.demandLogger(Logger.java:448)
        at java.util.logging.Logger.getLogger(Logger.java:502)
        at org.apache.juli.logging.DirectJDKLog.<init>(DirectJDKLog.java:68)
        at org.apache.juli.logging.DirectJDKLog.getInstance(DirectJDKLog.java:188)
        at org.apache.juli.logging.LogFactory.getInstance(LogFactory.java:131)
        at org.apache.juli.logging.LogFactory.getInstance(LogFactory.java:155)
        at org.apache.juli.logging.LogFactory.getLog(LogFactory.java:210)
        at org.apache.catalina.startup.Bootstrap.<clinit>(Bootstrap.java:52)

Found 1 deadlock.

@trask trask requested a review from a team as a code owner April 14, 2022 06:30
@trask trask added this to the v1.13.0 milestone Apr 14, 2022
@trask trask merged commit 4879ced into open-telemetry:main Apr 14, 2022
@trask trask deleted the prevent-possible-deadlock branch April 14, 2022 22:00
RashmiRam pushed a commit to RashmiRam/opentelemetry-auto-instr-java that referenced this pull request May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants