# [OmniOS-discuss] 64/libjvm_db.so in runtime/java on bloody does not seem to be built correctly, pstack doesn't work

Christopher Siden christopher.siden at delphix.com
Thu Jul 11 20:15:45 UTC 2013

- Previous message: [OmniOS-discuss] Zones failing to start after upgrade
- Next message: [OmniOS-discuss] 64/libjvm_db.so in runtime/java on bloody does not seem to be built correctly, pstack doesn't work
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

    Hi,
    I was recently playing around with the OpenJDK build in the bloody
    repo and noticed that pstack was only able to resolve Java stack
    frames when I used the 32-bit binary (/usr/bin/i86/pstack):

     fa00f402 * java/lang/Thread.sleep(J)V+0
     fa00367b * org/apache/tomcat/util/net/JIoEndpoint$AsyncTimeout.run()V+13
    (line 294)

    But the 64-bit pstack binary (/usr/bin/amd64/pstack) just prints
    question marks instead of class and method names:

     fa0f08a6 ???????? (85f4a28, 0, 858a778, 0, 0, 0)
     fa0fae7c ???????? (0, a4d, 0, 4c3, 0, 9ca8ab8f)

    Looking at the code in illumos's pstack.c it seems like pstack looks
    for a library called "libjvm.so" loaded into the target Java process,
    then looks for a library called "libjvm_db.so" in the same directory
    as libjvm.so and opens it with dlopen(3). libjvm_db.so provides pstack
    with the jvm-specific knowledge it needs to read java stack frames
    from the target process. When you are using a 64-bit pstack to debug a
    32-bit java process it uses a special 64-bit version of the
    libjvm_db.so library stored in a sub directory called "64", the
    directory structure looks like this:

    /usr/java/lib/i386/server/libjvm.so
    /usr/java/lib/i386/server/libjvm_db.so
    /usr/java/lib/i386/server/64/libjvm_db.so

    Note that this 64-bit libjvm_db.so is still meant for debugging a
    32-bit JVM, it's just compiled as a 64-bit library so that a 64-bit
    pstack process is capable of using it.

    The problem seems to be that the 64-bit version of libjvm_db.so in the
    runtime/java package in bloody right now is actually a 32-bit library:

    $ file /usr/java/lib/i386/server/64/libjvm_db.so
    /usr/java/lib/i386/server/64/libjvm_db.so: ELF 32-bit ...

    So the call to dlopen(3) from a 64-bit pstack process fails. The
    32-bit Java 7 JDK distributed by Oracle does not have this problem:

    $ file /usr/java/jre/lib/i386/server/libjvm_db.so
    /usr/java/jre/lib/i386/server/libjvm_db.so: ELF 32-bit
    $ file /usr/java/jre/lib/i386/server/64/libjvm_db.so
    /usr/java/jre/lib/i386/server/64/libjvm_db.so: ELF 64-bit

    I was able to use both 64-bit and 32-bit pstack to see the Java stack
    frames of 32-bit Java processes running with Oracle's binaries.

    I downloaded and started building OpenJDK myself mimicing what it
    looks like the omni-build repository scripts do (I didn't run the
    scripts themselves). I let the build run until the 64-bit library was
    created and confirmed that it was in fact a 64-bit library:

    $ file ./build/solaris/solaris_i486_compiler2/product/libjvm_db.so
    ./build/solaris/solaris_i486_compiler2/product/libjvm_db.so: ELF 32-bit
    $ file ./build/solaris/solaris_i486_compiler2/product/64/libjvm_db.so
    ./build/solaris/solaris_i486_compiler2/product/64/libjvm_db.so: ELF 64-bit

    This seems like there is a problem with the way the 64-bit
    libjvm_db.so files currently in the runtime/java package were built,
    but I'm not familiar with how OmniTI builds those packages. Could it
    be they were built on a 32-bit system or something? Otherwise there
    might be some difference between OpenJDK and whatever repo Oracle
    bases their builds on?

    Thanks,
    Chris

- Previous message: [OmniOS-discuss] Zones failing to start after upgrade
- Next message: [OmniOS-discuss] 64/libjvm_db.so in runtime/java on bloody does not seem to be built correctly, pstack doesn't work
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the OmniOS-discuss mailing list