ASM & Javaagent bytecode instrumentation: ClassFormatError: StackMapTable format error: bad offset for Uninitialized

user_11495796 0
Viewed 120

What am I doing

I am using ASM and javaagent to instrument classes to report their coverage (why am I not using jacoco? Well it is nothing to do with this question), the basic logic is that, every time visitLineNumber is called, I instrument some method invocation (right before visiting the next instruction) to record the hit line number.

Problem description

With such a simple logic, a class got ClassFormatError:

java.lang.ClassFormatError: StackMapTable format error: bad offset for Uninitialized in method org.apache.commons.math.ode.ContinuousOutputModelTest.buildInterpolator(D[DD)Lorg/apache/commons/math/ode/sampling/StepInterpolator;
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    ...

The bytecode before the instrumentation is shown below. The instructions where the stack map frames are located are @16(offset_delta = 16) and @17(offset_delta = 0).

private org.apache.commons.math.ode.sampling.StepInterpolator buildInterpolator(double, double[], double);
    descriptor: (D[DD)Lorg/apache/commons/math/ode/sampling/StepInterpolator;
    flags: ACC_PRIVATE
    Code:
      stack=7, locals=7, args_size=4
         0: new           #66                 // class org/apache/commons/math/ode/sampling/DummyStepInterpolator
         3: dup
         4: aload_3
         5: dload         4
         7: dload_1
         8: dcmpl
         9: iflt          16
        12: iconst_1
        13: goto          17
        16: iconst_0
        17: invokespecial #67                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator."<init>":([DZ)V
        20: astore        6
        22: aload         6
        24: dload_1
        25: invokevirtual #68                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator.storeTime:(D)V
        28: aload         6
        30: invokevirtual #69                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator.shift:()V
        33: aload         6
        35: dload         4
        37: invokevirtual #68                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator.storeTime:(D)V
        40: aload         6
        42: areturn
         ...
      StackMapTable: number_of_entries = 2
        frame_type = 255 /* full_frame */
          offset_delta = 16
          locals = [ class org/apache/commons/math/ode/ContinuousOutputModelTest, double, class "[D", double ]
          stack = [ uninitialized 0, uninitialized 0, class "[D" ]
        frame_type = 255 /* full_frame */
          offset_delta = 0
          locals = [ class org/apache/commons/math/ode/ContinuousOutputModelTest, double, class "[D", double ]
          stack = [ uninitialized 0, uninitialized 0, class "[D", int ]

After the instrumentation, the bytecode becomes:

private org.apache.commons.math.ode.sampling.StepInterpolator buildInterpolator(double, double[], double);
    descriptor: (D[DD)Lorg/apache/commons/math/ode/sampling/StepInterpolator;
    flags: ACC_PRIVATE
    Code:
      stack=10, locals=7, args_size=4
         0: ldc_w         #264                // String org/apache/commons/math/ode/ContinuousOutputModelTest
         3: ldc_w         #344                // String buildInterpolator
         6: ldc_w         #345                // int 169
         9: invokestatic  #272                // Method org/test/cov/CoverageCollector.reportCoverage:(Ljava/lang/String;Ljava/lang/String;I)V
        12: new           #66                 // class org/apache/commons/math/ode/sampling/DummyStepInterpolator
        15: dup
        16: aload_3
        17: dload         4
        19: dload_1
        20: dcmpl
        21: iflt          28
        24: iconst_1
        25: goto          29
        28: iconst_0
        29: invokespecial #67                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator."<init>":([DZ)V
        32: astore        6
        34: ldc_w         #264                // String org/apache/commons/math/ode/ContinuousOutputModelTest
        37: ldc_w         #344                // String buildInterpolator
        40: ldc_w         #346                // int 170
        43: invokestatic  #272                // Method org/test/cov/CoverageCollector.reportCoverage:(Ljava/lang/String;Ljava/lang/String;I)V
        46: aload         6
        48: dload_1
        49: invokevirtual #68                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator.storeTime:(D)V
        52: ldc_w         #264                // String org/apache/commons/math/ode/ContinuousOutputModelTest
        55: ldc_w         #344                // String buildInterpolator
        58: ldc_w         #347                // int 171
        61: invokestatic  #272                // Method org/test/cov/CoverageCollector.reportCoverage:(Ljava/lang/String;Ljava/lang/String;I)V
        64: aload         6
        66: invokevirtual #69                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator.shift:()V
        69: ldc_w         #264                // String org/apache/commons/math/ode/ContinuousOutputModelTest
        72: ldc_w         #344                // String buildInterpolator
        75: ldc_w         #348                // int 172
        78: invokestatic  #272                // Method org/test/cov/CoverageCollector.reportCoverage:(Ljava/lang/String;Ljava/lang/String;I)V
        81: aload         6
        83: dload         4
        85: invokevirtual #68                 // Method org/apache/commons/math/ode/sampling/DummyStepInterpolator.storeTime:(D)V
        88: ldc_w         #264                // String org/apache/commons/math/ode/ContinuousOutputModelTest
        91: ldc_w         #344                // String buildInterpolator
        94: ldc_w         #349                // int 173
        97: invokestatic  #272                // Method org/test/cov/CoverageCollector.reportCoverage:(Ljava/lang/String;Ljava/lang/String;I)V
       100: aload         6
       102: areturn
        ...
      StackMapTable: number_of_entries = 2
        frame_type = 255 /* full_frame */
          offset_delta = 28
          locals = [ class org/apache/commons/math/ode/ContinuousOutputModelTest, double, class "[D", double ]
          stack = [ uninitialized 0, uninitialized 0, class "[D" ]
        frame_type = 255 /* full_frame */
          offset_delta = 0
          locals = [ class org/apache/commons/math/ode/ContinuousOutputModelTest, double, class "[D", double ]
          stack = [ uninitialized 0, uninitialized 0, class "[D", int ]

I didn't see any problem with the StackMapTable. Any ideas about why this StackMapTable format is invalid?

The following is my instrumentation code:


class CoverageMethodVisitor extends MethodVisitor {

    private String slashClassName;
    private String methodName;
    private int currentLine;
    private boolean isJUnit3TestClass;
    private boolean hasTestAnnotation;
    private boolean isTestMethod;
    private int classVersion;

    private boolean isRightAfterLabel;

    protected CoverageMethodVisitor(MethodVisitor methodVisitor, String className, String methodName, boolean isJUnit3TestClass, int classVersion) {
        super(ASM_VERSION, methodVisitor);
        this.slashClassName = className;
        this.methodName = methodName;
        this.isJUnit3TestClass = isJUnit3TestClass;
        this.classVersion = classVersion;
    }

    private void instrumentReportCoverageInvocation() {
        super.visitLdcInsn(slashClassName);
        super.visitLdcInsn(methodName);
        super.visitLdcInsn(currentLine);
        super.visitMethodInsn(INVOKESTATIC, "org/test/cov/CoverageCollector",
                "reportCoverage", "(Ljava/lang/String;Ljava/lang/String;I)V", false);
    }

    @Override
    public void visitInsn(int opcode) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitInsn(opcode);
    }

    @Override
    public void visitIntInsn(int opcode, int operand) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitIntInsn(opcode, operand);
    }

    @Override
    public void visitVarInsn(int opcode, int varIndex) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitVarInsn(opcode, varIndex);
    }

    @Override
    public void visitTypeInsn(int opcode, String type) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitTypeInsn(opcode, type);
    }

    @Override
    public void visitFieldInsn(int opcode, String owner, String name, String descriptor) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitFieldInsn(opcode, owner, name, descriptor);
    }

    @Override
    public void visitMethodInsn(int opcode, String owner, String name, String descriptor, boolean isInterface) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitMethodInsn(opcode, owner, name, descriptor, isInterface);
    }

    @Override
    public void visitInvokeDynamicInsn(String name, String descriptor, Handle bootstrapMethodHandle, Object... bootstrapMethodArguments) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitInvokeDynamicInsn(name, descriptor, bootstrapMethodHandle, bootstrapMethodArguments);
    }

    @Override
    public void visitJumpInsn(int opcode, Label label) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitJumpInsn(opcode, label);
    }

    @Override
    public void visitLdcInsn(Object value) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitLdcInsn(value);
    }

    @Override
    public void visitIincInsn(int varIndex, int increment) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitIincInsn(varIndex, increment);
    }

    @Override
    public void visitTableSwitchInsn(int min, int max, Label dflt, Label... labels) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitTableSwitchInsn(min, max, dflt, labels);
    }

    @Override
    public void visitLookupSwitchInsn(Label dflt, int[] keys, Label[] labels) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitLookupSwitchInsn(dflt, keys, labels);
    }

    @Override
    public void visitMultiANewArrayInsn(String descriptor, int numDimensions) {
        if (isRightAfterLabel) instrumentReportCoverageInvocation(); isRightAfterLabel = false;
        super.visitMultiANewArrayInsn(descriptor, numDimensions);
    }

    @Override
    public void visitMaxs(int maxStack, int maxLocals) {
        super.visitMaxs(maxStack+3, maxLocals);
    }

    /**
     * Should not report line coverage immediately after the visitLineNumber. visitLineNumber is called right after
     * visitLabel, but it is very possible that a stack map frame is after the label, if insert instructions right
     * after the label, the original stack map frame will be messed up. So instead, insert instructions before the
     * first instruction after the label. */
    @Override
    public void visitLineNumber(int line, Label start) {
        super.visitLineNumber(line, start);
        currentLine = line;
        isRightAfterLabel = true;
    }
}

The comment in /** */ is actually my guess, not sure if it is correct.

EDIT:

I'm sorry I misunderstood the meaning of offset_delta = 0, as the java spec mentions:

The bytecode offset at which a frame applies is calculated by adding offset_delta + 1 to the bytecode offset of the previous frame, unless the previous frame is the initial frame of the method, ...

Summary

Reason of the error

The Uninitialized in the bad offset for Uninitialized refers to the uninitialized object produced by the NEW instruction (the first instruction in the original bytecode). Since the original stack map frame needs to use the label (let's say L0) before the NEW instruction to represent the uninitialized object in its locals and stack, and the L0 in the instrumented code no longer represents the NEW instruction, such error is thrown.

Solution

Since the L0 in the instrumented code no longer represents the NEW instruction, we need to create a new label for that NEW instruction, and replace the old label L0 with the new label in both stack map frames. If COMPUTE_FRAME is specified, such stack map frame (re)computation will be done automatically, but here we need to do it manually since COMPUTE_FRAME is not used to avoid other latent problems.

java-bytecode-asm jvm-bytecode instrumentation javaagents java
2 Answers

As Rafael Winterhalter said, labels are used to refer to NEW instructions, therefore placing instructions between the label’s location and the instruction will break this reference. Such references are required by stack map frames when an uninitialized instance created by the instruction is on the stack or in a local variable.

Since you want to keep the original location as branch target or line number start when inserting code before the NEW instruction, you have to create a new label for the code location between the injected code and the old NEW instruction, then replace the labels used by stack map frames (and only those used by stack map frames).

Note that this wouldn’t be necessary when using the COMPUTE_FRAMES option which will recompute the frames from scratch and not use the old labels, however, it’s actually a reasonable strategy not to use this option, as it is expensive and error prone. For injecting simple logging statements without branches, we can keep the original frames, even when checking for the special case of NEW instructions makes it slightly harder.

The following method visitor implements the strategy described above. For my testing, I just injected a plain System.out.println(…); statement.

class CoverageMethodVisitor extends MethodVisitor {
    private final String slashClassName;
    private final String methodName;
    private final Map<Label,Label> translateForUninitialized = new HashMap<>();
    private Label lastLabel;
    private int newLineNumber = -1;

    protected CoverageMethodVisitor(MethodVisitor mv,String clName,String methodName){
        super(Opcodes.ASM9, mv);
        this.slashClassName = clName;
        this.methodName = methodName;
    }

    @Override
    public void visitLineNumber(int line, Label start) {
        newLineNumber = line;
        super.visitLineNumber(line, start);
    }

    @Override
    public void visitLabel(Label label) {
        lastLabel = label;
        super.visitLabel(label);
    }

    @Override
    public void visitTypeInsn(int opcode, String type) {
        if(instrumentReportCoverageInvocation() && opcode == Opcodes.NEW) {
            if(lastLabel != null) {
                Label newLabel = new Label();
                super.visitLabel(newLabel);
                translateForUninitialized.put(lastLabel, newLabel);
            }
        }
        super.visitTypeInsn(opcode, type);
    }

    @Override
    public void visitFrame(int type,
        int numLocal, Object[] local, int numStack, Object[] stack) {
        switch(type) {
            case Opcodes.F_NEW, Opcodes.F_FULL -> {
                local = replaceLabels(numLocal, local);
                stack = replaceLabels(numStack, stack);
            }
            case Opcodes.F_APPEND -> local = replaceLabels(numLocal, local);
            case Opcodes.F_CHOP, Opcodes.F_SAME -> {}
            case Opcodes.F_SAME1 -> stack = replaceLabels(1, stack);
            default -> throw new AssertionError();
        }
        super.visitFrame(type, numLocal, local, numStack, stack);
    }

    private Object[] replaceLabels(int num, Object[] array) {
        Object[] result = array;
        for(int ix = 0; ix < num; ix++) {
            Label repl = translateForUninitialized.get(result[ix]);
            if(repl == null) continue;
            if(result == array) result = array.clone();
            result[ix] = repl;
        }
        return result;
    }

    private boolean instrumentReportCoverageInvocation() {
        int lineNumber = newLineNumber;
        if(lineNumber < 0) return false;
        newLineNumber = -1;
        super.visitFieldInsn(Opcodes.GETSTATIC,
             "java/lang/System", "out", "Ljava/io/PrintStream;");
        super.visitLdcInsn(slashClassName + "." + methodName + " line " + lineNumber);
        super.visitMethodInsn(Opcodes.INVOKEVIRTUAL,
            "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false);
        return true;
    }

    @Override
    public void visitLdcInsn(Object value) {
        instrumentReportCoverageInvocation();
        super.visitLdcInsn(value);
    }

    @Override
    public void visitInsn(int opcode) {
        instrumentReportCoverageInvocation();
        super.visitInsn(opcode);
    }

    @Override
    public void visitIntInsn(int opcode, int operand) {
        instrumentReportCoverageInvocation();
        super.visitIntInsn(opcode, operand);
    }

    @Override
    public void visitVarInsn(int opcode, int varIndex) {
        instrumentReportCoverageInvocation();
        super.visitVarInsn(opcode, varIndex);
    }

    @Override
    public void visitFieldInsn(int opcode,
        String owner, String name, String descriptor) {
        instrumentReportCoverageInvocation();
        super.visitFieldInsn(opcode, owner, name, descriptor);
    }

    @Override
    public void visitMethodInsn(int opcode,
        String owner, String name, String descriptor, boolean isInterface) {
        instrumentReportCoverageInvocation();
        super.visitMethodInsn(opcode, owner, name, descriptor, isInterface);
    }

    @Override
    public void visitInvokeDynamicInsn(
            String name, String descriptor, Handle bootstrapMethodHandle,
            Object... bootstrapMethodArguments) {
        instrumentReportCoverageInvocation();
        super.visitInvokeDynamicInsn(
            name, descriptor, bootstrapMethodHandle, bootstrapMethodArguments);
    }

    @Override
    public void visitJumpInsn(int opcode, Label label) {
        instrumentReportCoverageInvocation();
        super.visitJumpInsn(opcode, label);
    }

    @Override
    public void visitIincInsn(int varIndex, int increment) {
        instrumentReportCoverageInvocation();
        super.visitIincInsn(varIndex, increment);
    }

    @Override
    public void visitTableSwitchInsn(int min, int max, Label dflt, Label... labels) {
        instrumentReportCoverageInvocation();
        super.visitTableSwitchInsn(min, max, dflt, labels);
    }

    @Override
    public void visitLookupSwitchInsn(Label dflt, int[] keys, Label[] labels) {
        instrumentReportCoverageInvocation();
        super.visitLookupSwitchInsn(dflt, keys, labels);
    }

    @Override
    public void visitMultiANewArrayInsn(String descriptor, int numDimensions) {
        instrumentReportCoverageInvocation();
        super.visitMultiANewArrayInsn(descriptor, numDimensions);
    }

    @Override
    public AnnotationVisitor visitInsnAnnotation(
        int typeRef, TypePath tp, String desc, boolean visible) {
        instrumentReportCoverageInvocation();
        return super.visitInsnAnnotation(typeRef, tp, desc, visible);
    }
}

I used a straight-forward class visitor

class CoverageClassVisitor extends ClassVisitor {
    private String className;

    CoverageClassVisitor(ClassVisitor cv) {
        super(Opcodes.ASM9, cv);
    }

    @Override
    public void visit(
        int version, int acc, String name, String sig, String superName, String[] ifs) {

        className = name;
        super.visit(version, acc, name, sig, superName, ifs);
    }
    @Override
    public MethodVisitor visitMethod(int access,
        String name, String descriptor, String signature, String[] exceptions) {
        return new CoverageMethodVisitor(
            super.visitMethod(access, name, descriptor, signature, exceptions),
            className,
            name);
    }
}

…and the following code for testing it

public class ReportLineNumbers {
    public static void main(String[] arg) throws IOException, IllegalAccessException {
        String className = ReportLineNumbers.class.getName() + "$Example";
//        ToolProvider.findFirst("javap")
//            .ifPresent(p -> p.run(System.out, System.err, "-c", className));
        ClassReader cr = new ClassReader(className);
        ClassWriter cw = new ClassWriter(cr, ClassWriter.COMPUTE_MAXS);
        CoverageClassVisitor trans = new CoverageClassVisitor(cw);
        cr.accept(trans, 0);
        MethodHandles.lookup().defineClass(cw.toByteArray());
        Example.method();
    }
    static class Example {
        static void method() {
            for(int i = 0; i < 3; i++) {
                System.out.println(
                    i < 2?
                    new String(switch(i) {
                        case 0 -> {
//                            try {
                                yield "0";
//                            } catch(Throwable t) {
//                                yield "e";
//                            }
                        }
                        default -> {
                            for(int j = 1; j < 3; j++) {
                                System.out.println(j);
                            }
                            yield "3";
                        }
                    }):
                    new String(i == 2?
                        "4".toCharArray():
                        "5".toCharArray())
                );
            }
        }
    }
}

The Example class has been designed to be as challenging as possible, covering different branch types (forward, backward, switch) before and after NEW instructions. In fact, it turned out to be so challenging that I had to comment out one construct, as with Eclipse, it produced invalid bytecode even without the transformer. But when using javac, you can enable this inlined try … catch … construct, to see that the transformer will even handle this correctly.

user_27114880

ASM uses labels to mark offsets in byte code. If you place an instruction between a label and the original NEW instruction, you are breaking the reference.

Therefore, you need to avoid placing instructions directly after labels or issue a new label and replace the original one in the stack map frame.

user_12375750
Related