Implementing indirect instruction execution in processor plugin

[shuffle2]

The Andes V3 instruction set has an extension named EX9.IT.
EX9.IT extension adds a system register ITB (Instruction_Table Base) which points to a memory location which will be treated as a 512-entry array of 32bit values.
The EX9.IT instruction takes an immediate value which is used to index into this table.
32bits from the given index are fetched, and a single instruction is executed (if the 32bit data begins with a 16bit instruction, only 1 instruction is executed).
The feature is designed to allow executing a 32bit instruction from a 16bit one (EX9.IT is 16bit insn), the idea being that the most frequently used 32bit instructions can be extracted from the main executable and put into Instruction_Table, resulting in overall binary size decrease.

Usage looks something like:

# set ITB=0x4040
MOVI      a0, 0x4040
MTUSR     a0, ITB
...
# execute 1 instruction from Instruction_Table[0x10] inline
EX9.IT    0x10
...

Enlightening a disassembler about this has 2 annoyances:

1. ITB register value must be tracked
2. Upon EX9.IT disassembly, parser must be redirected to handle value fetched from ITB, while still considering the resulting instruction as "inline".

1 seems a common problem, and can be handled in ghidra similar to tracking other global pointers.
2 Seems more tricky, I'm not sure how to tackle this in sleigh. What is a way to handle this?

[shuffle2]

I realized this construct is sort of like an inverted delayslot. However, I don't think ghidra's existing delayslot behavior can be abused to implement it.

[shuffle2]

I implemented it with callotherfixup dynamic pcode and it seems to work well enough (although would be nicer if didn't have to do it this way)

[GhidorahRex]

The only way I can think of would be having a hidden return address register and context value that flags an indirect execution:

register ... [ returnadd ];
indir=(0,0)
...
:EX9.IT ... [indir=1;] {
    addr = ITB + offset;
    returnadd = inst_next;
    call [addr];
}

and for every instruction that can potentially be used in the table, have a return check:

rtn_check: indir=0 {}
rtn_check:indir=1[indir=0;]{ return [returnadd];}
...
:add regA, regB is opcode=0x1 & regA & regB & rtn_check {
    regA = regA + regB;
    build rtn_check;
}

I haven't tested this, but regardless this will only work if the instructions used in the EXT9.IT instruction are used as one-off instructions. It definitely will not work if they are instructions that could be used in a non-inline context and an inline context.

[shuffle2]

Thanks for the suggestion. However, the instructions referenced by EX9.IT are used in both non-inline and inline contexts.

My pcode injection looks like this: (basically, using PseudoDisassembler to get the pcode of the referenced instruction)

public class InjectEX9IT extends InjectPayloadCallother {
	private PcodeOp[] EMPTY_PCODEOP = new PcodeOp[0];
	private int INSTRUCTION_TABLE_ENTRY_LENGTH = 4;
	private AddressSpace defaultSpace;
	private CodeUnitFormat codeUnitFormat;
	private Register itbReg;
	private Register contextReg;

	public InjectEX9IT(String sourceName, SleighLanguage language) {
		super(sourceName);
		itbReg = language.getRegister("ITB");
		contextReg = language.getRegister("contextreg");
		defaultSpace = language.getAddressFactory().getDefaultAddressSpace();
		codeUnitFormat = new CodeUnitFormat(new CodeUnitFormatOptions());
	}

	PseudoInstruction disasmAt(Program program, Address disasmAddr, Address fetchAddr) {
		BigInteger context = program.getProgramContext().getValue(contextReg, disasmAddr, false);
		if (context == null || context == BigInteger.valueOf(0)) {
			throw new IllegalArgumentException("ITMode 0 in EX9.IT");
		}
		PseudoDisassembler disassembler = new PseudoDisassembler(program);
		byte[] data = new byte[INSTRUCTION_TABLE_ENTRY_LENGTH];
		try {
			if (program.getMemory().getBytes(fetchAddr, data) != Array.getLength(data)) {
				return null;
			}
			return disassembler.disassemble(disasmAddr, data);
		} catch (Exception ex) {
			return null;
		}
	}

	@Override
	public PcodeOp[] getPcode(Program program, InjectContext con) {
		Address ex9itAddr = con.baseAddr;
		int imm9u = (int) con.inputlist.get(0).getOffset();
		int itOffset = imm9u * INSTRUCTION_TABLE_ENTRY_LENGTH;

		BigInteger ITB = program.getProgramContext().getValue(itbReg, ex9itAddr, false);
		if (ITB == null) {
			return EMPTY_PCODEOP;
		}
		long memOffset = (ITB.longValue() & ~0b11) + itOffset;

		Address fetchAddr = defaultSpace.getAddress(memOffset);
		PseudoInstruction insn = disasmAt(program, ex9itAddr, fetchAddr);
		// Could be bad ITB
		if (insn == null) {
			return EMPTY_PCODEOP;
		}

		// Set comment if there's a valid insn referenced.
		// TODO append the referenced instruction in a more disassembler-aware way
		Listing listing = program.getListing();
		if (listing.getComment(CodeUnit.EOL_COMMENT, ex9itAddr) == null) {
			// getRepresentationString is also slow
			String ex9itComment = codeUnitFormat.getRepresentationString(insn);
			program.withTransaction("set EX9.IT comment", () -> {
				listing.setComment(ex9itAddr, CodeUnit.EOL_COMMENT,
						String.format("%s {%s}", fetchAddr.toString(), ex9itComment));
			});
		}

		String mnem = insn.getMnemonicString();
		if (mnem == "EX9.IT") {
			// hw would generate Reserved Instruction Exception
			return EMPTY_PCODEOP;
		}

		// 32bit insns which use inst_next (PC + 4) need to be fixed up to use PC + 2,
		// since EX9.IT is 16bit. J/JAL also OR displacement instead of ADDing it.
		// if J : PC = concat(PC[31,25], (Inst[23,0] << 1)) // not signed? and OR instead of ADD
		// if JAL : R30 = PC + 2; PC = concat(PC[31,25], (Inst[23,0] << 1))
		// JRAL, JRAL.xTON, JRALNEZ, BGEZAL, BLTZAL: RT = PC + 2
		// These are taken care of in sleigh code.
		//String[] pcRel = { "J", "JAL", "JRAL", "JRAL.xTON", "JRALNEZ", "BGEZAL", "BLTZAL" };
		//if (Arrays.asList(pcRel).contains(mnem)) {
		//	return EMPTY_PCODEOP;
		//}

		return insn.getPcode();
	}

}

This is working well except for the issue of 32bit instructions which would normally use inst_next in their semantic code.

I had thought I could do something like this:

define register offset=0x110 size=4 [bi_addr contextreg];

define context contextreg
    ITMode = (0,0)
;

inst_next_it: addr is ITMode=0 [addr = inst_start + 4;] { export *:4 addr; }
inst_next_it: addr is ITMode=1 [addr = inst_start + 2;] { export *:4 addr; }

imm24s_rel: rel is s0_23 [ rel = inst_start + (s0_23 << 1); ] { export *:4 rel; }
:JAL imm24s_rel is u24_24=1 & imm24s_rel & inst_next_it & ITMode=0 {
    lp = inst_next_it;
    call imm24s_rel;
}
jal_dst_it: rel is s0_23 [ rel = (inst_start & 0xfe000000) | (s0_23 << 1); ] { export *:4 rel; }
:JAL jal_dst_it is u24_24=1 & jal_dst_it & inst_next_it & ITMode=1 {
    lp = inst_next_it;
    # XXX call pcodeop implicitly uses inst_next which generates improper code xref at +4
    call jal_dst_it;
}

:EX9.IT imm9u is op9_14=0b110101 & imm9u     [ ITMode=1; ] { ex9it(imm9u:2); }
:EX9.IT imm5u is op5_14=0b1011101010 & imm5u [ ITMode=1; ] { ex9it(imm5u:1); }

However, the contextreg doesn't seem to propagate to the program context state which gets accessed by the injection java code (the result of program.getProgramContext().getValue(contextReg, disasmAddr, false) is null).

The discrepancy in behavior is outlined here:

Is this a good approach? It seems almost working how I'd like it (at least decompiler receives proper pcode). There's just a few instructions which need to operate differently in case they're executed via EX9.IT. I had also considered generating the pcode for those instructions in java, but was hoping to keep it in sleigh if possible.

[shuffle2]

I eventually arrived at something that seems to produce good results, although I think it may highlight some issues in existing ghidra code.

I split the functionality for decoding the indirectly referenced instructions, then only return pcode in the injection hook, and perform the extra annotation in an Analyzer:

package plugin.core.analysis;
//...
public class EX9ITDisassembler {
	private Program program;
	private ProgramContext programContext;
	private Language language;
	private Memory memory;
	private final int INSTRUCTION_TABLE_ENTRY_LENGTH = 4;
	private Register itbReg;
	private Listing listing;
	private Address zeroAddress;
	private final String EX9IT_MNEMONIC = "EX9.IT";

	public EX9ITDisassembler(Program program) {
		this.program = program;
		memory = program.getMemory();
		language = program.getLanguage();
		programContext = program.getProgramContext();
		listing = program.getListing();
		itbReg = language.getRegister("ITB");
		zeroAddress = program.getAddressFactory().getDefaultAddressSpace().getAddress(0);
	}

	boolean instructionIsEX9IT(Instruction insn) {
		return insn.getMnemonicString().equals(EX9IT_MNEMONIC);
	}

	public PseudoInstruction disassemble(Address disasmAddr, byte bytes[]) throws InsufficientBytesException,
			UnknownInstructionException {
		PseudoDisassemblerContext disassemblerContext = new PseudoDisassemblerContext(programContext);
		MemBuffer memBuffer = new ByteMemBufferImpl(disasmAddr, bytes, language.isBigEndian());

		// check that address is defined in memory
		try {
			memBuffer.getByte(0);
		} catch (Exception e) {
			return null;
		}

		InstructionPrototype prototype = null;
		disassemblerContext.flowStart(disasmAddr);
		prototype = language.parse(memBuffer, disassemblerContext, false);
		if (prototype == null) {
			return null;
		}

		PseudoInstruction instr;
		try {
			// First, normal decode
			instr = new PseudoInstruction(program, disasmAddr, prototype, memBuffer, disassemblerContext);

			// hw would generate Reserved Instruction Exception
			if (instructionIsEX9IT(instr)) {
				return null;
			}

			// If it's branch, it's decoded as if current pc is 0
			// Must avoid passing program to PseudoInstruction, otherwise it will read from
			// program at the given addr - which we're trying to avoid
			FlowType flowType = prototype.getFlowType(instr);
			if (flowType.isCall() || flowType.isJump()) {
				instr = new PseudoInstruction(program.getAddressFactory(), zeroAddress, prototype, memBuffer,
						disassemblerContext);
			}
		} catch (AddressOverflowException e) {
			throw new InsufficientBytesException(
					"failed to build pseudo instruction at " + disasmAddr + ": " + e.getMessage());
		}

		return instr;
	}

	Instruction getEX9ITInstruction(Address address) {
		Instruction insn = listing.getInstructionAt(address);
		if (insn == null) {
			return null;
		}
		if (!instructionIsEX9IT(insn)) {
			return null;
		}
		return insn;
	}

	public PseudoInstruction getITInstruction(Instruction ex9itInsn) {
		Address ex9itAddress = ex9itInsn.getAddress();
		long itOffset = ex9itInsn.getScalar(0).getUnsignedValue() * INSTRUCTION_TABLE_ENTRY_LENGTH;

		BigInteger ITB = programContext.getValue(itbReg, ex9itAddress, false);
		if (ITB == null) {
			return null;
		}
		long memOffset = (ITB.longValue() & ~0b11) + itOffset;
		Address fetchAddress = ex9itInsn.getAddress().getNewAddress(memOffset);

		byte[] data = new byte[INSTRUCTION_TABLE_ENTRY_LENGTH];
		try {
			if (memory.getBytes(fetchAddress, data) != Array.getLength(data)) {
				return null;
			}
			return disassemble(ex9itAddress, data);
		} catch (Exception ex) {
			return null;
		}
	}

	public PseudoInstruction getITInstruction(Address ex9itAddress) {
		Instruction ex9Insn = getEX9ITInstruction(ex9itAddress);
		if (ex9Insn == null) {
			return null;
		}
		return getITInstruction(ex9Insn);
	}
}

package ghidra.app.util.pcodeInject;
//...
public class InjectEX9IT extends InjectPayloadCallother {

	public InjectEX9IT(String sourceName) {
		super(sourceName);
	}

	@Override
	public PcodeOp[] getPcode(Program program, InjectContext con) {
		EX9ITDisassembler disassembler = new EX9ITDisassembler(program);
		PseudoInstruction itInsn = disassembler.getITInstruction(con.baseAddr);
		// Could be bad ITB
		if (itInsn == null) {
			return null;
		}

		// NOTE SymbolicPropogator must be patched to allow STORE pcode ops
		return itInsn.getPcode();
	}

}

package plugin.core.analysis;
//...
public class AndeStarEX9ITAnalyzer extends AbstractAnalyzer {
    private final static String PROCESSOR_NAME = "AndeStar";
    private final static String NAME = "AndeStar EX9IT Analyzer";
    private final static String DESCRIPTION = "Annotates EX9.IT instructions";

    private final static CodeUnitFormat codeUnitFormat = new CodeUnitFormat(new CodeUnitFormatOptions());

    public AndeStarEX9ITAnalyzer() {
        super(NAME, DESCRIPTION, AnalyzerType.INSTRUCTION_ANALYZER);
        setDefaultEnablement(true);
    }

    @Override
    public boolean canAnalyze(Program program) {
        return program.getLanguage().getProcessor().equals(
                Processor.findOrPossiblyCreateProcessor(PROCESSOR_NAME));
    }

    @Override
    public boolean added(Program program, AddressSetView set, TaskMonitor monitor, MessageLog log)
            throws CancelledException {
        Listing listing = program.getListing();
        ReferenceManager refMgr = program.getReferenceManager();

        EX9ITDisassembler disassembler = new EX9ITDisassembler(program);

        for (Address addr : set.getAddresses(true)) {
            PseudoInstruction itInsn = disassembler.getITInstruction(addr);
            if (itInsn == null) {
                continue;
            }

            // Add a comment
            // TODO append to mnemonic instead?

            // itInsn will not have associated program if it's a branch
            if (itInsn.getProgram() != null) {
                String comment = codeUnitFormat.getRepresentationString(itInsn);
                listing.setComment(addr, CodeUnit.EOL_COMMENT, comment);
            } else {
                // dont really need extra comment - ghidra will add one because of the reference
                // comment = itInsn.getPrimaryReference(0).getToAddress().toString();
            }

            // Copy the references

            refMgr.removeAllReferencesFrom(addr);

            Reference[] refs = itInsn.getReferencesFrom();
            if (refs.length > 0) {
                CodeUnit cu = listing.getCodeUnitAt(addr);
                for (Reference ref : refs) {
                    cu.addMnemonicReference(ref.getToAddress(), ref.getReferenceType(), ref.getSource());
                }
            }
        }

        return true;
    }
}

The things to highlight:

• PseudoInstruction is apparently intended to allow my exact use case: disassembling an instruction as-if it exists at a given Address, while supplying arbitrary byte[] data to decode. However, it doesn't fully work. To actually use PseudoInstruction like this, you must not construct it with a Program reference - a path which isn't exposed by PseudoDisassembler. Additionally, PseudoInstructions created like this will still try to read data from program when e.g. getRepresentationString is called (instead of reading from the provided byte[] data). So, you either wind up with null deref exception or incorrect disasm output.
• I am patching some code in ghidra to allow injected pcode STORE ops to be processed. I'm not sure why they are unconditionally ignored by existing ghidra code.
• I really wish injected pcode were still visible in the gui (disasm view), like other pcode.