16-bit Windows applications and pointers

[Wall-AF]

Over the last few months I've been climbing a mountain (a nanometre at a time), but I’m wondering if the configuration in /Processors x86/data/languages/x86-16.cspec describing <pointer_size value="2" /> is correct. I’ve noticed that actually all pointers are 32-bit comprising of either an implicit or explicit segment followed by the offset. The implicit segment is defined in the assembly instruction (as one of DS, SS or ES).

Therefore, is the definition wrong???

[ghidra1]

The definition is incomplete but is limited by GHIDRA's current capabilities.

Ideally, GHIDRA needs to support multiple pointer sizes and types for a specific compiler spec, however the current GHIDRA Data Organization only supports the notion of a single pointer size and is generally assumed to refer to the default address space in many cases. There are a number of other pointer attributes which would ideally be supported as well, including the notion of address space a pointer variable references. In addition, size alone is insufficient to handle proper de-referencing in many cases (e.g., offset pointers, bit-shifted pointers, etc.).

[Wall-AF]

Is there anything in the pipeline to support this?

[ghidra1]

Nothing at this point. It's on the wish list.

[Wall-AF]

Is there some technical documentation (not generally available) that I could gain access to that would enable me to better understand Ghidra's internal design so that I could help in this quest???

So far, I've been stepping through the Java code in order to make the changes I've already made for myself!

[ghidra1]

I have not researched a solution for your specific use-case involving segmented addressing issues, but do realize that many architectures support the notion of multiple pointer sizes and characteristics which can influence pointer interpretation and de-referencing. The hope is the architectural changes we are currently considering will improve the "datatype attribute" storage capabilities to provide sufficient flexibility to more easily tackle new use cases such as yours without the need to incrementally change the storage schema and low-level API. Such changes always have a big impact on data migration, datatype archives, multi-user conflict detection/merge and diff. With such a capability in place we can more easily introduce new datatype attributes, although it is still likely that a significant retrofit effort would be required when introducing new attribute which must be handled by the GUI and analysis chain (e.g., decompiler).

I apologize for our lack of design documentation and a more immediate solution to your issue. Your willingness to assist is appreciated. We encourage your experimentation with solving your specific use case and passing along anything you find helpful. If you find a solution which does not entail new pointer attributes it may be possible to work them in sooner than later.

[Wall-AF]

Here's one that I've zipped to store!
VOICEBAR.EXE.gzf.zip
Hope this helps.

[ghidra1]

It appears the root cause of your issue is the default pointer size of 2-bytes vs the pointer size you are attempting to trigger on (i.e., 4-bytes).

If you are running from Eclipse from a source pull, you comment-out the following lines in DecompilerStructureVariableAction.java (~line 68) and similar lines in ListingStructureVariableAction.java (~line 80) - although the structure does not fill-out as expected. A non-default pointer size is likely causing an issue in the implementation.

[Wall-AF]

Superb. That works. Except, you end up with a 16-bit pointer which messes up the stack!!!

[Wall-AF]

In class FillOutStructureCmd method applyTo (~line 155), I found a new pointer is created. When I modify the creation to include the current size, the calculated references reappear. I'm back to square 1! It calls the decompiler code to create the syntax!

[ghidra1]

I am hoping @emteere or @caheckman can provide additional input/suggestion on this

[Wall-AF]

DecompilerDebugOutput_16-bit_vs_32-bit_pointers.zip
Above is a very simple example illustrating the issue, taken from the Decompiler's Debug output. @emteere or @caheckman do you have any suggestions please?

I am currently littering the C++ code with cout's in the vain hope of determining where the decompiler's process varies between the 2 examples, but it is proving mesmerisingly difficult!

[Wall-AF]

From what I'm seeing, it's as if the decompiler is able to effectively ignore the segment part of any address thus allowing near (16-bit) pointers, but when given an explicit far (32-bit) one the segment and (in my case) the (BX) register are treated as individual entities and not joined to make a 32-bit pointer.

How can this be rectified???

[Wall-AF]

Similarly, if you create a struct with just a 16-bit pointer to your object and use that as your parameter spec, that dereferences fine (leaving unresolved stack references). Adding anything to that struct (the appropriate size - 16 bits) to fix the stack alignment breaks the dereferencing!

I think a new Rule maybe needed, but I'm struggling to know exactly how and what to add!

[NancyAurum]

I think this asks for a general framework. Consider the int21:AH syscalls, That is basically the same as segmented calls, but with int instruction picking the segment. Ideally having say a DOS memory dump, you want to be able to jump into the syscall code without navigating through the dispatcher. Then both the analyzer, the scripts and the users could use it.