Need a way to handle the 2/3 byte VEX in C# assembler

user_3931344 0
Viewed 110

Hello there I have fully ported the x86 assembler (and auto assembler) from cheat engine to c#.

Everything works fine for the most part however the "VEX" prefix instructions are currently broken.

There is 2 sections of code I wasn't able to convert just yet so I wonder if anybody can provide a solution to it.

Commented out sections are the ones that I'm not sure how to deal with.

2 byte vex

//2byte vex
bytes.SetLength(bytes.Length + 2);
for (i = bytes.Length - 1; i >= RexPrefixLocation + 2; i--)
    bytes[i] = bytes[i - 2];
bytes[RexPrefixLocation] = 0xc5; //2 byte VEX
//pvex2byte(&bytes[RexPrefixLocation + 1])->pp = (int) (Assembler.OpCodes[j].VexLeadingOpCode);
//pvex2byte(&bytes[RexPrefixLocation + 1])->l = Assembler.OpCodes[j].VexL;
//pvex2byte(&bytes[RexPrefixLocation + 1])->vvvv = vexvvvv;
//pvex2byte(&bytes[RexPrefixLocation + 1])->r = RexR ? 0 : 1;
if (RelativeAddressLocation != -1)
    RelativeAddressLocation += 2;

3 byte vex

//3byte vex
bytes.SetLength(bytes.Length + 3);
for (i = bytes.Length - 1; i >= RexPrefixLocation + 3; i--)
    bytes[i] = bytes[i - 3];
bytes[RexPrefixLocation] = 0xc4; //3 byte VEX
//pvex3byte(&bytes[RexPrefixLocation + 1])->mmmmm = (int)(Assembler.OpCodes[j].VexLeadingOpCode);
//pvex3byte(&bytes[RexPrefixLocation + 1])->b = RexB ? 0 : 1;
//pvex3byte(&bytes[RexPrefixLocation + 1])->x = RexX ? 0 : 1;
//pvex3byte(&bytes[RexPrefixLocation + 1])->r = RexR ? 0 : 1;
//pvex3byte(&bytes[RexPrefixLocation + 1])->pp = (int)(Assembler.OpCodes[j].VexLeadingOpCode);
//pvex3byte(&bytes[RexPrefixLocation + 1])->l = Assembler.OpCodes[j].VexL;
//pvex3byte(&bytes[RexPrefixLocation + 1])->vvvv = vexvvvv;
//pvex3byte(&bytes[RexPrefixLocation + 1])->w = RexW ? 1 : 0; //not inverted
if (RelativeAddressLocation != -1)
    RelativeAddressLocation += 3;

The original assembly/auto assembler is made in pascal (lazarus) and here is the definition of the pvex3byte

  TVEX3Byte=bitpacked record
    mmmmm: 0..31;
    B: 0..1;
    X: 0..1;
    R: 0..1;
    pp: 0..3;
    L: 0..1;
    vvvv: 0..15;
    W: 0..1;
  end;
  PVEX3Byte=^TVEX3Byte;

  TVEX2Byte=bitpacked record
    pp: 0..3;
    L: 0..1;
    vvvv: 0..15;
    R: 0..1;
  end;
  PVex2Byte=^TVex2Byte;

Can somebody solve how to fill this part in? If possible a complete class or extension for the "bytes" (byte[]) array to allow editing/reading the vexs properly?

I figure its something to do with bits, unfortunately there's no bit struct in c# (none that work in this way).

bit-fields bit-manipulation machine-code assembly c#
1 Answers

Finding this on github

    /* VEX 2 byte form */
    /*  7                           0    */
    /* +---+---+---+---+---+---+---+---+ */
    /* |~R |     ~vvvv     | L |   pp  | */
    /* +---+---+---+---+---+---+---+---+ */

    /* VEX 3 byte form */
    /*   7                           0     7                           0    */
    /* +---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+*/
    /* |~R |~X |~B | map_select        |   |W/E|    ~vvvv      | L |   pp  |*/
    /* +---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+*/

Then some help from a friend and my custom vec() class was quickly able to solve the problem.

The end result was a new multiByte class that handles the vex2/3 with simple properties.

    public class AVex3Byte : AMultiByte
    {
        #region Constants
        private const int SIZE = 2;
        #endregion
        #region R
        public Byte R
        {
            get => (Byte)Get(7, 1, true);
            set
            {
                Set(7, 1, true, value);
                Apply();
            }
        }
        #endregion
        #region X
        public Byte X
        {
            get => (Byte)Get(6, 1, true);
            set
            {
                Set(6, 1, true, value);
                Apply();
            }
        }
        #endregion
        #region B
        public Byte B
        {
            get => (Byte)Get(5, 1, true);
            set
            {
                Set(5, 1, true, value);
                Apply();
            }
        }
        #endregion
        #region Mmmmm
        public Byte Mmmmm
        {
            get => (Byte)Get(0, 5, true);
            set
            {
                Set(0, 5, true, value);
                Apply();
            }
        }
        #endregion
        #region W
        public Byte W
        {
            get => (Byte)Get(15, 1, true);
            set
            {
                Set(15, 1, true, value);
                Apply();
            }
        }
        #endregion
        #region Vvvv
        public Byte Vvvv
        {
            get => (Byte)Get(11, 4, true);
            set
            {
                Set(11, 4, true, value);
                Apply();
            }
        }
        #endregion
        #region L
        public Byte L
        {
            get => (Byte)Get(10, 1, true);
            set
            {
                Set(10, 1, true, value);
                Apply();
            }
        }
        #endregion
        #region Pp
        public Byte Pp
        {
            get => (Byte)Get(8, 2, true);
            set
            {
                Set(8, 2, true, value);
                Apply();
            }
        }
        #endregion
        #region Constructor
        public AVex3Byte()
            : base(SIZE)
        {
        }
        public AVex3Byte(AByteArray bytes, int index)
            : base(SIZE, bytes, index)
        {
        }
        public AVex3Byte(IntPtr bytesPointer)
            : base(SIZE, bytesPointer, 0)
        {
        }
        public AVex3Byte(IntPtr bytesPointer, int index)
            : base(SIZE, bytesPointer, index)
        {
        }
        #endregion
    }

Heres how the original question code looks upon fixing

//2byte vex
bytes.SetLength(bytes.Length + 2);
for (i = bytes.Length - 1; i >= RexPrefixLocation + 2; i--)
    bytes[i] = bytes[i - 2];
bytes[RexPrefixLocation] = 0xc5; //2 byte VEX
var vex2 = new AVex2Byte(bytes, RexPrefixLocation + 1);
vex2.Pp = (Byte)Assembler.OpCodes[j].VexLeadingOpCode;
vex2.L = Assembler.OpCodes[j].VexL;
vex2.Vvvv = (Byte)vexvvvv;
vex2.R = (Byte)(RexR ? 0 : 1);
if (RelativeAddressLocation != -1)
    RelativeAddressLocation += 2;

//3byte vex
bytes.SetLength(bytes.Length + 3);
for (i = bytes.Length - 1; i >= RexPrefixLocation + 3; i--)
    bytes[i] = bytes[i - 3];
bytes[RexPrefixLocation] = 0xc4; //3 byte VEX
var vex3 = new AVex3Byte(bytes, RexPrefixLocation + 1);
vex3.Mmmmm = (Byte)(Assembler.OpCodes[j].VexLeadingOpCode);
vex3.B = (Byte)(RexB ? 0 : 1);
vex3.X = (Byte)(RexX ? 0 : 1); 
vex3.R = (Byte)(RexR ? 0 : 1);
vex3.Pp = (Byte)(Assembler.OpCodes[j].VexLeadingOpCode);
vex3.L = Assembler.OpCodes[j].VexL;
vex3.Vvvv = (Byte)vexvvvv;
vex3.W = (Byte)(RexW ? 1 : 0); //not inverted
if (RelativeAddressLocation != -1)
    RelativeAddressLocation += 3;

What I needed was the correct information about what bits and where they are and how big they are and how to read/write to/from it.

user_39313440
Related