< Home

Challenge

What does asm3(0xd73346ed,0xd48672ae,0xd3c8b139) return? Submit the flag as a hexadecimal value (starting with ‘0x’).

asm3:
    <+0>:   push   ebp
    <+1>:   mov    ebp,esp
    <+3>:   xor    eax,eax
    <+5>:   mov    ah,BYTE PTR [ebp+0xa]
    <+8>:   shl    ax,0x10
    <+12>:  sub    al,BYTE PTR [ebp+0xc]
    <+15>:  add    ah,BYTE PTR [ebp+0xd]
    <+18>:  xor    ax,WORD PTR [ebp+0x10]
    <+22>:  nop
    <+23>:  pop    ebp
    <+24>:  ret

Let’s go over it, let us first think about the parameter locations.

|ebp+8 |ebp+C |ebp+10 |
|---|---|---|
|0xd73346ed|0xd48672ae|0xd3c8b139|

Now, in our code we don’t see 8, we do see A which is 0x8 + 0x2. So we are going to take the BYTE stored at this location and it seems that the first 2 bytes are thrown away. Also, another thing I forgot at the start but remembered mid-way. These are stored as little endians so we must read them from right to left!

asm3:
    <+0>:   push   ebp                      ; Push previous frame base pointer onto the stack
    <+1>:   mov    ebp,esp                  ; Set new base pointer to stack pointer
    <+3>:   xor    eax,eax                  ; Clear EAX registry
    <+5>:   mov    ah,BYTE PTR [ebp+0xa]    ; Move a BYTE into EAX high (0x0000AH00)
    <+8>:   shl    ax,0x10                  ; Shift AX (lower 16 bits of EAX) bitwise left ten times
    <+12>:  sub    al,BYTE PTR [ebp+0xc]    ; Subtract a byte from the lower 8 bits of EAX
    <+15>:  add    ah,BYTE PTR [ebp+0xd]    ; Add to the upper lower bits of EAX
    <+18>:  xor    ax,WORD PTR [ebp+0x10]   ; XOR from the lower 16 bits from EAX
    <+22>:  nop
    <+23>:  pop    ebp
    <+24>:  ret

Cool, let us fill in the registry every step

asm+3 -> EAX = 00000000
asm+5 -> EAX = 00003300 (33 comes from d7 33 46 ed where BYTE ebp+8->ed, BYTE ebp+9->46, BYTE ebp+a->33, BYTE ebp+b->d7)
asm+8 -> Sneaky! SHL does not overflow so we must see this as purely ah so 0011 0011 0000 0000 << 10 just empties it... so EAX is back to 00 00 00 00 as AH is cleared
asm+12 -> As EAX is completely zeroes again, the lower 8 bits will be subtracted with 0xae (BYTE ebp+C is 0xae, remember little endian) so 0x0 - 0xAE = 0x52, EAX is now 00 00 00 52
asm+15 -> Here we add BYTE to AH which is 00 at the moment. What we add is 0x72 which makes EAX  = 00 00 72 52
asm+18 = XORing AX (0x7252) with WORD ebp+0x10 (0xb139) => 0xc36b

Now we have EAX = 0x0000c36b which is also the answer

This is the “I have no tools available way except for my trusty calculator”. Next method is much easier.

Let our computer do it

Another way to do this is through live analysis. We have the source code, why not let our computer do these calculations and we won’t get stuck on some part we don’t fully understand.

Let us create some code that calls our challenge code.

global _main
section .text

_main:                     
    push 0xd3c8b139
    push 0xd48672ae
    push 0xd73346ed
    call asm3
    nop
asm3:
    push   ebp
    mov    ebp,esp
    xor    eax,eax
    mov    ah,BYTE [ebp+0xa]
    shl    ax,0x10
    sub    al,BYTE [ebp+0xc]
    add    ah,BYTE [ebp+0xd]
    xor    ax,WORD [ebp+0x10]
    nop
    pop    ebp
    ret

As I am running on Windows I use nasm with the linker of Visual Studio (took me awhile to find out how to compile asm on Windows, next time I’ll use the WCL Ubuntu). We also must make sure we utilize Win32 as this code was written for 32 bit.

> nasm -fwin32 asm3.asm
> link /subsystem:console /nodefaultlib /entry:main asm3.obj

Ok, now we’ve got asm3.exe. Let us open it with x32dbg.

x32dbg will automatically put breakpoints at the process start and at our main function (or point of entry). To skip the first one we just Run/F9 and then we arrive at the breakpoint of our main function.

Tightly packed

We put a breakpoint at our nop instruction (also the reason why we added nop) and after that we just Run/F9 our code again!

We hit our breakpoint again and now we can just read out EAX for the solution!

Here is where I bumped against a small problem. picoCTF is case senstive. When I entered 0xC36B I got that the flag was wrong. That is because picoCTF expects 0xc36b.

So, always try your answer multiple times 😊

< Home