NASM on Linux: System Calls, Linking and Debugging.

A NASM source file is not itself a finished executable program. It must first be assembled into an object file and then linked into an executable. Once executed, the program runs in a particular operating-system environment and may request services from the Linux kernel.

Understanding this complete path makes assembly programming much easier to reason about:

NASM source
     ↓
Object file
     ↓
Linker
     ↓
Executable
     ↓
Process
     ↓
Linux system calls

This page connects those stages with practical debugging techniques.

A Linux x86-64 program cannot simply perform every privileged operation directly. Instead, it can request operating-system services through defined system-call interfaces.

For the x86-64 Linux system-call convention, the system-call number is placed in RAX and arguments are passed through designated registers.

The exact system-call number and argument requirements depend on the particular system call being requested.

section .data
    message db "Hello from NASM!", 10
    length equ $ - message

section .text
    global _start

_start:
    mov rax, 1
    mov rdi, 1
    mov rsi, message
    mov rdx, length
    syscall

    mov rax, 60
    xor rdi, rdi
    syscall

The example demonstrates an important combination of concepts. The program has data in a separate section, calculates its length, places the required values into registers, requests the Linux write operation, and then exits.

Notice that RSI contains the address of the message rather than the message itself. This connects directly to the memory addressing concepts covered in the previous guide.

In a high-level language, a debugger can show variables, source-level expressions, function calls, and other abstractions that are familiar to the programmer. In assembly, many of those abstractions are much closer to the underlying processor.

This makes debugging both more challenging and more informative. You can observe the exact registers and memory locations being modified as instructions execute.

A useful debugging question is not simply: "Why did the program fail?"

Instead ask: "At which instruction did the processor state stop matching what I expected?"

When debugging an assembly program, it is useful to build the executable with debugging information where appropriate. The debugger can then provide more useful source-level context while still allowing the underlying machine state to be inspected.

gdb ./program

Once inside GDB, commands can be used to establish breakpoints, execute instructions step by step, inspect registers, examine memory, and inspect the current instruction.

The exact commands and debugging workflow depend on the executable and how it was assembled and linked, but the central principle remains the same: pause execution, inspect state, advance execution, and compare the observed state with the expected state.

Suppose a register is expected to contain a particular value before an arithmetic operation. A breakpoint placed before that instruction allows you to inspect the register and determine whether the problem occurred earlier or during the operation itself.

Single-stepping then allows the next instruction to execute while you observe how the processor state changes.

mov rax, 10
add rax, 20
cmp rax, 30
je  correct

A debugger can help determine whether RAX contained the expected value before ADD, whether ADD produced the expected result, whether CMP affected the expected flags, and whether the conditional jump behaved accordingly.

If a register contains a pointer, inspecting the register alone may not explain the problem. You may also need to inspect the memory at that address and determine whether the expected data is actually present.

This is particularly important when debugging:

• Arrays and buffers
• Strings
• Stack variables
• Pointers
• Structures and data layouts
• System-call arguments

Memory debugging connects directly to the addressing concepts discussed in the previous NASM guide.

mov rax, 10
mov rbx, 20

add rax, rbx

cmp rax, 30
jne error

mov rdi, 0
jmp done

error:
    mov rdi, 1

done:
    mov rax, 60
    syscall

If this program unexpectedly follows the error path, the debugging process should examine the values in RAX and RBX, verify the result of ADD, inspect the comparison, and determine why the conditional branch did not behave as expected.

This method scales to much larger programs. Instead of changing several instructions at once, isolate the point where the program's actual state first differs from the expected state.