binutils

Most architectures encode direct branch/call instructions with a PC-relative displacement. This post discusses a specific category of branch relocations: those used for direct function calls and tail calls. Some architectures use two ELF relocation types for a call instruction:

# i386, x86-64
call foo              # R_386_PC32, R_X86_64_PC32
call foo@plt          # R_386_PLT32, R_X86_64_PLT32

# m68k
bsr.l foo             # R_68K_PC32
bsr.l foo@plt         # R_68K_PLT32

# s390/s390x
brasl %r14, foo       # R_390_PC32DBL
brasl %r14, foo@plt   # R_390_PLT32DBL

# sparc
call    foo, 0        # not PIC: R_SPARC_WDISP30
call    foo, 0        # gas -KPIC: R_SPARC_WPLT30

Branch instructions on most architectures use PC-relative addressing with a limited range. When the target is too far away, the branch becomes "out of range" and requires special handling.

Consider a large binary where main() at address 0x10000 calls foo() at address 0x8010000-over 128MiB away. On AArch64, the bl instruction can only reach ±128MiB, so this call cannot be encoded directly. Without proper handling, the linker would fail with an error like "relocation out of range." The toolchain must handle this transparently to produce correct executables.

This article explores how compilers, assemblers, and linkers work together to solve the long branch problem.

• Compiler (IR to assembly): Handles branches within a function that exceed the range of conditional branch instructions
• Assembler (assembly to relocatable file): Handles branches within a section where the distance is known at assembly time
• Linker: Handles cross-section and cross-object branches discovered during final layout

This post explores how GNU Assembler and LLVM integrated assembler generate relocations, an important step to generate a relocatable file. Relocations identify parts of instructions or data that cannot be fully determined during assembly because they depend on the final memory layout, which is only established at link time or load time. These are essentially placeholders that will be filled in (typically with absolute addresses or PC-relative offsets) during the linking process.

Relocation generation: the basics

Symbol references are the primary candidates for relocations. For instance, in the x86-64 instruction movl sym(%rip), %eax (GNU syntax), the assembler calculates the displacement between the program counter (PC) and sym. This distance affects the instruction's encoding and typically triggers a R_X86_64_PC32 relocation, unless sym is a local symbol defined within the current section.

Both the GNU assembler and LLVM integrated assembler utilize multiple passes during assembly, with several key phases relevant to relocation generation:

GNU ld's output section layout is determined by a linker script, which can be either internal (default) or external (specified with -T or -dT). Within the linker script, SECTIONS commands define how input sections are mapped into output sections.

Input sections not explicitly placed by SECTIONS commands are termed "orphan sections".

Orphan sections are sections present in the input files which are not explicitly placed into the output file by the linker script. The linker will still copy these sections into the output file by either finding, or creating a suitable output section in which to place the orphaned input section.

GNU ld's default behavior is to create output sections to hold these orphan sections and insert these output sections into appropriate places.

Orphan section placement is crucial because GNU ld's built-in linker scripts, while understanding common sections like .text/.rodata/.data, are unaware of custom sections. These custom sections should still be included in the final output file.

• Grouping: Orphan input sections are grouped into orphan output sections that share the same name.
• Placement: These grouped orphan output sections are then inserted into the output sections defined in the linker script. They are placed near similar sections to minimize the number of PT_LOAD segments needed.

Updated in 2025-05.

This article describes linker notes about Portable Executable (PE) and Common Object File Format (COFF) used on Windows and UEFI environments.

In ELF, an object file can be a relocatable file, an executable file, or a shared object file. On Windows, the term "object file" usually refers to relocatable files like ELF. Such files use the Common Object File Format (COFF) while image files (e.g. executables and DLLs) use the Portable Executable (PE) format.

A look at linkers

MSVC Linker is the reference linker on Windows.

GNU ld supports Windows thanks to the MinGW effort. Here are some common BFD emulations for PE targets:

• i386pe: For 32-bit Intel x86 architectures.
• i386pep: For 64-bit Intel x86-64 architectures.
• arm64pe: For 64-bit ARM (AArch64) architectures.

For instance, you can use a command like ld.bfd -m i386pep a.o b.o to link two object files into a PE executable. Many options on Linux can also be used when targeting PE/COFF.

The LLVM linker offers two modes of operation that are relevant to PE/COFF:

• lld-link: In this mode, lld emulates the MSVC linker, making it a fast viable alternative.
• ld.lld: When invoked with a PE emulation (like -m i386pep), lld emulates GNU ld. Code in lld/MinGW translates GNU ld options to link.exe options.

GNU ld implements relocatable linking that is not available in other linkers.

Input files

The input files to the linker can be object files, archive files, and import libraries. GNU ld and lld-link allow linking against DLL files without an import library.

DSO undef and non-exported def

If a DSO has an undefined STB_GLOBAL symbol that is defined in a relocatable object file but not exported, should the --no-allow-shlib-undefined feature report an error? You may want to check out Dependency related linker options for a discussion of this option and the symbol exporting rule.

This article describes SHF_ALLOC|SHF_COMPRESSED sections in ELF and lld's linker option --compress-sections to compress arbitrary sections.

Updated in 2025-05.

This article provides a description of popular assemblers and their architecture-specific differences.

Assemblers

GCC generates assembly code and invokes GNU Assembler (also known as "gas"), which is part of GNU Binutils, to convert the assembly code into machine code. The GCC driver is also capable of accepting assembly input files. Due to GCC's widespread use, GNU Assembler is arguably the most popular assembler.

Within the LLVM project, the LLVM integrated assembler is a library that is linked by Clang, llvm-mc, and lld (for LTO purposes) to generate machine code. It supports a wide range of GNU Assembler syntax and can be used as a drop-in replacement for GNU Assembler.

On the Windows platform, the Microsoft Macro Assembler (MASM) is widely used.

On the IBM AIX platform, the AIX assembler is used. In 2019, IBM developers started to modify LLVM integrated assembler to support the AIX syntax.

On the IBM z/OS platform, the IBM High Level Assembler (HLASM) is used. In 2021, IBM developers started to modify LLVM integrated assembler to support the HLASM syntax.

Concepts

Sections are named, contiguous blocks of code or data within an object file. They allow you to logically group related parts of your program. The assembler places code and data into these sections as it processes the source file.

Symbols are names that represent memory addresses or values.

Updated in 2022-10.

In January I wrote Compressed debug sections. The venerable zlib shows its age and there are replacements which are better in every metric except adoption and a larger memory footprint. The obvious choice was Zstandard, but I was not so confident about adoptinig it and solving the ecosystem issue. At any rate, I slowly removed some legacy .zdebug support from llvm-project so that a new format could be more easily introduced.

On 2022-07-07, I added a RISC-V linker relaxation framework in ld.lld and implemented R_RISCV_ALIGN/R_RISCV_CALL/R_RISCV_CALL_PLT relaxation. The changes will be included in the next llvm-project release 15.0.0. This post describes the implementation.

See The dark side of RISC-V linker relaxation for more information about RISC-V linker relaxation.

Programming language behavior

FORTRAN 77 COMMON blocks compiled to COMMON symbols. You could declare a COMMON block in more than one file, with each specifying the number, type, and size of the variable. The linker allocated enough space to satisfy the largest size.

Updated in 2024-05.

Binary sizes are important. Filesystem compression is ergonomic but typically does not leverage application information well. Compressing allocable sections (text, data) increases program startup time and introduces memory overhead. In addition, filesystem compression is not sufficiently portable.

Debug sections are large and contribute to a significant portion of the binary size. Therefore, it is appealing to compress debug sections.

Here is a -DCMAKE_BUILD_TYPE=Debug build directory of llvm-project where I just ran ninja clang (on 2022-10-21). Here are the total sizes of .o files, text sections, and debug sections. It is typical that the debug information is often much larger than text sections.

% stat -c %s **/*.o | awk '{s+=$1} END{print s}'
1464767464
% readelf -WS **/*.o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.text/{d += strtonum("0x"$6)} END{print d}'
210026370
% readelf -WS **/*.o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.debug_/{d += strtonum("0x"$6)} END{print d}'
631069751
% readelf -WS **/*.o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.rela\.debug_/{d += strtonum("0x"$6)} END{print d}'
78448968

Some assemblers and linkers offer a feature to compress debug sections.

llvm-objcopy supports --compress-debug-sections=zlib to compress debug sections. We can use the option to check what if we compress debug sections for the assembler.

% for i in **/*.o; do /tmp/Rel/bin/llvm-objcopy --compress-debug-sections=zlib $i /tmp/c/o && readelf -WS /tmp/c/o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.debug_/{d += strtonum("0x"$6)} END{print d}'; done | awk '{s+=$1} END{print s}'
161691798

For debug sections, we have a compression ratio of 3.90! The total .o size is 995438992 bytes, 68% of the original.

Then let's check zstd.

% for i in **/*.o; do /tmp/Rel/bin/llvm-objcopy --compress-debug-sections=zstd $i /tmp/c/o && readelf -WS /tmp/c/o | awk 'BEGIN{FPAT="\\[.*?\\]|\\S+"} $2~/\.debug_/{d += strtonum("0x"$6)} END{print d}'; done | awk '{s+=$1} END{print s}'
159341878

To check whether an object file has compressed debug sections, we can use readelf.

% readelf -S a.o
...
Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
...
  [ 5] .debug_abbrev     PROGBITS        0000000000000000 000080 000087 00   C  0   0  8

In the readelf -S output, the Flg column describes sh_flags where C indicates the SHF_COMPRESSED flag.

% readelf -t a.o
...
Section Headers:
  [Nr] Name
       Type            Address          Off    Size   ES   Lk Inf Al
       Flags
...
  [ 5] .debug_abbrev
       PROGBITS        0000000000000000 000080 000087 00   0   0  8
       [0000000000000800]: COMPRESSED
       ZLIB, 0000000000000093, 1

History

In 2007-11, Craig Silverstein added --compress-debug-sections=zlib to gold. When the option was specified, gold compressed the content of a .debug* section with zlib and changed the section name to .debug*.zlib.$uncompressed_size.

Updated in 2026-02.

.a archives

Unix-like systems represent static libraries as .a archives. A .a archive consists of a header and a collection of files with file member header. Its usage is tightly coupled with the linker. An archive almost always contains only relocatable object files and the linker has built-in support for reading it.

% as /dev/null -o a.o
% rm -f b.a && ar rc b.a a.o
% ar t b.a
a.o

One may add other types of files to .a but that is almost assuredly a bad thing.

% rm -f a.a && ar rc a.a a.o b.a  # archive in archive, bad
% ar t a.a
a.o
b.a
% echo hello > a.txt
% rm -f a.a && ar rc a.a a.o a.txt  # text file in archive, bad

The original linker designers noticed that for many programs not every member was needed, so they tried to allow the linker to skip unused members. Therefore, they invented the interesting but confusing archive member extraction rule. See Symbol processing#Archive processing for details.

Updated in 2023-03.

In C++, dynamic initializations for non-local variables happen before the first statement of the main function. All (most?) implementations just ensure such dynamic initializations happen before main.

As an extension, GCC supports __attribute__((constructor)) which can make an arbitrary function run before main. A constructor function can have an optional priority (__attribute__((constructor(N)))).

Updated in 2024-02.

(In celebration of my 2800th llvm-project commit) Happy Halloween!

This article describes relative relocations and how the RELR format can greatly decrease file sizes.

An ELF linker performs the following steps to process an absolute relocation type whose width equals the word size (e.g. R_AARCH64_ABS64, R_X86_64_64).

if (undefined_weak || (!preemptible && (no_pie || is_shn_abs)))
  link-time constant
else if (SHF_WRITE || znotext) {
  if (preemptible)
    emit a symbolic relocation (e.g. R_X86_64_64)
  else
    emit a relative relocation (e.g. R_X86_64_RELATIVE)
} else if (!shared && (copy_relocation || canonical_plt_entry)) {
  ...
} else {
  error
}

Note: in FDPIC ABIs, there is no single base address. Relative relocations can still be used, but the runtime operation is not simply *loc += base.

Updated in 2024-01.

Branch target

Many architectures encode a branch/jump/call instruction with PC-relative addressing, i.e. the distance to the target is encoded in the instruction. In an executable or shared object (called a component in ELF), if the target is bound to the same component, the instruction has a fixed encoding at link time; otherwise the target is unknown at link time and there are two choices:

• text relocation
• indirection

In All about Global Offset Table, I mentioned that linker/loader developers often frowned upon text relocations because the text segment will be unshareable. In addition, the number of relocations would be dependent on the number of calls, which can be large.

LLD is the LLVM linker. It started at the end of 2011 as a work-in-progress rewrite of ld64 for the Mach-O binary format based on the atom model. COFF and ELF ports based on the atom model were contributed subsequently. They shared one symbol resolution model. (IMO due to Mach-O's unfortunate limitation of 255 section .subsections_via_symbols was invented. The atom model was an incarnation of the concept but it did not fit into ELF/PE where sections are the better basic units.)

In 2015, both COFF and ELF ports were rewritten. (See "LLD improvement plan") Today, LLD is a mature and fast linker supporting multiple binary formats (ELF, Mach-O, PE/COFF, WebAssembly). FreeBSD, Android, and Chrome OS have adopted it as the main linker.

As a main contributor of LLD's ELF port who has fixed numerous corner cases in recent years, I consider that its x86-64 support has been mature since the 8.0.0 release and is in a great shape since 9.0.0. The AArch64 and PowerPC32/PowerPC64 support has been great since the 10.0.0 release. The 11.0.0 release has very solid linker script support. (When people complain that GNU ld's linker script is not immediately usable with LLD, it is almost assuredly the problem of the script itself.) So, what's next? Build glibc with LLD!

Updated in 2025-05.

In ELF, the Global Offset Table is a memory region indexed by offsets, with entries holding constants and symbol addresses.

Symbol address

In an executable or shared object (called a component in ELF), a text section may need the absolute virtual address of a symbol (e.g. a function or a variable). The reference arises from an address taken operation or a PLT entry. The address may be:

• a link-time constant
• the load base plus a link-time constant
• dependent on runtime computation by ld.so

In ELF linkers, undefined symbols from DSO participate in symbol resolution, just like undef from .o. The differences with undef from .o are:

• undef purely from DSO do not need .symtab entries
• --no-allow-shlib-undefined can error for such undef (-rpath-link ("load dependent libraries") behavior can suppress some errors.

ELF linkers extract an archive member to satisfy an undefined symbol from a shared object. Here is an example:

Updated in 2025-06.

Some random notes about toolchain testing.

Classification

Testing levels.

Updated in 2025-09.

The blog post explores how COMDAT (in the PE format) and section groups (in the ELF format) provide a mechanism to group related sections for efficient deduplication and garbage collection.

Vague linkage

In C++, inline functions, template instantiations, and a few other things can be defined in multiple object files but need deduplication at link time. In the dark ages the functionality was implemented by weak definitions: the linker does not report duplicate definition errors and resolves the references to the first definition. The downside is that unneeded copies remained in the linked image.

The main task of a linker script is to define extra symbols and define how input sections map into output sections. It has other miscellaneous features which can be implemented via command line options:

• The ENTRY command can be replaced by --entry.
• The OUTPUT_FORMAT command can usually be replaced by -m.
• The SEARCH_DIRS command can be replaced by -L.
• The VERSION command can be replaced by --version-script.
• The INPUT and GROUP commands can add other files as input. This provides a mechanism to split an archive/shared object into multiple files.

UNDER CONSTRUCTION (COFF, Mach-O)

Symbol processing is a major step in a linker. In most binary formats, the linker maintains a global symbol table and performs symbols resolution for each input file (object file, shared object, archive, LLVM bitcode file). Some command line options can define/undefine symbols as well. The symbol resolution can affect archive processing and many subsequent steps (LTO, relocation processing, as-needed shared objects, etc).

Updated in 2026-03.

C/C++

GCC and Clang support __attribute__((weak)) which marks a symbol weak. The same effect can be achieved with a preprocessor directive #pragma weak symbol.

Object file format

In ELF, there are three main symbol bindings. The ELF specification says:

• STB_LOCAL: Local symbols are not visible outside the object file containing their definition. Local symbols of the same name may exist in multiple files without interfering with each other.
• STB_GLOBAL: Global symbols are visible to all object files being combined. One file's definition of a global symbol will satisfy another file's undefined reference to the same global symbol.
• STB_WEAK: Weak symbols resemble global symbols, but their definitions have lower precedence.

Updated in 2025-07.

This article introduces RISC-V linker relaxation and describes the downside.

Linker optimization/relaxation

Because the linker has a global view and layout information, it can perform some peephole optimizations which are difficult/impossible to do on the compiler side. Generic link-time code sequence transformation is risky, because semantic information is lost and what the linker sees are byte streams. However, if every instruction in the candidate code sequence is associated with one ore more relocations, the ABI and the implementation can assign (additional) semantics to the relocation types and make such transformation safe. This technique is usually called linker optimization or linker relaxation. It seems that the term "linker optimization" is often used when the number of bytes does not change while "linker relaxation" is used when the number of bytes decreases.

A program may have a lot of unused code and data. There can be many reasons:

Updated in 2026-01.

Thread-local storage (TLS) provides a mechanism allocating distinct objects for different threads. It is the usual implementation for GCC extension __thread, C11 _Thread_local, and C++11 thread_local, which allow the use of the declared name to refer to the entity associated with the current thread. This article will describe thread-local storage on ELF platforms in detail, and touch on other related topics, such as: thread-specific data keys and Windows/macOS TLS.

As mainly an LLVM person, I occasionally contribute to GNU toolchain projects. This is sometimes for fun, sometimes for investigating why an (usually ancient) feature works in a particular way, sometimes for pushing forward a toolchain feature with the mind of both communities, or sometimes just for getting sense of how things work with mailing list+GNU make.

Updated in 2026-07.

GNU indirect function (ifunc) is a mechanism making a direct function call resolve to an implementation picked by a resolver. It is mainly used in glibc but has adoption in FreeBSD.

Updated in 2024-06.

Subtitle: Is ld.lld a drop-in replacement for GNU ld?

The motivation for this article was someone challenging the "drop-in replacement" claim on LLD's website (the discussion was about Linux-like ELF toolchain):

LLD is a linker from the LLVM project that is a drop-in replacement for system linkers and runs much faster than them. It also provides features that are useful for toolchain developers.

99.9% pieces of software work with ld.lld without a change. Some linker script applications may need an adaption (such adaption is oftentimes due to brittle assumptions: asking too much from GNU ld's behavior which should be fixed anyway). So I defended for this claim.

中文版

Updated in 2026-01.

Many people just want to know how to define or reference versioned symbols properly. You may jump to Recommended usage below.

In 1995, Solaris' link editor and ld.so introduced the symbol versioning mechanism. Ulrich Drepper and Eric Youngdale borrowed Solaris' symbol versioning in 1997 and designed the GNU style symbol versioning for glibc.

gnu-gabi specification: https://sourceware.org/gnu-gabi/program-loading-and-dynamic-linking.txt#:~:text=versioning

中文版

Updated in 2025-02

(首先庆祝一下LLVM 2000 commits达成！)

Compiler driver options

Before describing the linker options, let's introduce the concept of driver options. The user-facing options of gcc and clang are called driver options. Some driver options affect the options passed to the linker. Many such options have the same name as the linker's, and they often have additional functions in addition to the options of the same name passed to the linker, such as: