6.1 Compile-Time Metaprogramming: Macros, Annotation Processing, and Code Generation

When ten data types need serialization, validation, and builder code, macros or generators can produce the repeated structure from declarations. The saved copying is useful, but the generator joins the build chain. Its inputs, outputs, errors, and versions need maintenance too. Code that writes code still needs someone responsible for it.

First, Ask: When Does Code Generation Occur?

"Code generates code" is too broad. A more useful classification is:

Source code text
  ↓ Preprocessing / macro expansion
Syntax or language model
  ↓ Annotation processing / derived code
Compilation product
  ↓ Bytecode weaving / linking
Runtime
  ↓ Reflection / dynamic proxies / class creation
Behavior

The stage determines what tools can see, and when errors are revealed:

• Text preprocessors focus on tokens or text fragments;
• Syntax macros can match language structures;
• Annotation processors see parsed elements and type models;
• Bytecode tools operate on the compiled output;
• Runtime metaprogramming can inspect actual objects and configuration, but errors are exposed later.

Don’t classify only by language name. The same language may offer mechanisms across multiple stages.

C Preprocessor Macros: Powerful but Lacking Type Information

#define SQUARE(x) ((x) * (x))

Parentheses can avoid some precedence issues, but they don't solve the problem of parameters being evaluated twice:

int i = 3;
int value = SQUARE(i++); // Two unsequenced modifications of i: undefined behavior in C

This is an invalid example, so do not infer a fixed value from a particular run. C does not sequence the operands of multiplication; the two modifications of i after expansion cause undefined behavior. An ordinary function avoids repeated argument evaluation:

static inline int square(int x) { return x * x; }

int i = 3;
int value = square(i++); // value == 9, i == 4

The product must still fit in int; single evaluation does not solve signed overflow. The preprocessor operates before C semantic analysis and has no knowledge of the type of x. It also can't guarantee that an expression is evaluated only once, as a function would. Macros remain useful for conditional compilation and minimal boilerplate, but error messages, debugging, and scope management require extra care.

Rust Declarative Macro Token Tree Matching

macro_rules! hashmap {
    ($( $key:expr => $value:expr ),* $(,)?) => {{
        let mut map = ::std::collections::HashMap::new();
        $( map.insert($key, $value); )*
        map
    }};
}

fn main() {
    let ports = hashmap!["http" => 80, "https" => 443];
    assert_eq!(ports["http"], 80);
    println!("{}", ports.len());
}

Save this as ports.rs, compile with rustc --edition 2021 ports.rs, and run ./ports to print 2. A later duplicate key replaces the earlier value. Empty input needs a type annotation, such as let empty: std::collections::HashMap<&str, u16> = hashmap![];.

The matcher and transcriber of macro_rules! operate on a Token Tree, using fragments like expr, ty, and pat to specify expected grammar categories. Calling this "direct AST manipulation" is misleading: matching occurs at boundaries defined by the macro system's tokens and fragments, not directly on the AST structure itself.

Rust declarative macros exhibit mixed-site hygiene: local variables and labels use definition-site name resolution, while other symbols use invocation-site resolution. Hygiene reduces unintended name capture, but does not eliminate the need for macro authors to consider names and paths. Library macros often require explicit use of $crate or fully qualified paths to ensure proper resolution when called from other crates.

When designing declarative macros, prioritize:

• Keeping the input syntax short and unambiguous;
• Moving complex business logic into regular functions, with macros responsible only for structural expansion;
• Providing compile-time errors as close as possible to the call site when input shapes are unsupported;
• Using expansion tools to validate the actual output, while ensuring callers do not depend on internal expansion details.

Rust Procedural Macros Receive and Return TokenStream

Procedural macros fall into three categories: function-like macros, custom derive macros, and attribute macros. The entry point is a Rust function executed at compile time. This block shows only the API shape, not a compilable implementation. Real code must import proc_macro::TokenStream, live in a proc-macro crate, and implement parsing and token generation:

#[proc_macro_derive(Validate, attributes(validate))]
pub fn derive_validate(input: TokenStream) -> TokenStream {
    // Parse input TokenStream, Generate implementation, then return TokenStream
}

Procedural macros are unhygienic. The code they generate can be influenced by imports at the call site, or may introduce naming conflicts. Macro authors should use absolute paths, low-conflict internal names, and preserve reasonable spans to improve error diagnostics.

Since procedural macros run at compile time, they should be treated as part of the supply chain: they can consume CPU and memory, and may affect build reproducibility. Generators must not secretly access the network, rely on the current time, or embed machine-specific paths into the output artifact.

Java Annotation Processors Use the Language Model

Java annotation processing provides Element, TypeMirror, Filer, and Messager. This experiment generates only a type-name constant, making the declaration-to-source-to-compilation path visible before adding JSON mapping or other complex logic. Use a JDK that supports --release 17, and place these three source files in one empty experiment directory.

EmitInfo.java declares the annotation:

package example;

import java.lang.annotation.*;

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.SOURCE)
public @interface EmitInfo {}

InfoProcessor.java checks inputs and generates source:

package example;

import java.io.IOException;
import java.util.Set;
import javax.annotation.processing.*;
import javax.lang.model.SourceVersion;
import javax.lang.model.element.*;
import javax.tools.Diagnostic;

@SupportedAnnotationTypes("example.EmitInfo")
@SupportedSourceVersion(SourceVersion.RELEASE_17)
public final class InfoProcessor extends AbstractProcessor {
    @Override
    public boolean process(Set<? extends TypeElement> annotations,
                           RoundEnvironment roundEnv) {
        if (roundEnv.processingOver()) return true;
        for (Element element : roundEnv.getElementsAnnotatedWith(EmitInfo.class)) {
            TypeElement type = (TypeElement) element;
            if (type.getNestingKind() != NestingKind.TOP_LEVEL
                    || (type.getKind() != ElementKind.CLASS
                        && type.getKind() != ElementKind.RECORD)) {
                processingEnv.getMessager().printMessage(
                    Diagnostic.Kind.ERROR, "top-level class or record required", type);
                continue;
            }
            String pkg = processingEnv.getElementUtils()
                .getPackageOf(type).getQualifiedName().toString();
            String name = type.getSimpleName() + "Info";
            String qualified = pkg.isEmpty() ? name : pkg + "." + name;
            try (var writer = processingEnv.getFiler()
                    .createSourceFile(qualified, type).openWriter()) {
                if (!pkg.isEmpty()) writer.write("package " + pkg + ";\n");
                writer.write("public final class " + name + " {\n"
                    + "  public static final String TYPE = \""
                    + type.getQualifiedName() + "\";\n}\n");
            } catch (IOException error) {
                processingEnv.getMessager().printMessage(
                    Diagnostic.Kind.ERROR, error.toString(), type);
            }
        }
        return true;
    }
}

User.java uses the generated type:

package example;

@EmitInfo
public record User(String name) {
    public static void main(String[] args) {
        System.out.println(UserInfo.TYPE);
    }
}

mkdir -p processor-classes generated classes
javac --release 17 -proc:none -d processor-classes EmitInfo.java InfoProcessor.java
javac --release 17 -cp processor-classes -processorpath processor-classes -processor example.InfoProcessor -s generated -d classes User.java
java -cp classes example.User

Expect example.User and the generated file generated/example/UserInfo.java. Compile the processor separately, then enable it explicitly with -processor. Since JDK 23, builds cannot rely on the old default of automatic classpath discovery. --release 17 fixes this experiment's language and API target; it does not turn the compiler running the processor into JDK 17.

Processing may involve multiple rounds: types generated in one round can participate in subsequent rounds. Filer manages the generation location, and within a single processing run, the same path must not be created repeatedly. Here, generated UserInfo lacks @EmitInfo, so later rounds do not regenerate it. A user-declared type with the same name produces a file-conflict diagnostic.

Standard annotation processors typically generate new files rather than arbitrarily modifying existing source code ASTs. Relying on internal compiler packages to rewrite the AST binds the processor to a specific javac implementation; the official documentation explicitly states that internal classes of com.sun.tools.javac may change at any time.

A reliable processor should:

• Use Messager to associate errors with specific elements;
• Pass originating elements to Filer to help incremental builds determine dependencies;
• Produce stable, formattable, checkable source; the build system must also remove generated files whose inputs were deleted;
• Handle repeated generation and round termination in an idempotent manner;
• Be tested across multiple JDK and build tool versions to ensure compatibility with different execution environments.

Generate Source Code or Generate Bytecode

Generating source code offers advantages in readability, debuggability, and ease of navigation within IDEs. However, it increases the number of build steps and generated files. Directly generating class files or modifying bytecode provides greater flexibility, but comes with higher debugging overhead, compatibility risks, and greater responsibility for validation.

When making a decision, consider the following trade-offs:

Requirement	More Suitable Mechanism
Generate repeated implementations based on type declarations	Annotation processing / derive macros
Define compact, expressive syntax patterns	Declarative macros
Generate client or protocol models at build time	Independent code generators
Observe method calls at runtime	Proxies / instrumentation
Simply want to reuse common business logic	Functions, composition, or generics, typically no need for metaprogramming

Generating Code Also Requires Product-Grade Interfaces

A generator's public contract includes annotations, macro inputs, generated type names, and error messages. During upgrades, at a minimum, the following must be verified:

• Whether previous inputs still produce compatible APIs;
• Whether the generation order is deterministic;
• Whether the output is consistent across different machines;
• Whether generated files include version and source information;
• Whether caching or incremental compilation might leave behind stale files;
• Whether generator dependencies have undergone secure review to align with compiler plugin requirements.

Completion Checklist

Select a requirement for generating JSON mappers for data types:

1. Clearly specify input declarations and the generated API;
2. Decide whether to use Rust derive, Java annotation processing, or an independent generator;
3. Define at which stage errors are reported;
4. Include two invalid input examples;
5. Demonstrate that successive builds produce consistent artifacts and that old artifacts are properly cleaned when inputs are deleted.

References

• Rust Reference: Macros
• Rust Reference: Macros by Example
• Rust Reference: Procedural Macros
• Java SE 26: Annotation Processing
• JDK 26: javac
• Java SE 26: Filer