Split the token, blind the head

Models Don't Read Words, They Read Integer Token IDs

Before your prompt ever touches Layer 1 of a Transformer, it passes through a deterministic Byte-Pair Encoding (BPE) Tokenizer (tiktoken in OpenAI models, SentencePiece / tiktoken in Llama/Mistral/Gemma).

The tokenizer chops your raw string into subword chunks and replaces each chunk with an integer index from a fixed vocabulary (typically 32,000 to 256,000 tokens).

Here is the crucial vulnerability: the embedding lookup table has zero built-in awareness of spelling or character similarity. A given word maps to a specific row vector that RLHF may have heavily associated with the refusal circuits. If you force the tokenizer to split that word into different subword pieces, the model starts Layer 1 with completely different embedding vectors that have no direct wire to the fast refusal head.

(redacted: a worked tokenization example and a diagram showing how splitting a specific offensive tool name into character tokens routes past the early refusal head.)

Technique 1: BPE Token Smuggling & Subword Splitting

Why does splitting a trigger word into sub-tokens bypass alignment if the model eventually figures out what the word means anyway?

Because of Layer Asymmetry:

1. External Guardrail Classifiers (Layer 1) are tiny models with shallow attention that rely heavily on whole-token embeddings. Splitting a token blinds them completely.
2. Internal RLHF Refusal Circuits fire in specific early-to-middle layers based on pattern recognition over standard BPE token sequences. When a word arrives fragmented across several character tokens, the model has to spend its early layers mentally reassembling the spelling, delaying semantic recognition until layers where the completion trajectory is already underway.

(redacted: a list of concrete methods for forcing subword splits, including intra-word delimiters, runtime string concatenation of sensitive binary/API names, byte-array initialization, and typoglycemia, with code.)

Technique 2: Unicode Homoglyphs & Script Mixing

Unicode contains thousands of characters across Cyrillic, Greek, and Mathematical Alphanumeric blocks that look identical (or near-identical) to Latin ASCII letters to the human eye, but map to completely different UTF-8 byte sequences and BPE token IDs. A word spelled with a few look-alike characters tokenizes into a different ID sequence than its plain-Latin form, even though a human reads them the same.

(redacted: a table giving the exact character/byte substitutions for a homoglyph bypass and a note on which named guardrail classifiers it defeats.)

Technique 3: Gradient-Guided Adversarial Suffixes (GCG)

What if, instead of splitting tokens inside the prompt, we append a short run of carefully calculated tokens to the end of the prompt that mathematically hijack the attention heads feeding the refusal layer?

In Universal and Transferable Adversarial Attacks on Aligned Language Models, Zou et al. introduced GCG (Greedy Coordinate Gradient)¹.

How GCG Computes an Adversarial Suffix

Given a restricted prompt and a desired affirmative opening ("Sure, here is..."), GCG appends a placeholder suffix of k tokens (initialized as "! ! ! ! ..."). In each optimization step on a white-box model:

1. Backward Pass: Compute the gradient of the cross-entropy loss of generating the affirmative opening with respect to the one-hot token indicator at each suffix position.

g_i = gradient wrt token-indicator at position i of L( y_target | prompt + suffix )

1. Top-K Candidate Selection: The negative gradient tells you which vocabulary replacement tokens at position i will most strongly push the model toward the affirmative opening.
2. Greedy Substitution: Test a batch of random top-K single-token swaps in a forward pass, keep the swap that achieves the lowest loss, and repeat for 250–500 steps.

Why Do GCG Suffixes Look Like Gibberish, And Why Do They Transfer?

A converged GCG suffix looks like random noise to a human, but it is a dense mashup of markdown/comment delimiters, continuation commands (Now write, revert with), and structural contrast tokens.

(redacted: a concrete converged GCG suffix string.)

Because different models (Llama, Mistral, GPT, Claude) were pretrained on similar web/code corpora and aligned with similar SFT templates, a GCG suffix optimized simultaneously across an ensemble of open-weight models exploits shared representation geometry, allowing the suffix to transfer zero-shot to closed-box frontier models. This transferability is why GCG matters for defenders: a suffix found on open weights may work on a model whose weights you never see.

Field Exercise 3.05: BPE Smuggling vs. Guardrail Classifiers

(redacted: an exercise that ran against third-party hosted services (Lakera Gandalf, Google AI Studio, OpenAI Playground) and used token splitting and string concatenation of sensitive tool names to defeat input/output filters. Replace with a version run on a filter stack you operate, using a benign protected string.)

Next Lesson

In Lessons 01–05, our goal was to make a model comply when it wanted to refuse. What if we flip the goal upside down, and force an enterprise AI agent to refuse when it is supposed to work? In 06 Turn over-refusal into a weapon, we wrap up Card 3 by exploiting false-positive refusals (XSTest) for reconnaissance and Alignment Denial-of-Service (DoS).