PROMPT SOURCE

Text:     Charles Darwin, "On the Origin of Species By Means of Natural Selection" (John Murray, 1859).
Edition:  Project Gutenberg eBook #1228, https://www.gutenberg.org/ebooks/1228
File:     https://www.gutenberg.org/cache/epub/1228/pg1228.txt
          fetched 2026-10-03, sha256 ededa9c0bf8761efed092c303b46c1c92de956838cba6249a33bedfd6d7363b4
          (header: "Release date: March 1, 1998 [eBook #1228]; Most recently updated: October 29, 2024")
Licence:  Public domain. Darwin died in 1882; the 1859 text is out of copyright worldwide.
          Only the book text is used. The Project Gutenberg header, licence and footer are removed,
          and nothing here is distributed under the Project Gutenberg name or trademark.

Extraction (mkprompts.sh):
  1. keep the lines from "When on board H.M.S. 'Beagle,' ..." (start of the Introduction) up to the
     "*** END OF THE PROJECT GUTENBERG EBOOK" marker;
  2. collapse every run of whitespace (line breaks included) to one space;
  3. for each N, take the longest prefix, in characters, whose token count is exactly N.
     Counts were taken from `veizik run <model> --tokens 1` ("prompt : <bytes> bytes -> <N> tokens"),
     with qwen2.5-0.5b for *_qwen25 and qwen3-1.7b for *_qwen3, and checked independently with the
     Hugging Face tokenizers of Qwen/Qwen2.5-0.5B and Qwen/Qwen3-1.7B (transformers AutoTokenizer,
     encode without a chat template): every file has exactly N tokens under both.

The Qwen2.5 and Qwen3 tokenizers split this text identically, so p<N>_qwen25.txt and p<N>_qwen3.txt
are byte-identical for every N. Both names are kept so each model reads its own file.

file                   characters  bytes   tokens
p16384_qwen25.txt           80365   80546   16384
p24576_qwen25.txt          121311  121553   24576
p30000_qwen25.txt          147608  147876   30000
p3968_qwen25.txt            19896   19943    3968
p512_qwen25.txt              2351    2365     512
p8192_qwen25.txt            40503   40577    8192
p16384_qwen3.txt            80365   80546   16384
p24576_qwen3.txt           121311  121553   24576
p30000_qwen3.txt           147608  147876   30000
p3968_qwen3.txt             19896   19943    3968
p512_qwen3.txt               2351    2365     512
p8192_qwen3.txt             40503   40577    8192
