Read Large Line-Oriented Input Safely in Go with bufio.Scanner
Line-oriented input looks simple until one record is much larger than expected. A program may process thousands of ordinary log lines correctly, then stop on a generated stack trace, a large JSON record, or a malformed input that contains no newline for megabytes. Go’s bufio.Scanner is convenient for this job because it handles tokenization and defaults to scanning lines. But that convenience comes with an important boundary: a scanner has a maximum token size. If a token cannot fit within that limit, scanning stops with an error.