Locate the Extra Document Token
Context
A proofreading tool compares a clean token list with a revised list. The revised document contains every token from the clean list in the same order, plus one additional token. The tool needs the index where the aligned sequences first separate, using zero-based positions.
Problem
Given a reference token sequence and a revised token sequence, return the earliest zero-based index at which their aligned tokens differ. The revised sequence contains exactly one additional token while preserving the order of every reference token. Compare positions from the beginning and stop at the first mismatch. If all positions of the reference sequence match the corresponding revised positions, return the reference sequence length. This boundary result also applies when the reference sequence is empty or the additional token is at the end. Repeated tokens are allowed; report the first mismatch produced by the forward alignment rule, not an inferred insertion among indistinguishable repeated tokens.
Examples
reference_tokens = ["the", "quick", "brown", "fox"]revised_tokens = ["the", "quick", "small", "brown", "fox"]2reference_tokens = ["read", "write", "share"]revised_tokens = ["read", "write", "share", "learn"]3reference_tokens = ["go", "go", "home"]revised_tokens = ["go", "stop", "go", "home"]1Constraints
0 <= len(reference_tokens) <= 1001 <= len(reference_tokens[i]) <= 15 for every ireference_tokens[i]: alphabet lowercaselen(reference_tokens) + 1 <= len(revised_tokens) <= len(reference_tokens) + 11 <= len(revised_tokens[i]) <= 15 for every irevised_tokens[i]: alphabet lowercasereference_tokens and revised_tokens contain only lowercase letters.- Types:
reference_tokensis str[],revised_tokensis str[]; result is int
Function signature
def find_extra_token_position(reference_tokens: list[str], revised_tokens: list[str]) -> int
reference_tokenslist[str]- The original document tokens in order.
revised_tokenslist[str]- The tokens after one additional token was inserted.
- returns int
- The zero-based position of the first aligned mismatch, or the length of reference_tokens when every reference position matches.
Adapted from MBPP problem task_890 (CC BY 4.0). Rewritten, extended and verified by Iksha.