Data Validation and Cleaning

From Mixed Text to Useful Data

Real-world data often arrives in mixed formats. A single line can contain an email address, a timestamp, and other information together. Parsing means breaking that larger string down to isolate the exact substring you need. In Python, a practical way to do this is to use find() to locate landmarks and string slicing to extract the characters between those landmarks.

Reading Positions with find()

The find() method locates a substring inside a string and returns the substring's starting index. An index identifies a character position in the larger string. For example, when find('@') returns 21, the @ character begins at position 21. The returned number is a position, not the substring itself.

The two-argument form find(substring, start_pos) begins searching at the specified position and continues onward. This is useful when the same kind of character or substring may appear more than once and you need the occurrence after a known landmark.

Turning Landmarks into a Slice

String slicing with [start:end] extracts characters from start up to but not including end. In this example, the start is atpos+1, which skips the @ symbol, and the end is sppos, which excludes the space. The indices therefore describe exactly which characters become the extracted value.

Tracing a Parsing Failure

When parsing does not produce the expected result, trace the indices step by step. First inspect the intermediate values returned by find(). Confirm that each landmark is at the position you expect. Then inspect the slice start and end positions. A mistake in either landmark can cause the slice to include the wrong characters or omit needed characters.

Mistakes with Indices

• Treating the value from find() as the extracted text

  The returned value is the starting index of the @ character, not the domain or another substring.

  Fix: Use the returned index as a landmark for a later slice.

• Including the delimiter in the extracted value

  The desired domain begins one position after the @ character.

  Fix: Use atpos+1 when the delimiter itself should be excluded.

• Assuming the slice includes its end position

  A slice extracts up to but not including its end index.

  Fix: Remember that [start:end] stops before end.

• Using a -1 result as though it were a valid landmark

  The expected substring was not found, so the planned boundary is unavailable.

  Fix: Check the result before using it in a slice and inspect intermediate values while debugging.

• Searching from the beginning when a later occurrence is needed

  The search may locate a space before the relevant landmark.

  Fix: Pass the known landmark as the starting position, as in find(' ', atpos).

Guided Practice

Parsing Workflow

1. Identify a meaningful landmark in the larger string with find().
2. Store or inspect the starting index returned by find().
3. Use find(substring, start_pos) when the search must begin after a known position.
4. Convert the landmark positions into slice boundaries.
5. Remember that [start:end] includes start and excludes end.
6. Check every find() result for -1 before using it in a slice.
7. Print intermediate index values when debugging unexpected parsing results.

Parsing becomes manageable when you treat each index as a boundary in the original string. find() tells you where a landmark begins; a starting position narrows a later search; slicing uses the resulting boundaries to isolate the needed characters. If the result is wrong, inspect those boundaries in order rather than guessing at the final substring.