A reference record pairs an example input with the output you expect. It makes a useful starting point for testing a scraper because the expected behavior is visible in the test itself.
As we develop BatScrape’s Actors, we plan to use small, reviewable fixtures to check transformations and output shape. This is an approach under development, not a claim about an established production test suite.
Pick examples that reveal assumptions
Start with an ordinary record, a record missing optional fields, and a record with an unusual but valid value. Include the source context needed to explain why the expected output is correct.
Use synthetic or appropriately permitted examples, and avoid including credentials or private data in a fixture.
Fail when there is nothing to check
A test loader that finds no fixtures should report a failure. Otherwise, a renamed directory can turn a useful check into a successful run that checked nothing.
The same principle applies when the expected number of cases changes: make that change visible for review.
Keep the limits clear
Saved examples can verify parsing logic without making live requests. They cannot prove that a source still returns the same structure today. Live checks and fixture tests answer different questions, so we plan to document what each check covers as the Actors are released.
BatScrape Team
Building BatScrape
