When writing software systems, we are usually taught to ensure that code has unit and integration tests. In core curriculum, it’s not often taught that live data audits and debuggability are also important to ensure code quality.
Unit and integration tests is well known in the software engineering community. Unit tests run subsystems in isolation of other running subsystems, usually in a non-production environment. They ensure that the subsystem behaves according to predefined expectations through assertion statements (e.g., assertTrue, assertFalse, assertEqual). Integration tests verify that the composition of several subsystems work together according to predefined expectations, also in a non-production environment.
When deploying production software systems, data inconsistencies can get introduced into a database from bad user input or buggy code. Data inconsistencies aren’t usually tested in unit and integration testing, because most developers assume clean data in non-production environment. It’s also hard to imagine all sorts of data inconsistencies that can arise. To ensure a software system is running correctly in production, live data audits should run on a production server. Live data audits verify expectations of data to ensure data integrity. Example live data audits are internal-facing dashboard web pages and batch reports on a production server.
Another approach to diagnose problems on production server systems is to ensure debuggability. Unlike a local environment, production servers cannot be interrupted during execution with breakpoints and step-through code. To figure out what the code is thinking, developers have to write debugging statements (e.g., print statements, log commands) that is captured in log files. These log files are made accessible to developers to trace unexpected branches of execution.
In conclusion, building software systems requires more than unit and integration testing to ensure code quality. Live data audits and debuggability are just as important to ensure code is working.