Auditing Tools in Software Delivery: Following the Evidence from Request to Production

A release can reach production and pass its health checks while leaving one question unanswered: was this the change the organization reviewed and approved? A successful deployment shows that the software ran. An auditor needs more to determine whether the change followed the required controls.

The answer is spread across several tools. A planning system records why the work was requested. Source control shows what changed and who reviewed it. The pipeline records tests and security checks. An artifact repository stores the release package. Change management records authorization, and the production platform records what was deployed. The auditor follows the connections among those records, including the ones that are missing.

These records show how auditing tools work together. The development team uses tools to deliver software and run controls. The auditor compares their records, often with the help of data analysis, to test whether those controls worked for the releases being reviewed. ISACA’s COBIT for DevOps Audit Program covers planning, deployment, monitoring, and management. The IIA’s Global Internal Audit Standards require relevant, reliable, and sufficient information to support an audit conclusion.

Tools that produce evidence, and tools that test it

Role in the audit	Examples	What the auditor tests or does
Describe and authorize the change	Jira or another planning tool; ServiceNow or another change-management system	Compare the business reason, risk, testing evidence, approvals, implementation plan, and exceptions with the release actually made.
Show what was built and checked	GitLab or GitHub; Jenkins or another continuous integration and delivery (CI/CD) platform; scanners such as Veracode; Nexus or another artifact repository	Link the reviewed code change to the build, required checks, package, and deployment. Inspect failures, overrides, and changes to the rules themselves.
Show what happened in production	Deployment platform records, application monitoring, and, where relevant, AWS CloudTrail	Make a complete list of production releases and check who or what performed relevant actions. Investigate deployments that do not match approved changes.
Help the auditor analyze records	SQL, scripts, spreadsheets, or an audit analytics tool such as Caseware IDEA	Compare exports from different systems, find releases without matching records or duplicate identifiers, check dates and approvals, and select cases to investigate.

The last row matters because an auditor needs to compare the tools’ records, not just read each report separately. Caseware IDEA can join data sets and find duplicate identifiers or gaps; an auditor can run similar tests with SQL. A successful pipeline run only shows that the checks set up for that run passed. It does not prove that every release used the pipeline or that nobody changed or bypassed its rules.

Start with what actually reached production

Consider a hypothetical update to a customer portal. The team records a requirement in Jira, reviews a merge request in GitLab, builds and tests a ZIP in Jenkins, stores it in Nexus, obtains a production approval in ServiceNow, and deploys the package. Each record looks reasonable on its own. The audit question is whether they all refer to the same release.

The auditor first obtains a complete list of production deployments from a source independent of the change-ticket list. It should include emergency changes, manual deployments, and applications that use a different release process. Starting only with approved tickets would miss a deployment that never had one.

For a selected deployment, the auditor works backward:

Audit question	Records to reconcile	Example of an exception to investigate
Was there an approved reason to release?	Requirement, change request, and approval history	The ticket describes a different change or approval came after deployment without an authorized emergency process.
Was the released code reviewed?	Merge request, commit SHA, reviewers, and branch rules	The released commit differs from the reviewed commit.
Were required checks performed?	Pipeline run linked to the commit; test and scan results	A required scan failed, did not run, or was waived without an approved exception.
Was the approved package deployed?	Build record, artifact digest, change record, and deployment record	The package in production has a different digest from the one built and approved.
Was the outcome followed up?	Monitoring, incident, rollback, and change-closure records	A failed release was closed without the required investigation.

A matching version label, such as 2.4.1, helps, but labels can be reused. Where the systems record it, a cryptographic digest acts like a digital fingerprint of the package and allows a stronger comparison. A mismatch needs investigation: the team may have documented a legitimate rebuild poorly, or the package may have changed after approval. The auditor should find out what happened before calling it a control failure.

The auditor also tests in the other direction. Starting from approved changes and looking for their implementation records may reveal tickets marked complete even though nothing was deployed. Neither direction replaces the other.

Check the records behind the records

A complete-looking record can be misleading if someone can change the rules or erase the history. The auditor checks who can change branch protections or pipeline settings, approve their own code, bypass a failed security check, replace a package, approve a production change, or deploy outside the usual process. Service accounts matter too: four accounts do not create separation of duties if one person controls all four.

Repository audit logs can help test changes to settings and access, subject to the events the platform records and the organization’s retention. If the application runs on AWS, CloudTrail can provide another view of relevant account activity. Its coverage depends on the event types collected and the services involved; it is not automatically a record of every application deployment or of the package digest. The auditor should reconcile these sources with the deployment system rather than assume any one log is complete.

Security checks need the same treatment. Static application security testing examines code for selected weaknesses; software composition analysis examines known risks in dependencies. Infrastructure as Code scanning is relevant when the release includes versioned infrastructure definitions. The auditor asks whether the required checks covered the repository and release in scope, whether exclusions or failed jobs were visible, and who accepted unresolved findings. An empty report says little if the application was never scanned.

An artifact repository, digest, software bill of materials (SBOM), or signed build provenance can strengthen the trail when the organization uses them. Each answers a different question: an SBOM identifies components, a digest helps compare package contents, and verified provenance can help establish how a package was built. None independently proves that the application is secure or that the change was properly authorized. The controls chosen should fit the organization’s technology and risk.

Turn the trail into an audit test

An auditor can use SQL or an audit analytics tool to compare all production deployments in a period with change requests and pipeline records. Unmatched deployments, approvals after implementation, repeated gate overrides, and missing build identifiers become leads for follow-up. Before reporting a percentage such as “98% of deployments had approval,” the auditor must verify that the denominator includes the releases outside the normal pipeline.

For a sample of possible exceptions, the auditor reads the original records, confirms who acted and when, checks the policy in effect at the time, and discusses the result with the person responsible for the control. Screenshots and dashboards can help locate a record, but the conclusion should rely on the original records when available.

Delivery teams may monitor builds and deployments continuously. Internal audit may use those feeds for more frequent testing, but management’s monitoring and the auditor’s independent assessment have different responsibilities. The IIA’s guidance on continuous auditing and monitoring makes that distinction clear.

Conclusion

The strongest tool in this audit is the ability to reconcile a real production release with its reason, reviewed code, checks, authorization, package, and outcome. Jira, GitLab, Jenkins, Nexus, ServiceNow, production logs, and audit analytics can each contribute. Their value comes from what the auditor can verify across them, including exceptions that the usual workflow did not capture.

References

1. The Institute of Internal Auditors (IIA). Global Internal Audit Standards, Standard 14.1: Gathering Information for Analyses and Evaluation.
2. ISACA. COBIT for DevOps Audit Program.
3. The IIA. Continuous Auditing and Monitoring.
4. Caseware. Introduction to IDEA Data Analysis.
5. GitHub. Reviewing the Audit Log for Your Organization; GitLab. Audit Events.
6. AWS. Understanding CloudTrail Events and Logging Data Events.
7. NIST. Secure Software Development Framework, SP 800-218.
8. SLSA. Verifying Artifacts and Build Provenance.