ARTICLE / 01
Auditing Tools

Auditing Tools in Software Delivery: Tracing a Release to Production

How auditors reconcile planning, source control, pipelines, artifacts, approvals, and production records to verify that software releases followed required controls.

Published
Reading time
8 min read
Author
Angélica Rosario Maisonet
↓

When a release reaches production, the team usually checks that the application is working and that no new errors have appeared. For an IT audit, I would also want to know which code went live, who reviewed it, and how it was approved. Finding those answers may require records from several systems.

The requirement might be in Jira, the code review in GitLab, the build and tests in Jenkins, the package in Nexus, and the change approval in ServiceNow. To review the release, the auditor needs to connect these records to the deployment. An approval in ServiceNow is useful, but it needs to apply to the version that actually reached production.

Auditing tools help organize and compare this information. Some platforms support the development process and keep records of the controls performed. Others, such as SQL, spreadsheets, and audit analytics software, help the auditor examine those records. ISACA’s COBIT for DevOps Audit Program covers audit work across planning, deployment, monitoring, and management. The IIA’s Global Internal Audit Standards establish the evidence requirement: information used to support a conclusion must be relevant, reliable, and sufficient.

What each tool contributes

The tools available will vary by organization. Platforms for continuous integration and delivery (CI/CD), such as Jenkins, can provide build and test records. In a software delivery audit, I would group the tools by the information needed for the review:

Area Examples Information to review
Request and approval Jira, ServiceNow Reason for the change, risks, testing plans, approvals, and authorized exceptions.
Code, checks, and package GitLab or GitHub, Jenkins, Veracode, Nexus Reviewed commit, build results, required tests and security scans, and the package stored for deployment.
Production activity Deployment records, application monitoring, and relevant AWS CloudTrail events Version deployed, person or account that performed the change, and what happened after deployment.
Audit analysis SQL, scripts, spreadsheets, Caseware IDEA Records that match across systems, duplicates, missing records, and differences that need investigation.

Caseware IDEA, for example, can help join data sets and find duplicate or missing records. SQL can support similar comparisons. The auditor needs identifiers that connect the data, such as a change ID, commit identifier, or build number. If each export uses a different name for the same release, those differences need to be understood before treating an unmatched record as a problem.

Reviewing a release

Consider an update to a customer portal. The team records a requirement in Jira and reviews a merge request in GitLab. Jenkins runs the tests and produces a ZIP package, which is stored in Nexus. The team then obtains approval in ServiceNow and deploys the package. For the audit, the question is whether each record belongs to the release being reviewed.

I would begin with a list of production deployments from a source other than the change tickets. Starting only with approved tickets could leave out a deployment that never had one. The list should also include emergency changes, manual deployments, and applications that follow another release process within the audit’s scope.

For each selected deployment, the auditor can check the change ticket and its approval history, then locate the merge request and the commit SHA, the identifier for that code revision. The build record should show which commit was used. Test and scan results need to belong to that build, and any failed or skipped required check needs an explanation and the approval required by policy. The stored package and deployment record complete the comparison.

A version label such as 2.4.1 helps identify a release, although the same label can be reused. A cryptographic digest, or hash, provides a more precise way to compare package contents. If the digest recorded for the deployed package differs from the approved package, the auditor should investigate. The team may have rebuilt it for a legitimate reason and failed to update the documentation. It is also possible that the contents changed after approval. The records need to explain which occurred.

Approval dates deserve a separate check. An approval entered after deployment may point to a problem, but the auditor first needs to determine whether the change followed an authorized emergency process. The review should also cover the outcome: whether the release caused an incident, whether a rollback was needed, and whether the team completed the required follow-up before closing the change.

It helps to check in the other direction too. An approved ticket marked as complete should have a corresponding implementation record. Otherwise, the ticket may describe work that never reached production.

Checking access and security evidence

The reliability of the records depends partly on who can change them. Someone with access to approve code, edit pipeline settings, replace a package, and deploy it could bypass several controls. The auditor should review branch protections, approval permissions, service accounts, and access to logs. Four separate accounts still provide little separation of duties if one person controls all four.

Repository audit logs can help identify changes to access and settings. Their usefulness depends on the events recorded and how long the organization keeps them. In AWS, CloudTrail may provide additional information about relevant account activity. Its coverage depends on the services and event types collected, so the auditor should compare it with the deployment system. CloudTrail does not automatically record every application release or its package digest.

Security reports require a similar review of coverage. Static application security testing (SAST) checks code for certain weaknesses. Software composition analysis (SCA) looks for known issues in dependencies. Infrastructure as Code scanning may also apply when a release includes versioned infrastructure definitions. For each required check, the auditor should confirm which repository and version were examined, whether anything was excluded, and how open findings were handled.

A green pipeline needs that context. It reflects the jobs and rules configured for that run. If a security job was skipped, allowed to fail, or run against another version, the final pipeline status may give an incomplete picture. The auditor needs to review those details and identify who could approve an exception.

Some teams also keep a software bill of materials (SBOM), package digests, or signed build provenance. These records serve different purposes: an SBOM lists software components, a digest helps compare package contents, and verified provenance can help establish how a package was built. Their relevance depends on the technology, the risk, and the organization’s process. The auditor still needs evidence that the release was authorized and that the required controls ran.

Comparing releases over time

For a period of releases, the auditor can export deployment records, change tickets, and pipeline results and compare them with SQL, a spreadsheet, or an audit analytics tool. Useful checks include deployments without tickets, approvals dated after implementation, repeated overrides, and builds that cannot be linked to a change.

The list of deployments needs to be complete before calculating results. Reporting that 98% had approval is misleading if the total excludes manual deployments. The working papers should identify the source of that total and any gaps in coverage.

Each difference selected for follow-up needs to be checked against the original records and the policy in effect at the time. The control owner may be able to explain an unfamiliar status or an approved exception. Screenshots can support the review, but the underlying records, when available, provide the details needed to assess what happened.

Development teams may already monitor builds and deployments every day. Internal audit can use that information to perform tests more often while maintaining an independent assessment. The IIA’s guidance on continuous auditing and monitoring explains the distinction between management’s monitoring and internal audit’s work.

For me, the value of these tools is being able to explain a release with evidence: what was requested, what was reviewed, what was approved, and what went live. When those records do not agree, there is a specific issue to investigate and discuss with the team responsible.

References

  1. The Institute of Internal Auditors (IIA). Global Internal Audit Standards, Standard 14.1: Gathering Information for Analyses and Evaluation.
  2. ISACA. COBIT for DevOps Audit Program.
  3. The IIA. Continuous Auditing and Monitoring.
  4. Caseware. Introduction to IDEA Data Analysis.
  5. GitHub. Reviewing the Audit Log for Your Organization; GitLab. Audit Events.
  6. AWS. Understanding CloudTrail Events and Logging Data Events.
  7. NIST. Secure Software Development Framework, SP 800-218.
  8. SLSA. Verifying Artifacts and Build Provenance.