03.04. Effective Audit Procedures – Evidence-gathering Techniques

Briefly reflect on the following before we begin:

• Why is it essential for IS Auditors to gather reliable and relevant evidence during an audit?
• Can you explain the concepts of sufficiency and appropriateness of audit evidence?
• What strategies can IS Auditors use to make sure that they collect the most reliable and comprehensive evidence during an audit?

Comprehensive, detailed, and diligent documentation is vital in creating a robust audit trail. Documentation is not merely about recording facts; it’s about weaving a narrative that captures the essence of the audit process. This includes various materials, from policy documents and system logs to user manuals and transaction records. We will start our discussion in this section by considering how adequate documentation can illuminate the path of an audit, providing clarity and direction.

Understanding the nature of audit evidence is equally crucial, as sufficient and appropriate evidence is the foundation upon which IS auditors build their conclusions. It’s not just about gathering enough evidence; it’s about collecting the right kind of evidence. The quality of evidence is paramount. It must be relevant, reliable, and sufficient to support audit findings. It comes in various forms, each carrying its weight and relevance. We will highlight the difference between qualitative and quantitative evidence, stressing the need for a balanced approach. IS Auditors must be adept at evaluating both types of evidence and understanding their unique characteristics and implications for the audit.

Finally, we will examine the primary evidence-gathering techniques or the tools of the trade for every IS auditor. Each technique has its place and purpose, from interviews and document reviews to technical testing and data analytics. We will explore these techniques in detail, underlining their relevance and application in modern IS audits. Drawing from real-world experiences, this section aims to comprehensively understand effective audit procedures to conduct thorough, efficient, and effective IS audits.

The Role of Documentation in IS Auditing

Documentation and working paper management in IS auditing is far more than a collection of papers or digital files. It is the tangible representation of the audit’s journey, encompassing various forms and functions. Effective documentation is a meticulous process of capturing, organizing, and presenting information vital to the audit’s success. It includes policies, procedures, system logs, correspondence, and transaction records, each serving a unique purpose.

An audit trail is a chronological record providing a step-by-step account of the audit process, decisions, and actions taken. It is essential for ensuring transparency and accountability. Given the extent of professional judgment to be applied across various stages of an IS Audit, the need to create a clear, comprehensive audit trail to justify the professional judgment applied by IS Auditors cannot be understated. It facilitates the audit process and serves as a critical tool for any disputes or follow-up inquiries. Different types of documentation in IS audits serve different purposes. Policy documents, for example, provide insight into the organization’s regulatory compliance and governance standards. System logs offer a technical perspective, revealing user activities and system performance. Transaction records are crucial for verifying the accuracy and integrity of financial data. Each type of document contributes a piece to the puzzle, helping auditors form a complete picture of the IS environment they are examining.

Evaluating the reliability and relevance of documentation is a skill honed with experience. IS Auditors often encounter situations where documentation appears comprehensive but is outdated or not aligned with current practices. Diligent auditors must critically assess every piece of documentation, ensuring it is current, accurate, and relevant to the audit’s objectives. This evaluation forms the basis for sound audit conclusions and recommendations. Documentation standards and best practices in IS auditing are not merely guidelines; they are the principles that uphold the integrity of the audit process. These standards ensure that documentation is consistent, complete, and adheres to professional and regulatory requirements.

The Nature of Audit Evidence

Audit evidence is the raw material gathered during an audit to support the auditor’s observations, findings, and opinions. A wide range of evidence may be obtained during an IS audit, each type offering unique insights into the audit process. Understanding these different types of evidence is key to conducting a thorough and effective audit. Audit evidence may include the following:

• An IS auditor’s observations (presented to management),
• Notes taken from interviews,
• Material extracted from correspondence and internal documentation or contracts with external partners,
• Results of independent confirmations obtained by an IS auditor from different stakeholders, and/or
• The results of audit test procedures.

One must discern between qualitative and quantitative evidence in IS audits. Qualitative evidence, often narrative in nature, provides context and understanding of the processes and controls within an organization. This includes observations, interviews, and written explanations. Quantitative evidence, on the other hand, is numerical. It is derived from data sets, financial records, and transaction logs analysis. Both types of evidence are crucial, and a skilled auditor knows how to balance and integrate them to form a comprehensive audit perspective. Direct and indirect evidence also play significant roles in IS audits. Direct evidence is obtained through firsthand observation or interaction, such as inspecting a system configuration or reviewing a transaction record. Indirect evidence, conversely, is evidence that is inferred or deduced, such as conclusions drawn from analyzing trends in data logs. Understanding the impact of these evidence types on audit conclusions is a critical skill. Direct evidence often carries more weight, but indirect evidence can provide crucial corroborative support.

The digital nature of IS audits presents unique challenges, such as data volatility, systems’ complexity, and the need for specialized tools and skills to extract and analyze evidence. Navigating these challenges requires technical expertise and a keen understanding of the legal and ethical considerations involved in handling digital evidence. Generally, the reliability of audit evidence must include an evaluation of:

• Independence of the evidence provider: Evidence obtained from outside sources is more reliable than from within the organization. This is why confirmation letters are used for verification of accounts receivable balances. Additionally, signed contracts or agreements with external parties could be considered reliable if the original documents are available for review.
• Qualifications of the individual providing the information/evidence: IS auditors must verify whether the providers of the information/evidence are inside or outside of the organization, an IS auditor should always consider the qualifications and functional responsibilities of the persons providing the information.

Sufficiency and Appropriateness of Audit Evidence

Sufficiency and appropriateness are the two important drivers of the reliability of audit evidence.

Sufficiencyrefers to the quantity of evidence, while appropriateness pertains to the quality (reliability and relevance) of the evidence gathered. More evidence does not necessarily equate to better evidence. The focus should be on gathering enough relevant and reliable evidence to form a solid foundation for audit findings.

Relevance of information means there is a logical connection to the audit areas. Therefore, evidence is considered relevant if it provides confirmation about an area most at risk. For example, if the auditor determines that the primary assertion at risk is the security of the network firewall, it would not be appropriate to spend more time gathering evidence about the appropriateness of data back-ups. By identifying the key risk areas for the auditee, an IS auditor can focus on gathering more (sufficient) high-quality (appropriate) evidence where the risk of material misstatement is believed to be most significant.

Reliability refers to whether the evidence reflects the true state of the information. In terms of the reliability of information, the auditor should consider the following:

• The source of the information—it is important for the evidence to be unbiased. Information from external third parties is generally reliable because the respondent or the person from whom the information is sought is independent of the organization.
• The consistency of the information—evidence that is consistent from one source to another is more reliable than inconsistent evidence from one source to another. For example, if responses to inquiries of operational management and risk management functions are not consistent, the reliability of the information will be reduced.
• The source of information and whether it is produced where internal controls operate effectively—for example if there are robust controls over the change management cycle, then change tickets, user testing and post-implementation records will provide more reliable evidence than if the controls are ineffective.
• The form of the evidence—paper and electronic is more reliable than verbal evidence. For example, inspecting a service-level agreement to support lease commitment disclosure provides more reliable evidence than discussing the lease requirements with management.
• The way the documents were created and maintained—original documents are less likely to be altered, and therefore, they are considered more reliable than photocopied, scanned, or other transformed documents.
• The way the evidence is collected—evidence gathered directly by the IS auditor is considered more reliable than evidence gathered indirectly. For example, a bank confirmation sent directly to the auditor provides more reliable audit evidence than an online bank statement provided by the client.

Balancing the quantity of evidence with audit efficiency is a challenge every auditor faces. In the fast-paced environment of IS auditing, where technology and systems rapidly evolve, time is a precious resource. Auditors must be adept at collecting sufficient evidence promptly, ensuring that audits are both thorough and efficient. This requires a strategic approach to evidence gathering, prioritizing areas of higher risk and materiality. Lastly, overcoming limitations in audit evidence is part of the auditor’s expertise. In my years as an auditor, I have encountered various challenges, such as incomplete data, inaccessible information, or difficult to interpret evidence. Developing the skill to navigate these limitations is essential. It involves creative problem-solving, leveraging technology, and sometimes seeking alternative forms of evidence.

Primary Evidence-gathering Techniques

Audit procedures are the processes, techniques, and methods auditors perform to obtain audit evidence, enabling them to conclude on the set audit objective and express their opinions. IS Auditors prepare audit procedures at the planning stages once they identify audit objectives, scope, approach, and risks. Auditors design audit procedures to detect all kinds of identified risks and ensure that the required audit evidence is obtained sufficiently and appropriately. Audit procedures might be different across various functions and periods. This is because internal controls differ from one function to another, and the controls may change from time to time.

Having said that, IT auditors typically use the following six basic types of evidence-gathering techniques:

IS Auditing standards require that sufficient appropriate audit evidence must be gathered to enable an IS auditor to draw a conclusion on which to base their opinion regarding the fair presentation of the management IS operations. However, the decision as to what constitutes sufficient appropriate audit evidence is a matter of professional judgement, as it is based upon an auditor’s understanding of management’s IS processes and the significant risks identified when planning the audit and evidence gathered when executing the audit. Thus, it is essential for an IS auditor to not only be familiar with these primary evidence-gathering techniques but also employ them in the right situation, recognizing the strengths and limitations of such techniques. Generally, the following hierarchy of evidence can be used to judge reliability.

• Most Reliable: Physical inspection, Confirmation, External documentation, and Reperformance
• Less Reliable: External-internal documentation, Observation, and Analytical procedures
• Least Reliable: Internal documentation (poor controls), Inquiry, and Broad analytical procedures

This does not imply that the IS auditor would never or rarely use inquiries, broad analytical procedures, or observation. Each of these techniques is relevant in specific situations. Applying them justly and appropriately stems from a combination of the focus of the audit, adequate technical knowledge, a deep understanding of management’s process, the IS auditor’s experience, and professional judgment. A snapshot of the degree of reliability of each evidence-gathering technique is presented below for your reference.