03. Planning an IS Audit

03.04. Effective Audit Procedures – Evidence-gathering Techniques

Credit : Three people working in the office by Yan Krukau, used under the Pexels License.

Briefly reflect on the following before we begin:

  • Why is it essential for IS Auditors to gather reliable and relevant evidence during an audit?
  • Can you explain the concepts of sufficiency and appropriateness of audit evidence?
  • What strategies can IS Auditors use to make sure that they collect the most reliable and comprehensive evidence during an audit?

Comprehensive, detailed, and diligent documentation is vital in creating a robust audit trail. Documentation is not merely about recording facts; it’s about weaving a narrative that captures the essence of the audit process. This includes various materials, from policy documents and system logs to user manuals and transaction records. We will start our discussion in this section by considering how adequate documentation can illuminate the path of an audit, providing clarity and direction.

Understanding the nature of audit evidence is equally crucial, as sufficient and appropriate evidence is the foundation upon which IS auditors build their conclusions. It’s not just about gathering enough evidence; it’s about collecting the right kind of evidence. The quality of evidence is paramount. It must be relevant, reliable, and sufficient to support audit findings. It comes in various forms, each carrying its weight and relevance. We will highlight the difference between qualitative and quantitative evidence, stressing the need for a balanced approach. IS Auditors must be adept at evaluating both types of evidence and understanding their unique characteristics and implications for the audit.

Finally, we will examine the primary evidence-gathering techniques or the tools of the trade for every IS auditor. Each technique has its place and purpose, from interviews and document reviews to technical testing and data analytics. We will explore these techniques in detail, underlining their relevance and application in modern IS audits. Drawing from real-world experiences, this section aims to comprehensively understand effective audit procedures to conduct thorough, efficient, and effective IS audits.

The Role of Documentation in IS Auditing

Documentation and working paper management in IS auditing is far more than a collection of papers or digital files. It is the tangible representation of the audit’s journey, encompassing various forms and functions. Effective documentation is a meticulous process of capturing, organizing, and presenting information vital to the audit’s success. It includes policies, procedures, system logs, correspondence, and transaction records, each serving a unique purpose.

An audit trail is a chronological record providing a step-by-step account of the audit process, decisions, and actions taken. It is essential for ensuring transparency and accountability. Given the extent of professional judgment to be applied across various stages of an IS Audit, the need to create a clear, comprehensive audit trail to justify the professional judgment applied by IS Auditors cannot be understated. It facilitates the audit process and serves as a critical tool for any disputes or follow-up inquiries. Different types of documentation in IS audits serve different purposes. Policy documents, for example, provide insight into the organization’s regulatory compliance and governance standards. System logs offer a technical perspective, revealing user activities and system performance. Transaction records are crucial for verifying the accuracy and integrity of financial data. Each type of document contributes a piece to the puzzle, helping auditors form a complete picture of the IS environment they are examining.

Evaluating the reliability and relevance of documentation is a skill honed with experience. IS Auditors often encounter situations where documentation appears comprehensive but is outdated or not aligned with current practices. Diligent auditors must critically assess every piece of documentation, ensuring it is current, accurate, and relevant to the audit’s objectives. This evaluation forms the basis for sound audit conclusions and recommendations. Documentation standards and best practices in IS auditing are not merely guidelines; they are the principles that uphold the integrity of the audit process. These standards ensure that documentation is consistent, complete, and adheres to professional and regulatory requirements.

The Nature of Audit Evidence

Audit evidence is the raw material gathered during an audit to support the auditor’s observations, findings, and opinions. A wide range of evidence may be obtained during an IS audit, each type offering unique insights into the audit process. Understanding these different types of evidence is key to conducting a thorough and effective audit. Audit evidence may include the following:

  • An IS auditor’s observations (presented to management),
  • Notes taken from interviews,
  • Material extracted from correspondence and internal documentation or contracts with external partners,
  • Results of independent confirmations obtained by an IS auditor from different stakeholders, and/or
  • The results of audit test procedures.

One must discern between qualitative and quantitative evidence in IS audits. Qualitative evidence, often narrative in nature, provides context and understanding of the processes and controls within an organization. This includes observations, interviews, and written explanations. Quantitative evidence, on the other hand, is numerical. It is derived from data sets, financial records, and transaction logs analysis. Both types of evidence are crucial, and a skilled auditor knows how to balance and integrate them to form a comprehensive audit perspective. Direct and indirect evidence also play significant roles in IS audits. Direct evidence is obtained through firsthand observation or interaction, such as inspecting a system configuration or reviewing a transaction record. Indirect evidence, conversely, is evidence that is inferred or deduced, such as conclusions drawn from analyzing trends in data logs. Understanding the impact of these evidence types on audit conclusions is a critical skill. Direct evidence often carries more weight, but indirect evidence can provide crucial corroborative support.

The digital nature of IS audits presents unique challenges, such as data volatility, systems’ complexity, and the need for specialized tools and skills to extract and analyze evidence. Navigating these challenges requires technical expertise and a keen understanding of the legal and ethical considerations involved in handling digital evidence. Generally, the reliability of audit evidence must include an evaluation of:

  • Independence of the evidence provider: Evidence obtained from outside sources is more reliable than from within the organization. This is why confirmation letters are used for verification of accounts receivable balances. Additionally, signed contracts or agreements with external parties could be considered reliable if the original documents are available for review.
  • Qualifications of the individual providing the information/evidence: IS auditors must verify whether the providers of the information/evidence are inside or outside of the organization, an IS auditor should always consider the qualifications and functional responsibilities of the persons providing the information.

Sufficiency and Appropriateness of Audit Evidence

Sufficiency and appropriateness are the two important drivers of the reliability of audit evidence.

Sufficiencyrefers to the quantity of evidence, while appropriateness pertains to the quality (reliability and relevance) of the evidence gathered. More evidence does not necessarily equate to better evidence. The focus should be on gathering enough relevant and reliable evidence to form a solid foundation for audit findings.

Relevance of information means there is a logical connection to the audit areas. Therefore, evidence is considered relevant if it provides confirmation about an area most at risk. For example, if the auditor determines that the primary assertion at risk is the security of the network firewall, it would not be appropriate to spend more time gathering evidence about the appropriateness of data back-ups. By identifying the key risk areas for the auditee, an IS auditor can focus on gathering more (sufficient) high-quality (appropriate) evidence where the risk of material misstatement is believed to be most significant.

Reliability refers to whether the evidence reflects the true state of the information. In terms of the reliability of information, the auditor should consider the following:

  • The source of the information—it is important for the evidence to be unbiased. Information from external third parties is generally reliable because the respondent or the person from whom the information is sought is independent of the organization.
  • The consistency of the information—evidence that is consistent from one source to another is more reliable than inconsistent evidence from one source to another. For example, if responses to inquiries of operational management and risk management functions are not consistent, the reliability of the information will be reduced.
  • The source of information and whether it is produced where internal controls operate effectively—for example if there are robust controls over the change management cycle, then change tickets, user testing and post-implementation records will provide more reliable evidence than if the controls are ineffective.
  • The form of the evidence—paper and electronic is more reliable than verbal evidence. For example, inspecting a service-level agreement to support lease commitment disclosure provides more reliable evidence than discussing the lease requirements with management.
  • The way the documents were created and maintained—original documents are less likely to be altered, and therefore, they are considered more reliable than photocopied, scanned, or other transformed documents.
  • The way the evidence is collected—evidence gathered directly by the IS auditor is considered more reliable than evidence gathered indirectly. For example, a bank confirmation sent directly to the auditor provides more reliable audit evidence than an online bank statement provided by the client.

Balancing the quantity of evidence with audit efficiency is a challenge every auditor faces. In the fast-paced environment of IS auditing, where technology and systems rapidly evolve, time is a precious resource. Auditors must be adept at collecting sufficient evidence promptly, ensuring that audits are both thorough and efficient. This requires a strategic approach to evidence gathering, prioritizing areas of higher risk and materiality. Lastly, overcoming limitations in audit evidence is part of the auditor’s expertise. In my years as an auditor, I have encountered various challenges, such as incomplete data, inaccessible information, or difficult to interpret evidence. Developing the skill to navigate these limitations is essential. It involves creative problem-solving, leveraging technology, and sometimes seeking alternative forms of evidence.

Primary Evidence-gathering Techniques

Audit procedures are the processes, techniques, and methods auditors perform to obtain audit evidence, enabling them to conclude on the set audit objective and express their opinions. IS Auditors prepare audit procedures at the planning stages once they identify audit objectives, scope, approach, and risks. Auditors design audit procedures to detect all kinds of identified risks and ensure that the required audit evidence is obtained sufficiently and appropriately. Audit procedures might be different across various functions and periods. This is because internal controls differ from one function to another, and the controls may change from time to time.

Having said that, IT auditors typically use the following six basic types of evidence-gathering techniques:

 

Table: Evidence-Gathering Techniques
Technique Description Example
Inquiry Inquiry is often the starting point in evidence gathering. It involves engaging with personnel to gain insights and information. This includes formal interviews, casual conversations, and questionnaires. Inquiry is more than just asking questions; it’s about listening and interpreting the responses to form a broader understanding of the audit area. However, it is important to remember that information obtained through inquiry needs to be corroborated with other evidence forms, as it is subject to biases and misunderstandings. An IS auditor interviews the IT staff to understand the procedures for system updates and patches. The auditor inquires about how often these updates occur, how they are documented, and how they are approved. This helps assess the organization’s current approach to maintaining system security and software.
Observation Observation is another fundamental technique, where the IS auditors observe processes, operations, and activities to understand how systems and controls are implemented and functioning. Observation provides real-time evidence, offering a snapshot of the activities under review. It’s particularly useful in understanding workflows and identifying deviations from prescribed procedures. However, the limitation of observation is that it only provides evidence for the observed period. The IS auditor observes the backup process in real-time to ensure that data backup procedures follow the policy. This includes verifying that backups are taken at scheduled times and that the correct data sets are being backed up, providing IT assurance on data integrity and availability.
Analysis Analysis involves scrutinizing data and information to identify patterns, anomalies, and trends. This often entails analyzing system logs, financial records, and transaction data in IS auditing. The power of analysis lies in its ability to transform raw data into meaningful insights. With advanced analytical tools and techniques, auditors can analyze large datasets more efficiently and effectively. However, interpreting the results correctly requires a deep understanding of both the business and the technology. The IS auditor analyzes system logs to identify unusual or unauthorized access attempts. By reviewing these logs, the auditor can spot patterns that might indicate security breaches or attempts at data theft. This analysis helps in evaluating the effectiveness of the organization’s network security measures.
Inspection Inspection involves the examination of records, documents, and tangible assets. This could include reviewing contracts, policies, system configurations, and physical verification of assets. Inspection provides concrete evidence and is essential in verifying the existence and accuracy of assets and information. The meticulous nature of inspection demands a keen eye for detail and a thorough understanding of valid and reliable documentation. The IS auditor inspects access control lists to ensure that permissions and roles are appropriately assigned. By examining these lists, the auditor can verify that users have access rights consistent with their job functions, reducing the risk of unauthorized access to sensitive information.
Confirmation Confirmation is a technique used to obtain a direct response from a third party verifying the accuracy of information. This could involve confirming account balances, contractual terms, or the authenticity of transactions. Confirmation serves as an independent and objective source of evidence, often providing a high level of assurance. However, the challenge lies in ensuring that responses are received from the appropriate and authoritative sources. The auditor sends a confirmation request to a third-party service provider to verify the terms of service and data handling procedures. This is especially relevant for cloud-based services where the organization’s data is stored off-site. Confirmation from the service provider helps assess compliance with data privacy and security standards.
Reperformance Reperformance is a technique where the auditor independently executes procedures or controls to validate their effectiveness. This includes recalculating financial figures or reprocessing transactions. Reperformance provides a high level of assurance as it allows the auditor to directly assess the reliability of controls and procedures. However, it requires a comprehensive understanding of the systems and processes being audited. The auditor reperforms a sample of transactions to verify the effectiveness of application controls. This could include reprocessing transactions through the system to ensure the controls correctly capture, process, and report data. Reperformance assures the auditor that the application controls are functioning as intended.

IS Auditing standards require that sufficient appropriate audit evidence must be gathered to enable an IS auditor to draw a conclusion on which to base their opinion regarding the fair presentation of the management IS operations. However, the decision as to what constitutes sufficient appropriate audit evidence is a matter of professional judgement, as it is based upon an auditor’s understanding of management’s IS processes and the significant risks identified when planning the audit and evidence gathered when executing the audit. Thus, it is essential for an IS auditor to not only be familiar with these primary evidence-gathering techniques but also employ them in the right situation, recognizing the strengths and limitations of such techniques. Generally, the following hierarchy of evidence can be used to judge reliability.

  • Most Reliable: Physical inspection, Confirmation, External documentation, and Reperformance
  • Less Reliable: External-internal documentation, Observation, and Analytical procedures
  • Least Reliable: Internal documentation (poor controls), Inquiry, and Broad analytical procedures

This does not imply that the IS auditor would never or rarely use inquiries, broad analytical procedures, or observation. Each of these techniques is relevant in specific situations. Applying them justly and appropriately stems from a combination of the focus of the audit, adequate technical knowledge, a deep understanding of management’s process, the IS auditor’s experience, and professional judgment. A snapshot of the degree of reliability of each evidence-gathering technique is presented below for your reference.

Table: Reliability of Evidence-Gathering Techniques
Types of Evidence &  
Extent of Reliability* 
Independence  of Provider  Effectiveness 
of Auditee’s Internal Control 
Auditor’s Direct Knowledge  Qualifications of Provider  Objectivity of Evidence 
Inquiries (*) Low (Client provides) Not Applicable Low Varies Varies
(low to high)
Analysis (*) High (Auditor does)

Low (Client provides)

Varies High Not Applicable Low
Observation (*) High
(Auditor does)
Varies High Normally High  (Auditor does) Medium
Inspection (**) High
(Auditor does)
Varies High Normally High (Auditor does) High
Confirmation (***) High Not Applicable Low Varies
(Usually High)
High
Recalculation/ Reperformance (***) High
(Auditor does)
Varies High High
(Auditor does)
High

In the Spotlight

For additional context on the nature, role, and types of audit evidence, please read the article “What are the types of audit evidence?” [opens a new tab].

RiskOptics. (2023). What are the types of audit evidence? https://reciprocity.com/blog/what-are-the-types-of-audit-evidence/

 

Key Takeaways

Let’s recap the key concepts discussed in this section by watching this video.

Source: Mehta, A.M. (2023, December 6). AIS OER ch 03 topic 04 key takeaways [Video]. https://youtu.be/9sRffp30Fto

 

Knowledge Check

 

Review Questions

  1. Explain why documentation is crucial in IS auditing and list two types of documents typically reviewed during an IS audit.
  2. Distinguish between qualitative and quantitative evidence in IS auditing and give an example of each.
  3. What do ‘sufficiency’ and ‘appropriateness’ of audit evidence mean, and why are they important in IS auditing?
  4. Describe the technique of ‘Reperformance’ in IS auditing and explain its significance.

 

Mini Case Study

XYZ Corporation, a mid-sized manufacturing company, recently implemented a new Enterprise Resource Planning (ERP) system. As part of the IS audit, you are tasked with assessing the effectiveness of specific IT controls. The controls you need to assess include the following:

  • User access controls to ensure only authorized personnel can access the ERP system.
  • Change management controls for any modifications to the ERP system.
  • Backup procedures to ensure data integrity and availability.

Required: Develop test of controls audit procedures using one or more evidence-gathering techniques (Inquiry, Analysis, Observation, Inspection, Confirmation, Reperformance) discussed in this section.

definition

License

Icon for the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License

Auditing Information Systems Copyright © 2024 by Amit M. Mehta is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, except where otherwise noted.

Share This Book