07.05. Follow-up and Monitoring of IS Audits

Briefly reflect on the following before we begin:

• Why is reporting on follow-up activities important?
• How should the implementation of audit recommendations be monitored?
• What metrics can be used to evaluate the effectiveness of post-audit changes?

The follow-up on the IS audit findings phase is where the audit cycle comes full circle. It needs to be more to identify issues and recommend solutions; the real impact of an audit is seen in how these findings are addressed and resolved. In this section, we will go through the various follow-up stages, emphasizing the auditor’s role in ensuring that recommendations are implemented and that the audited entity achieves the desired improvements. We will go over establishing a structured approach to monitor and review the actions taken by the auditee in response to the audit findings. We will learn about setting timelines, defining responsibilities, and creating tracking and reporting progress mechanisms.

The section will also go over the practical aspects of ensuring that the corrective actions are implemented and effective in addressing the identified issues. This involves continuously engaging with the auditee, assessing the progress, and providing any necessary guidance or clarification on the recommendations. Next, we will evaluate the effectiveness of implemented changes or remedial actions. Here, we will explore methods to assess whether the changes made have achieved their intended objectives to ascertain the value and impact of the audit. It also involves revisiting the original audit objectives and criteria.

Lastly, we will explore the criteria and processes for formally closing audit findings, highlighting the importance of documenting lessons learned for the audited entity and the audit team. This reflection and documentation are essential for continuous improvement in future audits.

Establishing Follow-up Procedures

IS Auditing standards require IS auditors to monitor and periodically report to the Board of Directors and the audit committee on management’s progress on the IS audit findings and recommendations. The reporting is expected to include a conclusion on whether management has planned and taken appropriate, timely action to address reported audit findings and recommendations. Follow-up activity is a process through which the IS Auditors determine the adequacy, effectiveness and timeliness of actions taken by management on reported observations and recommendations, including those made by external auditors and others.

Effective follow-up procedures serve as the bridge between audit recommendations and their implementation. The first step in establishing these procedures is to define clear objectives that align with the audit’s initial goals. These objectives guide the entire follow-up process, ensuring it remains focused and effective. Once objectives are set, developing a detailed follow-up plan is essential, which should articulate the process and timeline and assign responsibilities. This plan acts as a roadmap, providing structure and consistency to the follow-up efforts. Setting specific, measurable, and relevant criteria for evaluating the implementation of recommendations is another crucial step, as they offer a solid foundation for assessing the progress and effectiveness of the implemented changes.

Assigning clear responsibility for the follow-up activities, whether to an individual or a team, is vital for ensuring accountability. This assignment is complemented by establishing a realistic and feasible timeline, aiding in tracking progress and maintaining momentum. Alongside this, creating a system for monitoring the implementation of recommendations is beneficial by providing a centralized view of progress and streamlining the monitoring process. Transparency and clarity are enhanced by communicating the follow-up procedures to all relevant stakeholders to ensure that everyone is on the same page and foster cooperation and support. Given the dynamic nature of technology and business operations, regularly reviewing and updating the follow-up plan allows the procedures to adapt and remain effective under varying circumstances.

Incorporating a process for escalating unresolved issues helps ensure critical risks and challenges receive the necessary attention. Planning for interim check-ins or milestones within the follow-up timeline is also strategic, offering opportunities to assess progress and identify issues early on. Similarly, developing a robust reporting mechanism is essential for maintaining transparency and keeping all stakeholders informed about the status of recommendation implementations. Flexibility in the follow-up procedures is critical, allowing adaptation to different audits and findings. It’s also important to consider external factors, such as technological changes, regulations, or business operations, which might impact follow-up activities.

Lastly, documenting all follow-up activities is vital for maintaining a transparent, accountable record that can be referenced in the future. Training the staff involved in these activities ensures they have the necessary skills and knowledge. Feedback mechanisms within the follow-up procedures can contribute significantly to continuous improvement. This feedback can be sourced from auditors, auditees, or other stakeholders, offering diverse perspectives on the process.

Monitoring Implementation of Recommendations

Monitoring the implementation of recommendations is aimed at verifying whether the suggested changes are effectively executed. It begins with establishing a clear and structured approach for tracking these implementations by creating a detailed monitoring plan outlining specific steps, timelines, and responsibilities associated with each recommendation. This plan guides, ensuring that every suggestion is tracked meticulously. Assigning responsibility for monitoring to particular individuals or teams is crucial in ensuring that someone is accountable for regularly checking the progress of each implementation. Similarly, setting up a systematic tracking system involving a combination of software tools and manual checklists allows for recording progress against predefined benchmarks. IS Auditors should schedule regular status updates and be involved in reviewing the progress made on each recommendation.

The monitoring process should also include interim evaluations of the implemented changes to help understand whether the changes are on track to meet the audit’s objectives. They also provide an opportunity to make necessary adjustments in response to unforeseen challenges or changes in the organization’s environment. Addressing implementation challenges promptly might include resource constraints, resistance to change, or technical difficulties. A proactive approach to solving these problems ensures that the overall progress is maintained. This can be achieved through several ways, including collaboration between the audit team and the organization’s staff to facilitate a better understanding of the practical aspects of implementation. Such partnership also helps foster a positive attitude towards the changes the audit recommends.

The monitoring process should be flexible and adaptable to the context of each recommendation since different recommendations might require different approaches and levels of scrutiny. Such flexibility ensures that the monitoring process is effective and efficient. Moreover, the monitoring process should also include a mechanism for escalating issues that need to be addressed adequately to ensure that significant issues are brought to the attention of higher management and dealt with appropriately. IS Auditing standards require that when the risk related to a finding has been accepted and is greater than the enterprise’s risk appetite, this risk acceptance should be discussed with senior management. The acceptance of the risk (particularly failure to resolve the risk) should be immediately brought to the attention of the Board of Directors and the Audit Committee. Beyond such escalation, periodic reporting to the audit committee on the status of implementations is crucial. It provides an overview of progress, highlights any areas of concern, and ensures that the Audit Committee is aware of the audit’s impact and the status of its recommendations.

From a value-added perspective, incorporating feedback from the staff involved in implementing the recommendations can provide valuable information on the implementation process’s practical aspects and lead to improvements in future audits. Similarly, considering the need for additional training or support for staff involved in the implementation can address skill gaps and ensure that staff are well-equipped to implement the recommendations effectively. Finally, aligning the monitoring process with the organization’s strategic goals and audit objectives is crucial. This alignment ensures that the monitoring efforts contribute to improving the organization’s information systems and controls.

Evaluating the Effectiveness of Implemented Changes

Evaluating the effectiveness of implemented changes involves assessing whether the changes made post-audit have met the intended objectives and contributed to enhancing the organization’s information systems. Some of the commonly performed procedures in such evaluation include:

• The recording of a time frame within which management should respond to agreed-on recommendations.
• An evaluation of management’s response through follow-up work using a methodology similar to the standard IS audit fieldwork.
• A communication procedure that escalates outstanding and unsatisfactory responses and actions to the appropriate management levels and those charged with governance.
• A process for obtaining management’s assumption of associated risk if corrective action is delayed or not proposed to be implemented.

The evaluation begins with defining clear criteria against which the changes will be measured. These criteria should be directly linked to the objectives outlined in the original audit report and provide a basis for a systematic and objective evaluation. Using qualitative and quantitative measures for a comprehensive assessment is essential. Quantitative data might include metrics like system downtime reduction or improved transaction processing speed, while qualitative measures could encompass user satisfaction or improved process understanding.

This is followed by developing a structured evaluation plan outlining the methods and tools to be used in the assessment, the timeline for the evaluation, and the individuals responsible for conducting it. Using a combination of techniques, such as surveys, interviews, system performance analysis, and documentation review, can provide a well-rounded view of the impact of the changes. Engaging with different stakeholders (users, IT staff, and management) can provide insights into the practical effectiveness of the changes and help identify any unintended consequences or areas that need further improvement. Similarly, comparing the post-implementation state with the pre-implementation state helps determine the extent of progress and whether the changes have successfully addressed the issues identified in the audit. It is also beneficial to benchmark the organization’s performance against industry standards or similar organizations, as it provides an external perspective on the effectiveness of the changes.

Finally, sharing the evaluation findings with all relevant stakeholders is essential to ensure that everyone is aware of the outcomes and can contribute to any necessary further actions. Through practical evaluation, organizations can ensure that the changes made following an IS audit are genuinely beneficial and contribute to enhancing their information systems and processes.

Reporting on Follow-up Activities

Reporting on follow-up activities involves communicating the progress and outcomes of the follow-up actions taken after the initial IS audit, as it ensures transparency and accountability and provides valuable insights for all stakeholders. Such reporting should provide a comprehensive overview of the follow-up activities, including a summary of the original audit findings, recommendations, and a detailed account of the actions taken in response to each recommendation. It is essential to present this information in a structured and logical way, making the report easy to follow and understand. Incorporating both quantitative and qualitative data is necessary for a balanced report. Quantitative data might include metrics such as the number of recommendations fully implemented, while qualitative data could cover the overall impact of the changes on the organization’s operations.

The report should also assess the progress against the planned timeline and objectives, highlighting deviations from the plan to help stakeholders understand the status of the follow-up activities and whether any adjustments are needed. It is crucial to maintain a factual and objective tone throughout the report. Avoiding bias and ensuring the report is based on evidence and data enhances its credibility and builds stakeholder trust. This includes discussing challenges encountered during the follow-up process, their reasons, and the steps taken to address them. It demonstrates the organization’s commitment to addressing issues and improving its systems and processes.

The report’s distribution strategy should be carefully planned to ensure that the report reaches all relevant stakeholders promptly and efficiently. Depending on the audience and the report’s content, this may involve using different communication channels, such as email, presentations, or physical meetings. Regular reporting is often beneficial, especially for more extended follow-up periods. Periodic updates can keep stakeholders informed and engaged throughout the process and provide opportunities to make course corrections.

Maintaining a repository of all follow-up reports for a historical record of the organization’s IS audit and follow-up activities is also essential. It can be a valuable resource for future IS audits and for understanding the evolution of the organization’s IS and controls. Finally, soliciting feedback on the report from its audience can provide insights for improving future reports, help understand the report’s effectiveness, and facilitate identifying areas where the reporting process can be enhanced.

Closing Audit Findings and Lessons Learned

Closing audit findings and extracting lessons learned solidifies the value gained from the audit and ensures continuous improvement in managing information systems. In cases where findings cannot be closed, it’s necessary to understand and document the reasons. These might include resource constraints, technical challenges, or strategic decisions. Understanding why specific findings remain open provides valuable insights into the organization’s risk management and decision-making processes.

Once findings are closed, the closure report should summarize the actions taken, the outcomes of these actions, and the status of each audit finding. The closure report serves as a formal record of the completion of the audit process. The process of extracting lessons learned is equally essential. It involves reflecting on the entire audit process, from planning to follow-up, and identifying what worked well and could be improved. Engaging a broad range of stakeholders in this reflection process can provide diverse perspectives and a more comprehensive understanding of the lessons learned.

Documenting these lessons learned should be performed in a way that is accessible and useful for future audits. It might take the form of lessons-learned databases, reports, or updates to audit guidelines and procedures to ensure that the knowledge gained from one audit contributes to the success of future audits. The lessons learned should cover various aspects of the audit process, including the effectiveness of audit planning, the adequacy of audit procedures, the response to audit findings, and the follow-up process. It should also consider the communication and reporting aspects, as well as the overall impact of the audit on the organization’s information systems. Sharing the lessons learned with relevant stakeholders promotes a culture of continuous improvement and learning within the organization and ensures that the insights gained are applied to enhance future audits and information systems management.

It is also beneficial to consider the broader implications of the lessons learned as they might provide insights into emerging risks, industry trends, or areas where the organization needs to develop additional competencies. These broader insights can inform strategic planning and decision-making. In some cases, the lessons learned also highlight the need for additional training or development for the audit team or other staff. Addressing these needs can strengthen the organization’s audit capabilities and overall information systems management. Finally, closing audit findings and extracting lessons learned should be considered an integral part of the organization’s governance and risk management framework as it contributes to the organization’s ability to manage risks effectively and continuously improve its information systems and controls.