Measuring Effectiveness and Efficiency of Internal Audit
At this year's IIA International Conference, I received an audience question about how to measure effectiveness of internal audit while leveraging agile ways of working. This is a really important topic, regardless of your ways of working. As the saying goes "what gets measured gets done". Because of that, we need to be careful about the behaviors we incentivize through our performance measures.
With the few minutes we had left before the next speakers took the stage, I provided a brief answer. Without the time limitations of a brief Q&A session at the end of a breakout session, let's dive deeper into this and explore potential internal audit performance measures.
How are Effectiveness and Efficiency Measured Today?
Performance of internal audit is often measured in terms of outputs. While outputs can be a helpful data point, internal audit deals in value, not widgets, so outputs alone usually aren't a great way to measure performance.
The Institute of Internal Auditors (IIA) issued guidance in December 2010 on measuring the effectiveness and efficiency of internal audit. This guidance suggests performance measures such as:
- Level of contribution to the improvement of risk management, control, and governance processes
- Achievement of key goals and objectives
- Evaluation of progress against audit activity plan
- Improvement in staff productivity
- Increase in efficiency of the audit process
- Increase in the number of action plans for process improvements
- Adequacy of engagement planning and supervision
- Effectiveness in meeting stakeholders' needs
- Results of quality assurance assessments and internal audit activity's quality improvement programs
- Effectiveness in conducting the audit
- Clarity of communications with the audit client...and the board
The guidance goes on to provide example metrics organizations may use to measure the effectiveness and efficiency of internal audit. Many of these examples are output-focused, such as:
- Number of audits scheduled vs number of audits completed
- Number of significant audit findings
- Number of unsatisfactory internal audit opinions
- Audit report cycle time elapsed (defined as "elapsed time from opening conference to fieldwork completion and elapse[d] time from fieldwork completion to final report")
Let's explore why these alone might not be the best measures of internal audit efficiency or effectiveness.
Number of audit scheduled vs number of audits completed
By comparing scheduled and completed audits, stakeholders can hold internal audit accountable to their commitments. It can also help identify areas of inefficiency or development opportunities. For example, if internal audit is unable to complete the audit plan, it may uncover the need for additional training, staff development, or a different mix of auditor skills to be effective and efficient.
This metric may also drive unwanted behavior. If measured primarily on completing the plan, auditors may rush to complete audits simply to meet the performance metric at the expense of quality. This metric is also often used with a static, annual audit plan, which isn't as effective in today's dynamic business environment.
Number of significant audit findings
Measuring performance based on the number of significant findings gives audit clients the impression (either real or perceived) that auditors are incentivized to find gaps and that they get paid by the finding. It literally reinforces that bad stereotype that auditor performance, which is often tied to pay, is based on the number of findings raised. Besides, what is the right number to target for this metric...5? 50? 500? This outdated metric needs to go.
Number of unsatisfactory internal audit opinions
Similar to the last metric, measuring performance based on the number of unsatisfactory internal audit opinions is a bad measure of internal audit performance. Not only does it incentivize auditors to issue unsatisfactory opinions and encourages people to think of auditors as out to get you, an unsatisfactory internal audit opinion also says more about the business's control environment than it does about internal audit's performance.
Audit report cycle time
The IIA's guidance defines this metric as:
"elapsed time from opening conference to fieldwork completion and elapse[d] time from fieldwork completion to final report"
This metric has some promise. The intent of it is to incentivize internal audit to get results into their clients' hands sooner, so audit clients can take any necessary action sooner (and better-manage overall risk to the organization). This metric, as is, encourages the traditional waterfall approach to auditing and misses the opportunity to deliver results to audit clients during planning or fieldwork (or at least misses the opportunity to encourage auditors to deliver results to their clients during planning and fieldwork). Although inherently flawed, this metric is headed in the right direction... especially compared to the previous metrics.
A Shift to Outcomes and Value
The IIA's guidance also includes examples of performance measures more closely aligned with outcomes and organizational value. Examples from the guidance include:
- Number of management requests
- Responsiveness to management requests
- Amount of identified cost savings and percent of recoveries
These metrics are a solid start in moving from output-focused measures that could drive unwanted behaviors (or give the wrong impression to our clients) to valuable, outcome-focused performance measures. Let's double-click into each of them.
Number of management requests
At a recent speaking engagement, I asked the audience how their clients react when they find out the auditors are going to do work in their space. Many audience members responded with words like "fear", "anxiety", and even "run and hide". When we as auditors are truly adding value to our organizations, and our clients see the value we're adding, we can influence a completely different response from our clients: requests for audit work. When our clients proactively request our help, we can feel confident that they see the value we bring to the organization. By changing our ways of working and adding more value, while decreasing the pain inflicted on our clients (unplanned work, surprise findings, etc.), we become the trusted partner and sought-out advisor of our organization. Measuring internal audit performance based on the number of management requests (as one of many data points), we're incentivizing healthy behaviors that drive greater value to the organization.
Responsiveness to management requests
Not only is it valuable to receive requests from audit clients, but it is also important for auditors to be responsive to those requests. That doesn't mean that every request is accepted and acted upon. It does mean that when a client requests internal audit's assistance, the client gets a timely response. That response can indeed be "yes we're going to prioritize this and get started on it right away", or it may be something like "I really appreciate you reaching out and requesting this work, but we're in the middle of some very high-priority work at the moment, and it doesn't make sense to deprioritize our current work to accommodate this just now. Let's reconnect on this in a few weeks and discuss potential timing." If the work isn't aligned with the scope of internal audit's work and is better aligned with an existing second line function, the auditors' response could be a redirection to the appropriate key contact in the second line, and an introduction between the two parties.
A response to a client's request may not always result in a full-blown audit engagement. And that's okay too! It could be as simple as a quick follow-up call to answer questions or having auditors sit in on a few of the client's working sessions to provide real-time feedback and advice. Regardless of the response auditors provide, it needs to be timely and genuine. This encourages more management/client requests in the future, further strengthens the relationships between auditors and clients, and increases the value added to the organization.
Amount of identified cost savings and percent of recoveries
While this metric may be difficult to quantify in many instances, it can drive behaviors like identifying opportunities to reduce waste and increase efficiencies. Instead of demanding additional controls in a well-controlled environment (like what you might encounter when the auditors' performance is based on the number of significant findings raised), you start to experience auditors looking for opportunities to identify instances of duplicative controls or over-controlled environments and recommend more efficient ways of managing the risk. When you combine the identified cost savings and percent of recoveries as a performance metric, it measures not only the identified opportunities, but those the organization has taken action on and actually realized benefits, going from theory to actual cost savings. This also encourages auditors and clients to work more closely together through the recovery exercise, which further strengthens the relationship between the two parties and adds even more value to the organization, creating more positive momentum along the way.
Other Measures of Success
In addition to the IIA's guidance, I asked my network how they measure internal audit's effectiveness. My network came through for me with an overwhelmingly helpful response. They suggested the following measures of success:
- Feedback from management and the Board on helping the organization be efficient, focusing on assurance over more significant risks, helping management sleep at night, doing work management would pay for, and contributing to the organization's success (suggested by Norman Marks [1])
- Stakeholder feedback on whether internal audit is covering the right things
- Assurance maps demonstrating alignment between audit coverage and key stakeholders' view of strategic risk2
- Timeliness of insights measured by "[q]uality and timing of insights, improved collaboration, improved customer feedback, [and] reduced time from issue identification to remediation"2
- Cost Savings1
- Number of audits requested by management2
- Net Promoter score3
- Human touchpoints4
- Control coverage4
Another overwhelming piece of advice given was that each of these are data points. One data point alone is typically not enough to tell the whole story. When determining how to measure success (e.g., effectiveness and efficiency) of internal audit, first determine your desired outcome, and then determine how best to measure progress toward those outcomes.
This is a pretty hearty list of ideas on how to measure success, effectiveness, and efficiency of internal audit, using outcomes rather than outputs. Personally, I’m looking forward to experimenting with these and figuring out which combination works best for me. I'm also looking forward to learning about other potential performance measures.
What metrics have you found success with? And which ones are you going to start using going forward?
In Beyond Agile Auditing: Three Core Components to Revolutionize Your Internal Audit Practices, readers gain additional insight into why a few more outdated metrics are no longer effective measures of audit effectiveness or efficiency. The book further explores how to shift from an output-focused mindset to an outcomes-focused operating model. Order your copy today to learn more!
1 - https://normanmarks.wordpress.com/
2 - https://www.auditboard.com/blog/modern-metrics-for-high-value-internal-audit/
4 - https://itrevolution.com/articles/governance-engineering/