The AFM’s Sector in Beeld 2023 (Accountancy en Verslaggeving, November 2023) reported continued quality deficiencies across statutory audits, including monitoring deficiencies at smaller firms. Not because partners weren’t reviewing work. Because the monitoring itself had no measurable indicators attached to it. Firms had quality policies on paper and no way to tell whether those policies were actually working. That gap between documented procedures and evidence of their effectiveness is where most ISQM 1 findings originate.
Audit quality indicators (AQIs) are quantifiable metrics that a firm tracks over time to evaluate whether its system of quality management is operating as designed, as required by ISQM 1.30-31. A properly constructed AQI framework selects indicators linked to specific quality risks, sets thresholds for investigation, assigns monitoring responsibility, and feeds results back into the firm’s risk assessment at least annually.
Key takeaways
- How to select AQIs that connect directly to your firm’s identified quality risks under ISQM 1.30-31
- How to set meaningful thresholds that trigger investigation rather than generating noise
- How to build a monitoring process that satisfies ISQM 1.48-49 without consuming your entire quality function
- How a mid-tier Dutch firm implemented a 12-indicator dashboard from scratch
What ISQM 1 actually requires for monitoring
ISQM 1.30 requires the firm to establish a monitoring and remediation process. That process has to include monitoring activities that provide information about the design and operation of the system of quality management: information that is relevant, reliable, complete, and timely enough to act on. Paragraph 31 goes further. The firm must design and perform monitoring activities, evaluate findings, identify deficiencies, and evaluate both the severity and pervasiveness of those deficiencies.
The standard doesn’t prescribe specific indicators. It doesn’t say “track partner-to-staff ratios” or “measure realization rates.” What it requires is that your monitoring produces evidence that the system works. AQIs are the mechanism most firms use to produce that evidence, and they’re what inspectors look for when they test your monitoring process.
The IAASB’s implementation guide for ISQM 1 distinguishes between inspection of completed engagements and ongoing monitoring of the system as a whole. AQIs sit in the second category. They don’t replace file inspections under ISQM 1.37. They supplement them with continuous data points that can surface problems between inspection cycles. Think of file inspections as the annual health check. AQIs are the blood pressure readings you take every quarter.
The distinction between inspection and monitoring matters for how you resource the quality function. File inspections require qualified reviewers to spend days in completed files. AQI monitoring requires someone to collect data, compare it to thresholds, document the results, and escalate when something is off. The second task is less skilled, more automatable, less time-intensive, and should happen more frequently. Most firms that struggle with monitoring are trying to do both tasks with the same people on the same annual schedule.
Why most existing monitoring fails inspection
Most mid-tier firms already track some form of quality data. Completion rates. Review turnaround times. Staff utilization. Training hours.
The problem isn’t that firms have no data. The problem is that the data they track wasn’t selected because it connects to a quality risk. It was selected because it was easy to measure or because a Big 4 firm published a list and it got copied without adaptation. ISQM 1.30 requires the connection to be deliberate: each indicator should trace back to a specific quality risk identified in your risk assessment under ISQM 1.23-24.
The AFM’s inspection approach tests this chain directly. Inspectors pull the firm’s risk assessment, then pull the monitoring plan, then check whether thresholds are defined, then ask: which indicator monitors which risk? If the answer is a generic list of KPIs with no mapping to the risk assessment, the monitoring process fails regardless of how much data the firm collects. The data may be excellent. It just can’t demonstrate that it serves the purpose ISQM 1.30 requires.
A second common failure is firms that track indicators but set no thresholds. They produce annual reports that say “the system operated effectively” based on a qualitative narrative with no defined tolerance levels and no documented basis for the conclusion. ISQM 1.A77 expects the firm to use professional judgment in setting tolerance levels, and an annual narrative with no quantitative criteria doesn’t demonstrate that judgment was applied.
Both failures share the same root cause. The firm treated ISQM 1 monitoring as a documentation exercise rather than an operational process. Documentation exercises produce binders. Operational processes produce decisions.
Selecting indicators that connect to quality risks
The selection process starts with your risk assessment, not with a list of popular AQIs. ISQM 1.23 requires firms to identify quality risks that have a reasonable possibility of occurring and, if they occur, a reasonable possibility of individually or in combination adversely affecting the achievement of quality objectives. Your AQIs should map directly to those quality risks.
Here’s what that looks like in practice. Suppose your firm’s risk assessment identified “insufficient engagement partner involvement during planning” as a quality risk under the governance and leadership component. A useful AQI for this risk might be the percentage of engagement files where the partner signed the planning memorandum before fieldwork began. If that number drops below 80%, something in your system isn’t working.
Contrast that with tracking total billable hours per partner. That’s a useful management metric. It tells you nothing about whether partners are involved at the right stages of the engagement. The metric measures activity. What the AQI should measure is the quality behaviour you’re trying to ensure.
The distinction matters because inspectors (the AFM in particular) now test the logic chain from risk assessment to monitoring indicator to remediation action. If they can’t trace your AQIs back to specific quality risks, the monitoring process fails regardless of how much data you collect.
A practical starting point: print your firm’s risk assessment. Next to each quality risk, write down what would change in a measurable way if that risk materialised. That measurable change is your candidate indicator. If you can’t identify a measurable change, either the risk is too abstract (rewrite it more specifically) or you need a different type of monitoring activity for that particular risk, perhaps a targeted file inspection rather than an ongoing AQI. Not every risk needs an AQI. Some are better addressed through periodic deep reviews.
The four categories of useful indicators
Most firms need a mix of indicators across four categories. No single category covers all quality risks, and relying on only one type leaves gaps in your monitoring.
Engagement-level indicators measure what happens on individual files. The percentage of files where the audit materiality calculation was revised at completion per ISA 320.12 is a common one. Others include the number of files with open review points at signing, the percentage of files where the going concern assessment was completed before the audit report date, and the average number of days between fieldwork completion and partner sign-off. These indicators are the most direct measure of audit quality but can only be collected after engagements close or at defined milestones during the engagement.
Firm-level resource indicators measure whether the firm has the capacity to deliver quality work. Average hours of technical training per qualified staff member per year. Staff turnover rate among seniors with more than two years’ experience. The number of engagements per partner during peak season. These indicators are leading rather than lagging. A spike in turnover predicts future file quality problems before those problems appear in individual engagements.
Resource indicators deserve special attention at smaller firms, where the departure of a single experienced senior can degrade file quality across an entire portfolio within one busy season.
Compliance indicators measure adherence to firm policies and professional standards. Examples include the percentage of engagement quality reviews completed where required under ISQM 1.34(f), independence breaches reported and remediated within 30 days, the percentage of engagements where the risk assessment was documented before fieldwork commenced, and the number of overdue client confirmation requests at the end of fieldwork. These tend to be the easiest indicators to automate because most engagement management systems already capture the relevant dates and sign-offs.
Outcome indicators measure the results of the audit process after the fact. The number of material misstatements identified after the audit report was issued, the rate of restatements for audited financial statements, client complaints relating to audit quality, and the number of inspection findings per engagement all fall here. Outcome indicators are the most powerful evidence of whether the system works, but they’re lagging by definition. By the time a restatement appears, the quality failure happened months or years earlier.
If your entire dashboard is engagement-level metrics with no resource indicators, you’re blind to the structural problems (understaffing, inadequate training) that cause engagement-level failures. If you only have outcome indicators, you’ll catch problems too late to prevent them. The balance matters more than the total count.
Setting thresholds that mean something
An indicator without a threshold is just a number on a page. ISQM 1.31(d) requires the firm to evaluate findings from monitoring activities and determine whether deficiencies exist. You can’t evaluate a finding if you haven’t defined what “acceptable” looks like.
Thresholds come in two forms. Investigation triggers are the levels at which the firm must look into what’s happening. Remediation triggers are the levels at which the firm must take corrective action. Not every breach of an investigation trigger leads to remediation. Sometimes the investigation reveals a reasonable explanation (a cluster of partner transitions, for instance, or a data collection error). But every remediation trigger breach must produce a documented response under ISQM 1.42.
Setting thresholds is where most firms go wrong in one of two directions. Over-engineering looks like 47 indicators with tolerances defined to one decimal place and a 30-page quarterly report that nobody reads. Under-engineering looks like tracking indicators with no defined thresholds at all, reviewing them once a year, and writing “results are satisfactory” without stating what satisfactory means.
The practical approach: set thresholds based on your own historical data where available. Where your data is thin, use regulatory expectations as a proxy. If the AFM’s 2023 report noted that partner involvement at planning was deficient in 35% of inspected files (AFM, Sector in Beeld 2023, Accountancy en Verslaggeving, November 2023), and your firm’s internal data shows partner sign-off before fieldwork in 90% of cases, then 85% is a reasonable investigation trigger and 75% is a reasonable remediation trigger. Document why you chose those numbers. The documentation of that judgment is what gets tested at inspection. Inspectors aren’t looking for mathematical precision. They’re looking for evidence that someone thought about what “good” looks like and wrote it down before the results came in.
For new indicators with no historical baseline, set an initial threshold based on professional judgment, measure for two reporting periods, then recalibrate. State this approach explicitly in your monitoring plan. Inspectors accept “we’re establishing a baseline” in year one. They don’t accept it in year four.
One more practical point: don’t set thresholds so tight that every quarter triggers an investigation. If your investigation trigger fires every period, the indicator is either miscalibrated or the underlying process is genuinely broken. Constant alerts produce alert fatigue, and eventually no one investigates at all. Set triggers that fire occasionally but meaningfully. Two to four investigations per year across your full dashboard is a reasonable expectation for a firm with 10-15 indicators.
Worked example: Van Leeuwen Accountants builds an AQI dashboard
Scenario: Van Leeuwen Accountants is a 14-partner firm in Utrecht with 85 audit clients (annual revenues ranging from €8M to €120M). The firm completed its ISQM 1 risk assessment in 2023 and identified 11 quality risks across four components. The quality management partner, Anneke de Groot, needs to build an AQI dashboard that connects to those risks and satisfies the AFM’s expectations for monitoring evidence.
Map each quality risk to a measurable indicator
Anneke maps each of the 11 quality risks to at least one measurable indicator. For the risk “insufficient partner involvement at planning,” she selects “percentage of files with partner-signed planning memo before fieldwork start date.”
Documentation note: Record the mapping in a matrix. Column A: quality risk ID from the risk assessment. Column B: AQI description. Column C: data source. Column D: measurement frequency. Column E: investigation trigger. Column F: remediation trigger.
Set thresholds from historical baseline
She sets thresholds using the firm’s 2023 internal inspection data as a baseline. The 2023 data shows partner sign-off before fieldwork in 88% of files. She sets the investigation trigger at 82% and the remediation trigger at 70%.
Documentation note: Record the baseline data, the threshold chosen, the rationale, and the data source in the monitoring plan. “2023 baseline: 88%. Investigation trigger set at 82% to allow for normal variation while catching systematic decline. Remediation trigger set at 70% based on the AFM’s observed deficiency rate at Tier 2 firms.”
Assign data collection and automate where possible
She assigns data collection responsibility. The engagement management system already captures partner sign-off dates and fieldwork start dates. She asks the IT manager to create a quarterly report that calculates the percentage automatically rather than relying on manual file reviews.
Documentation note: Record who collects the data, how it’s collected, and how often. Automated collection is preferred where possible because manual collection creates its own quality risk.
Investigate when a trigger fires and diagnose root cause
After the first quarter of 2024, the report shows 79%. Below the investigation trigger but above the remediation trigger. Anneke investigates and discovers that four files involved partner transitions mid-engagement. The incoming partners signed the planning memo after fieldwork had started because they were appointed late.
Documentation note: Record the investigation, its findings, the root cause identified, and whether remediation is needed. Anneke concludes that the root cause is the partner transition process, not partner disengagement. She recommends a policy change: when a partner transition occurs mid-engagement, the incoming partner must review and re-sign the planning memo within five working days of appointment, regardless of fieldwork status.
Close the loop with a documented remediation action
She presents the Q1 results and the policy recommendation to the firm’s quality management team. The policy change is approved and added to the firm’s procedures manual.
Documentation note: Record the remediation action, the approval, and the follow-up timeline. ISQM 1.42 requires the firm to evaluate whether remedial actions are appropriately designed and to determine that they were implemented. The Q2 report will measure whether the new policy resolves the decline.
Practical checklist for your firm
- Pull your ISQM 1 risk assessment and list every quality risk your firm identified. If any risk has no measurable indicator attached to it, that’s the first gap to close.
- For each indicator, confirm the data source exists and can produce results at least quarterly. If data collection requires more than 30 minutes per indicator per quarter, automate it or pick a simpler indicator.
- Set investigation and remediation triggers for every indicator, documented with rationale. Use your own historical data as a baseline where available. Where you have no baseline, state that you’re establishing one and set a provisional threshold.
- Assign a named person (not a committee) responsible for collecting each indicator’s data and escalating breaches to the quality management partner (ISQM 1.20(b)).
- Build a one-page quarterly AQI summary that the quality management partner can review in under 15 minutes. If the dashboard takes longer than that, you have too many indicators or too little automation.
- After two complete reporting cycles, review whether each indicator actually detected a problem or provided useful trend data. Drop indicators that generated zero signal and replace them with indicators that address your current risk profile.
Common mistakes
- Firms copy Big 4 AQI lists without mapping indicators to their own quality risks. The AFM’s 2023 inspection report specifically noted that monitoring programs at smaller firms often lacked a clear connection between the indicators tracked and the risks identified in the firm’s own assessment.
- Thresholds exist on paper but never trigger action. A 2022 FRC thematic review of audit quality monitoring (FRC, Audit Quality Inspection and Supervision: Results for 2021/22, July 2022) found that firms frequently tracked indicators below threshold levels for multiple quarters without initiating investigation, undermining the entire monitoring framework.
- Monitoring reports conclude that “the system operated effectively” without presenting the underlying data. ISQM 1.48 requires the firm to communicate monitoring results to the engagement partner and other appropriate personnel. A conclusion without supporting data doesn’t satisfy the communication requirement under paragraph 49.
- Collecting data annually instead of quarterly. An annual data point tells you the system failed at some point during the year. Quarterly data tells you when and lets you intervene before the problem compounds across an entire busy season. ISQM 1.31 doesn’t specify frequency, but the requirement for “timely” information under ISQM 1.30(a)(ii) is difficult to meet with annual collection.
Related content
- Quality management glossary entry covers the full ISQM 1 framework and how AQI monitoring fits within the broader system of quality management.
- The audit materiality calculator is one of the engagement-level data sources most commonly tracked as an AQI (percentage of files with materiality revised at completion).
- ISQM 1 quality management guide covers the full ISQM 1 implementation process, including how to structure the risk assessment that feeds this AQI framework.
Related tools and reading
Put audit concepts into practice with these free tools:
Related guides:
Frequently asked questions
What does ISQM 1 require for monitoring audit quality?
ISQM 1.30 requires the firm to establish a monitoring and remediation process that provides information about the design and operation of the system of quality management. Paragraph 31 requires the firm to design and perform monitoring activities, evaluate findings, identify deficiencies, and evaluate their severity and pervasiveness. AQIs are the mechanism most firms use to produce the required evidence.
Why do most existing audit quality monitoring programs fail inspection?
Most firms track data that wasn’t selected because it connects to a quality risk. The AFM tests the chain from risk assessment to monitoring indicator to remediation action. If the answer to “which indicator monitors which risk?” is a generic list of KPIs with no mapping to the risk assessment, the monitoring process fails regardless of how much data the firm collects.
What are the four categories of useful audit quality indicators?
The four categories are: engagement-level indicators measuring what happens on individual files, firm-level resource indicators measuring capacity to deliver quality work, compliance indicators measuring adherence to firm policies and standards, and outcome indicators measuring results of the audit process after the fact. A balanced dashboard needs indicators from all four categories.
How should a firm set AQI thresholds?
Set thresholds based on your own historical data where available. Where data is thin, use regulatory expectations as a proxy. Define two levels: investigation triggers and remediation triggers. Document the rationale for chosen thresholds. For new indicators, set an initial threshold based on professional judgment and recalibrate after two reporting periods.
How many audit quality indicators should a firm track?
Most firms need 10–15 indicators across the four categories. The key is that each indicator traces back to a specific quality risk from your ISQM 1 risk assessment. Over-engineering is as problematic as under-engineering. Aim for two to four triggered investigations per year across the full dashboard.
Source references
- ISQM 1, Quality Management for Firms that Perform Audits or Reviews of Financial Statements (effective 15 December 2022)
- IAASB, Implementation Guide for ISQM 1
- AFM, 2023 Inspection Cycle Results
- FRC, Thematic Review of Audit Quality Monitoring, 2022