TMCnet News
Measuring and Communicating Audit Quality: The New AQIs [CPA Journal, The](CPA Journal, The Via Acquire Media NewsEdge) Audit quality has been described as the raison d'être of the audit profession, for if there is no public trust in the quality of the audit, it is of no value (S. Maijoor and A. Vanstraelen, "Research Opportunities in Auditing in the EU, Revisited," Auditing: A Journal of Practice & Theory, vol. 31, no. 1, 2012, pp. 115-126). Yet, in spite of decades of research, there is little agreement on how to define audit quality, let alone how to measure it (M. L. Murphy, "Improving Audit Quality: An Interview with Cynthia M. Fomelli, Executive Director of the Center for Audit Quality," The CPA Journal, February 2014, pp. 24-31). Some suggest audit quality is a function of the auditor's ability to discover and report misstatements; others define it as the auditor's ability to meet legal and professional requirements (e.g., following generally accepted auditing standards and firm quality control standards); still others consider it the ability of the auditor to meet the needs of investors [see L.E. DeAngelo, "Auditor Size and Audit Quality," Journal of Accounting and Economics, vol. 3, no. 3, December 1981, pp. 183-199; J.R. Francis, "What Do We Know about Audit Quality?," British Accounting Review, vol. 36, no. 4, December 2004, pp. 345-368; and Discussion of the Standing Advisory Group (SAG) of the PCAOB, May 15-16, 2013]. Most do agree that audits contribute to the efficient functioning of capital markets-therefore, enhancing audit quality is an important objective. Audit quality has historically been assessed ex post. That is, it has largely been measured using outcome measures, such as whether a company's financial statements require restatement, or, if a business fails, whether the auditor's opinion included a going concern modification. More recently, audit quality has also been measured based upon the number and nature of findings identified during PCAOB inspections. Unfortunately, in spite of the efforts of Congress to regulate audit quality (e.g., the Sarbanes-Oxley and DoddFrank Acts), the trends in these outcomebased measures have not been favorable. Restatements of public company financial statements have not declined (per the Audit Analytics Database, 2010 through 2013), nor have reported deficiencies of inspections conducted by the PCAOB (E. K. Davis and L. J. Moravy, "Update: a Brief Overview of Trends in PCAOB Inspection Reports. What Every Practitioner Should Know," Finance Scholars Group, Mar. 5, 2013). In part, as a result of continued deficiencies in inspection reports, the PCAOB initiated a project in 2013 to identify both ex post (output) and ex ante (input) indicators of audit quality. The PCAOB's objectives in identifying audit quality indicators (AQI) are threefold: * Better inform regulators about audit quality * Aid the decision making of audit committees, investors, and managers * Positively influence the practices of auditing firms. At least two other organizations have also added AQI projects to their agendas: the Center for Audit Quality (CAQ) and the PCAOB's Investor Advisory Group (IAG). Both the CAQ and IAG are in process of vetting their respective measures of audit quality, while the PCAOB expects to issue a concept release during 2014 to solicit input on its AQIs and how those AQIs might be used and communicated. It has been suggested that the identification of AQIs be required for audits of publicly traded companies-similar to, or in connection with, the requirements of PCAOB Auditing Standard 16, Communications with Audit Committees. As such, the AQIs would be available to audit committees for their annual evaluation of their external auditor, and would also be available to the PCAOB during its inspections of registered auditing firms. It is not unreasonable to expect that AQIs, some of which will be based on sensitive firmand engagement-specific data, will ultimately be made public, going beyond the purview of audit committees and standards setters. Although the debate around AQIs currently only directly impacts auditors of publicly traded companies, it is not unreasonable to expect a trickle-down effect. If the PCAOB adopts or modifies auditing standards to require the reporting of AQIs, it is possible that the AICPA will adopt similar reporting requirements for nonpublic entities. The identification of AQIs raises three questions. First, is it possible to actually predict audit quality based on inputs? Second, what is the cost of reporting AQIs, in terms of gathering and reporting the required data as well as the potential consequences for auditing firms and their clients? Third, will the anticipated benefits of reporting AQIs outweigh the costs? By providing a comparison of recent AQIrelated activities by the PCAOB, CAQ, and IAG, the authors hope to start answering these questions. Summary of Recent AQI Activities Since 2008, the European Union (EU) has required all auditors of EU public interest entities to annually disclose a great deal of information about their firm, including the items listed in Exhibit 1, most of which was previously regarded as confidential operating information, unavailable to the public (e.g., continuing education policies, basis of partner compensation). These reports are generally referred to as "transparency reports." Believing that market participants benefit from the transparency of auditing firm disclosures (such as those required by the EU), the U.S. Department of the Treasury's Advisory Committee on the Auditing Profession (ACAP) issued a report in 2008 advocating similar firm disclosures for auditors of U.S. publicly traded companies (http://www.treasury.gov/ about/organizational-structure/offices/ Documents/fmal-report.pdi). The report recommended that the PCAOB develop AQIs, that firms publish the AQIs, and that the PCAOB monitor the reported indicators. Much of the basis for the ACAP's AQI discussion and recommendations stemmed from the frameworks developed by the IAASB (http://www.ifac.org/sites/ default/files/meetings/files/20111205IAASB-Agenda_Item_6-F-Revised_ Draft_AQ_Fwk_Paper.pdf) and the U.K.'s Financial Reporting Council [https:// frc.org.uk/Our-Work/Publications/FRCBoard/The-Audit-Quality-Framework(l)-File.pdf). The ACAP suggested that AQIs should include both qualitative and quantitative measures, as well as inputbased (e.g., training) and output-based (e.g., PCAOB inspection findings) measures. The ACAP's recommendation was aimed at both enhancing the quality of audits and increasing the ability of smaller auditing firms to compete with larger firms on the basis of audit quality. In response, the SAG provided input later that year on how to best address the ACAP's recommendations (http://pcaobus .org/N ews/Events/Documents/10222008_ SAGMeeting/BP_Feasability_AQI.pdí). A key input was that AQIs should include both firmand engagement-level indicators. The SAG suggested that while indicators of audit firm quality are important, investors are more likely to be interested in the quality of an auditor's work at specific companies in which they have a vested interest, thus the need for engagementlevel AQIs. AQIs and the reporting of audit firm quality have since been on the meeting agendas of the IAG and the Technical Committee of the International Organization of Securities Commissions (IOSCO). In 2012, the SEC announced its own project to develop an Accounting Quality Model intended to detect ex post earnings management. The SEC's proposed model is based, in part, upon predicting earnings management based on discretionary, or abnormal, accruals (see http:// www.sec.gov/News/Speech/Detail/Speech/ 1365171491988#.Uvjp0mJdUz4). Academic studies frequently use "discretionary accruals" as a measure of audit quality. This measure considers whether management has altered year-end accruals with the intent of manipulating reported net income. Discretionary accruals are measured as the difference between a company's actual accruals (net income compared to cash provided by operations), and expected accruals, which are estimated based upon the change in sales and balance of property, plant and equipment. As announced in its 2013-2017 Strategic Plan, the PCAOB has begun vetting numerous possible measures of audit quality (http://pcaobus.org/About/Ops/ Documents/Strategic%20Plans/2013 - 2017.pdf). The value of any particular AQI is to be considered in light of its usefulness to audit committees and investors, potential for unintended consequences, availability of data, and its scalability across auditing firms (Greg Jonas, "Update on Quality Indicators," Dec. 10, 2013, http://pcaobus.org/News/Speech/Pages/ 12102013_Jonas_AICPA.aspx). The PCAOB's framework for defining the essential elements of audit quality is divided into two primary categories, inputand output-based indicators. Input-based indicators attempt to measure the auditor's ability to supply a quality audit. Input-based factors are divided into: operational inputs, which largely reflect personnel-driven factors; and process inputs, which largely reflect firm attributes and are intended to roughly align with five components of the Committee of Sponsoring Organizations (COSO) Internal Control Framework (i.e., control environment, risk assessment, control activities, information and communication, and monitoring). Proposed operational AQIs include a number of common measures, including the firm's ratio of partners to staff and training hours per audit professional. They also include several measures that might be considered more controversial, such as average compensation of partners and managers, "excessive" turnover and transfers of audit personnel, "fly-in" partners and managers involved in an audit, and technical competency testing of auditors. Proposed process AQIs also include several typical measures, such as the firm's number and frequency of communications on audit quality and investors' interests, as well as metrics pertaining to independence, testing, and compliance. Some process AQIs are less intuitive, such as compensation trends of prematurely rotated partners, credentials of new hires, and file firm's level of investment in supporting infrastructure (e.g., technology, risk management). Proposed output-based AQIs are the more commonly referenced results of audit quality, such as the form and accuracy of the audit opinion, reports on internal controls over financial reporting, results of PCAOB inspections, and the cost of litigation. Depending upon the source, the PCAOB is evaluating between 40 to 70 AQIs, which will be narrowed to a subset of 10 to 20 meaningful measures. The narrowing process is critical to avoid both an undue burden on the audit firm and information overload for AQI users. Various public and private organizations, and individuals, have started commenting on the PCAOB's AQIs and their means of dissemination. In October 2013, a report from the IAG's Working Group on AQIs stressed, among other things, the importance of AQIs to investors (http://pcaobus .org/N ews/Events/Documents/10162013_ IAGMeeting/AQI Report.pdf). The report stated that to be informative, AQIs should be capable of forecasting risks and problems (i.e., predicting audit quality). The IAG was somewhat critical of the fact that the PCAOB's AQIs relate more to an auditing firm's quality and processes than the financial statement audit quality of a particular client. The report also presented the IAG's own list of AQIs, divided into engagement team-level measures, firmlevel measures, and both. Because many of the IAG's AQIs are client-specific, they may be viewed as more controversial, or at least more burdensome in terms of record-keeping, than the AQIs under consideration by the PCAOB. For example, the IAG's engagement-level AQIs include hours spent by the partner addressing the engagement's identified key audit risks, engagement consultations with national technical experts and industry experts, and the name of the lead engagement partner (to assess participation in other audits whose credibility has been questioned). Examples of the IAG's firmlevel AQIs include compensation policies for partners and employees, average salary for new audit staff hires, and percentage of audits judged to be high risk. Beyond debating whether these influence audit quality, they may also be viewed as competitive information of the audit firm. AQIs to be reported at both the engagementand firm-level include violations of the independence mies, and hours of professional education of all personnel. The CAQ, while affiliated with the AICPA, is an autonomous public policy organization dedicated to enhancing investor confidence and public trust in global capital markets. The CAQ has formed an AQI Task Force to consider ways to provide capital market participants with an understanding of some key elements of audit quality, as well as to develop indicators that provide information about the performance of those elements over time at a profession-wide level (thecaq.org/docs/default-document-library/ caqauditqualitylettertopcaobmayl32013.pdf?). Most notably, the CAQ's AQIs differ from those of the PCAOB and IAG because they are both quantitative and descriptive, and because audit committees (as opposed to market participants) are the targeted audience of AQIs. The CAQ's AQIs are divided into four components: firm leadership and tone at the top; engagement team knowledge, experience, and workload; monitoring; and auditor reporting. Examples of indicators being considered by the CAQ include a description of how the audit firm establishes its tone at the top; metrics intended to measure the knowledge, experience, and workload of engagement team members; and trends in audit report reissuances. The CAQ's task force is in process of soliciting feedback on its (and other entities') AQIs. At the AICPA's 2013 Current SEC and PCAOB Developments Conference, presenters from a variety of organizations discussed their views on AQIs. Former SEC Chief Accountant Paul Beswick praised the PCAOB's AQI project as a help to audit committees in their evaluation of auditors (http://www.sec.gov/News/ Speech/Detail/Speech/1370540488257). Beswick suggested that audit committees might be overly focused on audit fees in their decision to retain or fire an auditor. He proposed that the auditor engagementdecision should focus on which auditor is best qualified to protect shareholders' interests, something which can be assessed, in part, by AQIs. AQIs and Current Practice Comparisons As discussed, the AQIs being developed by various organizations vary widely in their number and content. Based on the authors' recent count, the PCAOB's list includes 40 AQIs, the IAG's list includes 28, and the CAQ's list includes 13. (For a complete listing and comparison, see http://www.ecu.edu/business/AQI.cfin.) Of the AQIs under consideration, only the nine listed in Exhibit 2 are currently available from public sources. One potential problem with the proposed AQIs is that most of the measurements imply that less (i.e., a lower frequency) is better. Examples include the number of PCAOB inspections and the number of auditor resignations. Given that many of the AQIs are likely highly correlated with the size of the auditor's practice (number of clients), "less" does not necessarily mean that audit quality is higher, and comparisons across firms are not informative. Furthermore, with regard to reporting the number of client material weaknesses, it is unclear whether fifis AQI is meant to imply that more adverse opinions on the effectiveness of internal control over financial reporting is an indicator of "better" or "worse" audit quality. Should auditors be seeking to issue more adverse internal control opinions, or should they adopt risk-minimization policies that discourage accepting or retaining clients with material internal control issues? Based on the authors' analysis, only seven AQIs are common among all three organizations. These are listed in Exhibit 3. The first five of the AQIs listed in Exhibit 3 are inputs to the audit process, thought to be predictors of audit quality, while the last two are outputs of the audit process. At least one of the input measures, industry expertise, has been empirically demonstrated to be highly correlated with audit quality (e.g., S. Balsam, J. Krishnan, and J. S. Yang. Auditor industry specialization and earnings quality, Auditing: A Journal of Practice & Theory, vol. 22, no. 2,2003, pp. 71-97). Several contentious issues arise from this list. First, several measures suggest that more is better-more chargeable hours, more consultations, and more involvement of specialists. Adding engagement personnel may or may not enhance audit quality, but it will certainly drive up cost. Second, to develop future managers and partners, and to manage the cost of audits, the audit model relies on a pyramid of professionals; therefore, an AQI which implies that the delegation of routine audit tasks to less experienced personnel reduces audit quality is not tenable. Interestingly, although the audit firm transparency reports required by the EU explicitly require information about continuing education policies and quality assurance reviews, including PCAOB inspections, the 2013 transparency reports of the seven largest auditing firms provide few, if any, of the quantitative AQIs recommended by the PCAOB, CAQ, and IAG. It may be that collection and summarization of these data are hampered by the firms' form of operation. Most operate as a network of country-based firms that, as cited in PricewaterhouseCoopers' 2013 Transparency Report (http://www.pwc .com/en_US/us/about-us/assets/pwc-llpfyl3-transparency-report.pdf), are responsible for monitoring their own systems of quality control. Most international networks of firms are members of the International Federation of Accountants' Forum of Firms, which does not currently mandate reporting AQIs, but does require that member firms adhere to the International Auditing and Assurance Standards Board's International Standard on Quality. Even if AQIs are ultimately consolidated to 10 or 20 measures that most everyone agrees are important, the collection and reporting of the quantitative metrics could be costly; that cost could account for their lack of inclusion in transparency reports. Worse yet, a number of the AQIs under consideration may not be particularly good measures of audit quality. A 2012 study of the relationship between "transparency report disclosure scores" (calculated from the EU Directive transparency reports of 103 audit firms) and audit quality (proxied by discretionary accruals) found that the two are not highly correlated (R. Deumes, C. Schelleman, H. Vander Bauwhede, and A. Vanstraelen, "Audit Firm Governance: Do Transparency Reports Reveal Audit Quality?," Auditing: A Journal of Practice & Theory, vol. 31, no. 4, 2012, pp. 193-214). Because reporting requirements are very broad, all audit firms (including those with weaker governance) are able to provide extensive discussions of the required elements of the transparency reports, thereby potentially distorting the signal of audit quality. Rather than providing value about audit quality, it might be argued that transparency reports are marketing tools for auditing firms. The authors conclude by stating, "If the goal of such [transparency reporting] requirements is to provide available indicators of audit quality, more work seems necessary to develop such indicators." Conspicuously absent from the AQIs under consideration are two measures that have been debated in both the public press and by researchers as being predictors of audit quality: auditor tenure and the auditor's provision of nonaudit services. Although it has been essentially established that audit quality is diminished during the first two to three years of an auditor's appointment (e.g., J. N. Myers, L. A. Myers, and T. C. Omer, "Exploring the Term of the Auditor-Client Relationship and the Quality Of Earnings: A Case For Mandatory Auditor Rotation?" Accounting Review, vol. 78, no. 3,2003, pp. 779-799), standards setters such as the EU and PCAOB continue to debate the benefits of mandating audit firm rotation. Furthermore, the provision of nonaudit services is highly regulated, despite a lack of evidence suggesting that nonaudit services reduce audit quality (e.g., M. L. DeFond, K. Raghunandan, and K. R. Subramanyam, "Do Non-Audit Fees Impair Auditor Independence? Evidence From Going Concern Audit Opinions," Journal of Accounting Research, vol. 40, no. 4, 2002, pp. 1247-1273). Finding Answers There is no question that all market participants, including auditors, benefit when restatements of financial statements are minimized and when auditors are able to identify all material financial statement misstatements. Nevertheless, care must be taken when expecting certain inputs to the audit process to be highly correlated with, or to create, audit quality. Auditors are not the only party responsible for audit quality; after all, the auditor's report explicitly states: 'These financial statements are the responsibility of management." The results of extant research suggest many client-specific measures (e.g., size and number of institutional investors) are indicators of financial reporting quality, absent any consideration of the quality of the auditor. Furthermore, in light of the inherent weaknesses in any system of internal controls, such as the possibility of collusion, management overrides, and human error, one could argue that even the best auditor with the highest quality process inputs could be the victim of management fraud and inadvertently fail to detect a material misstatement in a company's financial statements. This article began with three questions. First, is it possible to actually predict audit quality based on inputs? The authors believe the answer to this question is maybe. Research suggests that some inputs, such as industry expertise, are highly correlated with audit quality, but correlation is not prediction. Client-specific characteristics (e.g., quality of finance personnel) may be equal, if not greater, predictors of audit quality. In addition, due to a lack of available public information, most of the input-AQIs have yet to be tested rigorously in terms of their impact on audit quality. Second, what is the cost of reporting AQIs in terms of gathering and reporting frie required data, as well as the potential consequences to auditing firms and their clients? As discussed, the costs could be significant, depending upon the number of AQIs and level at which they are reported (at the audit firm level, at the clientengagement level, or both). For example, while firms currently track the continuing education hours of each auditor, a mandate to disclose average education hours per engagement would require an investment in time or technology to capture that data. Disclosure of certain AQIs may lead to unintended consequences. For example, if firms are required to justify decisions related to changes in staffing, they might choose to forego actions that might enhance audit quality to avoid justifying their decisions. This complication may be exacerbated if AQIs are disclosed beyond audit committees and regulators. For example, the disclosure of salary information at the engagement level provides an unprecedented opportunity for staff and other employees to compare their salaries, not only to their firm's or national averages, but also to their peers working on the same engagement. Research suggests this may lead to decreased pay and job satisfaction for employees falling below the median, which may in turn lead to increased turnover and detrimental behavior (J. L. Cotton and J. M. Tuttle, "Employee Turnover A Meta-Analysis and Review with Implications for Research," Academy of Management Review, vol. 11, no. 1, 1986, pp. 55-70; T. A. Judge, C. J. Thoresen, J. E. Bono, and G. K. Patton, "The Job Satisfaction-Job Performance Relationship: A Qualitative and Quantitative Review," Psychological Bulletin, vol. 127, no. 3,2001, pp. 376-407; S. C. Currall, A. J. Towler, T. A Judge, and L. Kohn, "Pay Satisfaction and Organizational Outcomes," Personnel Psychology, vol. 58, no. 3, pp. 613-640). Of additional concern for companies is the likelihood of auditors passing along the cost of measuring and reporting AQIs in the form of higher audit fees (e.g., more engagement hours and an increase in required consultations to conduct the audit according to the new quality measures). Third, will the anticipated benefits of increased audit quality outweigh the costs of reporting AQIs? Ultimately, audit quality must be measured in terms of outputs. Although the number of financial statement restatements has not declined in recent years, restatements still only represent a very small percentage of total filings. It is, therefore, hard to imagine that reporting AQIs will significantly reduce the number of restatements. Importantly, while the number of inspection deficiencies may be positively correlated with the possibility of issuing an incorrect audit opinion (i.e., unmodified in the presence of a material misstatement), in most cases, inspection deficiencies do not lead to financial statement restatements. Nevertheless, it is possible that reporting AQIs could reduce the number of PCAOB inspection deficiencies or provide meaningful context in evaluating noted deficiencies. At this point, the cost of measuring AQIs is largely unknown, and the benefits are not easily derived; thus, this is a matter of debate. Despite many issues, including the ones raised above, the authors propose that increasing transparency by reporting a consensus list of 10 to 20 AQIs to audit committees is a reasonable way to address the stated goals of the PCAOB's AQI project. But care must be taken when compiling and interpreting AQIs. Guidance about calculations should be specific, to increase the likelihood that AQIs will be comparable across firms. AQIs should take into consideration firm size, and descriptions or explanations should accompany most quantitative measures, as suggested by the CAQ. Standards setters should also consider the cost of accumulating and tracking AQIs. AQIs that pose an undue burden on smaller auditing firms should be dismissed, as they would impose a competitive disadvantage and may drive smaller auditing firms out of tiie market for audits of publicly traded companies for reasons other than an inability to provide quality audits. The debate over AQIs is just heating up. CPAs should keep up with developments in the drafting process and make their voices heard by standards setters. ? It is not unreasonable to expect that AQIs, some of which will be based on firmand engagementspecific data, will ultimately be made public. EXHIBIT 1 Summary of Required Transparency Disclosures of the European Union * Legal structure and ownership of the auditing firm * Legal and structural arrangement of any network of the auditing firm * Governance structure of the firm * Internal quality control system of the firm, together with a statement on its effective functioning * Date of the latest quality assurance review * List of public-interest entity audit clients * Independence practices, including a statement confirming an internal review of independence compliance * Continuing education policies * Financial information depicting the importance of the firm * Information concerning the basis of partner remuneration Source: Article 40 of the EU Eighth Directive, pp. L 157/102-103, http://www.esma.europa.eu/system/files/dir_2006_43_EN.pdf The CAQ's AQIs differ from those of the PCAOB and IAG because they are both quantitative and descriptive, and because audit committees (as opposed to market participants) are the targeted audience. EXHIBIT 2 AQIs under Consideration Currently Available from Public Sources * Number and nature of PCAOB inspection deficiencies * Number and size of auditor resignations * Number of first-year audit engagements where the prior auditor resigned, or where there was a reported disagreement with the prior auditor * Frequency and market impact of financial statement restatements * Number and percentage of unqualified opinions on internal controls over financial reporting, which had material financial statement errors in the following year * Number of reported material weaknesses in internal control over financial reporting * Number of audit reports including a going concern opinion that did not have a subsequent bankmptcy * Number of audit reports lacking a going concern opinion that had a subsequent bankmptcy * Number and trends of PCAOB and SEC enforcement actions, including independence violations Source: http://ww.ecu.edu/business/AQI.cfm EXHIBIT 3 AQIs Common to the PCAOB, Investor Advisory Group, and Center for Audit Quality biputs * Chargeable hours of partners and employees (IAG and CAQ specify engagementlevel metrics) * Average years of experience of partners and employees (IAG-specifies engagement-level and firm-level metrics; CAQ-specifies engagement-level metrics) * Industry expertise (CAQ-specifies engagement-level metrics) * Training hours of partners and employees (IAG-specifies engagement-level and firm-level metrics; CAQ-specifies engagement-level metrics and descriptions) * Consultations with national technical experts and specialists (IAG and CAQ-specify engagement-level metrics and descriptions) Outputs * Restatements (IAQ-requires by industry; CAQ-requires descriptions) * Number and nature of PCAOB deficiencies Source: http://ww.ecu.edu/business/AQI.cfm Denise Dickins, PhD, CPA, CIA, is an associate professor of accounting, Rebecca G. Fay, PhD, CPA, is an assistant professor of accounting, and John Reisch, PhD, CPA, is an associate professor of accounting, all at East Carolina University, Greenville, N.C. (c) 2014 New York State Society of Certified Public Accountants |