Performance Audit Guidelines – Key Principles

INTRODUCTION

  1. INTOSAI’s fundamental auditing principles recognise that due to the differing approaches and structures of Supreme Audit Institutions (SAIs), not all auditing standards apply to all aspects of their work[1]. Furthermore, on the basis of the terms of the audit mandate with which SAIs are empowered, any auditing standards external to the SAI cannot be prescriptive, nor have a mandatory application to the work of the SAI[2]. However, in order to promote high quality work across its members, INTOSAI advocates that each SAI should establish a policy which has regard to INTOSAI standards, and other specific professional standards, which should be followed in carrying out various types of work that the organisation conducts. This audit guideline of key principles outlines a common understanding of what defines high quality work in performance auditing.
  2. Comparisons between the practices of performance auditing in different countries show considerable variations depending on the mandate, organisation and methods used by the SAIs. The legal, administrative and economic environment can have a bearing on the nature of performance audits conducted and how they are carried out. The maturity of public sector administration also impacts on the extent and nature of performance audits that can be performed.
  3. Performance auditing generally follows one of three approaches in examining the performance of the audited entity. The audit may take a result-oriented approach, which assesses whether pre-defined objectives have been achieved as intended, a problemoriented approach, which verifies and analyses the causes of a particular problem(s), or a system-oriented approach which examines the proper functioning of management systems: or a combination of the three approaches.
  4. Performance audit may also adopt one of two perspectives for the audit: a top-down perspective, which focuses on the requirements, intentions, objectives and expectations of the Legislature, Executive and/or regulatory body, or a bottom-up perspective, that focuses on the effects of the activity on the audited entity and the larger community[3]. In the case of the former performance audit does not question the intentions and decisions of the legislature, but instead examines whether possible shortcomings in the laws and regulations have affected those intentions being met. Depending on their mandate, SAIs may audit the assumptions on which policy decisions were based and the impact of such policy decisions. The audit provides an objective assessment to inform the legislature on such issues as how to enhance policy target achievement and/or how to accomplish objectives more efficiently and effectively.
  5. Whichever approach or perspective is adopted, performance audit aims mainly towards examining the economy, efficiency and effectiveness of the audited entity in the performance of its functions and activities, not excluding the verification of the audited entity’s compliance with established legislation and regulations[4]. Where appropriate, the impact of the regulatory or institutional framework on the performance of the entity should also be taken into account. Performance audit often achieves this by attempting to answer two basic questions: are the right things being done, and are things being done in the right way?
  6. As performance auditing can deal with all facets of the public sector, it would not be possible or appropriate to propose detailed common auditing standards to cover all situations. Accordingly, auditors are required to apply their own professional judgments and applicable professional standards to the diverse situations that arise in the course of performance auditing. This document is largely based upon the concepts contained in ISSAI 3000 – Implementation guidelines for Performance Auditing, to which auditors should refer for additional guidance.

2. KEY PRINCIPLES OF PERFORMANCE AUDITING

 

2.1              Definitions

  1. Performance auditing is an independent and objective examination of government undertakings, systems, programmes or organisations, with regard to one or more of the three aspects of economy, efficiency and effectiveness, aiming to lead to improvements5.
  2. The performance audit task is a separately identifiable piece of audit work, typically resulting in the issuing of a statement, or report. It should have clearly identifiable objectives and pertain to a single or clearly identifiable group of activities, systems, programmes or bodies know as the “audited entity”.

2.2.             Performance audit objective

 

  1. According to ISSAI 100[5], an individual performance audit should have the objective of examining one or more of these three assertions:
  • the economy of activities in accordance with sound administrative principles and practices, and management policies;
  • the efficiency of utilisation of human, financial and other resources, including examination of information systems, performance measures and monitoring arrangements, and procedures followed by audited entities for remedying identified deficiencies; and
  • the effectiveness of performance in relation to the achievement of the objectives of the audited entity, and the actual impact of activities compared with the intended impact.

 

  1. The audit objectives are usually expressed in the form of one overall audit question and a limited number of subsidiary questions that the audit will answer and conclude against. Such questions are thematically related, complementary, not overlapping and collectively exhaustive in addressing the overall question. The audit questions addressed by performance audit do not have to be exclusively based on a retrospective audit approach. In a performance audit, SAIs can take an early initiative and furnish proactive audit findings, and/or recommendations, where appropriate, if this is explicitly allowed by their legal mandate. Furthermore, financial and compliance audit aspects[6], including environmental considerations in the context of sustainable development, can also be included in a performance audit. Finally, the perspective of the citizen that is related to the performance of the audited entity should be taken into account where appropriate.

2.3 Selecting audit topics

 

  1. Auditors should select audit topics that are significant, auditable, and reflect the SAI’s mandate8. The audit should lead to important benefits for public finance and administration, the audited entity, or the general public. Where there is an overlap between other types of audit and performance auditing, classification of the audit engagement will be determined by the primary purpose of that audit[7]. Aside from audits carried out under legal mandate at the request of the Parliament or other empowered entity, performance audit topics should be selected on the basis of problem and /or risk assessment and materiality or significance (not only financial significance, but also social and/or political significance), focusing on the results obtained through the application of public policies. The selection process for audit topics should aim to maximise the expected impact from the audit while taking account of audit capacities. The processes of strategic planning[8] and establishing the annual audit programme, are useful tools for setting priorities.

2.4              The audit process

 

2.4.1           Planning an audit

  1. The auditor should plan the audit in a manner which ensures that it is of high quality and is carried out in an economic, efficient and effective way and in a timely manner[9]. The audit planning documents should contain:
  2. background knowledge and information needed to understand the entity to be audited, to allow an assessment of the problem and risk, possible sources of evidence, auditability, and the materiality or significance of the area considered for audit12;
  3. [10] the audit objective, questions or hypotheses, criteria, scope and period to be covered by the audit, and methodology (including techniques to be used for gathering evidence and conducting the audit analysis);
  4. an overall activity plan which includes staffing requirements, i.e. sufficient competencies (including the independence of engagement staff), human resources, and possible external expertise required for the audit, an indication of the sound knowledge of the auditors in the subject matter to be audited[11];
  5. the estimated cost of the audit, the key project timeframes and milestones, and the main control points of the audit.
  6. Performance audits should have suitable audit criteria that focus the audit and provide a basis for developing audit findings. The audit criteria, which can be of a qualitative or quantitative nature, should be reliable, objective, useful, and complete. It should be possible to identify the source of the audit criteria used.
  7. The audit scope should clearly define the extent, timing and nature of the audit to be carried out. When laws, regulations, and other compliance requirements pertaining to the audit entity have the potential to significantly impact on the audit questions, then the audit should be designed to address these issues in order to conclude on the audit questions14.
  8. In determining the extent and scope of the audit, auditors often need to assess the reliability of internal controls that assist in conducting the business of the audited entity15. The extent of that assessment depends on the objectives of the audit. Moreover, they should be alert to situations or transactions that could be indicative of illegal acts or abuse and should determine the extent to which such acts affect the audit findings[12].
  9. When designing audit procedures, the auditor should determine the means for gathering sufficient appropriate audit evidence to conclude against the objectives, answer the audit questions, or confirm the hypotheses. Since auditors seldom have the opportunity to consider all information about the audited entity, data collection methods and sampling techniques should be carefully chosen. The planning phase should always involve certain research efforts, with the aim of building knowledge, testing various audit designs; and checking whether data needed is available. This makes it easier to choose the most appropriate audit method.
  10. Performance audits can draw upon a large variety of data-gathering and analysis techniques, such as surveys, interviews, focus groups, observations, documentary analysis, transaction testing, as well as the analysis of economic, financial and performance data. Audit methods should be chosen which best allow the gathering of audit data in an efficient and effective manner. While the aim of auditors should be to adopt best practices, practical reasons such as availability of data may restrict the choice of methods. Therefore, as a general rule, it is advisable to be flexible and pragmatic in the choice of methods. For this reason, performance audit procedures should not be standardised in all their terms, as being too prescriptive may hamper the flexibility, professional judgement, and high levels of analytical skills required, in a performance audit[13].
  11. Auditees should be notified of the key aspects of the audit, including the audit objective, questions, criteria, and scope, before the start of the data collection phase[14] or after the completion of the audit planning.

2.4.2           Conducting the Performance Audit

  1. Audit examination work takes place on the basis of audit planning already undertaken, and the planning documents thereby developed. Audits should be performed with due care, with an objective state of mind, and with appropriate supervision. The audit team should collectively possess adequate knowledge of the subject matter and audit techniques.
  2. The auditor should obtain sufficient and appropriate audit evidence to satisfy the audit objective and questions, to be able to draw conclusions and, if appropriate, to issue recommendations. The nature of the audit evidence required in performance audit is determined by the subject matter, the audit objective, and the audit questions. Under normal circumstances, performance audits require significant judgement and interpretation in concluding against the audit questions, due to the fact that audit evidence is more persuasive (“points towards the conclusion that…”) than conclusive (“right/wrong”) in nature[15].
  3. Evidence may be categorized as physical, documentary, testimonial, or analytical. The types of evidence to be obtained should be explainable and justifiable in terms of sufficiency, validity, reliability, relevance, and reasonableness. Audit evidence should be competent, relevant and reasonable in order to support the auditor’s judgment and conclusion regarding the audit questions[16]. All audit findings and conclusions must be supported by audit evidence.
  4. Performance auditors should be resourceful, flexible and systematic in their search for sufficient evidence. They must also be receptive to alternative views and arguments and seek data from different sources and stakeholders[17]. Auditors should always try to be practical in their efforts to collect, interpret and analyze data. While primary or own source data is usually the most reliable, secondary data which is collected and/or analysed by others (e.g. performance evaluation reports, internal audit reports, etc.), can be an important source of information in performance audits. It is important, that the reader of the audit report is informed about the source and quality of the data, particularly when it contains estimations.[18]
  5. The analysis of data involves combining and comparing data from different sources. It is important that the auditor works systematically and carefully in interpreting the data and arguments collected23. The audit team should document all matters which in its professional judgement are important in providing evidence to support the audit findings and the conclusions to be expressed in the audit report.
  6. The auditor needs to produce audit documentation to fully record the preparation, conduct, contents and findings of the audit in a meaningful way. They should be sufficiently complete and detailed to enable an experienced auditor having no previous connection with the audit to subsequently determine what work was performed in support of the audit findings, conclusions, and recommendations24. In general, the organisation of the audit should also satisfy the requirements of good project management.
  7. The development of good and proper external relations is a key factor in achieving effective and efficient performance audit results. Auditors should seek to maintain good professional relationships with all stakeholders involved, promote a free and frank flow of information in so far as confidentiality requirements permit, and conduct discussions in an atmosphere of mutual respect and understanding of the respective role and responsibilities of each stakeholder. The communication process between the auditor and auditee begins at the planning stage of the audit and continues throughout the audit process, by a constructive process of interaction, as different findings, arguments and perspectives are assessed. Where important audit findings are made during an audit these should be communicated to those charged with corporate governance in a timely manner.
  8. Auditors should not communicate to third parties, neither in writing nor orally any information they obtain in the course of audit work, except where doing so is necessary to discharge the statutory or otherwise prescribed responsibilities of the SAI in question. Any such communication of information should be governed by the statutory or other rules of procedure in force for the respective SAI[19]. Auditors however, may exchange information regarding management deficiencies with internal auditors, should this information not be of a data security or other confidential nature, for the purposes of ensuring that any identified shortcomings are addressed. Auditors should also report any financial irregularities to the authorities concerned, where appropriate.

2.4.3           Reporting

  1. In a performance audit, the auditor reports on the economy and efficiency with which resources are acquired and used, and the effectiveness with which objectives are met. Such reports may vary considerably in scope and nature, for example covering whether resources have been applied in a sound manner, commenting on the impact of policies and programmes and recommending changes designed to result in improvements[20].
  2. For all audit assignments any limitations to the audit, such as restrictive regulations, or limitations concerning access to information or reporting requirements, should be disclosed to users of the audit report. The report should also disclose the standards that were followed and audit criteria applied in carrying out the performance audit.
  3. The auditor is not normally expected to provide an overall opinion on the achievement of economy, efficiency and effectiveness by an audited entity in the same way as the opinion on financial statements[21]. Where the nature of the audit allows this to be done in relation to specific areas of an entity’s activities, the auditor is expected to provide a report which describes the circumstances and context to arrive at a specific conclusion rather than a standardised statement.
  4. The audit report should include information about the audit objective, audit questions, audit scope; audit criteria, methodology, sources of data, any limitations to the data used, and audit findings. The findings should clearly conclude against the audit questions, or explain why this was not possible. The audit findings should be put into perspective and congruence should be ensured between the audit objective, audit questions, findings and conclusions. The report should, where appropriate, include recommendations.
  5. The report should be timely, complete, accurate, objective, convincing, constructive, and as clear and concise as the subject-matter permits[22]. It should also be readerfriendly, well structured, and contain unambiguous language. Overall, it should contribute to better knowledge and highlight improvements needed[23]. The audit findings and conclusions should be based on evidence and should be clearly distinguishable in the report[24]. All relevant viewpoints should be considered in the report and the report should be balanced and fair[25].
  6. Recommendations, where provided, should be presented in a logical, knowledgebased and rational fashion, and be based on competent and relevant audit findings[26]. They should be practicable, add value and address the audit objective and questions.

They should be addressed to the entity(ies) having responsibility and competence for their implementation.

  1. Auditors should refer to all significant instances of non-compliance and significant instances of abuse[27] that were found during or in connection with the audit34. Where such instances are not pertinent to the audit questions, it is envisaged that they would nevertheless be communicated to the auditee preferably in writing at the appropriate level.
  2. Unless prohibited by legislation or regulations, before publishing a performance audit report, the SAI should always give the auditee(s) the opportunity to comment on the audit findings; conclusions, and recommendations[28]. Where disagreements occur they should be analysed and factual errors corrected. The examination of feedback received should be recorded in working papers so that any changes to the draft audit report, or reasons for not making changes, are documented.
  3. Distributing audit reports widely can support the credibility of the audit function. SAIs should decide about the method of distribution in conformity with their respective mandates. The reports should be distributed to the auditee, the Executive and/or the Legislature, and where relevant, made accessible to the general public directly and through the media and to other interested stakeholders[29], unless prohibited by legislation or regulations.

2.4.4           Follow-up

  1. Follow-up of the audit report should be part of the audit process as it is an important tool used to strengthen the impact of the audit and improve future audit work. The priority of follow-up tasks should be assessed as part of the overall audit strategy of the SAI[30]. Sufficient time should be allowed for the audited entity to implement appropriate action.
  2. When conducting follow-up of audit reports, the auditor should adopt an unbiased and independent approach. The focus should be to determine whether actions taken on findings and recommendations remedy the underlying conditions[31]. The results of the follow-up should be reported appropriately in order to provide feedback to the legislature, together if possible, with the conclusions and impacts of the corrective actions taken where relevant.

2.5 Quality system

 

  1. Performance audits should be subject to a system of quality control[32], incorporating processes for supervision and monitoring of quality, quality assurance, and external quality and peer reviews, in order to provide reasonable assurance that the audit has been conducted in accordance with professional standards and regulatory and legal requirements, and that the reports are appropriate. In this regard SAIs should apply the provisions of ISSAI 40 which provides a framework in relation to establishing and maintaining an appropriate system of quality control which covers all audit assurance and other work performed by SAIs. For performance audit, the following issues are important to ensure the quality of audit work:
  2. to the extent possible and needed, performance audits should be carried out by teams since, as a rule, they address complex questions[33]. All audit team members should understand the audit questions, the terms of reference of the work assigned to them, and the nature of responsibilities required of them by auditing standards;
  3. experts participating in an audit should have the necessary competence required for the purposes of the audit. The audit team should ensure that the expert is independent of the activity/programme, and that (s)he is informed about the conditions and the ethics required[34];
  4. decisions should be properly documented regarding the audit objective and questions, and criteria of the audit, resources to be applied to the audit in terms of skills and qualifications, arrangements for reviews of progress at appropriate points, and the dates by which fieldwork is to be completed and a report on the audit is to be provided[35].

 

[1] ISSAI 100/13

[2] ISSAI 100/17

[3] ISSAI 3000/1.8/ page 27

[4] ISSAI 4000-series 5 ISSAI 3000/1.1.

[5] ISSAI 100/40.

[6] ISSAI 4000-series 8 ISSAI 100/34.

[7] ISSAI 100/41.

[8] ISSAI 3000/3.2.

[9] ISSAI 300/1.1.

[10] ISSAI 300/1.3-1.4.

[11] ISSAI 3000/2.2, page 38. 14 ISSAI 4000 series 15 ISSAI 300/3.1.

[12] ISSAI 300/0.3(d).

[13] ISSAI 3000/1.8, page 29.

[14] ISSAI 300/1.4(g).

[15] ISSAI 3000/4.2.

[16] ISSAI 300/5.4.

[17] ISSAI 3000/4.2.

[18] ISSAI 3000/appendix 3/5 23 ISSAI 3000/4.5. 24 ISSAI 300/5.7

[19] ISSAI 200/2.46.

[20] ISSAI 400/4.

[21] ISSAI 400/23

[22] ISSAI 400/7(a).

[23] ISSAI 3000/5.3.

[24] ISSAI 400/7.

[25] ISSAI 400/24.

[26] ISSAI 3000/4.5.

[27] ISSAI 1240/P6, “Abuse involves behavior that is deficient or improper when compared with behavior that a prudent person would consider reasonable…”. 34 ISSAI 400/7.

[28] ISSAI 3000/4.5.

[29] ISSAI 3000/5.4.

[30] ISSAI 3000/5.5. 

[31] ISSAI 400/26.

[32] ISSAI 40

[33] ISSAI 3000/2.2.

[34] ISSAI 200/1.18 and 2.43-44.

[35] ISSAI 200/1.24 and 1.26.

(Visited 94 times, 1 visits today)
Share this:
Share

Written by 

One thought on “Performance Audit Guidelines – Key Principles”

  1. Pingback: INTERNATIONAL STANDARDS IN AUDITING (ISA) - MASOMO MSINGI PUBLISHERS

Leave a Reply