Home

Subjectivity and Objectivity in Auditing

Presented by: John H. Johnson

J-E-T-S Consultants

First presented at ASQ National Audit Conference, Charlotte

 Introduction:

Experienced practitioners of quality auditing have struggled with subjectivity and objectivity for a long time. As a result, we now see audit results that range from totally objective to totally subjective, depending on the auditor's approach and the sponsoring organization's expectations of what an audit should be. In this paper, we will review what should be expected from an audit, where subjectivity and objectivity fit, and lessons learned in dealing with these issues in actual audit practice.

Discussion:

To get a feel for where subjectivity and objectivity fit into the audit process, we should first look at the purpose of an audit. Audits are conducted to provide feedback on the relative success of a process. (In this paper, process is a term we will use to define activities, programs, or other methods of accomplishing work.) This feedback must be provided to someone, but who is most appropriate? This question may be answered by asking a second, more pointed question: who are the customers of an audit? In most cases there are not more than two primary customers -- first, auditee management; and second, management of the audit sponsor.

Under the best conditions, the audit sponsor is the auditee management. In this case, an organization recognizes the need for an independent assessment and requests the audit. The auditee is now sponsoring the audit and there is one primary customer -- the auditee management. At other times, an audit is requested by an group other than the auditee. For example, a large organization may want to know how well some satellite group is doing. In this case, there are two customers: the auditee management and the sponsor's management. This can cause problems for the auditor, because each group may have a different expectation of what the audit results should be.

There also may be several secondary customers of the audit. These may include the following:

Auditee supervisors;

Auditee line personnel;

Regulatory agencies;

Users of the audited process;

Supporting product or service suppliers; and

News or media concerns.

Note that the auditor is not included on the list. The auditor is the supplier, not a customer. In some organizations, the customer of the audit has become the auditor group. These groups do audits because management has committed to do them, and that's that! "Audits will be scheduled and conducted on a planned basis" and the auditor's job is to satisfy the schedule. The audit becomes a way to satisfy the schedule rather than a process that is useful to audit customers.

To be truly useful, the auditor must determine who the primary and secondary customers are and how to address each group's needs and expectations. This is where subjectivity and objectivity enter the picture. Different customers expect different results. The audit must be planned, conducted, and presented in a manner that can meet these different expectations.

Let's take the customers defined earlier and identify what they want from an audit:

Customer: Customer wants:
Audit sponsor    How well is auditee doing?
Auditee management  How are we doing? What is the big picture?
Auditee supervisors  Where are our problems? What can be improved?
Auditee line personnel  How can we be more effective in our jobs?
Regulatory agencies  Is the auditee meeting regulations?
Users   Can auditee provide a quality product?
Supporting suppliers  Are we part of a problem?
News and media Is this something the public wants to know?

Note that all audits do not include all the customers listed above. However, to be truly useful, the audit must fulfill what the applicable customers need. That word "useful" keeps coming up! Information is of no value if it cannot be used. Let's revise the purpose of the audit discussed earlier to reflect this important word: Audits are conducted to provide useful feedback on the relative success of a process.

This is where subjectivity and objectivity come into play in the audit process. Let's look at some definitions that we will call the subjectivity/objectivity scale:

Term: Definition:
Totally Subjective Opinion
Subjective Professional opinion, based on facts and experience
Somewhat Subjective Logical assumption based on facts
Somewhat Objective Summary of different factual data or analyses
Objective Data analysis
Totally Objective  Objective Evidence, data

For audits to provide useful feedback to customers, there must be a place for each of these in auditing! Different customers need different information depending on their responsibilities. J-E-T-S' experience in dealing with many client companies has proven two fundamentals of subjectivity in relation to customer needs:

1. The level of detail desired decreases as responsibilities increase.

2. The level of subjectivity needed increases as level of detail decreases.

These two factors demonstrate that customers need different levels of objectivity depending on their organizational responsibilities. To be useful, audit products must satisfy these varying needs. Let's look at each of the customers listed above again, this time including the level of detail each needs:

Customer:    Customer wants: Detail Level Needed:
Audit sponsor How well is auditee doing?   Big picture, not detail
Auditee management How are we doing? What is the big picture? Summary, not detail
Auditee supervisors Where are our problems? What can be improved? More detail, specifics
Auditee line personnel How can we be more effective in our jobs? Detailed information
Regulatory agencies Is the auditee meeting regulations? Limited to regulations
Users/customers Can auditee provide a quality product/service? Summary
Supporting suppliers  Are we part of a problem? Specific to supplier
News and media Is this something the public wants to know? Specific to public impact

The information listed above demonstrates the first fundamental -- the level of detail needed from an audit is inversely proportional to responsibility. A line worker needs to know specifically what individual actions are needed for process improvement. A vice president needs to know what the trend is or if management actions support process improvement. For example, while the line worker is wondering if elimination of step 3.2a of the work procedure would improve the process, the VP is wondering if workers have adequate training to evaluate and improve their procedures. The line worker needs objective information; the VP needs much more subjective information. Recall again our description of "Subjective." Note it is based on factual data and experience. This brings us to the only rule of subjectivity in auditing:

All subjective audit statements must be clearly supported by objective evidence.

Auditors who use subjectivity and do not follow this rule are headed for disaster. Total subjectivity, as in a statement of opinion not based on factual data, has no place in auditing. Auditors not following the single, simple rule of subjectivity has resulted in embarrassing situations with management trying to support an auditor's opinion that has no basis. This is the reason that some organizations have removed subjectivity from the audit process. Without some level of subjectivity, however, audits cannot provide useful information to their customers. These are no longer audits, but have instead become inspections, or as Alan Sayles puts it -- Inspection Nouveau.

Subjectivity is therefore an important part of the audit process, but must always be used with the rule that all subjectivity be clearly supported by objective evidence. Note the word clearly. The customer should not have to stretch to arrive at the same subjective conclusion as the auditor. Analyses used to support subjective statements should be provided or described to establish a clear link between data and results.

How can we reasonably do this? Many auditors and audit programs already do. The audit report structure provides information in a format that meets the needs of the customers. Each customer can easily find the level of detail and subjectivity that they need. Let's look at the different parts of the audit report and apply the subjectivity/objectivity scale:

Report Section S/O Scale Description:
Executive Overview Subjective Professional opinion, based on factual data and experience
Recommendations Subjective Professional opinion, based on factual data and experience
Findings Somewhat Subjective (or) Logical conclusion based on facts
  Somewhat Objective Summary of different factual data or analyses
Checklist Questions Objective Data analysis
Objective Evidence Totally Objective Objective Evidence, data

Note that each part of the audit report is more subjective than the one that follows. Each subjective statement is thus supported by more objective information. In following our only subjectivity rule, statements in the executive overview that are subjective must be clearly supported by findings, checklist results, and ultimately, objective evidence.

Now let's take our customer list and apply it to the parts of the audit report that satisfy the customer needs:

Customer:  Detail Level Needed: Applicable Report Sections
Audit sponsor Big picture, not detail Executive Summary, Findings
Auditee management Summary, not detail Executive Summary, Recommendations
Auditee supervisors More detail, specifics Findings, Recommendations
Auditee line personnel Detailed information Checklist Results, Findings
Regulatory agencies Limited to regulations   Summary, Relevant Findings
Users Summary   Executive Summary
Supporting suppliers   Specific to supplier   Applicable Findings
News and media  Specific to public impact Executive Summary, "Interesting" items

As you can see, a well-written audit report can satisfy the needs of a variety of customers. The level of subjectivity is appropriate to the customer's need. If additional information is needed for any subjective statement, there is always more objective or detailed information available to support it.

One aspect that relates to subjectivity in auditing is the use of recommendations. Since recommendations are professional opinion based on factual data and experience, they are subjective. Our one rule for subjectivity applies -- the recommendation must be supported by objective evidence. Badly written recommendations carry the same stigma as any other subjective statement in an audit report. Auditors presenting subjective recommendations without clear objective basis results in arguments or embarrassing situations that should be avoided. To prevent these unfortunate situations, some audit programs do not allow the use of recommendations. This restricts the auditor's ability to facilitate process improvement and reduces the effectiveness of the audit. Recommendations can be a very valuable asset to auditing when used correctly.

In order to be used effectively, recommendations must also fulfill one additional criterion -- they must not represent corrective action. The auditor is not responsible for corrective action, the auditee is. A recommendation should therefore not relate to a proposed "fix" for an identified deficiency.

Our experience has proven that recommendations can be very beneficial if used within careful boundaries. Those boundaries are:

1. The recommendation must not represent a deficiency. Deficiencies are findings.

2. The recommendation must not be corrective action to an identified finding.

3. The recommendation must clearly state:

The current condition and what can be improved;

Specifically what is recommended to be done;

What the benefit of implementing the recommendation will be; and

Who will benefit from the implementation of the recommendation.

If recommendations are kept within these boundaries, they can be a major force in leading an organization toward excellence. As operations undergo continuous improvement, the number of clear deficiencies or findings will decrease. Improvement will go from giant steps to baby steps. Recommendations are a method of providing management with the benefit of a knowledgeable auditor's expertise that will facilitate continuous improvement.

Summary:

Subjectivity, in the form of professional opinion based on factual data and experience, should not be avoided in auditing. In fact, for audits to be truly useful for all the customers of the audit, some level of subjectivity is necessary. Audits that do not include subjectivity are reduced in their ability to provide useful feedback for improvement and, therefore, become inspections. The rule of subjectivity is simple: All subjective statements must be clearly supported by objective evidence. This one rule of subjectivity must always be followed; otherwise the subjective statements may lead to disastrous results. Different parts of the audit report should contain varying levels of objectivity versus subjectivity to respond to the needs of different customers of the audit.

When used carefully by knowledgeable auditors, subjectivity can greatly improve the benefit of the audit to auditee management. The ability to search for common causes, provide comparative analyses, and identify management issues all require some level of subjectivity, but all can also contribute to the audit's purpose -- providing useful feedback.

Note:  The author welcomes comments or suggestions!  Comments on this article may be e-mailed to john@jetsquality.com.

Note:  This article is presented for your personal use only.  Do not reproduce, revise, or distribute this article without permission of J-E-T-S!

Home ] Up ]

J-E-T-S celebrates over 30 years of service excellence in 2017!               Phone:  704-987-9858
Send mail to webmaster@jetsquality.com with questions or comments about this web site.
Copyright 1984-2017 
Last modified: December 05, 2016