Makes Sense to Me (The Beginning)

It is amazing what our minds do to us. And we are never quite so obstinately unintelligent as when we think — nay, we know — that our conclusions based on numbers and statistics are absolutely true. 

Those numbers lie.

I started thinking about this when I had cause to revisit the book Inevitable Illusions: How Mistakes of Reason Rule Our Mind by Massimo Piattelli-Palmarini. This is an excellent book about people’s tendency to go down mental tunnels — logical journeys to erroneous facts — and I would advise anyone to pick it up. In addition, some co-workers and I also recently were talking about the “Monty Hall Problem.” (We will revisit that one later.)

To explore this, I’m going to use my next few posts to pose some questions and provide real-life examples of how our logic and common sense can so easily mislead us. Let’s start with an easy one — ripped from the pages of actual audit work.

An auditor is evaluating the depositing practices of a payment processing center and wants to determine if it is meeting its goal (90 percent of all collections deposited the same day they are received in the department). For the two-week period under review, he obtains the following percentages.

  Week 1 Week 2
Mon 80% 85%
Tue 95% 95%
Wed 95% 95%
Thu 99% 98%
Fri 100% 100%

 

By adding the percentages and dividing by 10, he determines that 94 percent of payments are processed the same day as received. Since the department’s objective is 90 percent, he concludes that they are effectively meeting their objective.

The Auditor-in-charge knows this isn't right, but can't quite put his finger on why. Where is the fallacy in the auditor's thinking and why might the department not be meeting that objective?

Go ahead and post your thoughts below. The answer and the next problem in a couple of days.

 

Posted on Feb 8, 2010 by Mike Jacka

Share This Article:    

  Comments (16) un-categorized
  1. Comment by: D. Wanner   (2/9/10 06:07 PM)   

    The auditor in incorrect in his thinking because he is assuming that the department recieves around the same number of payments each day.  If the department happens to received most of the payments on Monday then the department would be out of compliance.   For example, on Mondays they recieve 100 payments and 80 are compliant and on Tues, Wed, Thur, and Friday they only receive 10 to 15 payments thier overall compliance would be less than 90%. 

  1. Comment by: Chris Abbott   (2/11/10 08:20 AM)   http://chrisabbott.wordpress.com/

    The first question I ask myself, is what are they measuring when they report say, 80% for Monday?  Is that 80% of what came in that day, or 80% of all the work they have including carryover from other days?  Is it 80% provided the payments arrived by 9:00am and had all the necessary information?  Does anyone hold work back?  Is it 80% of the money, or 80% of the number of payments?

    Typical distribution for departments handling payments received via snail mail from the public, is to receive at least 3 times the amount of mail for other days of the week on a Monday, sometimes, it is over half the total volume. So as previous commenter stated, the volumes for each day is required for even a half-hearted attempt at arriving at an accurate result.

  1. Comment by: Tom Donahue   (2/11/10 08:30 AM)   

    The focus of compliance should be each day (90% of all collections deposited each day).  The 94% represents an average over 10 days.  However, when you look at each day it is apparent that Mondays are bad days.  In the scheme of things, 20% of the time, 1 day a week, collections are in noncompliance.  At 94% the auditor would conclude that everything is okay.  However, by looking at the day, in this case Mondays, the auditor can focus his or her attention on the reasons why Mondays are so difficult to accomplish 90% compliance.  And perhaps make some recommendations for improvement.

  1. Comment by: Leya Neizvest   (2/11/10 08:57 AM)   

    The department reaches their goal of 90% only 80% of the time.  The auditor should look at the fact that only 4 out of 5 days show deposits of 90% or more of items received on that day.

  1. Comment by: Willie Sparkman   (2/11/10 09:04 AM)   

    Looking at the percentages presented and the goal, I would say the goal was not met.  I agree with Tom, Donahue, looking at it on a weekly basis, they were in compliance 80% of the time.  Monday appear to be when a lot of mail is received due to the weekend and maybe staffing is an area they could look in to improve the percentages to meet the goal

  1. Comment by: Michael Lenox   (2/11/10 09:39 AM)   

    The simple answer is that volume isn't taken into consideration. Percentages can lie when that occurs. Based on the statistics, Monday is a high volumn day, where as on Friday, there may only be a few receipts.

  1. Comment by: Kevin   (2/11/10 09:57 AM)   

    Agree the measurement is being met 80% of the time based on obtained numbers.  So, first off how do they arrive at the obtained numbers.  Also, is weekend mail received and deposited Monday?  Is weekend mail received on the weekend and processed, then deposited on Monday?  What up? 

  1. Comment by: Bruce Wilbat, CFE, CGAP, CICA   (2/11/10 10:56 AM)   

    In two words - WEIGHTED AVERAGE.     And then, Mondays appear to be "penalized", most likely due to having to process payments from over the weekend.     And please notice that all percentages increase as they go through the week, with 100% obtained by Friday. 

    95-100% on 8 of ten days could be giving a false sense of appropriate processing.    i.e. if only ten remittances come in on those days, whereas, the days that show 80 and 85% could have 50 on each day and as such would be more liekly to have lower percentages.   Q: How are the remittances from the days before considered in the percentages for the following day?

  1. Comment by: Suman Roy   (2/11/10 03:16 PM)   

     The only way to get the right answer would be to have two columns-

    Col A for payments received and Col. B for  payments processed for all the 10 days and then totaling both columns and then taking the % for the whole period of 10 days. Averaging percentages does not work, because the denominators are different.

  1. Comment by: Dave M   (2/11/10 03:25 PM)   

    Need number of collections per day to determine population and number of exceptions.  Some replies guess that lower compliance comes on days with heavy volume, but we don't have that data.

    Weekends are irrelevant if department is not open ("received in dept.").

  1. Comment by: Leif N   (2/12/10 09:46 AM)   

    The actual overall average needs to be calculated using the actual "numbers" not the "percentages"!

  1. Comment by: Deb   (2/15/10 12:08 AM)   http://sites.google.com/site/debashisgupta/

    While most comments have pointed out the fallacies in the conclusion, there appears to be a misplaced emphasis on number of collections in some.  This may be quite misleading, since you could deposit more than 90% of the collections which are below a certain amount - say $100 - and not deposit about 5% of the collections which could be the bulk by amount.  This could lead to additional risks of potential pilferage, crime or even short insurance cover.

    So a good way would be to always take the weighted average of amounts deposited.

  1. Comment by: Nicole   (2/16/10 01:43 AM)   

    Simple maths - you cant add percentages and come to a logical conclusion?

  1. Comment by: Terry Thompson   (2/17/10 01:25 PM)   

    Adding the percentages is an erroneous approach.  The auditor should have divided the total number of timely deposits by the total number of deposits to determine the accurate compliance percentage.

  1. Comment by: Terry Thompson   (2/17/10 02:19 PM)   

    I retract my original statement.  After re-reading the audit objectives more closely, I agree with Tom Donahue's post on 2/11/10.  If the objective is to determine whether 90% compliance exists daily, the auditor would have to conclude that this is the case 80% of the time based on calculations for individual days. 

    One thing that is missing is determining whether having 90% compliance 80% of the time is acceptable to management.  Even if management accepted this condition, the auditor should determine the root cause with regard to why the goal is not being achieved on Mondays - especially since this day is non-compliant 100% of the time - to recommend opportunities for improvement.

  1. Comment by: Mike Jacka   (2/17/10 06:07 PM)   

    Lots of good comments.  And you've all gotten to varius pieces of the problem.  Now, check up on the next two installments and let us know what you think about the next problems.

Leave a Reply