Benjamin Disraeli was the Victorian era Prime Minister of the United Kingdom.  He famously said “there are three kinds of lies: lies, damned lies and statistics”.  He died in 1881.  This was after Sir Francis Galton coined the term standard deviation, but before he popularized concepts like of correlation and the Central Limit Theorem with his publication of Nature in 1889.

Perhaps Disraeli was witness to how misleading statistics could be without an understanding of sample size requirements.  Most people wander about in the same fog that engulfed Disraeli.

The Central Limit Theorem states that a sample size equal to or greater than 30 is required to make credible assertions about a population.  “In practice, the Central Limit Theorem allows us to make inferences about population means relying on the normal distribution when a) the population is normal or b) when n ≥ 30. As a practical matter, the sampling distribution of the mean will be approximately normal when n ≥ 15 and the population is symmetrically distributed. However, appraisers usually know very little about the shape of population distributions of price, property attributes, financing arrangements, and the like. Therefore, the n ≥ 30 criterion generally applies to real property valuation work.”[1]

In general, if the mean and the median of a population differ, the distribution is not normal and you need a sample size of 30 or greater.

[1] Marvin L. Wolverton, PhD, MAI, An Introduction to Statistics for Appraisers (Chicago, The Appraisal Institute)

5 replies
  1. Dan Forrester
    Dan Forrester says:

    15 sounds about right. Back about 50+years ago in a stat 101 class,at BGSU, the number necessary for a reasonable outcome of reliability was 17, minimum. How many times does the market present enough data points in the population? Unless you are in a densely populated area, that answer should be seldom. That’s where knowing the market place and being competent enough to select the right comps, makes you an appraiser, You know, the part where appraisal is, also, an art.

  2. Old Appraiser
    Old Appraiser says:

    In a large metro area there ‘should’be sufficient sample size one would think? Not so fast! An appraiser could sample the entire metropolis or he could choose MLS or Assessor boundaries or whatever he subscribes to. He would still need to separate investor distressed buys from investor renovated turnkey sales from multi-parcel or atypical sales and owner-occupants lender based v cash so on and on. We receive Courthouse Retrieval market report that provides increasing or decreasing stats based on 1 or 2 sales in 0-3 mo. After our individual research corrects the ‘documented’ but Misleading impression, we still often find sample size insufficient to support trends. All this takes hours of research to complete the MCR, which supports nothing other than what a client wants to satisfy QC.

  3. Jim Baumberger
    Jim Baumberger says:

    Thanks, AeL, and Mr. Cullen for sharing some wisdom that is time-tested and still just as true today!! I concur with Suki that characterizing trends — especially in a 1004MC — with a sample size less than 30 is usually unreliable. I say usually because even a broken watch is right twice a day 😉

    I have seen appraisal reports that have concluded property values are declining with a sample size of 3-5 sales total across the three time periods of the 1004MC. Even more questionable, the characterizations that values are declining are sometimes asserted in a market where property values are known to be increasing — based on MLS, the FHFA’s Home Price Index, Case-Schiller’s statistics, etc., that are based on much larger sample sizes.

    Of course, the strongest support for any market conditions conclusions is an adequate sample size of data on comparable and competitive properties located in the Subject’s neighborhood. If you have an inadequate sample size, though, you can either incorporate similar property data from a comparable and competitive market area, with disclosure and explanation, or you can supplement your initial research.

    Fortunately, I see many admirable appraisal reports where Appraisers with small sample sizes in the 1004MC strengthen their trends analysis with supplemental information. Many Appraisers do a terrific job of additional analysis and additional explanatory comments when the boxes in the 1004MC do not reliably reflect accurate trends. Appraisers frequently cite multiple data sources for their conclusions on one-unit housing trends and explain to the Intended User(s) the logic, reasoning and support for their market conditions analysis and conclusions. Keep up the good work!

    Happy appraising,

    Jim in Portland

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply