From: Adam Rifkin -4K (adam@XeNT.ics.uci.edu)
Date: Tue May 02 2000 - 03:02:40 PDT
I'm acknowledging that this is "old bits", but I'd always wanted to FoRK
a reference to Benford's Law and conveniently a tight little explanation
of it came out two weeks ago.
> http://www.ireland.com/newspaper/science/2000/0417/science5.htm
>
> Monday, April 17, 2000
>
> Some Numbers Are Less Random Than Others
>
> Random numbers occur less randomly than you might think, with numbers
> beginning with one and two occurring much more frequently than any
> other, according to Benford's Law. Fintan Gibney explains how this law
> has been used to uncover fraud.
>
> Everyone who buys a quick-pick in the Lotto is making an act
> of faith in the randomness of the distribution of the winning
> numbers. Surprisingly, some numbers that we would intuitively
> expect to be random are not.
>
> In 1881, an American astronomer Simon Newcomb reported
> in the American Journal of Mathematics that, for some reason,
> the initial pages of logarithm books were more thumb-smudged
> than the later ones. This seemed to indicate that people did
> more calculations involving the numbers one and two than with
> the numbers eight and nine.
>
> Not having any convincing argument as to why this should be
> so, the matter failed to arouse much interest, and was forgotten
> until 1938 when Frank Benford, a physicist with General
> Electric, reopened the file. He quantified the occurrence of
> numbers used in many everyday applications. He found that 30
> per cent of numbers from any source begin with the digit one,
> 18 per cent with the digit two, 13 per cent with three
> decreasing to about 5 per cent for eight and nine.
>
> Benford analysed data such as the drainage area of rivers,
> numbers drawn from the front pages of newspapers, even
> specific heat capacities of materials. All conformed to the same
> pattern. Most sets of figures that you meet in everyday life will
> conform to this law, which has now become known as
> Benford's Law or the First Digit Law.
>
> The law is remarkably durable and persistent. When applied to
> data that has dimensions, for example the drainage areas of
> rivers, it still works no matter what measuring units are used. If
> you convert from acres to square kilometres for river drainage,
> the digits still obey the distribution, a phenomenon referred to
> as scale invariance by mathematicians. Scale invariance seems
> to suggest that the law is fundamental.
>
> Explanations as to why everyday sets of numbers should
> behave in this way were slow in coming, and it wasn't until
> 1996 that a satisfactory mathematical explanation was
> provided by Theodore Hill of the Georgia Institute of Technology.
>
> Suppose that most sets of numbers under consideration are not
> infinite, but bounded by an upper limit. As we start to count the
> natural numbers from one upwards looking for those which
> begin with the digit one, by the time we reach 20 the number
> one has a definite lead. It is responsible for 11 of the numbers
> used to count from one to 20 - one, 10 and all the teens. By
> the time we reach 100, things have evened up a bit, but the
> number one gets the upper hand again as we count from 100
> to 200.
>
> True, the other numbers even things up as we reach 1,000, but
> now the next 1,000 numbers will begin with one. Each time the
> number one stretches into the lead, it takes longer for the
> others to catch up. If we call a halt to the proceedings by
> having an upper limit, then the chances are that number one will
> be in the lead.
>
> Recently auditors have begun to apply Benford's Law in an
> attempt to uncover fraud. People inventing numbers for the
> purposes of fraud will not do so in accordance with Benford's law.
>
> In one instance, an insurance worker who had the power to
> sanction claims of up to $500 was caught because a Benford
> analysis revealed an inappropriate spike of numbers beginning
> with the digit four. Claims in the $400 range were too tempting
> for the fraudster, as they maximised his take without having to
> involve his superiors.
>
> The digital analysis technique has been adopted by the world's
> leading auditing and consulting groups. Fraudsters hoping to
> evade detection will find the job difficult, as the second digit
> obeys a similar though not identical distribution to the first.
>
> It would be interesting to see how horseracing results fare in a
> Benford analysis. Every horse in a race has an assigned
> number. Perhaps someone might be able to put to profitable
> use the fact that the probability of the leading digit in a Benford
> distribution being either a one or a two is close to 50 per cent.
>
> gibney@ireland.com Fintan Gibney is an IT consultant with the
> Irish software company SmartForce.
---- Adam@4K-Associates.com.sig double play!
I used to think that assholes became business people but now I realize that the causation works in the other direction. Now when I find a deranged person shouting at someone, I ask "Oh, does he have employees?" If you have high standards and you grow fast you will inevitably find yourself having to tell people how they're not meeting your standards. -- Philip Greenspun
I'm finding this whole process super irritating because I'm coming to realize that people in other lines of work just have no sense of urgency at all. At every job I've ever had, doing the impossible was kind of the whole point. "Oh, everyone knows that that kind of thing takes two years? Interesting, because we need to do it in four months. Now all we have to do is figure out how." Nobody, and I mean nobody, I have dealt with on this project understands that attitude. They go "ha ha ha" and assume I'm joking. -- Jamie Zawinski, 27-Apr-2000, http://www.dnalounge.com/log.html
This archive was generated by hypermail 2b29 : Tue May 02 2000 - 03:03:00 PDT