Distribution of Sudoku Solution Times

Websudoku.com has a density plot of solution times. If you don't know what sudoku is, then you have a life, so not to worry. On the right is a copy of their plot of solution times for evil difficulty puzzles. The height of the density crushes down to the y-axis as the time gets small, suggesting a very small left tail.  On the right, the height of the density goes to zero very very slowly as the solution time gets large, suggesting a very large right tail. The density reminds me of an inverse gamma density, and I realized I had probably never used an inverse gamma density to model real data before. 

I measured two points on the plot, mode, and the point to the right where the density crosses the bottom grey grid line, I got (.21, 1.46) for the mode and (.63, .18) on the right. The right hand side of the y-axis at (90,0) is measured to be (0,2.21) and the top of the x-axis is measured to be (0,1.47).  From this info, I fit gamma (red) and inverse gamma (green) densities which are plotted on the left. 

The gamma does not look as much like the density on the right as the inverse gamma density does. The gamma goes gently to zero and has positive probability at all points near zero, while the inverse gamma doesn't appear to have any probability mass below a certain point. The gamma right tail area goes to zero sooner than the inverse gamma, and the inverse gamma matches the solution times distribution a touch better on the right tail as well. I therefore declare the inverse gamma density the winner in this bake-off. Checking left tail areas, and subject to the usual high percentage of mistakes that I normally make, I checked left tail areas for 2 min 50 sec and 6 min 55 sec, which according to websudoku, come in as top 1% and top 14% respectively. I got respectively 2.2% and 20% from the gamma, and .012%  and 11% from the inverse gamma, which matches websudoku's numbers perhaps a touch better, though hardly perfectly.

A real maven would have digitized the sudoku curve directly and copied it into the R plot. And one could consider other distributions like the log normal or the inverse Gaussian. Readers? 

 

Happy Christmas for all, and for all a good-night! Part 2

This blog post is asking permission to update.  

Got this wrong the first time. Apologies.    

\documentclass[12pt]{article}

\usepackage{times}
\usepackage{relsize}
\usepackage{amsmath}

\title{Happy Christmas for all, and for all a good-night! }

\author{Clement Clark Moore}

\date{December 25, 1822}

\begin{document}
\maketitle

\begin{align*}
\mathlarger{\Gamma} \left( \left( \left( \mbox{Merry}, \mbox{ Happy} \right) \otimes \left( \mbox{Chistmas}, \mbox{ Holidays}, \mbox{ New Year} \right)^T \right)_{(1,2)} \mathlarger{\forall}, \; \mathlarger{\wedge} \; \mathlarger{\forall} \; \mbox{a} \; \mbox{good} - \mbox{night} + \mathlarger{1} \right) 
\end{align*}

where the argument of the Gamma function is a positive integer. With apologies to Clement Moore.  

\end{document}

Brief Review of Malcolm Gladwell's David and Goliath

The sub-title is Underdogs, Misfits, and the Art of Battling Giants. I think D&G (David and Goliath, not Dolce 'n' Garbanzo) doesn't quite match Gladwell's quartet of grand slams of Outliers, Blink, The Tipping Point and What the Dog Saw. Blink and David and Goliath are going to have something in common: Gladwell is going to be disappointed, either in his readers, or in his writing because people are going to come away with the wrong message. In Blink, Gladwell thought his message was that we are very likely to draw the wrong conclusions in the first few seconds, when we're judging people/things/situations. And I think he was disappointed or surprised (this is from me remembering Gladwell's comments at UCLA a year ago, Oct 24, 2012 in the IMED seminar series), that many peoples' takeaway from Blink was that we judge really well in the first few seconds, in that first blink of an eye. Now, partly, this may be a 6 of one, glass half full in the other situation. Blink I thought made the case initially that we can judge some situations very well very quickly, and that additional time doesn't help the judgment. Then Gladwell went over a number of situations where people made terrible errors in those first few seconds. David and Goliath has a similar two (or maybe more?) part theme. He makes the case that David, playing by David's rules in the David and Goliath encounter, actually had a very serious advantage over Goliath. Now had David fought according to Goliath's rules, Goliath would have had a substantial advantage.  As we know, David had his choice of rules under which to operate and won, fairly easily. Those of us without David's skills or imagination would have fought essentially on Goliath's terms, and we would have lost. Perhaps this was rock paper scissors (a point Gladwell makes in the book), but not a version of the game where both sides show their choices simultaneously; Goliath was chosen first; only then David volunteered. 

In other parts of the book Gladwell makes a case that people with what are considered substantial cognitive deficits are or can be extremely successful in our society. For example, a ridiculous number of presidents spent much of childhood without two parents, including two of our recent greater presidents, Clinton and Obama. He gives examples of people who learned to read late, or almost not at all, or that huge numbers of entrepreneurs have dyslexia. Two of our other recent presidents, Bush the Herbert Walker and Bush the just Walker have been identified as possibly/probably being dyslexic. Neil Bush, George W.'s brother and HW's son was actually diagnosed as dyslexic. As dyslexia tends to run in families, this might be considered evidence about W and HW. George W. has denied being dyslexic.  

Periodically in the book, Gladwell reminds us, "that there are a remarkable number of dyslexics in prison" or that every dyslexic that Gladwell asked has said absolutely that they wouldn't wish dyslexia on their own children. But over all the book seems to stress the advantage of being the underdog, or challenged, and the limits of power. An important point is that often we don't judge situations correctly: competitors may not be as disadvantaged or unequal as we think. Someone can be lucky. The rich may ruin their children by spoiling them. Someone may not play by the rules. But, often we do judge the situation correctly, the superior force overwhelms the inferior force, rich people produced children who in turn do extremely well. Poor people who receive lousy educations end up working dead end, low paying jobs or in jail. 

Another popular bestseller, The Black Swan, by Nassim Nicholas Taleb, makes a related point, that we often mis-under-estimate very small or very large probabilities. Taleb has made a specialty of identifying these situations and betting in the stock market to take advantage of situations where traders are underestimating the chance of a disaster.  When the disaster happens, Taleb makes a fortune. But he loses a little bit on normal days. The fortunes overwhelm the normal small losses.  Supposedly. 

I highly recommend Gladwell's first four books. If after reading those, you are game for more, by all means read D&G. It's still Gladwell, and the writing is still vintage Gladwell. But be prepared to have to process the information, and come to a more nuanced conclusion that the obvious but incorrect conclusion. 

This topic deserves further discussion, I'll follow-up in a further blog post. 

Happy Christmas for all, and for all a good-night!

\documentclass[12pt]{article}

\usepackage{times}
\usepackage{relsize}
\usepackage{amsmath}

\title{Happy Christmas for all, and for all a good-night! }

\author{Clement Clark Moore}

\date{December 25, 1822}

\begin{document}

\maketitle

\begin{align*}
\begin{pmatrix}
{\left( \left( \mbox{Merry}, \mbox{ Happy} \right) \otimes \left( \mbox{Chistmas}, \mbox{ Holidays}, \mbox{ New Year} \right)^T \right)_{(1,2)} \mathlarger{\forall}, \; \mathlarger{\wedge} \; \mathlarger{\forall} \; \mbox{a} \; \mbox{good} - \mbox{night} }
\\
\mathlarger{1}
\end{pmatrix}
\end{align*}

\end{document}

Research as a Job

One of my favorite sayings:

To work, perchance to dream.  

with apologies to Shakespeare.  Research, perchance one of the best jobs ever.  And no perchance about it, to research is to dream.  

Filed Under

On Getting Stuff Accomplished: Doodle Alley

Doodle Alley, subtitled Comics that Nourish, by Stephen McCranie is a comic_book/web_site/series_of_essays that is an inspirational instructional web_book/how_to/co[s]mic_essay on creativity and getting things done. It's written around his own experiences learning to draw, develop a comics website and generally getting things done. Particularly worth reading if you are having trouble of your own getting things accomplished. You can read most of the essays on the web. With help from kickstarter, he has recently published the comics in book form (5$) with three additional essays available from his online store. I found it easy to agree with and to empathize with his suggestions and discusions; his suggestions match my own experiences. The book is called Brick by Brick. From the image, it looks like his book might be called Brick by Brick Table of Contents, but it's not.  smiley 

Resistance Training Study

I saw a study on resistance training, and it came with a free evaluation. I wanted to know how many ohms I was and how to make it higher or lower, so I tried to enroll. Might prove handy in case of an electrical storm out on the golf course.  Unfortunately they wanted smokers and wanted to help them quit. I went out and bought some cigarettes, but I couldn't stand to smoke them. So I tried a cigar, figuring I just had to keep the smoke going until my clothes were saturated, but I couldn't manage to keep the cigar lit. Won't be learning about my resistance. Fortunately, after I do a load of clothes, the smell will be gone.

Diet Study 1

I'm interested in losing weight; read an ad in the stairwell for a diet study.  They want people with a BMI of at least 30.  I'm at 28 or so.  So this Christmas I'm eating all the chocolate ice cream I can manage.  Plan to hit 30 in the middle of next week, no problem.  

My Advisor and Me

My thesis advisor was Dennis Cook.  Was back visiting the U of Minnesota this Fall and took the opportunity to get a jointly authored photograph.

Dennis has made major [fundamental] contributions to regression diagnostics, regression modeling, model checking, statistical graphics, normal theory nonlinear regression modeling, graphical regression, envelope models and a bunch of additional topics I've omitted.  He has just taken over as director of the school of statistics.

[Photograph by anon U of M Stat grad student.]

Filed Under

The Purpose of Blog

At least, the purpose of this blog.

Advice and speculation on most any related subject. Many questions and occasional answers. Send an email if you like!

There will be commentary. However, dear reader, your help is solicited immediately, this minute. From the very humble beginning, even before the first spirit attends its chore this night. Commentary is entirely your responsibility. Be brave. Be prolific. Contribute. Your community awaits.

With grateful regards for reading this, I remain your one true servant.

Robert Weiss

Filed Under
Subscribe to