Does the argument about p-values miss the big problem? One might argue that blaming p-values for our mis-uses of p-values is akin to blaming the messenger for bad news.
How do we use p-values? Prior to publishing, we use p-values as gate keepers. A small p-value indicates a 'significant' and therefore 'real' result, while large p-values indicate that a particular effect is not there or not worth mentioning. Researchers emphasize their results that have small p-values, while ignoring or not mentioning results with large p-values. Often results with large p-values are omitted from a paper. Having at least one small p-value is considered de rigueur to bother submitting a paper, and a paper with no significant results will either be reworked until
some p-value is small (torturing the data until the data speaks) or the paper will be laid to rest without being submitted (the file drawer problem).

It seems to me the problem isn't the p-value. The problem isn't even that a lot of people don't quite understand the underlying logic of what a p-value is. The problem is that p-values are used to determine which results are presented to the broader scientific community and which results are ignored. But when we read a result
in an article in the scientific literature, we typically assume the canonical (but false) idea that the authors wanted to test that specific result or hypothesis, and then reported that specific result or hypothesis.

The truth is that there are quite a few results that authors could report in most studies. Authors choose the results that are actually reported in the paper. And this choosing is closely correlated with the p-value. This is commonly called selection bias.

To over-simplify slightly, there are two ways to get a significant p-value. One way is to have an underlying effect that is strong with a study that has the power to identify that strong effect. Another way is to have an unusually strong result that isn't warranted by the underlying truth. The first way is that there is an actual effect, and the second way is 'something unusual' occurred. Testing lots of hypotheses gets you lots of chances to get a significant result. With each test you may have identified something real, or you may have gotten lucky.

Unfortunately, when researchers get lucky, society gets unlikely. It's a situation where society's utility function and researchers' utility functions may be at odds with each other. Researchers with few exceptions do want to actually identify real effects. However, researchers operate in a milieu that rewards significant results. And especially, journal publishing rewards splashy and unusual results more than steady straightforward routine results or worse, non-results which means not-significant results. There are stories (of real researchers) getting wealthy by marketing modest (and probably lucky) but splashy findings into big businesses.

Scientists test a lot of hypotheses. Some of these tests are due to direct interest as when we assess a treatment's effect on a sample of subjects. Much of it is exploratory, as when we try to find the demographic characteristics that lead to higher values of some outcome. In addition, some tests of hypothesis are not straightforward, as when it requires exploratory analysis to build a model. When we build a complicated model to identify the effects of a given hypothesis, then we are subject to selection bias that occurs when we add model terms that distinguish a treatment's effects and we omit model terms that do not seem to distinguish treatment from control.

The problem is that, due to selection bias, our scientific literature is likely filled with results whose strength has been
over-inflated. Now how over-inflated the typical result is depends on the underlying ground truth that is being explored in the literature. But when we have a large literature, and many researchers looking for many different effects, and few people checking results (it's less prestigious after all!) we're likely to have lots of candidate results that are not all correct but are all treated as real.

There have been a few attempts at validation studies, where researchers have gone back and tried to duplicate a number of past results in the literature. These validation studies don't validate all the past findings. Is this due to the original studies identifying false effects? Or is it because the follow-up study got unlucky and didn't manage to identify an actual effect? Perhaps a combination: the original study over estimated the effect size, and the validation study was under powered due to the original over estimation.

So should we toss out p-values? As a Bayesian, I made my peace with p-values a while ago. Now, two sided p-values are a bit weird and possibly a touch creepy from a Bayesian perspective. However, a one-sided p-value has a simple Bayesian interpretation that is actually useful. Suppose our estimate of a treatment effect is that treatment has a positive effect on the outcome. Then the Bayesian interpretation of a one-sided p-value (the one-side smaller than 0.5) is the probability, given the data, that the treatment effect in this study is actually harmful. It's a useful summary of the
results of an analysis. Its not a sufficient statistic in either the English or statistical senses of the word with regard to how we should interpret the particular result.

It seems to me that the problem is more of selection. We select for strong results. We pay attention to strong results. We really pay attention to unusual and weird results. The popular press picks up unusual results and magnifies their impact immensely, without evaluating whether the reported result is likely to be true or not. Thus we propagate ideas based on something distinct from their likelihood of being true.

So don't shoot the messenger, neither messenger RNA nor messenger pidgeon. Oops too late, we already shot that last one. Consider how we can solve the crisis of scientific selection.

You have finished all your courses. You have passed your written comprehensive exams. Congratulations! What’s next? If you haven’t already (and you should have), you pick an advisor and start to work on your doctoral dissertation. Writing a dissertation and finishing your doctoral degree involves several steps. Two important steps in finishing your PhD are partially bureaucratic in nature: the preliminary oral and the final oral. These may well be the last two exams of your academic career. This blog post is about the preliminary oral exam and the dissertation proposal.

Depending on your university rules and department traditions, the specific steps in preparing a thesis will vary. Here I talk about my university, which is UCLA, and my department, which is biostatistics and this moment in time, which is late 2018. But please realize: the rules, procedures, and customs surrounding prelim oral exams and dissertation proposals have evolved over the decades. They are not fixed in stone, except by university and department written rules which can and do change. You can expect procedures to evolve over time and to vary by committee and most especially by dissertation advisor.

At UCLA, you have an advisor (you can have two, but one is most common) and you will pick a dissertation committee in conjunction with your advisor. You will prepare a dissertation proposal and hand (or email) it out to your dissertation committee members in advance of the exam. Two weeks in advance is a courtesy (we used to say 3 weeks). The committee meets, you present a talk on your proposal. The committee members ask you questions about your proposal and talk which you answer to the best of your ability. The committee supplies feedback to you and your advisor about your proposed dissertation and makes a decision about you passing or failing the exam.

The preliminary oral exam in our department usually is scheduled to take 2 hours. We’re allowed 3 hours by University policy, but 3 hours is a lot of time for faculty these days, and 3 hours is exhausting for students to present and answer questions. And we haven’t noticed any benefit to the extra hour. So we try to restrict the exam to 2 hours. It is mostly your advisor’s responsibility to make sure the exam runs on time, but you may need to help your advisor out with this. If there are a lot of questions, and usually there are, then you won’t finish all your slides. Not finishing all your slides is possibly the norm, not the exception. You should prepare ahead of time various cuts that you can take without harming the narrative of your talk.

Talk to the student affairs officer to schedule a room for the exam. Often the exam takes place in our biostatistics library, but occasionally it can be an outside room depending on scheduling conflicts. Typically the room is scheduled for half an hour before the exam so that you can set up your computer and display equipment. The exam is for 2 hours, then you have the room for a half an hour to dismantle the equipment, though usually it shouldn’t take that long.

The preliminary oral exam is closed, meaning only the student and the dissertation committee members are allowed in the room during the exam. One committee member who is not the chair or co-chair may connect remotely – I really don’t recommend that – but it is allowed.

Once the committee is fully assembled, the exam starts by you stepping out of the room while the committee meets without you. During this time, the chair will provide an assessment of how it has been working with you. The committee will discuss your academic background, your academic progress, what they can expect from you in terms of progress and development. They may provide feedback on your proposal to your chair. The committee may also discuss the weather. This initial meeting may take from 5 to 20 minutes. When the committee is done, someone will call you back into the room. You will now start presenting. After the presentation and questions are finished, you will step out of the room once again while the committee discusses your presentation and provides your advisor with guidance. After the exam you will typically meet with your advisor to discuss the results of the exam and any comments from the committee.

During your presentation, the committee members will typically interrupt with questions. There are many purposes for these questions:

• To clarify meaning, as in a short clarification or asking for a definition;
• To see if you understand what you are saying;
• To see if you can think on your feet and respond to something different;
• To see if you understand or are aware of a particular reference;
• To see if you can extend your work in response to a new idea;
• To see if you can explain what you just said in a different way;
• To see if you can answer a question in the middle of a talk.

Students often make assumptions about what the questions mean or imply, but this is usually a mistake and these assumptions are usually incorrect. Do not assume that a question means that the committee member disagrees with something you said. Nor should you assume that the committee member doesn’t understand you, even if the question starts with “I don’t understand …”. An angry question doesn’t actually mean the faculty member is actually angry with you – it more likely means they didn’t get enough sleep the night before or that that is their style. Far and away best is to take the question at face value and answer as best possible.

It used to be that the committee had 5 members, but in these busy times, it has become impossible to get 5 faculty members into a single room at the same time, so the University has reduced the committee size to 4 faculty members. The rules on who can be on a committee are surprisingly complicated. For UCLA’s rules, see https://grad.ucla.edu/academics/doctoral-studies/minimum-standards-for-…. The general intent is to get enough faculty on the committee to supply additional expertise should the student need it, to provide faculty expertise to be able to confirm that your dissertation will be a new contribution to knowledge, and to insure that the committee members have sufficient seniority to provide sensible guidance. At the same time, there is flexibility to find additional expertise if needed, including potentially going outside the university to find a committee member with needed expertise.

The biostatistics department has placed an additional constraint on dissertation committees: one of the faculty members must have a primary appointment from outside the department. This is intended to mean that someone with subject matter scientific expertise is included in the committee, not that someone from mathematics, statistics or computer science is placed on your committee. Our intent is that you explain to someone who is a scientist, and who is specifically not a statistician, what the tools you will develop in your dissertation are, and why they might potentially benefit the scientist. Explaining to a scientist what your tools are teaches you to explain your statistical tools non-technically and requires that you think about the scientist while you work on your dissertation. It’s not enough to say that you’ve improved the root mean square of some estimator – what good will you do for the scientist, and by extension, society with your dissertation work?

The dissertation proposal is a document that you write that tells the committee where you plan to go with your dissertation. This document can take many forms, and it may range in length from arbitrarily short to arbitrarily long, with 40 to 100 double-spaced pages being pretty common. The dissertation proposal has several purposes, though an individual document may not serve all purposes:

• To show what you know. What you know might be demonstrated by a literature review for example. A long literature review is definitely not required, and is becoming rarer.
• To show that you can handle the thesis topic. For example, by illustrating the results of example calculations that are similar to what needs to be done in the topic.
• To show that you can do research.
  • The easiest way to do this is to actually show some novel model or novel results in the proposal.
  • For example, your advisor might have you start working on writing a first paper and that material would then show up in the dissertation proposal.
  • However, you might demonstrate research competence by demonstrating your awareness of past research and knowledge of currently unanswered questions.
• To show the committee an outline of the future research you intend to undertake in your dissertation. This is the proposal part of the dissertation proposal.
  • This includes any novel research already undertaken that might be given in the proposal.
  • But typically this is unfinished work and is in outline form and shows the committee that you have an idea of where you are going and what you will do.
  • Outline form may mean a paragraph or two on each idea that you propose to execute during your dissertation research.

A very important part of the proposal is where you indicate what is old and what is novel. That is, what old material has already occurred in the literature, and what new material is your own novel work. You will be receiving a PhD because of your novel contributions to biostatistics. If you don’t indicate what is novel and what is old, you cannot expect your committee to understand this distinction. If you don’t indicate what is new, then it becomes up to the committee to figure out what is new, and they might err on the side that everything you said is review. Much easier if you tell them what is new.

In your proposal you should have a section that outlines the planned future research: a “proposal” section. The proposal section will sketch projects that you plan to tackle in your dissertation. I consider it important to have a worked numerical example that shows that you are able to compute with the sort of data and models and methods that you intend to develop in the dissertation. If you’ve submitted your first paper prior to the preliminary oral, you can add an introduction, a non-technical discussion of your paper and proposed additional work, and a proposal section and your proposal is ready for the committee.

Your committee members are expected to read your proposal but might not. If they read the proposal, you can assume they will read or skim the proposal the night before. Thus there is little value in checking in with faculty about issues or comments prior to your talk. Your talk needs to be self-contained, and should not depend on the committee having read the proposal.

The member who is not from biostatistics may well have difficulty reading the mathematical statistical portion of your proposal. So why did the department require you to have a non-statistician scientist on your committee? The reason is that we want you to be able to communicate the value of your statistical research to a non-statistician. Biostatistics has a substantial collaborative aspect to it. Thus, as part of your proposal, you should have a section that explains the value of your work in layman’s terms. This is a courtesy to the outside member of your committee, as well as being important in its own right. Similar, as part of your dissertation, you should have a section or chapter that describes your contributions to biostatistics and science in non-technical language. This section should be completely accurate but not rely on mathematical notation or technical statistical jargon to make its points.

A dissertation can take many forms. A common form that is increasingly popular is to write three separate research papers and then bind them in dissertation format and submit these as the final dissertation. This is not required, and is decided upon primarily by the advisor, with input from the student and possibly the committee. The dissertation is supposed to be publishable, but if one merely writes a dissertation that contains three publishable ideas then it can take a long while to turn the dissertation into the three papers. In contrast, if one writes three papers, then it is quite quick to turn three papers into a dissertation, taking perhaps a few weeks at most, with time mostly spent on formatting your papers into the UCLA dissertation style. For students interested in academia these days, substantial ability to publish must necessarily be demonstrated, so having a good CV out of grad school with a number of publications published or in submission is necessary. The three papers model of a dissertation is required for those students. Similarly, many faculty require the three papers model. I assume this model for the dissertation in the remainder of this discussion.

When writing your proposal, there are a number of technical issues. You may be learning LaTeX, or even if you know LaTeX, you will need to learn new features to format your proposal properly. Similarly, you need to learn bibtex to format your bibliography.

You may not be used to reading technical papers in the statistics literature and you need to start doing this immediately. Those papers can be models for what your dissertation papers will be like. Further, these papers illustrate how to write technical material. Some papers are better written than others, so learn to be critical so that you can learn to write well. Well written technical prose is a signal to the reader that you take your job seriously as an author and it signals that your underlying work may well be worth their time to read. Similarly, formatting your text properly flags to the reader that you take your job of presenting your work seriously and that the underlying work is worth the reader’s time to read. Not formatting your proposal properly signals to your committee that either (a) you don’t take your work seriously, or (b) you don’t understand your tools (LaTeX and English) very well. Or both. And either of those highly correlates with weak or bad statistics. [Bad = wrong, weak = very little new.]

I have read a large number of papers submitted for publication in my lifetime. Poorly written usually (not always, but usually) translates to uninteresting work and it certainly can mean unintelligible. Similarly, in submitting a paper for publication, sloppy formatting is a strong indicator to me that the underlying material is not publishable. Editors of journals have choices of many papers to publish. They don’t mind if they don’t publish the next great paper from you, because they can publish many other people’s next great paper. If you don’t take your work seriously, why should they take your paper seriously? Also, refereeing a statistics paper is hard and if they can take a short cut by recognizing that the paper is poorly written and formatted, they may reject a paper without making a serious determination as to the quality of the underlying work.

The goal of a paper is to communicate new methodology. Similarly the goal of your proposal is to communicate to your committee that you can write a dissertation. The skills needed to write a good proposal will translate to writing good papers and to writing a good dissertation. So take the formatting seriously and take the writing seriously. At the same time, once the preliminary oral exam is over, and assuming you passed, then the proposal is of little interest to anybody. The amount of work in the proposal that you can re-use in the dissertation and in your papers translates to time saved. Hence the advantage of the form where most of the proposal is a start on your first submitted paper. But any time spent on learning to format the proposal is time well spent. And time spent on learning to write technical prose is time well spent. You will spend your life writing technical prose. The better you write, the more useful you will be to your employer, whether you end up self-employed, a professor or go into industry or government.

The preliminary oral exam is a pass-fail exam. The purpose is to confirm that you can do research, that the research topic you have chosen is worth researching, and that you can do the project. The committee will advise your advisor or you on whether you are proposing to do too little or too much, or that the project is too hard for you.

There are many resources on the web about preliminary exams and proposals. The statistics department at UCLA has a nice discussion of the oral exam at http://answers.stat.ucla.edu/groups/answers/wiki/abdb2/Taking_the_Oral_… and a quick check of google finds many resources at UCLA and around the United States.

Good luck!

Suppose you are the Nefarious Large Organization (NLO) and you want to kill lots of random Americans but you do not want your fingerprints on the trigger weapons.

How might you go about doing this? Having a long time frame helps. You're going to need a large political arm that can help with loosening laws to create the conditions you need. You'll need to grow your NLO organization.

A two prong approach can work wonders. You need to get large numbers of weapons into the hands of lots and lots of people.

First, which weapons? Knives don't kill many people, are very personal, and are hard to work with. You might get hurt trying to use a knife, especially if someone fights back. Bombs are difficult to make and have a habit of blowing up their amateur makers as much as blowing up the intended targets. Plus they need to be hidden. There is a military complex that makes bombs, and there is a cottage industry in bomb making in parts of the third world, but in the first world, bombs perhaps aren't optimal. You would need better bomb training and if your NLO started marketing bomb making courses, people might catch on when bombs with your designs started killing lots of people. Guns would be better, as other companies make them. Your NLO doesn't need to make guns, just advocate for their use and acquisition. Although guns take some skill, that skill is easily acquired. Or you can shoot into large crowds where it is hard to miss. But Saturday night specials and revolvers have a limited number of bullets. Having guns with lots of bullets that shoot quickly in the hands of many people is paramount for achieving your goal of killing lots of Americans. Thus the need for automatic firearms.

This is where your NLO political operation becomes paramount. It takes a long time frame and plenty of cash to get politicians to set the stage where you can get these automatic firearms into the hands of lots of people. This may take you decades. Advertising can induce a cachet to owning automatic firearms and after the population of owners gets large enough, network effects will work to spread their availability and ownership.

If you can get enough weapons into enough hands, the 'which people' part will solve itself as there are always some people willing to work with you in your goal of wanting to kill lots of Americans without your fingerprints on the weapons. You won't even need to communicate directly with those people.

Humans do not classify neatly into 'good people' or 'bad people'. We're all good some of the time and we're all bad at least occasionally. The fractions of good/bad vary from person to person and over time within a person. Toss in a rare event (what people call 'bad luck' or 'an accident') and something terrible can happen. You speed, you drink and drive, you walk atop a wall, you clamber up a steep cliff. People can be immature, then they grow up, get a job, get married and settle down. People can be fine, then get depressed or desperate. Your spouse or mother dies and you have no one caring for you, keeping you safe and on an even keel, keeping those mental demons quiet. You suddenly are isolated, alone. You have difficulty in school, get teased a lot.

It's a fiction that we just have to keep the guns out of the hands of the 'bad people'. Truth is that most of our killers are 'good people', legally speaking, right up until they actually start killing people. Medically, it's virtually impossible to distinguish people at risk of doing bad things some time in the future and those who are not. Certainly before they bought or inherited that first gun, before their friend or neighbor or local advertising circular introduced them to that gun, they were fine, legally speaking. They buy that first gun, they still classify as 'good people'. Before something bad happened to them, before they lost their job, before their last parent died, before they were picked on, before they became depressed or suicidal, people were 'good people'.

You the NLO need to get those guns into the hands of millions of people. Some of those people are going to have problems. They become sick. They may get depressed. They get in a car accident. Socially they are isolated, or they become isolated. They don't fit in. They get teased. They feel like they are teased, even if they are not. Mild paranoia may set in. They think strangers and foreigners are going to take what is rightfully theirs. They may not have been raised with a full set of social skills to handle modern society. These people aren't all that common, they're not the majority. Most of us humans are doing fine, are decent people and aren't teased so badly that we think about becoming killers or drug addicts. That's why you need millions of gun owners. Not all of those millions of gun owners are going to have good sense. They may give or sell their guns to some troubled soul. They may encourage some troubled soul to buy guns as a way to self-worth. They themselves may grow to have psychological problems themselves even though they've been fine for decades. Take millions of people, and some of them are just going to be 'off'.

You make things easy, those bad things you want to have happen will happen more often. Make it easy to commit suicide, more people will commit suicide. Make it easier to commit mass murder, more people will commit mass murder. We are not talking about making the average person commit suicide or mass murder. Just those in the extremes. Just loosen the laws enough so that one person in a million has the ability to shoot someone else or to shoot themselves.

Similarly, reduce health care, screw up schools, reduce social services, make lives more miserable, increase access to drugs, do what you can to cause more difficulties for more people. This will provide a lush environment to grow people with the potential to have further problems.

Maybe these troubled individuals are only one in a million. This is the problem and the issue of rare events. This is where you, the NLO can hide behind the problem of rare events. How could we foresee some random, rare event occurring? Who us? How could we have figured out that lone individual in Las Vegas or Parkland or Orlando or Sandy Hook might be that one individual who feels so aggrieved that we wish we could have seen them coming? Gee, not my fault you say.

If an event is one in a million, then you need millions of attempts to get a 'success'. If you're the NLO, you need millions of gun owners, so that your guns end up in the hands of that one person who decides that killing a lot of others is the solution to their problems. Low probability events happen when you have lots of chances. Someone always wins the lottery. Your NLO political arm has been working tirelessly for years. Loosening laws, fighting hard, fighting dirty, making glib arguments, buying off amenable politicians. You can tolerate rare events, because you've gotten your guns into the hands of millions of people. You've got millions of chances to hit the jackpot.

You make things easy, those bad things you want to have happen will happen more often. Make it easy to commit suicide, more people will commit suicide. Make it easier to commit mass murder, more people will commit mass murder. We are not talking about making the average person commit suicide or mass murder. Just those in the extremes. Loosen the laws enough so that one person in a million has the ability to shoot someone or to shoot themselves. Eliminate health care, screw up schools, reduce social services, make lives more miserable, do what you can to cause more difficulties for more people. This will provide a warm environment to grow people with the potential to have further problems. You don't need everyone to want to do your bidding, you just need one person. One person in a million.

Modeling is important and useful. Having killers getting lots of press attention in the popular media is very helpful to your NLO plan. More and more of your automatic rifle owners see that killing lots of people is a really neat way to get some attention. The media is on your side, NLO, because the media will spread around the how-to and the what-to and the fame of these killers.

And when you've done your job right, your NLO fingerprints aren't on the trigger. When the killing is over, there is no obvious link between killers. But you've achieved your purpose. Setting many many small probability events in motion, incubating, waiting to see who cracks next, who decides to shoot next.

As the smoke settles, as survivors decide whether to tear down or rebuild the building, as survivors make memorials and attend funerals, your spokespeople say: don't politicize this, don't make decisions in the heat of the moment, don't do something you'll regret later. Your political arm materials just write themselves, don't they?

Sadly the memorials need to be written too. She wanted to be a scientist. He was everyone's friend. They just wanted to watch a good movie. She was an inspiring teacher. He was a great football coach. He was here on vacation. She was in 1st grade. I don't suppose, NLO, that you'd like to contribute to writing the memorials for the people killed today? How about the people killed yesterday? Your machinations are working NLO, congratulations. Maybe you'll work on the future memorials for those killed tomorrow? We appreciate your assistance.

In executing a classical hypothesis test, a small $p$-value allows us to reject the null hypothesis and declare that the alternative hypothesis is true.

This classical decision requires a leap of faith: if the $p$-value is small, either something unusual occurred or the null hypothesis must be false.

These days we should add a third possibility. That we searched over several models and methods to find a small $p$-value. We need to update the $p$-value oath of decision making to state: Either something unusual happened, we searched to find a small $p$-value or the null hypothesis is false.

Note that being Bayesian doesn't necessarily avoid this problem. Suppose a regression model $Y = X\beta+ \mbox{error}$. Apologies for not defining notation, except that $\beta$ is a $p$-vector with elements $\beta_k$. One way to define a one-sided Bayesian $p$-value is the posterior probability that $\beta_k$ is less than zero. If this probability $P(\beta_k \lt 0 | Y)$ is near 0 or near 1, then we declare "significance". Basically the Bayesian $p$-value tells us how much certainty we have about the sign of $\beta_k$. The usual classical $p$-value is approximately twice the smaller of $P(\beta_k \lt 0 | Y)$ and $P(\beta_k \gt 0 | Y)$. How close the approximation is depends on the relative strength of the prior information to the information in the data, the observed Fisher information. The Bayesian $p$-value is subject to the same maximization by search over models as the classical $p$-value.

Bayesians have an alternative to merely searching over models however. We can do a mixture model (George and McCulloch 1993, JASA; Kuo and Mallick 1998, Sankhyā B) and incorporate all the models that we've searched over into a single model to calculate the $p$-value.

In answer to an old friend's question.

1. Bayesians have more fun.
1. Our conferences are in better places too.
2. It's the model not the estimator.
3. Life's too short to be a frequentist: In an infinite number of replications ...
4. Software works better.
1. Rather surprisingly, Bayesian software is a lot more general than frequentist software.
5. Small sample inference comes standard with most Bayesian model fitting these days.
1. But if you like your inference asymptotic, that's available, just not high on anyone's priority list.
2. We can handle the no-data problem, all the way up to very large problems.
3. Don't need a large enough sample to allow for a bootstrap.
6. Hierarchical random effects models are better fit with Bayesian models and software.
1. If a variance component is small, the natural Bayes model doesn't allow zero as an estimate, while the natural maximum likelihood algorithms do allow zero. If you get a zero estimate, then you're going to get poor estimates of standard errors of fixed effects. [More discussion omitted.]
2. Can handle problems where there are more parameters than data.
7. Logistic regression models fit better with Bayes
1. If there's perfect separation on a particular variable, the maximum likelihood estimate of the coefficient is plus or minus infinity which isn't a good estimate.
2. Bayesian modeling offers (doesn't guarantee it, there's no insurance against stupidity) the opportunity to do the estimation correctly.
3. Same thing if you're trying to estimate a very tiny (or very large) probability. Suppose you observe 20 out of 20 successes on something that you know doesn't have 100% successes.
4. To rephrase a bit: In small samples or with rare events, Bayesian estimates shrink towards sensible point estimates, (if your prior is sensible) thus avoiding the large variance of point estimates.
8. Variance bias trade-off is working in your favor.
9. Frequentists keep reinventing Bayesian methods
1. Shrinkage estimates
2. Empirical Bayes
3. Lasso
4. Penalized likelihood
5. Ridge regression
6. James-Stein estimators
7. Regularization
8. Pittman estimation
9. Integrated likelihood
10. In other words, it's just not possible to analyze complex data structures without Bayesian ideas.
10. Your answers are admissible if you're Bayesian but usually not if you're a frequentist.
1. Admissibility means never having to say you're sorry.
2. Alternatively, admissibility means that someone else can't prove that they can do a better job than you.
3. And if you're a frequentist, someone is clogging our journals with proofs that the latest idiocy is admissible or not.
4. Unless they are clogging it with yet more ways to estimate the smoothing parameter for a nonparametric estimator.
11. Bayesian models are generalizations of classical models. That's what the prior buys you: more models
12. Can handle discrete, categorical, ordered categorical, trees, densities, matrices, missing data and other odd parameter types.
13. Data and parameters are treated on an equal playing field.
14. I would argue that cross-validation works because it approximates Bayesian model selection tools.
15. Bayesian Hypothesis Testing
1. Treats the null and alternative hypotheses on equal terms
2. Can handle two or more than two hypotheses
3. Can handle hypotheses that are
1. Disjoint
2. Nested
3. Overlapping but neither disjoint nor nested
4. Gives you the probability the alternative hypothesis is true.
5. Classical inference can only handle the nested null hypothesis problem.
6. We're all probably misusing p-values anyway.
16. Provides a language for talking about modeling and uncertainty that is missing in classical statistics.
1. And thus provides a language for developing new models for new data sets or scientific problems.
2. Provides a language for thinking about shrinkage estimators and why we want to use them and how to specify the shrinkage.
3. Bayesian statistics permits discussion of the sampling density of the data given the unknown parameters.
4. Unfortunately this is all that frequentist statistics allows you to talk about.
5. Additionally: Bayesians can discuss the distribution of the data unconditional on the parameters.
6. Bayesian statistics also allows you to discuss the distribution of the parameters.
7. You may discuss the distribution of the parameters given the data. This is called the posterior, and is the conclusion of a Bayesian analysis.
8. You can talk about problems that classical statistics can't handle: The probability of nuclear war for example.
17. Novel computing tools -- but you can often use your old tools as well.
18. Bayesian methods allow pooling of information from diverse data sources.
1. Data can come from books, journal articles, older lab data, previous studies, people, experts, the horse's mouth, rats a** or it may have been collected in the traditional form of data.
2. It isn't automatic, but there is language to think about how to do this pooling.
19. Less work.
1. Bayesian inference is via laws of probability, not by some ad hoc procedure that you need to invent for every problem or validate every time you use it.
2. Don't need to figure out an estimator.
3. Once you have a model and data set, the conclusion is a computing problem, not a research problem.
4. Don't need to prove a theorem to show that your posterior is sensible. It is sensible if your assumptions are sensible.
5. Don't need to publish a bunch of papers to figure out sensible answers given a novel problem
6. For example, estimating a series of means $mu_1, mu_2, \ldots$ that you know are ordered $mu_j \le mu_{j+1}$ is a computing problem in Bayesian inference, but was the source of numerous papers in the frequentist literature. Finding a (good) frequentist estimator and finding standard errors and confidence intervals took lots of papers to figure out.
20. Yes, you can still use SAS.
1. Or R or Stata.
21. Can incorporate utility functions, if you have one.
22. Odd bits of other information can be incorporated into the analysis, for example
1. That a particular parameter, usually allowed to be positive or negative, must be positive.
2. That a particular parameter is probably positive, but not guaranteed to be positive.
3. That a given regression coefficient should be close to zero.
4. That group one's mean is larger than group two's mean.
5. That the data comes from a distribution that is not a Poisson, Binomial, Exponential or Normal. For example, the data may be better modeled by a t, gamma.
6. That a collection of parameters come from a distribution that is skewed, or has long tails.
7. Bayesian nonparametrics can allow you to model an unknown density as a non-parametric mixture of normals (or other density). The uncertainty in estimating this distribution is incorporated in making inferences about group means and regression coefficients.
23. Bayesian modeling is about the science.
1. You can calculate the probability that your hypothesis is true.
2. Bayesian modeling asks if this model describes the data, mother nature, the data generating process correctly, or sufficiently correctly.
3. Classical inference is all about the statistician and the algorithm, not the science.
4. In repeated samples, how often (or how accurately) does this algorithm/method/model/inference scheme give the right answer?
5. Classical inference is more about the robustness (in repeated sampling) of the procedure. In that way, it provides robustness results for Bayesian methods.
24. Bayesian methods have had notable successes, to wit:
1. Covariate selection in regression problems
2. Model selection
3. Model mixing
4. And mixture models
5. Missing data
6. Multi-level and hierarchical models
7. Phylogeny

The bottom line: More tools. Faster progress.

Today we have a guest post from a colleague named Mathprof. The pseudonym perhaps is needed as Mathprof's colleagues might not be pleased to read all mathprof's comments. I did some very minor editing, but otherwise the content is Mathprof's.

I asked Mathprof about Alexander Coward. You can read some background here: http://alexandercoward.com/BlowingTheWhistleOnUCBerkeleyMathematics.html

Mathprof's response to me:

Nationwide, mathematics departments, higher education or otherwise, mirror his [Alexander Coward's] interpretation and description of events. He perfectly described my department. Just today, some of my students alerted me that my peers have been complaining that I try too hard and that my approach is hurting students. In their opinion, it is not proper for the teacher to try to teach - it is the student’s responsibility to try to learn. I am, supposedly, doing my students a disservice as I raise their knowledge base through the use of sound, research based, pedagogical practices by letting my students become accustomed to a style of learning they will likely never experience again.

Most mathematicians at large universities are grounded in pedagogical, epistemological, ontological, and methodological paradigms that uphold and maintain the current mathematics education paradigm. There are deep seated beliefs about what is education, who can and should access it, how it should look, and to what end. Although the socialization in this manner of thinking occurs mainly in colleges, it is often introduced in early education, as mathematics undergraduates fill most of the K12 mathematics teacher positions, and they often bring this paradigm with them. This is how the system perpetuates itself and how it constructs continuity between K-12 and higher education.

What I have learned in the few years of reading and writing extensively on this topic is that the form and function of education are completely at odds with one another. This is particularly true in mathematics. If you ask math teachers what they hope to achieve and then observe their method of approach, it is quickly evident that for the majority of teachers, form rules over function. Strategies whose function is to increase learning are aggressively de-emphasized and de-legitimized simply because they clash with the dominant paradigm view of education. This occurs both in practice and in research. Recreating their learning experience in not a means to an end, but an end onto itself. The demonstration of a superior capacity and knowledge in mathematics is the form that must rule over function. For these teachers, learning is a mere externality. Those that are meant to learn math do - everyone else learns their place. There is no maximization of exposure, customization of approach, or an intellectualization of the process. It is complete madness.

I could go on forever but I will stop. I feel bad for the guy [Alexander Coward]. It sounds like he is doing it the right way. That said, welcome to education. Watch out for gravity - you never know which way it will pull tomorrow.

-- Mathprof

How would you advise an undergraduate interested in a PhD to prepare for studying biostats?

More math. You can't be too rich or know too much math. In terms of courses, take more mathematics, take as much as you can. When you get into our biostatistics graduate program, we teach you statistics, so taking more statistics now won't help you in the long run, plus we may have to un-teach something if you learn statistics badly.

In terms of what math courses to take, try to take real analysis. Advanced linear algebra is very helpful. Every part of math is useful somewhere in statistics, though connections may be obscure, or more likely, just not part of some current fad. Numerical analysis and combinatorics are also helpful, and everything else is helpful too. First though is Real Analysis and Advanced linear algebra.

What might I read outside of my courses?

Start picking up books and articles that relate to statistics, math, science, and public health and read them. Lately there have been a number of excellent popular science books that relate to science, statistics and statistical thinking. Anything and everything by Malcolm Gladwell I highly recommend. Other books that come to mind are things like Stephen Jay Gould's essays; The Signal and the Noise by Nate Silver; The Theory That Would Not Die by McGrayne; any of the books by Jared Diamond. A very important book for most anyone: Thinking, Fast and Slow by Kahneman. The Black Swan by Nassim Taleb; How to Lie With Statistics by Darrell Huff.

For someone not yet expert in statistics, books on statistical graphics are directly statistical and will be much more accessible than a technical book. Books on statistical graphics will directly make you a better statistician, now. These teach you both to look at data and how to look at data. There's a set of 4 books by Edward Tufte. See http://www.edwardtufte.com/tufte/ . Get all four, I recommend hardcover over paperback, and definitely I wouldn't get the e-books. Read all four! These are not always practical books, but they inspire us to do our best and to be creative in our statistical and graphical analyses.

Read Visualizing Data by William Cleveland (A+ wonderful book). Additional graphics books include Graphical Methods for Data Analysis by Chambers Cleveland Kleiner and Tukey if you can find a copy. A more statistical book that will help instill the proper attitude about data is Exploratory Data Analysis by Tukey.

Read anything else you can find to read. Read widely and diversely. As you get stronger in math and statistics, change the level of the books. Start exploring the literature. Dive into one area and read as much as you can. Then find another area and check it out.

Can I depend on the department to teach me everything I need to be a good statistician?

Of course not. Active learning is paramount. No graduate department will teach you everything. All departments teach a core set of material, and it is up to students to supplement that core with additional material. How you supplement that core determines what kind of statistician you can be, how far you can go. Some people might supplement their core material with an in depth study of non-parametrics; others with Bayesian methods, statistical computing, spatial data analysis or clinical trials. I supplemented my graduate education with statistical graphics, Bayesian methods, statistical computing, regression methods, hierarchical models, semi-parametric modeling, foundations and longitudinal data analysis. The semi-parametric modeling, graphics and computing mostly came from books. The longitudinal data analysis came from a mix of books and journal articles. Bayesian methods and hierarchical models I learned mostly from journal articles. Foundations came from talking to people and listening to seminars, as well as from journal articles and books. I also tried to learn additional mathematical statistics using various texts, but wasn't very successful; similarly with optimal design.

How you supplement your education depends on your interests and may help you refine your interests. I found I wasn't that interested in time series, survey sampling, stochastic processes or optimal design. If you're interested in working with a particular professor, you're going to need to supplement with books in her/his area, and you're going to need to read that professor's research papers to see what you're going to be getting into.

What programming language(s) should I learn?

R is growing fast and may take over, sort of like kudzu, so it is well worth your time to become expert in it. Definitely learn/use R Studio. Some folks make a living just off their R expertise. A lot more make a living off their SAS expertise. But I bet the R people are having a lot more fun. The rest of this is what I garner from others, not from direct knowledge. If you want less of a statistics specialty language and to be closer to the computer end of things, C++ or JAVA are extremely popular (you should figure out why). Python seems to be coming on very strong. So maybe R and Python? Depends on what you like. Learn something about algorithms and something about modern computer programming interfaces. And a little HTML.

Go learn latex now. Become at least a partial expert. Knowing latex before you come in to grad school is very helpful.

Seems that people with severe mental illness are able to easily purchase guns in this country. Seems like people (whether actually mentally ill or not) who are upset and angry and want to hurt someone also can purchase a gun easily.

Often we hear that gun advocates think that more guns is the solution. This seems to revolve on a world view where most all of us have our hands on our guns, all of us with guns are ready to take appropriate defensive action, all of us with guns can read a scene perfectly and all of us will come to a correct decision at all times and every time about how to react.

Hypothesize many many people walking and driving around every day carrying their gun with them. Suppose (just for supposes sake) that half the people wandering around have a gun within reach. Think about what is required here. No one with a gun should be angry. No one carrying a gun should have hormones raging out of control. No one carrying a gun has mental illness. No one carrying a gun had too little sleep last night. No one carrying a gun has depression, schizophrenia, mania or delusions. No one carrying a gun is temporarily irrational. Nothing bad happens to anyone carrying a gun that gets them mentally out of joint: a bump in the crowd, loud truck sounds, a boss with an unreasonable demand just before quitting time, missing the green light, a mentally ill homeless person smelling bad standing on the street corner asking for handouts.

Have you ever done something stupid that you later regretted, even though you didn't suffer consequences? Would having access to a gun made that situation better? Think of all the stupid things everyone else has ever done. Ever seen someone in a fender bender get out of their car and start screaming at the other person, no matter who was at fault?

Those of us in academia worry a lot about our students, both generally and specifically. Every now and again, someone with access to a gun or guns comes to campus or to the area near campus with intent, sometimes to shoot a specific person such as an adviser, colleague, competitor or department chair. Sometimes they come to campus or the area near campus with the intent to kill just any random person they can find.

Having a lot of people wandering around with guns, I now have to worry about the sanity and training and general level-headedness of each and every one of them. And we can't tolerate a situation where any one of these gun toting people anywhere in the country are temporarily off-balance, temporarily ill-tempered, temporarily irritated, or temporarily over-stressed.

In academia, we do put people under stress. There's paperwork and bureaucracy, bad food and traffic. We give tests. Sometimes hard ones. We have deadlines. We demand a lot from our students. We teach, and we expect them to learn. We deliver bad news: you didn't get the grade you hoped for, or the grade you thought you deserved or thought you needed. You didn't get admitted to the school, the major, the program, graduate school. All this puts stress on an individual. Some people cope fine, some learn coping mechanisms, some cope sufficiently, some grow as a person and rise to the occasion. Little of this is done without stress.

What about the stresses of interpersonal relationships? Students at college are meeting many new people, from many different walks of life and this can be a fresh challenge. Students are away from their comfort zone for the first, or if in grad school, second time; many students travel and some travel great distances to attend college. One develops boy/girl friends, close friends and people you can't stand, though they share a bathroom with you. Sometimes friendships fall apart, sometimes they go lax. A support group can grow and it can shrink suddenly at holiday time or at the end of the school year. People get isolated, sometimes for long periods of calendar time, some for shorter periods. All of us are isolated for hours in a given day.

Some students press themselves very hard. Some students expect more from themselves than they are able to deliver. Some have expectations out of sync with their abilities and interests. And just about everyone learns how to adjust to reality when reality bites back. People naturally get stressed in life and in academia. And this pushes some folks a bit far, and for a while they can break, until they mend and come back, hopefully stronger, ready to try new things. Or they change their goals and their objectives, and move to a different sphere in their lives.

And people less capable of handling limits, instructions, social interactions, deadlines are more likely to be stressed by these situations.

And into this vital, vibrant personal and academic stew, full of growth and challenge, despair, growth and success, you want to mix lethal weapons too?

I suspect that people who want freely available lethal gun weapons are also those who seem to think that illness stops at the neck, that mental illness doesn't exist, or doesn't happen. Mental illness is something that can strike most anybody. It may not strike you. But you'll want to be insured for it, just in case. Someone can be healthy, and mentally well balanced one day and go buy a gun. And pass any check on mental illness. And that person, at that time, I'm not worried about them going out and shooting someone. But that person can live for another 50 years with that gun. And in all that time, you want to guarantee me that they will never be stressed? Never depressed or despondent? Never make a stupid decision? Never be trod upon by life? By life, by a boss, by a spouse, by bureaucracy, by a neighbor, by politics, by money worries? By kids or parents? Never get in a bad way? Never suicidal, never angry, never unhappy, never mad, never irate? Never mentally ill? Never get Alzheimer's?

You want to assure me that this healthy, mentally stable person will never mis-use that gun for the rest of their life?

And their spouse, their kids, their parents, and their kid's friends will never mis-use that gun for the rest of their lives as well? And everyone that might accidentally or purposefully gain access to that gun?

What training is required to properly use a gun? Cleaning, handling, safe storage, target practice, accuracy, reading a scene, what else? Do you want to assure me that everyone who might ever come into contact with that gun will have the proper training to use that gun? That they remember their training? That they own a gun safe? That their weapons are stored in that gun safe? That the combo to the safe isn't written on a piece of paper in the kitchen drawer with the spare car keys?

Now multiply this by the estimated 100,000,000 people in the United States who own a gun. I don't think people understand what happens when you multiply a small problem, a small probability of a bad outcome by 100 million.

You want to guarantee me that all 100,000,000 people are not having such a bad day that they won't mis-use that gun? You want to guarantee me that all 100,000,000 people know how to use a gun properly, that they'll safely store it away from their kids and any other kids? You want to guarantee me that all 100,000,000 people won't have their house burgled and the gun stolen where it disappears into the underground economy? Not one person in those 100,000,000 won't get furious at someone or something in the next week?

You want to guarantee me that all 100,000,000 people won't commit suicide, tomorrow, this year, sometime in the next fifty years? In any given day, 58 of those 100,000,000 nice US citizens commit suicide using a gun. About 30 people use a gun to kill someone else each day. In the US.

So, can you guarantee me that out of those 100 million people, that all 100 million of them are going to behave properly? Right now, about 88 of them per day on average are killing themselves or someone else. But that adds up to quite a lot of dead Americans by December 31st.

Have a nice day. And please don't shoot me.

25 helpful pieces of advice.

1. Comportment:
1. Walk like you're walking away from an explosion in a Hollywood movie.
2. Tuck your chin in, tilt your head down and look at people from out of the top of your eyes.
3. Squint.
2. Lecturing:
1. Show up late, then run over time because there is so much to cover.
2. Bring to your lectures more hardware, dry erase markers, computers, tablets,
   cell phones and piles of paper and printed references than any one else.
3. Consider lecturing in Esperanto so that your international students can learn as much as your native students.
4. Liberally sprinkle your lectures with Fisher quotes in Latin and French.
3. Questions:
1. Take 5 minutes to respond to every question.
2. Always ask if there are any questions, but don't wait for people to raise their hands.
3. You have several options for answering questions:
1. Give a lot of fake life philosophy type advice, but don't actually answer the question.
2. Point out that you'll get to that answer later in the lecture. It doesn't matter if you actually do get to it later.
3. Explain how semantic deconstruction enables a pithier answer. Never give the pithier answer.
4. Give homeworks that are impossible to answer correctly, but don't grade them. Give everyone 100% but only after the quarter ends.
5. Do not follow the syllabus.
1. The syllabus should cover everything from Moby Dick to Playfair to Rise and Fall of the Roman Empire to the complete first edition of F. N. David's first book to John Graunt's tables of mortalities and a summary of Biometrika, but only through 1900.
2. Do not update your syllabus. The syllabus your mentor, who retired the year after you joined the faculty, first wrote during world war II should suffice. He worked hard enough on it after all.
3. If there are any pre-med students in the class, make sure you cover all the material on the MCAT.
4. If there are any pre-law students you need not cover the material on the LSAT as there are currently enough seats in first year law classes for all applicants in the United States.
6. Explain at length that the class really should satisfy distribution requirements in a different area.
7. Direct everyone to read your blog where you post:
1. Frequent links to XKCD and phdcomics.
2. On semantic deconstruction and quantum mechanics.
8. Teach your students how to do a better job of simulating real data.
9. Explain the grading policy in detail, but don't follow it.
10. Orthogonal decomposition theorem. Homeworks, lectures and tests should form an orthogonal basis in knowledge space.
11. Jokes. Humor always makes difficult material more interesting. If you don't know any statistics jokes, here are some that you can use, preferably without attribution.
1. Q: How many statisticians does it take to screw in a lightbulb? A: It depends on your model.
2. Q: What's purple and commutes? An Abelian Grape. (Yeah, well).
3. An applied statistician, an econometrician, a theoretical statistician and a psychometrician walk into the faculty bar on campus and don't talk to each other. Bose-Einstein statistics at work.
4. Q: Why don't theoretical statisticians play hide and seek? A: Because no one will look for them.
5. Q: How many econometricians does it take to make chocolate chip cookies? A: Ten. One to stir the batter and nine to peel the M&Ms.
6. A statistician came home and found his house burned to the ground. When he asked what happened, the police told him "Well, apparently the chair of the math department came to your house, and ...".
   
   The statistician's eyes lit up and he interrupted excitedly, "The chair? Of the math department? Came to my house?"
7. Q: Why do statistics departments put questions on asymptotics on comprehensive exams? A: Otherwise there would be no use for asymptotics at all.
8. Theoretical statisticians do it better, but only asymptotically. And in the long run, ... well, you know. But don't say anything, it makes them happier.
12. Prove everything:
1. Proofs are clearer when you use measure theory, even in an introductory class for biologists.
2. And martingales allow for more efficient proofs. After all, life is a martingale. Biologists need to learn to appreciate that.
3. Entropy means it really doesn't matter. Physicists should appreciate that.
4. Note that everything in class has been proved previously in Doob (1953) or in one of Herman Rubin's Annals of Mathematical Statistics papers from the 1950's.
5. Or was it the 1945 paper?
13. Develop the theory of minimal length confidence intervals for the mean of a normal when n=1 and the mean and variance are both unknown.
14. When students are confused, coding theory (Daleks and Ood 1985) provides an alternative way to derive most statistical models. Ontological science should be relegated to discussion sessions where the TA can handle the presentation.
15. Refer every questioner to Fisher's original publications on the subject for more more information.
16. No software need be installed the first two weeks of computer lab. Never use the same software package two weeks in a row.
17. Use data sets from your own collaborative papers as examples, with results given in class that exactly contradict what was published in the literature.
18. Due dates can be given in Julian date (in ISO-8601 format of course) during the first half of the semester and in Aztec calendar form for the second half. Dates outside the semester and during finals week should use Ptolemaic and Carbon dates.
19. Office hours: check the course schedule to maximize the number of conflicts of office hours with other courses.
20. Cancel class on leap days, during full and new moons, the 13th of every month, for faculty meetings, special seminars and Tuesday and Wednesday of Thanksgiving, Veteran's day and memorial day weeks. Go to at least two international conferences each quarter.
21. Announce every Friday that class is canceled for Saturday and Sunday.
22. Take attendance, but never get past the M's.
23. Always misspell your email address. If possible, delete your email address (there is so much spam after all) and acquire a new one after posting your syllabus to the web site.
24. Cover ethics, but don't cite your sources.
25. Per School of Medicine policy, you are required to hold several lectures each semester in rooms other than the scheduled room. Lecture days and room numbers are subject to change even after the lecture has started.
26. Play classical music during lectures and industrial grunge during midquarters. For a special treat (and personal favorite) arrange for a live performance of Karlheinz Stockhausen's Helicopter String Quartet during your final.
27. Love your students. Unless that's against school policy, in which case announce that your love is strictly Platonic. Unless that might be misinterpreted, in which case explain the 7 different forms of love, (Eros also known as sorE backwards, Philia, Ludic, Aghast, ACDC, Programa, Flotus, Read-Write, FIFO and GIGO) and after you get them all straight, the first class should be over and you still won't have covered the syllabus and course information sheets. Bring lots of paper towels when you cover GIGO. (For advanced classes: discuss FIFI and FISTR (First In, Screw The Rest) instead of FIFO.)
28. Finally, after you've mastered all that. Respect your students, love the material, and enjoy yourself.

Next week: Continued third order directed non-Riemannian fractional fast Fourier stochastic differential particle separating longitudinal Latin hyper-active swarm graphs in British bus queueing theory, and practice. Lecturer: Thorin "Missing Totally at Random" Oakenshield.

Someone asked this recently, so I am posting a response here.

For as long as I've been in the statistics/biostatistics profession, students haven't had difficulties finding jobs after graduation. There aren't enough biostat and stat MS and PhD grads to come close to satisfying current demand. Quite a number of companies have more than 100 PhD (bio)stat grads and would be willing to hire as many as they could find, if they could find more. Even if one industry might decrease their demand for our grads, a couple more industries crop up, needing our students ever more badly. Currently rapidly growing are the internet marketing companies (Amazon, Google) and the social-media companies (Facebook, Twitter). News organizations are beginning to hire some statisticians. And any company large enough to need one statistician is likely to need five more in short order. Lately HMO, health care and hospital type organizations have been increasing their share of our students. The traditional employers like Biotech and Biopharm have not slacked off their hiring. And more Biotech and Biopharm companies seem to be created every week. Lets not forget public policy, county, state and federal health departments, survey companies (political, marketing), JD Powers, Consumers Reports, and so on. Did I mention academia? Academia is still growing for statisticians too.

Outsourcing would have to go to people somewhere. Presumably the fear is that the jobs might go to China and India, having the largest populations out there. Chinese undergrads very much want to come here, apparently because there are many more jobs and much better jobs here than in China. I've heard that it pays US companies to outsource statistics but I find it very difficult to believe there are sufficient even partially trained people any where on the planet to be able to outsource many biostat jobs. And anyone you out-source to will need to be managed by someone who knows more statistics than they do. India and China are not yet training many statisticians, though both train some really great people.

For decades, many people with only basic statistical skills have found jobs as statisticians because there are so few statisticians. UCLA has thousands of researchers, most of whom could use a statistician to help with their research. There are now maybe 50 PhD statisticians, (not all faculty) on campus, and we don't begin to make a dent in the demand. Thus there are psychometricians, econometricians, quantitative social, educational and political scientists, computer scientists and many others who operate as statisticians because they can run software and fit basic and some complex models. The outside world isn't much different.

There has always been a good demand for statisticians, and the demand should be increasing as society is increasingly valuing the tools we understand and the services we supply.