Biographical Memoirs: Volume 82 (2003)

March 19, 1910–July 16, 1981

BY SHELEMYAHU ZACKS

JACOB WOLFOWITZ, A GIANT among the founders of modern statistics, will always be remembered for his originality, deep thinking, clear mind, excellence in teaching, and vast contributions to statistical and information sciences. I met Wolfowitz for the first time in 1957, when he spent a sabbatical year at the Technion, Israel Institute of Technology. I was at the time a graduate student and a statistician at the building research station of the Technion. I had read papers of Wald and Wolfowitz before, and for me the meeting with Wolfowitz was a great opportunity to associate with a great scholar who was very kind to me and most helpful. I took his class at the Technion on statistical decision theory. Outside the classroom we used to spend time together over a cup of coffee or in his office discussing statistical problems. He gave me a lot of his time, as though I was his student. His advice on the correct approach to the theory of statistics accompanied my development as statistician for many years to come. Later we kept in touch, mostly by correspondence and in meetings of the Institute of Mathematical Statistics. I saw him the last time in his office at the University of Southern Florida in Tampa, where he spent the last years

of his life. I regret that I did not have the opportunity to associate closer with this great man.

Jacob Wolfowitz was born in Poland on March 19, 1910. He immigrated to the United States with his parents, Samuel and Helen (Pearlman) in 1920. Jacob Wolfowitz got his education in New York City. Despite the depression he had the opportunity to get a college education in the City College of New York, and received his B.S. degree in 1931. The depression was then at its depth and jobs were scarce. Wolfowitz succeeded in becoming a high-school mathematics teacher and continued to work as a teacher until 1942. I was told that it was very difficult, especially for an immigrant candidate, to obtain a job as a high-school teacher in those years. People had to take very difficult certification exams. Wolfowitz achieved first place among hundreds of contestants. During these years he also studied mathematics as a part-time graduate student at New York University, from which he received his Ph.D. degree in mathematics in 1942. In 1934 Jack Wolfowitz married Lillian Dundes, who became his cherished companion for the rest of his life. Their children, Laura Mary and Paul Dundes, were born in 1941 and 1943. Laura is a biologist living in Israel with her family, and Paul is a political scientist serving as deputy secretary of defense in Washington, D.C.

In 1938 Wolfowitz met Abraham Wald, with whom he collaborated till the tragic death of Wald in 1950. The collaboration of Wald and Wolfowitz produced some of the most important results in theoretical statistics. Wald was a professor of statistics at Columbia University. During World War II Wolfowitz joined the Statistical Research Group at Columbia University, where the group worked for the war effort. The Wald sequential probability ratio test was developed there at that time. In 1945 Wolfowitz moved to the University of North Carolina at Chapel Hill as associate

professor but stayed only one year. In 1946 he joined the faculty of the Statistics Department at Columbia and worked there till 1951. In 1951 he joined the Mathematics Department at Cornell University as professor and stayed until 1970. In 1970 Wolfowitz moved to the University of Illinois as professor of mathematics. Upon his retirement from the University of Illinois in 1978 he moved to Tampa, Florida, where he accepted the position of distinguished professor at the University of South Florida. Professor Wolfowitz died of a heart attack in Tampa on July 16, 1981.

Professor Wolfowitz spent sabbatical years at various places. As mentioned earlier, in 1957–58 he was at the Technion, Haifa, Israel. In 1967 he visited both the Technion and the University of Paris. In 1969 he visited the University of Heidelberg. In 1966–67 he was a fellow of the Guggenheim Foundation. In 1975 he received an honorary doctorate from the Technion. Professor Wolfowitz was elected to membership in the National Academy of Sciences in 1974. He was also a fellow of the American Academy of Arts and Sciences, Institute of Mathematical Statistics, Econometric Society, and the International Statistical Institute. The following are some of the celebrated meetings in which Wolfowitz delivered honorary lectures: International Congress of Mathematics, American Mathematical Society, German Mathematical Society, and All-Soviet Congress of Mathematics. He gave the Rietz lecture and the Wald lecture at the Institute of Mathematical Statistics meetings and the Shannon lecture at the Institute of Electrical and Electronics Engineers. Professor Wolfowitz also served one term as president of the Institute of Mathematical Statistics.

Up to the sudden death of Abraham Wald (a plane crash while visiting India) in 1950, Wolfowitz collaborated in research mainly with Wald. Starting in 1952 he collaborated with professors Arye Dvoretzky of Hebrew University, Jack

Kiefer of Cornell and University of California at Berkeley, and Lionel Weiss of Cornell University. In the 1960s and 1970s Wolfowitz collaborated with R.Ahlswede in his research on coding theory. Lionel Weiss wrote an excellent summary of the research contributions of Wolfowitz, which can be found in the volume Leading Personalities in Statistical Sciences.¹ Another more comprehensive summary is given in the collection of Wolfowitz’s papers compiled by Jack Kiefer.² Wolfowitz contributed in his research to the following areas of statistical theory: nonparametric inference, sequential analysis, statistical decision theory, asymptotic statistical theory, maximum probability estimators, design of experiments, probability theory, queuing and inventory theory, and information theory. I will give here a short nontechnical summary of these accomplishments.

Nonparametric statistical inference is an area of estimation or testing hypotheses that does not assume a particular functional form of the distribution of the observed random variables. The nonparametric procedures are also called distribution free. A nonparametric estimator of the distribution function (cdf), F(x), of a random variable, based on n iid random variables X_1, X₂,…, X_n is, for example, the empirical distribution function F_n(x)=Σ_iI(X_i<=x)/n. Procedures based on linear functions of the ranks of the observations within the samples are another familiar type of distribution-free, or nonparametric, procedure. A considerable number of papers published by Wolfowitz (some are with Wald or with Kiefer) discuss various properties of nonparametric procedures. They deal with confidence intervals for the cdf F(x) (1939); asymptotic minimaxity of the empirical cdf (with Dvoretzky and Kiefer, 1956; with Kiefer, 1959, 1976); convergence of the empirical distribution (1960); run tests (1943, 1944); and permutation tests (1944). The collaborations with Kiefer in this general area

concentrated on procedures that are of the first of the two general types described above. They include the first general description of the so-called nonparametric maximum likelihood procedure that has become a popular modern technique. Wolfowitz’s famous paper (1952) generalizing the Robbins-Monro stochastic approximation procedure also belongs to the nonparametric domain, as well as the sequential domain discussed below.

Sequential analysis is a branch of statistical experimentation in which observations are taken sequentially, one at a time or in groups. After each observation a decision is made based on all previous results whether to continue sampling or stop. At termination an inference is made, for example, an estimate or hypothesis test, concerning the distribution of the observed random variables or some parameter(s) or functional(s) of it. Wald and Wolfowitz were the pioneers of modern sequential analysis. Wald developed the widely applied sequential probability ratio test (SPRT). Wolfowitz collaborated with Wald in proving the optimality of the procedure. As written by Kiefer in the introduction to the Selected Papers,² the proof of Wolfowitz and Wald (1945, 1966) “is one of the strikingly beautiful results of theoretical statistics.” The optimality result asserts that an SPRT with prescribed type I and type II error probabilities minimizes the expected sample size of all tests having error probabilities that do not exceed those of the SPRT. Wolfowitz was most proud of his contribution to Wald’s sequential test. In later years Wolfowitz worked further on stochastic approximations of the Robbins-Monro type. He proved the strong convergence of the procedure in a more general setup than the original one. In 1972 Wolfowitz and Lionel Weiss published a paper on the sequential fixed-width confidence interval estimation of a translation parameter. This paper treats the problem in a nonparametric setup and proves its

asymptotic optimality. They also published in 1972 on asymptotically efficient sequential procedure that is equivalent to the t-test.

The decision theoretic approach is found in many of Wolfowitz’s papers. A loss function is assumed in general form and a procedure is chosen to minimize the risk (expected loss). Often Wolfowitz applied the minimax principle to overcome the dependence of the optimal procedure on the unknown parameters of distributions. Bayes solutions to a decision theoretic formulation are a minimization of the prior risk, which is an average risk over the space of unknown parameters according to some (prior) distribution. Given the observations the Bayesian procedure is the one minimizing the posterior risk. Together with Wald (1949, 1950) Wolfowitz characterized Bayes solutions to sequential decision problems, which were mentioned in the previous section.

In 1953 Wolfowitz, with the collaboration of Dvoretzky and Kiefer, generalized the results to sequential methods with continuous time in place of the discrete time formulation of the previous results. Many of the results of discrete time sequential analysis are carried over. These two early papers in the field showed clearly the advantage of solving discrete time problems by working first in continuous time, a technique that has become a contemporary staple of many areas of statistics and applied probability.

Asymptotic statistical analysis is concerned with the limiting behavior of sequences of procedures (estimators or test functions), each one corresponding to a sequence of increasing sample sizes. Two criteria of asymptotic behavior are prevalent: consistency (involving convergence in probability to the true value) and efficiency (involving the rate of approach). Wald’s approach to the proof of consistency of the maximum likelihood estimator was extended in 1956

(with Kiefer as coauthor) to a general class of estimators, including nonparametric models. Wolfowitz developed the minimum distance method (1957) for estimating parameters or functionals of distributions by minimizing the distance between the empirical distribution and the family of distributions for the model under consideration. The method yields consistent estimators in complicated problems. Wolfowitz’s research on the asymptotic properties of the maximum likelihood estimator, with the collaboration of Weiss, led to the development (1969) of maximum probability estimators.

Let R be a specified region in the parameter space θ. Suppose that θ is the true value of a parameter. R could be a neighborhood set around this point. An estimator of θ, d_n, based on n observations is a maximum probability estimator (MPE) if limP_θ{k(n)(d_{n −}θ) ε R} is maximal in a class of estimators. k(n) is the rate of approach, usually k(n)=√n. Here the efficiency of an MPE is defined relative to R. Maximum likelihood estimators under the common regularity conditions are MPE. However, estimators could be MPE in more general models in which the common regularity conditions of the maximum likelihood estimator are not satisfied. The reader is referred to the Springer lecture notes (1974) on the subject, written jointly with L.Weiss.

In 1959, 1960, 1964, and 1965 Wolfowitz published a sequence of papers, written jointly with Kiefer, dealing with optimality of regression designs under least-squares estimation. The problem is to determine an optimal set of design points at which to make observations to attain certain optimality conditions. Several criteria of optimality are defined in these papers. Some of them are shown (surprisingly) to be equivalent under specified conditions. Some methods were also created for finding optimal designs. These papers created a sub-field of optimal design that is an active research area to this day.

From time to time Wolfowitz published papers on problems in probability theory, which arose out of his research in mathematical statistics and applied stochastic processes. An early one is his paper on the notion of recurrence (1949). With K.I.Chung he published a paper in 1952 on a limit theorem in renewal theory. In this paper they generalized an earlier result of Erdös, Feller, and Pollard. In 1967 Wolfowitz published a paper on the moments of recurrence times. Wolfowitz’s first paper in queuing theory (with Kiefer) appeared in 1955. They studied queues with many servers. In 1956 Wolfowitz and Kiefer published a joint paper on the characteristics of queuing processes, which yielded important results in random walk theory. Contributions to inventory theory were done in a series of three papers (with Dvoretzky and Kiefer) published in Econometrica in 1952 and 1953. These contributions were pioneering at the time. They proved, for example, the optimality of the celebrated (s, S) policy in inventory control.

The first paper of Wolfowitz’s on the coding of messages subject to random errors appeared in 1957. From that time till the end of his life Wolfowitz published vigorously on this subject. In 1961 he published a classical book in this area under the title of Coding Theorems of Information Theory. The book immediately became a great success; a second edition was published in 1964 and a third edition in 1978. At the time it was the only book concentrating on the probabilistic aspects of noisy channels. The book became indispensable for specialists in the field but also served well as an introductory book because of its brief and simple explanation of the problems and their solutions. The second edition was translated into Russian. In the preface to the Russian translation the editor wrote that the “exposition is compact and elegant; the system of notation is complicated but logical.” The reader is referred to the introduction of Selected Papers²

for additional explanations and exposition of the problems treated in Wolfowitz’s papers.

In addition to being prolific in research Jacob Wolfowitz was a very well read person. He was interested in current affairs and used to discuss issues of the day with his colleagues. He fought at the time for the liberation of Soviet Jewry. He was a friend and strong supporter of the state of Israel and had many friends and admirers there. We will always remember him as a great scholar, a principled person, and a charitable man.

I GRATEFULLY ACKNOWLEDGE the input and help of Professor Lawrence Brown.

1940 With A.Wald. On a test whether two samples are from the same population. Ann. Math. Stat. 11(2):147–62.

1943 With A.Wald. An exact test for randomness in the non-parametric case. Ann. Math. Stat. 14(4):378–88.

1944 Asymptotic distribution of runs up and down. Ann. Math. Stat. 15(2):163–72.

1947 The efficiency of sequential estimates. Ann. Math. Stat. 18(2):215–30.

1948 With A.Wald. Optimum character of the sequential probability ratio test. Ann. Math. Stat. 19(3):326–39.

1949 With A.Wald. Bayes solutions of sequential decision problems. Proc. Natl. Acad. Sci. U. S. A. 35(2):99–102.

1950 With A.Wald. Bayes solutions for sequential decision problems. Ann. Math. Stat. 21(1):82–99.

Minimax estimates of the mean of a normal distribution with known variance. Ann. Math. Stat. 21(2):218–30.

1951

With A.Wald. Two methods of randomization in statistics and the theory of games. Ann. Math. 53(3):581–86.

1952 With A.Dvoretzky and J.Kiefer. The inventory problem. I. Econometrica 20(2):187–222. II. 20(3):450–66.

With J.Kiefer. Stochastic estimation of the maximum of a regression function. Ann. Math. Stat. 23(3):462–66.

1953 With A.Dvoretzky and J.Kiefer. Sequential decision problems for processes with continuous time parameter. Testing hypotheses. Ann. Math. Stat. 24(2):254–64; 24(3):403–15.

With A.Dvoretzky and J.Kiefer. On the optimal character of the (s, S) policy in inventory theory. Econometrica 21(4):586–96.

Estimation by the minimum distance method. Ann. Inst. Stat. Math. 5(1):9–23.

1955 With J.Kiefer. On the theory of queues with many servers. Trans. Am. Math. Soc. 78(1):1–18.

1956 With J.Kiefer. Consistency of the maximum likelihood estimator in the presence of infinitely many incidental parameters. Ann. Math. Stat. 27(4):887–906.

1957 The minimum distance method. Ann. Math. Stat. 28(1):75–88.

1958 Information theory for mathematicians. Ann. Math. Stat. 29(2):351–56.

1959 With J.Kiefer. Optimum designs in regression problems. Ann. Math. Stat. 30(2):271–94.

1960 Contributions to information theory. Proc. Natl. Acad. Sci. U. S. A. 46(4):557–61.

1967 With L.Weiss. Maximum probability estimators. Ann. Inst. Stat. Math. 19(2):193–206.

1969 With L.Weiss. Asymptotically minimax tests of composite hypotheses. Z.Wahrscheinlichkeitstheorie 14(2):161–68.

1974 With L.Weiss. Maximum Probability Estimators and Related Topics. New York: Springer-Verlag.

1975 Signaling over a Gaussian channel with feedback and autoregressive noise. J. Appl. Probab. 12(4):713–23.

1978 Coding Theorems of Information Theory. 3rd ed. New York: Springer Verlag.