National Institute of Statistical Sciences
Cross-Disciplinary
Research
in the Statistical Sciences
Report of a Panel of the Institute of Mathematical Statistics
Ingram Olkin, Chair
Jerome Sacks, Cochair
September, 1988
CONTENTS
The Panel on Cross-Disciplinary Research in the Statistical Sciences wishes to acknowledge suggestions and comments from Samuel Greenhouse, Oscar Kempthorne, and Paul Meier. The panel is also most appreciative of the helpful input during its deliberations from Murray Aborn and Nancy Flournoy. Finally, the panel is most grateful to Constance Citro, whose editorial assistance was instrumental in completing this report.
The driving force behind the development of modern statistics has been the need to solve practical problems. Historically, statisticians have achieved signal advances in theory and methods as they worked on applications in agriculture, industrial production, medicine, and many other fields. In turn, statistical thinking and methodology, including the principles and methods of experimental design, statistical modeling, and simulation, have greatly influenced the development of virtually all areas of science. It is no surprise that the statistics profession almost universally accepts the need to encourage cross-disciplinary research. Yet, questions have been raised in recent years about the health of that part of the research enterprise devoted to cooperative efforts between statisticians and other scientists. These questions include:
The National Science Foundation (NSF) in 1985 funded a proposal by the Institute of Mathematical Statistics (IMS) to assess the current status of cross-disciplinary statistical research and to make recommendations for its future. The IMS formed a panel to carry out the project with Ingram Olkin and Jerome Sacks as cochairs.
The panel endorses the principles that advances in substantive knowledge and in statistical theory and methods are virtually inseparable and that the continued health of statistics depends strongly on continuing cross-disciplinary research in many fields. The panel's report reviews past successes in collaborative research among statisticians and scientists in other fields, drawing its examples from agriculture, medicine, military operations, and transportation and communication. The report also reviews important current problems that would greatly benefit from close collaboration among statisticians and other scientists to push forward the frontiers of theory, methods, and knowledge. The examples cited are in the fields of chemometrics, computer science, financial analysis for business decision making, food production, industrial production, and the use of complex materials and processes (random media).
Yet the panel finds that constrained resources and the existing infrastructure within government, academia, and industry thwart the growth and development of needed cross-disciplinary work. Interactive collaboration is not a natural phenomenon in the current environment of defined disciplines. The panel's report presents recommendations for concrete steps to help rectify this situation and to promote and encourage cross-disciplinary research in the statistical sciences.
The increasing complexity of scientific models, the necessity for major computational efforts, and the potentially large data management problems that confront many applications all combine to create a need for large-scale projects in order to address many cross-disciplinary research issues. A combination of resources must be attracted to support large-scale projects, or new resources must be allocated to them. Hence, the panel recommends that:
1.1 Funding agencies support large-scale cross-disciplinary research projects through inter-agency and intra-agency cooperation.
1.2 Funding agencies ensure that existing and proposed large-scale projects incorporate adequate statistical collaboration.
The panel also believes that a continuing effort is required to assess the state of cross-disciplinary statistical research and, especially, to identify areas where large-scale cross-disciplinary projects are needed. We recommend that:
1.3 The National Research Council, through its Committee on Applied and Theoretical Statistics and its Committee on National Statistics, establish subcommittees and convene working groups to monitor the health of cross-disciplinary statistical research and specifically to identify potential large-scale cross-disciplinary projects and mechanisms to fund them.
Structure of NSF for cross-disciplinary research in the statistical sciences. The panel believes that funding increments are needed for the growth and advancement of cross-disciplinary activities aimed at specific objectives. First and foremost, however, attention should be devoted to making the most effective use of existing resources.
In this regard, the panel is much encouraged by recent actions at NSF to restructure the way in which the statistical sciences are administered. Specifically, NSF is acting to turn the Measurement Methods and Data Improvement program in the Division of Social and Economic Science into a broadly-based applied statistical science program with cross-disciplinary emphasis. The Statistics and Probability program in the Division of Mathematical Sciences will continue to address the theoretical aspects of statistics. There is, however, as yet no clear path for research projects involving statistical science that cut across other divisions or directorates within NSF. Hence, we recommend that:
2.1 NSF establish a continuing task force with responsibility for the statistical sciences. The task force should consist of a core group - the program directors of Statistics and Probability and the new applied statistical science program - together with other program directors knowledgeable in the statistical sciences.
2.2 NSF give each member of the core group a liaison appointment in one of the other Directorates (Engineering, Computer Science, Geoscience, and Education).
Support for research by statisticians in primarily advisory roles. Statisticians working as consultants in major research organizations are in ideal locations to develop cross-disciplinary research programs. These consultants are often associated with well-funded organizations or projects, and, in university settings, they often have reduced teaching loads. However, in spite of the appearance of adequate support, consulting statisticians frequently do not have the time or the resources to carry out methodological and theoretical research. We recommend that:
2.3 Project directors and organization administrators regularly allocate funds for essential methodological and theoretical research in addition to those funds allocated for statistical consulting.
2.4 Funding agency program managers actively encourage and solicit research proposals that arise from consultative projects and give consideration to high-quality research proposals that are oriented toward applications.
Support for effective transfer of technical results. The ultimate success of cross-disciplinary statistical research is measured, not in terms of technical results, but in terms of the impact of these results on applications. Yet, today, communication problems often impede effective methodological transfer. As matters stand, there is a wide gap between the "state of the art" and the "state of the practice." To alleviate this situation, we recommend that:
2.5 Principal investigators of cross-disciplinary statistical research projects allocate adequate funds and personnel for the effective transfer of the resulting methods to user disciplines.
2.6 Funding agencies develop additional mechanisms to support the active dissemination of techniques developed as a result of cross-disciplinary research involving statistics.
The place of statistics in universities. The panel believes that universities can take active steps to improve the structure of support for cross-disciplinary research at their institutions. Universities can act as well to improve the training both of statisticians and of scientists in other fields in ways that will foster successful cross-disciplinary collaboration. We recommend that:
3.1 University faculties and officials establish joint appointments between statistics and other disciplines and ensure that well-defined and appropriate standards are used in evaluating the performance of such personnel.
3.2 Academic institutions develop cross-training programs for pre- and post-doctoral researchers in statistics and substantive disciplines. These training programs should be collaborative between a university statistics department or program and cooperative departments.
3.3 Academic institutions develop special programs for cross-disciplinary training for senior researchers in statistics and other disciplines.
The place of statistical methods and statisticians in industry. Pressures of world-wide competition are leading to the increased use of statistical methods in industry. Statisticians who work for industry must usually rely on the academic community to perform needed methodological research. Yet, academicians are often unaware of industry's problems, and academic curricula at many institutions do not adequately incorporate methodology vital for industrial applications. We recommend that:
3.4 Industrial organizations and academic institutions jointly create and support mutually beneficial collaborative cross-disciplinary statistical research projects.
3.5 Professional statistical societies form working groups representing both industry and universities to foster the creation of cross-disciplinary research projects by developing prototypes and guidelines.
3.6 Academic faculty in statistics create advisory committees with industrial representatives to aid them in developing statistical curricula including methodology vital to industrial needs.
3.7 Academic faculties in engineering, science, and management, with the aid of industrial advisory groups, develop curricula that include a statistical component sufficient to meet industrial needs.
Federal statistical agencies and national laboratories. The federal statistical agencies and the national laboratories are instructive both for their efforts to foster cross-disciplinary research and for the obstacles with which they contend. The conflict for their statistical staff members between acting as consultants and as researchers surfaces constantly. On the other hand, the Research Fellows programs at several agencies afford stellar examples of effective structures for breeding useful cross-disciplinary research. We recommend that:
3.8 Federal statistical agencies actively promote and support methodological research by statistical staff members.
3.9 Federal statistical agencies fund programs that promote collaborative research with statisticians from outside the agencies.
3.10 National laboratories provide seed money to initiate programs that involve collaboration in cross-disciplinary research.
Professional society support for cross-disciplinary research. Cross-disciplinary research must involve intensive interactions among scientists that take place in an atmosphere conducive to interdisciplinary thinking. There must also exist appropriate publication outlets that reach members of all the disciplines involved. We recommend that:
4.1 The statistical societies in cooperation with other professional societies sponsor research conferences and retreats held separately from existing meetings in order to emphasize their cross-disciplinary nature.
4.2 The cooperating professional societies publish jointly any conference proceedings in order to reach researchers in both disciplines.
4.3 Professional societies provide outlets for the publication of cross-disciplinary statistical research.
Institute for Statistical Sciences. A proper atmosphere is vital for encouraging cross-disciplinary research between statistics and other fields. The establishment of an Institute for Statistical Sciences that provides the kind of supportive atmosphere that we envision could be one of the single most effective means to bring to life the recommendations of this panel. Such an Institute would foster major collaborative efforts between statisticians and other scientists, provide the site for conferences and workshops devoted to cross-disciplinary research, create places for "transplant" fellowships from other disciplines, and sponsor related activities. Hence, we recommend that:
4.4 The statistical sciences community establish an Institute for Statistical Sciences with a strong cross-disciplinary component.
4.5 The Institute for Statistical Sciences undertake to:
4.6 The statistical societies assist in the development of such an Institute.
The development of modern statistics has its origins in applications. Driven by the need to solve practical problems in agriculture, industrial production, medicine, and many other fields, statisticians have achieved signal advances in theory and methods. In turn, statistical thinking has greatly stimulated the development of virtually all areas of science. The application of the principles and methods of experimental design, the role of statistical modeling, and the pervasive use of simulation methods are but three instances of statistical developments that have had a deep impact upon science. The widespread influence of interactions between statistics and other disciplines and the very nature of statistics as the science of the "meaning and use of data" establish the statistical sciences as the discipline with the most central and complex cross-disciplinary activity.
It is consequently no surprise that there is almost universal acceptance among the statistical profession of the need to nurture cross-disciplinary research. Yet, the last few years have witnessed growing concern about the health of that part of the research enterprise devoted to cooperative efforts between statisticians and other scientists. Questions raised include:
While theoretical developments and the application of statistical methods have proceeded rapidly over the course of this century, it is only recently that the advent of inexpensive and powerful computational resources has opened the way for major advances in the statistical study of complex models. At the same time, greatly expanded data collection and processing capabilities have created opportunities and challenges for analysis with large, multidimensional data sets.
Yet, the statistical and scientific communities appear to have paid insufficient attention to the opportunities presented by these advances in computational and data resources. Moreover, although it is increasingly evident that failure to adopt useful statistical methods and techniques is costly to science and industry, there has been a lack of a concerted effort to address the interactions between statistics and other disciplines.
In recognition of these circumstances and the need to answer the questions raised above, the National Science Foundation (NSF) in 1985 funded a proposal by the Institute of Mathematical Statistics (IMS) to assess the current status of cross-disciplinary research involving statistics and to make recommendations for its future. Four divisions within NSF Mathematical Sciences, Social and Economic Science, Chemistry, and cross-disciplinary Research (Engineering Directorate) provided funding. The IMS formed a panel to carry out the project whose members were Alfred Blumstein, Amos Eddy, William Eddy, Peter Jurs, William Kruskal, Thomas Kurtz, Gory McDonald, Ingram Olkin (cochair), Ronald Peierls, Jerome Sacks (cochair), Paul Shaman, and William Spurgeon.
We note that the terms cross-disciplinary, multidisciplinary, and interdisciplinary have at times been used interchangeably. We take advantage of these alternatives in order to draw some distinctions. Following Epton, Payne, and Pearson (1983), we adopt the definition: a research task requiring a combination of disciplines is cross-disciplinary. In contrast, the terms multidisciplinary and interdisciplinary refer to the organizational forms used to carry out cross-disciplinary research. The multidisciplinary form is that in which research tasks are carried out by separate single-discipline units and their results brought together by a coordinator. In the interdisciplinary form, the research is carried out collaboratively and interactively within a single unit representing all the necessary disciplines. Our report is about cross-disciplinary research that has a statistical component the work may have multiple objectives but includes the advancement of statistical science. One of the concerns of our study is the development and support of organizational structures that will foster successful cross-disciplinary work.
The panel developed a series of recommendations to enhance support for cross-disciplinary research. Part B of the report presents our recommendations, which are addressed to the National Science Foundation and other funding agencies, to research managers and members of the statistical community resident in academia, industry, national research laboratories and statistical agencies, and to the professional associations.
Part A of the report presents the findings that underlie our recommendations. The panel believes that advances in statistical theory and methods are stimulated by the need to solve problems in substantive areas and that these advances in turn stimulate the development of substantive knowledge. In support of this premise, Section A.1 summarizes and Appendix I describes more fully several prototype examples of past successes in collaborative research among statisticians and scientists in other fields.
The panel believes that there are important current problems in many disciplines that would greatly benefit from close collaboration among statisticians and other scientists to push forward the frontiers of theory, methods, and knowledge. Indeed, without adequate support for cross-disciplinary research, there is the risk that major opportunities for significant new developments in many fields will be lost. To indicate what is needed, Section A.2 summarizes and Appendix II describes more fully several examples of specific research areas that could greatly benefit from cross-disciplinary work. A persistent theme is the impact of large-scale computer technology and data sets, both of which afford great potential to researchers but present equally formidable problems of effective use. Indeed, the phenomenal amount of data being generated in all of the sciences is creating a need for new ideas to deal with the compression, presentation, and analysis of large data sets. The panel believes that statisticians must give priority to developing adequate theory and methodology for handling the complex models and large multidimensional data sets that characterize today's scientific research.
The panel finds constraints in funding and institutional support for cross-disciplinary statistical research on the part of NSF and other funding agencies. This conclusion rests in part on a review, presented in Section A.3.1 (with additional details in Appendix III), of reports from other committees and panels over the past ten years that considered the health of cross-disciplinary research. Uniformly, these reports document problems in obtaining support for work proposing to develop statistical theory and methods directed to the application needs of other disciplines. The panel also notes that recent funding problems experienced by federal agencies generally are having a deleterious impact on statistical research and particularly on collaborative work Section A.3.2 presents some data on this point. Although the panel calls for many of the same ameliorative efforts that earlier committees urged, we point to some approaches that can have a marked impact on the support and encouragement provided by funding agencies for cross-disciplinary research.
The panel further finds that institutional support for collaborative research between statisticians and other scientists in academia and industry is problematic and could be substantially improved. There are successful efforts by some government agencies such as the Bureau of the Census, the Bureau of Labor Statistics, and the National Center for Education Statistics to bring together multidisciplinary teams for needed research programs. The example set by these agencies and their approaches are worth transferring to other agencies and other arenas. Maintenance of these efforts and added support for them by other interests in and out of government are recommended.
Section A.4 summarizes the responses to a questionnaire sent by the panel to statistics departments and programs in colleges and universities across the nation. The information gathered from this survey provides useful insights into problems connected with cross-disciplinary activity. (Appendix IV gives a full description of the survey.) The panel has sought in its recommendations to take account of the perceptions of the community at large in identifying targets of opportunity and suggesting approaches to buttress and extend the support for cross-disciplinary research in all environments in which statisticians work.
Finally, in order to provide a necessary long-range view and to establish a structure for maintaining the health of the field and its cross-disciplinary character, the panel recommends the establishment of an Institute for Statistical Sciences. The panel calls upon professional societies, committees of the National Research Council, the National Science Foundation, and other units in academia, industry, and government that have responsibility for the well-being of the discipline to review the panel's findings and recommendations and to act upon them.
Reviewing previous successes of cross-disciplinary research can help motivate and guide programs to support the needed cross-disciplinary efforts of the future. We could easily cite success stories from many fields and time periods. For example, over 150 years ago, the needs of astronomy stimulated development of the classical least-squares methodology of Legendre and Gauss. In recent years, the role of statistics in achieving high quality and cost-effective performance of foreign manufacturing provides an entirely new perspective on the impact of statistical science in the industrial world. Below, we briefly present illustrations from recent history of successful cross-disciplinary research in the fields of agriculture, medicine. military operations, and transportation and communication. Appendix I describes these examples in greater detail. We also briefly review the role of survey research in the development of the social sciences.
The impact of statistical methodology on agricultural and medical research and the effects of such research on statistics are prominent events in 20th-century science. In both fields, the ability to carry out planned experiments that could test the effects of alternative treatments and procedures motivated the development of statistical methods to maximize the information obtained. Each discipline developed specialized designs and methods of analysis to suit its particular needs. For example, "split plot" designs were developed for agriculture to determine optimal mixes of fertilizers for crop yields, whereas in medicine bio-assay designs were developed to assess responses of patients to varying dosage levels and clinical trials initiated to evaluate the efficacy of different treatment regimes. Cross-disciplinary statistical research and experimentation in these fields has had profound impacts, representing a major factor in the expansion of the world's food supplies and in the medical treatments available to combat and, in some cases, wipe out disease.
Turning to the military arena, the Statistical Research Group (SRG) at Columbia University which served the Armed Forces during the Second World War achieved major methodological advances that had far-reaching impacts not only on military operations but in many other fields as well. Statisticians at the SRG helped improve military tactics and the design and production of combat weapons through such projects as determining optimal settings on proximity fuses for air bursts of artillery shells. In the course of addressing specific applications, the SRG staff pioneered in the development of quality control procedures for manufacturing processes and sequential analysis techniques for more efficient sampling-inspection plans. Sequential analysis methods have had pervasive effects on the entire discipline of statistical science and have proved of great practical importance in industrial production and other fields of application.
Modern transportation and communication systems have benefited greatly from advances in statistical methodology, particularly from practical applications of queuing theory. Researchers at the Copenhagen Telephone Company in the early 1900s pioneered the development of queuing theory through analysis of telephone traffic patterns and use of stochastic methods to model the phenomena. By 1960, the literature contained about 250 papers on the theory of queues and 500 papers on applications to telecommunications, transportation systems, maintenance and service systems, inventory systems, health care, hydrostorage (dams), and other fields. With the ever-increasing complexity of applications and the surge in the study of computer systems, the post-1965 developments in queuing theory have been largely oriented toward queuing network problems. These networks lead to very complicated models and have stimulated and employed research in such areas as simulation methods.
Finally, just as controlled experiments in agriculture and clinical trials in medicine were fundamental to the development of these fields, the sample survey became a basic fact-gathering mechanism in the political and social sciences. Sample surveys have been used throughout history. but the modern era of survey research began with the development of a theoretical structure at the Bureau of the Census in the 1940s. Further work led to the development of alternative sampling mechanisms and methods for analyzing their results. Today, we take for granted the scientific basis of the indices published by governmental agencies such as the Census Bureau and the Bureau of Labor Statistics indices such as the cost of living index, the unemployment rate, the poverty rate, and others. The theory and practice of sample surveys have not remained static. Public opinion polls are now conducted in a scientific manner and error estimates are routinely provided for them. A variety of technological mechanisms, such as computer-assisted telephone interviewing and random-digit dialing, have come into use to permit more extensive and cost-effective surveys.
Although special application needs in each of the areas discussed fostered the development of particular statistical techniques, the resulting methodological advances often proved beneficial for applications in other fields as well. Overall, these major success stories point to a process characterized by the following stages:
At the present time, important research areas in virtually every scientific discipline raise statistical issues that must be addressed in order for work in these areas to go forward. These statistical questions should be tackled via cross-disciplinary investigations with the goal of advancing knowledge and practice in both the substantive field and statistics. Below we briefly discuss research problems and the potential contributions of statistics in the areas of chemometrics, computer science, financial analysis for business decision making, food production, industrial production, and the use of complex materials and processes (random media). Appendix II discusses each example in detail.
Cross-disciplinary research in chemometrics - the use of mathematical and statistical methods to analyze data obtained from chemical measurements - should lead to fundamental advances in both chemistry and statistics. One needed area for research is to develop improved methodology to handle the large, multivariate data sets that are generated, often at an extremely rapid rate, by modern chemical instrumentation. Another area lies in the application of experimental design methods to several types of chemical experiments, including the search for optimum settings on complex instruments. Finally, simulation techniques offer a potentially useful approach to study the complex chemistry of events such as atmospheric reactions.
Computer science provides a rich arena for cross-disciplinary statistical research in large part because of the growing use of stochastic models to represent the systems being studied. Earlier deterministic models for computer systems are now being modified to include a stochastic component. Furthermore, the newness of the discipline presents a variety of novel, hardly-touched problems. Research is needed to develop stochastic models of computer systems to better understand and improve systems that use parallel processors, to analyze the behavior of algorithms through probabilistic methods, and to build appropriate numerical representations of uncertainty into rule-based expert systems.
The pressures on American business firms to improve their competitive posture in the world economy place a premium on effective decision making. Yet it appears that the accounting and auditing systems used in business firms are not appropriate instruments upon which to base sound management decisions. Most accounting systems, which are designed for purposes such as calculating federal and state taxes and systematizing annual reports, do not suitably address risk aspects inherent in management decisions. Moreover, they do not always provide adequate information for decision-making purposes. Statistical research is needed to further the development of sophisticated stochastic models that quantify risk factors present, for example, in a decision to invest in automated equipment or a pilot plant. Advances in statistical methodology, including complex sampling schemes and analysis of censored data, offer the potential to enhance current accounting and auditing practices and improve the usefulness of accounting information.
The need to understand the influences of global factors on food production and the quality of the environment is leading researchers in this field to develop very large complex computer models. The inadequacies and uncertainties in these models, which are being used to study the possible consequences of problems such as deforestation, pollution, acid rain, overgrazing, and the greenhouse effect, are not well known. Statistical analyses of these systems and the development of novel methods to cope with the complexity of the models, the input data, and their applications are essential.
Advances in statistical methodology for industrial applications are vitally important for the nation's pursuit of economic competitiveness. The proliferation of product lines and the increasing complexity of manufactured products have given rise to new problems in design and scheduling, process control, and quality assurance. New methods are required to maintain order and to obtain high yields, as well as to avoid increases in fixed costs, warranty and service costs, recall campaigns, and liability claims. The systems approach has evolved in manufacturing in response to these problems, and a parallel development of statistical tools is needed as an essential component.
Methodological advances in understanding the properties of materials and processes so complex that they can only be described statistically, random media are important for continued technological development in a number of fields. As just one example, effective medium theory. or the prediction of bulk properties of composite materials from the known statistics of their microstructure, is gaining increased importance in materials science, chemical engineering, and other technological disciplines. Similarly, models of interacting particle systems are being widely used, principally for physical processes but also in modeling biological populations and the spread of epidemics and forest fires. Clearly, the analysis of random media offers many opportunities for cross-disciplinary research to develop statistical theories and methods with widespread practical application.
The topics we have singled out for discussion are only illustrative of a much broader class of areas that could greatly benefit from cross-disciplinary research. What emerges from a review of these and other areas is the urgent need for the statistical research community to:
A clear priority for statistical researchers is to work on developing adequate methodology for handling complex models, large data sets, and multidimensional data in today's environment of large-scale powerful computing technology.
Over the past decade the statistical profession has evinced continued concern for the state of cross-disciplinary research. Two previous external oversight committees that reviewed the Statistics and Probability program in the Division of Mathematical Sciences at NSF raised the issue of inadequate support for cross-disciplinary projects. (Appendix III provides excerpts from their deliberations. We note that these and other committees often used the terms cross-disciplinary, interdisciplinary, and multidisciplinary interchangeably.)
In 1979 an external oversight committee consisting of Herbert A. David, Peter Huber, Ingram Olkin, Ronald Pyke, Frank Spitzer, and William E. Strawderman noted that the NSF mathematical sciences program did not adequately cover applied statistics. It stated that "the vitality and importance of statistics are dependent upon an active accessible interface between it and many other disciplines."
Again, in 1981 an oversight committee consisting of Frederick Mosteller, Ingram Olkin, Steven Orey, Ronald Pyke, Frank Spitzer, and Grace Wahba took a similar position. In particular, it noted that "the general plan of funding individual research workers... may miss some excellent opportunities to open up new fields of research or to organize them by means of projects awarded to groups of scholars who would work together on a specific area." This committee endorsed a report by David Moore, then Program Director for Statistics and Probability, declaring that "coherent modes of support (the Mathematics Institutes) designed to encourage cross-fertilization of mathematical specialties are... inadequate for the special needs of statistics."
Oversight committees which assessed the programs of other NSF divisions similarly pinpointed cross-disciplinary research as a target of critical importance. In 1985 a committee reviewing the Measurement Methods and Data Improvement program in the Division of Social and Economic Science noted that the community of statisticians and social scientists concerned with the scope, quality, and efficiency of government statistics and the community working on graphical methods were integral parts of the program and had special importance in reference to the federal statistical system. The committee, chaired by William Kruskal and including Hayward Alker, Jr., Roy D'Andrade, Zvi Griliches, Albert Reiss, Jr., and Teresa Sullivan, stated that:
The multidisciplinary nature of the [Measurement Methods and Data Improvement] program, and the research it supports, are striking and valuable. Basic research related to social scientific data (including econometrics, statistics, demography, etc.), has always been multidisciplinary in motivation, development, and application. It is important to keep in mind that this multidisciplinary character of the program is not a forced marriage; on the contrary, it flows organically from the nature of basic science. A good example is the support of joint work by cognitive psychologists and statisticians on survey problems: that work should benefit both statistics and cognitive science... and indeed all of us by better understanding and practice of an important mode of measurement.
More recently, the Office of Interdisciplinary Research in NSF funded a panel to review methodology transfer in the social sciences. The members of the panel were Saul Amarel, Alfred Blumstein, Emilio Casetti (chair), Richard H. Day, Alexander H. Levis, Stephen R. Rosenthal, Jerome Sacks, and Christian Werner. Their report, issued in 1986, recommended a particular strategy to promote cross-disciplinary research. Specifically, the report called for workshops whose two-fold goal would be to keep social scientists abreast of recent technical and methodological developments in rapidly advancing areas such as artificial intelligence and statistics and to interest them in pursuing further cross-disciplinary efforts.
The theme of cross-disciplinary research recurs in other contexts. The 1987 Panel on Applied Mathematical Research Alternatives for the Navy, chaired by Gary C. McDonald and including Peter J. Bickel, Herman Chernoff, J. D. Cote, W.W. Cooper, Charles L. Fefferman, Ronald L. Graham, Alan J. Hoffman, Gerald J. Lieberman, R. J. Lipton, and M. P. Tulin, noted that the mathematical sciences contribute to and gain from interactions with other sciences and disciplines. The panel recommended further support opportunities for meaningful cross-disciplinary research.
The research community recognized early on the need for cross-disciplinary connections in the computational sciences. The Rheinboldt Report (1985) was influential in encouraging initiatives for interdisciplinary research teams in computational science and engineering. More recently, the Eddy Report (1986) provided a review of computers in statistical research.
The above panels generally focused on cross-disciplinary activities related to the mathematical, statistical, or computational sciences. The White House Science Council (1986) took a more global view of the need for cross-disciplinary activities in order to attack national and societal problems. Their report asserted that "... we must promote a broad interdisciplinary approach to problem-solving by focusing on university-based centers that will improve cooperative linkages between scientists, engineers, and industry," and that "... the time has come when a new partnership involving all three, the federal government, universities and the private sector, must be forged. And we must be realistic about the very real limitations on the extent to which industrial support of basic research, important as it is, can replace that from the federal government." In order to "maintain the strong science base essential to our national future," this council recommended that "the federal government support a major initiative to establish university-based interdisciplinary, problem-oriented research and technology centers directed to problems of broad national needs and relevant to industrial technology."
This White House Report served as a catalyst for an NSF study focused on the concept of research centers (Science, 1987). NSF Director Erich Bloch, in explaining what the centers mean, stated that they are needed "to tackle problems that you are not able to tackle on an individual grant basis," and that these problems require an interdisciplinary approach.
The impressions and concerns of the 1979 and 1981 committee reports remain salient today, while it is too soon to assess the impact of the more recent reports on cross-disciplinary research and methodology transfer in the social sciences. In the years since 1981, NSF provided additional funding for computing equipment, and program directors with ingenuity stretched limited resources to support a few projects with some of the characteristics of the cross-disciplinary research discussed by the earlier committees. However, the overall situation with respect to NSF support of cross-disciplinary research showed little improvement.
At the present time, funding constraints of other federal agencies pose grave problems for the health and development of the statistical sciences and for cross-disciplinary research involving statistics. Reallocations and decreased levels of funding within many agencies have considerably diminished the support for statistical research without adequate increases in the NSF budget to make up the difference.
The 1988 report of the Intersociety Working Group, convened by the American Association for the Advancement of Science, summarizes the situation with regard to federal funding for research and development in the period from 1980 to the present as follows:
The Administration's R&D policies have been remarkably consistent over the years, emphasizing heavy investment in military R&D, especially development; providing substantial increases to non-defense basic research, particularly in the physical sciences and engineering; showing resistance to increases in basic biomedical research; and proposing reductions in applied research in civilian agencies, including DOE, NOAA, the U.S. Geological Survey, the Bureau of Mines and others. Congress has responded equally consistently, reducing military R&D below the President's request, rapidly expanding biomedical research in NIH, and rejecting many of the proposed reductions in civilian applied R&D programs.
The report found that total federal spending for R&D increased by 26 percent in constant dollar terms between fiscal year 1980 and fiscal year 1988, but that the growth was very uneven. Defense R&D grew by over 80 percent in real terms, but civilian R&D declined by one-fourth. Moreover, within the Defense Department, defense development funding essentially doubled in real terms, but funding for basic research grew only 11 percent and funding for defense applied research declined about 7 percent.
These funding patterns do not indicate much support for cross-disciplinary statistical work which typically falls under the category of applied research. Indeed, reports made at recent meetings of the Committee on Applied and Theoretical Statistics of the National Research Council are that overall funding levels for statistical research at NSF, the Department of Defense, and the Department of Energy have been virtually constant for the last three years. Precise estimates of statistical research funding from all agencies are elusive and an item for more intensive inquiry. How much research funding is devoted to cross-disciplinary statistical research is even more difficult to assess. What is true is that there are few reports of projects of any scale that represent the type of collaborative effort that the panel believes is needed.
In order to obtain information about cross-disciplinary activity in the statistical sciences in academic settings, the panel circulated questionnaire letters to statistics departments and programs and to a large number of individual statisticians in colleges and universities across the country. The survey was not designed for quantification, but to gain a sense of the extent and variety of cross-disciplinary research under way in the academic world and to identify issues of concern to the academic community for the panel to consider. Consequently, the questionnaire was short, most questions were open-ended, and the sample was self-selected. (Appendix IV provides details of the survey methodology and results.)
The responses documented the great extent of collaborative research that is being carried out by statisticians across the country involving at least 30 major disciplines. Much of this work employs statisticians in an advisory capacity to answer specific questions raised in a substantive discipline. Typically, the statistician engaged in advisory work will adapt existing methodology to the problem at hand and create computable versions of known techniques. Another mode of collaboration is much more interactive in nature and involves work to develop novel techniques and methods to deal with broader substantive questions. This second type of collaboration leads to research on statistical issues that may subsequently advance knowledge both in the substantive field and in statistics itself.
Of course, it is not always possible to dichotomize the interchanges between statisticians and scientists so easily the boundaries are blurred, and one mode of activity can lead to the other. Nonetheless, we find it useful to make the distinction between the advisory and interactive forms of collaboration in order to examine perceptions in the community and focus the issues. For purposes of exposition, we label the two modes of collaboration "Type A" and "Type B," respectively.
Almost every statistician who has been consulted is familiar with examples of Type A research. Such work often begins with a request from a colleague, student, or professional in another field for some small amount of help in addressing a particular problem. Examples of Type B research may not be as familiar. Some of the well-known advances in methodology that resulted from the interactive form of collaboration are experimental design, isotonic regression, log-linear analysis, sequential analysis, and factor analysis. In addition to examples such as these, which involve rather large areas of research, there are many examples that deal with specific techniques, such as canonical correlations, order statistics and accelerated life testing, and calibration models.
The survey responses indicated a high frequency of Type A research. While sounding a common theme that Type B research does not receive sufficient time. money, or recognition of its value. The short-run "advisory consultation" rarely becomes the "long-range interactive collaboration." Yet it is the interactive mode that has the greater potential to break new ground and lead to statistical innovations of far-reaching significance for the future conduct of science, and it is this type of collaboration that, the panel feels must receive the attention of the disciplines and of NSF and other funding agencies.
One indicator of a supportive environment for cross-disciplinary research is the extent to which faculty hold joint appointments. Of the 115 statistics programs represented in the survey, only 33 have 2 or more faculty members with secondary appointments in other disciplines, and 82 programs have none or only one member with a secondary appointment. Of the largest programs, those with over 10 faculty members in statistics, less than half - 16 of 35 programs - have 2 or more faculty members with secondary appointments. Similarly, only 30 statistics programs include faculty members with joint appointments whose primary appointments are in substantive fields. Although secondary appointments can create problems in terms of salary, tenure, and promotion determinations, the relatively small number of faculty in statistics programs with such appointments gives cause for concern about the institutional support for cross-disciplinary research.
The replies from individual statisticians indicated, perhaps not surprisingly, that funding is insufficient to pursue cross-disciplinary research. Respondents also cited the need for longer-term and more stable funding than is now available, to permit making a sustained commitment to a research program.
The questionnaire contained an open-ended request for comments that yielded a number of telling observations. Respondents cited the difficulty of obtaining support for cross-disciplinary research given that funding agencies too often maintain that sources other than their own should provide the money. Some funding programs view statistical research as too applied, whereas other programs view the same research as too theoretical or inappropriate, so that the researcher "falls between the cracks." It has become increasingly difficult to obtain funding for methodological work in disciplines that combine statistics with a substantive field, e.g., biostatistics, psychometrics, and educational statistics. Respondents also noted misperceptions on the part of both statisticians and researchers in other fields that militate against cross-disciplinary research. Finally, in a positive vein, respondents singled out the fields of biology and medicine, computer science, engineering including industrial applications, and military applications as providing particularly fruitful opportunities for useful cross-disciplinary research in statistical science.
This panel's obligation is to direct attention to the opportunities in cross-disciplinary statistical research today and the barriers to such research. Although we call for many of the same efforts that earlier committees urged, our report points to several areas in which a great impact is possible now and suggests approaches to buttress and extend the support for cross-disciplinary work.
The panel endorses the following general principles:
The panel finds that constrained resources and the existing infrastructure within government, academia, and industry thwart the growth and development of needed cross-disciplinary research. Interactive collaboration is not a natural phenomenon in the current environment of defined disciplines. Since any new cross-disciplinary effort, virtually by definition, involves at least two researchers with different backgrounds and training, it is incumbent on at least one of them to learn a substantial amount of new material and methodology. In doing this, younger researchers may become too involved in research in a new discipline before they have fully mastered their own, and successful senior researchers may face abandoning productive lines of research within their own discipline. Couple these problems with day-to-day communication problems and with the perception that cross-disciplinary research is a dead end professionally or the "last refuge" of a mediocre researcher, and it becomes clear that special efforts must be made to foster this kind of research.
While these conditions are most apparent in academia, some of the same difficulties are found in other environments. In the national laboratories, federal agencies and industry, the problems emanating from the distinction between advisory and interactive research are often more pronounced. The statistics staff is often viewed at best as an advisory body rather than as a resource for collaborative research. Quality and productivity failures that have plagued American industry and some governmental agencies can be attributed in part to the corresponding failure to engage statistical scientists in the research and development process.
In order to successfully stimulate cross-disciplinary research, funding mechanisms must be in place that are supportive of cross-disciplinary work, interactions among scientists must be fostered that are more than superficial, and appropriate outlets must exist for the work that reach members of all the disciplines involved. The remainder of the report presents our recommendations for concrete steps to promote and encourage cross-disciplinary work in the statistical sciences. They are directed to NSF and other funding agencies and to all academic institutions, private firms, federal statistical agencies and laboratories, and professional organizations that house or support research with a statistical content. The recommendations cover the following topics:
Many of the opportunities for cross-disciplinary statistical research that are described in Section A.2 and Appendix II involve large-scale projects that require researchers from many disciplines. Other projects with less varied representation may require multiple investigators from one or more disciplines to ensure the necessary interchange of ideas and knowledge. In addition, the increasing complexity of scientific models, the necessity for major computational efforts, and the potentially large data management problems that confront many applications all combine to create a need for large-scale projects in order to address many cross-disciplinary research issues.
Such projects can place an undue strain on funding agencies. A combination of resources must be attracted to support large-scale projects, or new resources must be allocated to them. For these reasons, the panel recommends that:
1.1 Funding agencies support large-scale cross-disciplinary research projects through inter-agency and intra-agency cooperation.
1.2 Ending agencies ensure that existing and proposed large-scale projects incorporate adequate statistical collaboration.
In particular, the panel endorses the establishment of the NSF Science and Technology Centers and urges that they include cross-disciplinary statistical collaboration.
The panel also believes that a continuing effort is required to monitor the health of cross-disciplinary statistical research and, especially, to identify areas where large-scale cross-disciplinary projects are needed to further the development of statistical theory and methods and their application to substantive problems. The panel believes that the National Research Council can play an important role in this regard, and hence recommends that:
1.3 The National Research Council, through its Committee on Applied and Theoretical Statistics and its Committee on National Statistics, establish subcommittees and convene working groups to monitor the health of cross-disciplinary statistical research and specifically to identify potential large-scale cross-disciplinary projects and mechanisms to fund them.
Structure of NSF for cross-disciplinary research in the statistical sciences. Efficient utilization of funding resources devoted to science requires support of cross-disciplinary research in statistics. The panel believes that funding increments are needed for the growth and advancement of cross-disciplinary activities aimed at specific objectives. First and foremost, however, attention should be devoted to making the most effective use of existing resources.
In this regard, the panel is much encouraged by recent actions at NSF to restructure the way in which the statistical sciences are administered. Specifically, NSF is acting to turn the Measurement Methods and Data Improvement program in the Division of Social and Economic Science into a broadly-based applied statistical science program with cross-disciplinary emphasis.
As we have stressed throughout this report, statistical science by its nature involves three components theory, methodology, and cross-disciplinary interaction and the health of the statistical sciences depends on the continued viability of each component. At NSF, the Statistics and Probability program in the Division of Mathematical Sciences addresses the theoretical aspects in large part and at the same time, but with very limited resources, fosters some methodological and cross-disciplinary research. The new applied statistical science program can be regarded as oriented toward the methodological aspects, while at the same time addressing some cross-disciplinary efforts in connection with social science disciplines. There is, however, as yet no clear path for research projects involving statistical science that cut across other divisions or directorates within NSF. The panel believes that NSF can make some additional, relatively simple organizational changes to expand and improve the coordination of statistics research with other programs. The panel recommends that:
2.1 NSF establish a continuing task force with responsibility for the statistical sciences. The task force should consist of a core group the program directors of Statistics and the new applied statistical science program together with other program directors knowledgeable in the statistical sciences.
2.2 NSF give each member of the core group a liaison appointment inone of the other Directorates (Engineering, Computer Science, Geoscience, and Education).
NSF should also make available additional funds to stimulate and support cross-disciplinary projects that are bound to come to the attention of the liaison officers. It is the panel's view that the proposed structure together with resources to assist some well-defined research objectives with cross-disciplinary content will enable NSF to play a significant role in advancing cross-disciplinary statistical research.
The panel further urges other funding agencies to follow the lead of NSF and take steps to devise effective mechanisms for encouraging cross-disciplinary statistical research. At present, support for interactive research tends to rely on individual managers and ad hoc arrangements. Moreover, good working structures that facilitate inter-agency cooperation on overlapping research projects are rare. Since statistics plays a fundamental role in the research mission of every funding agency, the lack of such structures is a disservice to all the agencies. The funding agencies should address the problem of developing effective intra-agency and inter-agency structures to support cross-disciplinary research. The National Research Council Committees should address this issue as well in their continuing review of the "state of statistics."
Support for research by statisticians in primarily advisory roles. Statisticians working as consultants in major research organizations are in ideal locations to develop cross-disciplinary research programs and to serve as links between the statistical community and substantive disciplines. These consultants are often associated with well-funded organizations or projects, and, in university settings, they often have reduced teaching loads. However, in spite of the appearance of adequate support, consulting statisticians frequently do not have the time or the resources to carry out the methodological and theoretical research that should grow naturally out of their cross-disciplinary activity. Specifically, the demand for advisory consulting services usually leaves little time for research. Furthermore, funding agencies may perceive that statistical researchers in these settings are already well-funded, even though they may not be able to use the available funds to support methodological or theoretical research.
In order to encourage cross-disciplinary research on the part of advisory statisticians, we recommend that:
2.3 Project directors and organization administrators regularly allocate funds for essential methodological and theoretical research in addition to those funds allocated for statistical consulting.
2.4 Funding agency program managers actively encourage and solicit research proposals that arise from consultative projects and give consideration to high-quality research proposals that are oriented toward applications.
Support for effective transfer of technical results. The ultimate success of cross-disciplinary statistical research is measured, not in terms of technical results, but in terms of the impact of these results on applications. It is essential that the advances in statistical theory and methods that are achieved through interactive collaborative efforts be put into practice in the substantive field. Yet, today, communication problems often impede effective methodological transfer. On the one hand, statistical researchers frequently write for each other and publish in periodicals that specialize in statistical methods. They use their own technical jargon and symbols, which are becoming increasingly complex and require much time to learn. On the other hand, researchers and practitioners in other disciplines, who should be using statistical methods, often do not understand the more recent and complex research results and do not have the time to learn a new field. Only rarely can they articulate research needs. As matters stand, there is a wide gap between the "state of the art" and the "state of the practice."
The amount and complexity of technical knowledge in all fields. including statistical science, is increasing rapidly. To make matters worse, the literature has expanded, and the general ability to write technical papers well has diminished. Furthermore, the ability of researchers to deal with the information explosion and to keep abreast of important new findings in their own field, let alone another field such as statistics, has not improved over the years. Computer-aided searching has helped, but the rate of assimilation of knowledge remains fixed.
Francis Bacon, in the year 1627, drew attention to this problem in his New Atlantis. He described a kind of author known as the "compiler." In the "Advancement of Learning," he describes the compiler as a "profound interpreter or commenter," "a sharp champion or defender," and "a methodical compounder or abridger." Today, authors who can organize research results into forms that can be read, understood, and used with a minimum of effort have become rare, which is one reason for the wide, and widening, gap between the state of the art and the state of the practice in so many fields.
A primary reason for the lack of compilers, or translators, is that our institutions have no mechanisms for rewarding them for their efforts. Accordingly, we recommend that:
2.5 Principal investigators of cross-disciplinary statistical research projects allocate adequate funds and personnel for the effective transfer of the resulting methods to user disciplines.
2.6 Funding agencies develop additional mechanisms to support the active dissemination of techniques developed as a result of cross-disciplinary research involving statistics.
The Place of Statistics in Universities. The panel believes that universities can take active steps to improve the structure of support for cross-disciplinary research at their institutions. Universities can act as well to improve the training both of statisticians and of scientists in other fields in ways to foster successful cross-disciplinary collaboration.
The panel urges academic institutions to make more use of joint appointments as a means to support cross-disciplinary research. In many institutions, the difficulties that junior faculty members (and even senior ones ) face in gaining recognition, promotion, and rewards for cross-disciplinary research have been an obstacle to creating joint appointments and have often discouraged individuals from accepting such appointments. Moreover, failures or problems with joint appointments in certain disciplines in some institutions have created a negative bias against other joint appointments. However, a number of universities have successfully fostered joint appointments between statistics and other disciplines and can attest to their effectiveness in promoting and generating cross-disciplinary research projects and programs.
The panel feels strongly the need for removing organizational impediments of all kinds to cross-disciplinary research involving statistics and other disciplines. We recommend that:
3.1 University faculties and officials establish joint appointments between statistics and other disciplines and ensure that well-defined and appropriate standards are used in evaluating the performance of such personnel.
A major obstacle to the cross-disciplinary research envisioned by this panel is the education and training in the statistical sciences that researchers in other fields receive. Frequently, this training focuses on statistics as a set of rigid tools and fails to convey the idea that statistics is a dynamic and evolving discipline. A consequence is that scientific researchers view statistics as a necessary tool but are generally unaware of the cross-fertilization that can result from active collaboration with statisticians.
There is an equivalent and parallel difficulty in the education and training of many statistical researchers. This training focuses on statistical theory and methodology in the abstract and often fails to convey the idea that applications frequently drive the development of new theory and methods. A consequence is that statistical researchers are often not interested in collaborative work with researchers in other sciences and not knowledgeable enough of application areas to be able to carry out such work.
Cross-training programs are needed both to educate statisticians in depth about the particular discipline and to educate scientists in the other discipline in depth about statistics. We recommend that:
3.2 Academic institutions develop cross-training programs for pre- and post-doctoral researchers in statistics and substantive disciplines. These training programs should be collaborative between a university statistics department or program and cooperative departments.
3.3 Academic institutions develop special programs for cross-disciplinary training for senior researchers in statistics and other disciplines.
Cross-training programs for senior faculty should involve individual senior researchers (approximately ten years post-doctorate) spending one or more years in a department of statistics becoming re-educated about statistics and. conversely, individual senior statisticians spending one or more years in a discipline department receiving in-depth education about the discipline. We are unaware of many current programs of this type. One such successful program at the University of Illinois provides release time for faculty to study a second discipline within the university. Successful programs have existed in the past, for example, at the University of Chicago in the early 1950s. We believe that such programs can have a great impact on fostering cross-disciplinary research. However, it is essential that both the statistics and the discipline departments endorse this effort to ensure that the transplant individual will be welcome in the host department.
The place of statistical methods and statisticians in industry. Pressures of world-wide competition are leading to the increased use of statistical methods in industry. Large companies typically have a small group of professional statisticians located at corporate headquarters or in a central laboratory. Normally, this group has three functions: (1) to teach the elements of statistics to various other groups in the company; (2) to consult with scientists and engineers in the operating divisions who have some knowledge of and enthusiasm for statistical and (3) although to a much lesser extent to perform the research needed to deal with the problems that the members of the group encounter.
Industrial statisticians deal with three kinds of products - hardware, software, and service. Statistical methods have been developed and are widely used for hardware products, developed but often not widely used for service products, and neither developed nor used for software products. Industrial statisticians are also involved in all phases of a product's life cycle - design, engineering, manufacturing, and marketing. In the design and engineering phases, they deal with experimental design, simulations, and product reliability and failure analysis. In manufacturing, they deal with process capability, process control, and quality assurance, including computer-automated inspection and testing. In marketing, they deal with forecasts and with service costs for warranty, recalls, and liability.
In a few instances, industrial statisticians perform the research needed to support their other work. In general, however, they must rely on the academic community to perform the research function. A major problem is that the academics qualified to carry out the research are often unaware of the needs of industry. For example, a question currently of great concern is where do statistical methods fit into the total methodologies that companies use for product quality assurance. Another impediment to cross-disciplinary research directed to the needs of industry is that academic curricula at many institutions do not adequately incorporate methodology vital for industrial applications. (The Snee report in 1980 recommended steps to improve the training of statisticians for careers in industry and to encourage attention on the part of academic institutions to the statistical research problems that confront the business sector.)
Examples of successful cross-disciplinary research groups housed within industrial firms can be cited. In 1971, the General Motors Research Laboratories established a "societal analysis project" which resulted in the establishment of a Societal Analysis Department (later merged into the Operating Sciences Department). The department grew to 30 researchers, representing statistics, computation, physics, psychology, engineering, and other disciplines, who are engaged in cross-cutting issues such as risk assessment and consumer perception and behavior issues that require the ability to attack complex and large data sets.
Other organizations such as the National Opinion Research Center (NORC) similarly merge several interests and provide an environment where successful cross-disciplinary research is conducted. At NORC there is a need to blend the work of statisticians and social scientists in order to focus adequate attention on difficult sampling and survey methodology problems. The Quality Assurance Center at Bell Laboratories brings together computer scientists, statisticians, and engineers to form research teams that address problems of product design.
To help resolve the problems of cross-disciplinary research for industrial needs, we recommend that:
3.4 Industrial organizations and academic institutions jointly create and support mutually beneficial collaborative cross-disciplinary statistical research projects.
3.5 Professional statistical societies form working groups representing both industry and universities to foster the creation of cross-disciplinary research projects by developing prototypes and guidelines.
3.6 Academic faculty in statistics create advisory committees with industrial representatives to aid them in developing statistical curricula including methodology vital to industrial needs.
3.7 Academic faculties in engineering, science, and management, with the aid of industrial advisory groups, develop curricula that include a statistical component sufficient to meet industrial needs.
Federal statistical agencies and national laboratories. The federal statistical agencies (e.g., the Bureau of the Census, Bureau of Labor Statistics, Energy Information Administration) and the national laboratories (e.g., Brookhaven, Los Alamos, Oak Ridge) are instructive both for their efforts to foster cross-disciplinary research and for the obstacles with which they contend. The conflict for their statistical staff members between acting as consultants and as researchers surfaces constantly, and maintaining adequate balance is a difficult managerial task. Unlike academic institutions, there is often less reward for creative research than for filling the consultant role.
The Research Fellows programs at several statistical agencies, on the other hand, afford stellar examples of effective structures for breeding useful cross-disciplinary research. These programs, which are sponsored by the American Statistical Association with the support of the National Science Foundation, are designed to bring academics closer to government and simultaneously provide some infusion of academic thought into operating agencies. Such fellowship programs exist at the Bureau of the Census, the Bureau of Labor Statistics, the National Center for Education Statistics, and the Department of Agriculture. We believe that these programs provide a model for other agencies and the national laboratories to follow.
The issues raised in recommendations 2.3 and 2.4 regarding the importance of support for research by statisticians in primarily consultative roles are relevant to the statistical agencies and national laboratories. The recommendations in the preceding section regarding closer collaboration between academia and industry are also applicable here. In addition, we recommend that:
3.8 Federal statistical agencies actively promote and support methodological research by statistical staff members.
3.9 Federal statistical agencies fund programs that promote collaborative research with statisticians from outside the agencies.
3.10 National laboratories provide seed money to initiate programs that involve collaboration in cross-disciplinary research.
Professional society support for cross-disciplinary research. Cross-disciplinary research must involve interactions among scientists that are more than superficial and that take place in an atmosphere conducive to interdisciplinary thinking. There must also exist appropriate publication outlets for cross-disciplinary work that reach members of all the disciplines involved. We believe that the professional societies representing statistics together with societies representing other disciplines can play an active role in sponsoring meetings and supporting publications that will spotlight and encourage successful collaboration.
One way in which the professional statistical societies can provide support for cross-disciplinary research is to organize and co-sponsor cross-disciplinary meetings with other societies. To be effective, such meetings need to be held separately from existing, periodic meetings of professional societies in order to emphasize their interdisciplinary nature, and any proceedings need to be published jointly in order to reach members of both disciplines. We envisage meetings of several types, including two-or three-day symposia and week-long retreats.
Examples of alternative modes already exist. One successful cross-disciplinary meeting was the Symposium on Chemometrics sponsored by the National Bureau of Standards (NBS) in October 1985. This symposium included papers by both statisticians and chemists, which were published in a special issue of the Journal of Research of the NBS. Another example was the Mohonk conference of 1984, in which the Taguchi product design approach was brought to the statistics community (methodology transfer in the other direction!). Meetings of longer duration in a retreat atmosphere could also be organized, similar to the Oberwolfach mathematical conferences or the Gordon Research Conferences.
Another way in which the statistical professional societies can encourage cross-disciplinary research is to examine the existing outlets for research publication to determine if joint work with each of the other disciplines has an appropriate outlet. In those cases where it does not, the statistical societies should explore appropriate publication avenues with the societies representing the other discipline.
We recommend that:
4.1 The statistical societies in cooperation with other professional societies sponsor research conferences and retreats held separately from existing meetings in order to emphasize their cross-disciplinary nature.
4.2 The cooperating professional societies publish jointly any conference proceedings in order to reach researchers in both disciplines.
4.3 Professional societies provide outlets for the publication of cross-disciplinary statistical research.
Institute for Statistical Sciences. A proper atmosphere is vital for encouraging cross-disciplinary research between statistics and other fields. Such an atmosphere is also vital to fostering related activities, such as workshops, conferences, and training, that in turn will promote continued fruitful collaboration among statistical researchers and other scientists.
The mathematics community has developed a number of institutes, each with a special goal and focus, that provide the kind of supportive atmosphere that we have in mind. Examples are the Courant Institute, New York University; the Institute for Advanced Study, Princeton; the Mathematical Sciences Research Institute, Berkeley, the Institute for Mathematics and Its Applications, Minneapolis; and the Mathematics Research Center, Cornell University. In Europe, the mathematical center at Oberwolfach is very well-known, and there are many other institutes as well, such as those in Czechoslovakia, Hungary, and the Soviet Union. Other scientific research communities in the United States have also successfully established centers to promote cross-disciplinary activities. Among the newest such institutes is the Beckman Institute, University of Illinois, whose focus is cross-disciplinary research in human intelligence. More traditional centers such as the Center for Advanced Study in the Behavioral Sciences, Stanford, serve as other models of how cross-disciplinary research is carried forward.
The establishment of an institute to foster major collaborative efforts between statisticians and other scientists, provide the site for conferences and workshops devoted to cross-disciplinary research, create places for "transplant" fellowships from other disciplines, and sponsor related activities could be one of the single most effective means to bring to life the recommendations of this panel. Hence, we recommend that:
4.4 The statistical sciences community establish an Institute for Statistical Sciences with a strong cross-disciplinary component.
4.5 The Institute for Statistical Sciences undertake to:
4.6 The statistical societies assist in the development of such an Institute.
We note that the National Science Foundation has responded to the need for furtherance of interdisciplinary research in science and technology through its new program for the creation of Science and Technology Centers. This panel wholeheartedly endorses this initiative. The panel's recommendation for the establishment of an Institute for Statistical Sciences, though different from that of the NSF directives for establishing centers, should complement the efforts of NSF. Such centers in statistical science could either be collateral to or part of the Institute.
We further note that the National Science Foundation has funded the American Statistical Association to conduct a feasibility study for the establishment of an Institute for Statistical Sciences. Also, at ASA, the plans for the new Office of Scientific and Public Affairs (OSPA) include education functions like those proposed for the Institute in 4.5 (d) and 4.5 (e) above. These functions should be coordinated between the Institute and OSPA.
We believe that an Institute structure is needed to facilitate cross-disciplinary research that furthers statistical science, has relevance and importance in other disciplines, and that can provide timely response to pressing issues. Present academic and other institutional structures do not have the scope to manage the large-scale projects that are required in many fields of application, such as those discussed in Section A.2. Fund-raising and organizing diverse groups of researchers into effective teams are consuming tasks that cannot be managed on a continued basis at standard academic departments or at any existing institutions. With a chief purpose of overcoming such difficulties and providing avenues for stimulating responses to opportunities and needs, the Institute can become indispensable.
There is a need both for centralized and decentralized activities to encourage cross-disciplinary research. The creation of a central facility would provide a vehicle for furthering cross-disciplinary collaboration both directly and indirectly through its impact on the statisticians and scientists who spend time there and carry new ideas and behaviors back to their home institutions. However, the research activities sponsored by the Institute, whether short-term or long-term, would not have to be located at the Institute itself and should be strongly encouraged to be located at appropriate sites consistent with the goals of the research. Thus, for example, it would be natural for a geostatistics project to be situated at a laboratory or institution with a receptive and strong group of geoscientists. Such decentralized modes of operation reduce the potential for distortion of resources. The central role of the Institute as facilitator would be to spotlight opportunities and needs.
The Institute should undertake to manage transplant fellowships, i.e., programs for statisticians to work in other disciplines and substantive scientists to work in statistics, and industrial fellowships with funding from multiple external sources. In combination with post-doctorate and sabbatical programs, these activities will enable the Institute to focus attention on pertinent issues of cross-disciplinary research.
Methodology transfer and public education on the relevance and impact of statistical results and studies are overlapping concerns that bear on the use and acceptance of the results of cross-disciplinary work. The Institute should also be concerned with issues, such as statistical education, that are important to the future of statistical science and its applications. These concerns may be at some distance from cross-disciplinary research but are consistent with the need to promote such research.
It is important that the Institute nurture and advance the theoretical and methodological aspects of statistical science without downgrading its role to promote cross-disciplinary research. Adequate representation and careful management will be required, not to ensure that everyone gets a "slice of the pie," but rather to maintain awareness of the continued need for theoretical advances in all aspects of statistical research including cross-disciplinary work. The prospect of thereby engaging theoreticians in cross-disciplinary projects should be a major plus. Indeed, we believe that a strong statistical core together with outreach to other disciplines on the part of the Institute will serve to bring about the interactive process between theory, method, and application that is the very essence and life-blood of the statistical sciences.
In this appendix, we review at greater length past successes in collaborative research among statisticians and scientists in other fields. We have chosen only a few of the many historical examples where cross-disciplinary work directed to important practical problems had profound impacts on both statistical science and the field of application. Our examples pertain largely to the 20th century and are drawn from the fields of agriculture, medicine, military operations, and transportation and communication systems.
The introduction of experimental design for determining optimal combinations of plant nutrients and for gaining insight into basic growth processes was a major innovation in agricultural science that led to large-scale use of planned intervention studies in the field. These studies in turn constituted a major factor in the expansion of the world's food supplies.
The story of the development of planned experimentation in agriculture has its origins in the 1920s at the Rothamsted Agricultural Experiment Station in Harpenden, England. Here, Ronald A. Fisher formulated a theory of experimental design and developed methods for analysis of experimental data sets. He also developed methodology to provide reliable answers from small samples of observations. At about the same time, Jerzy Neyman, stimulated by problems in agriculture, developed theory leading to confidence intervals, one of the most important measures of uncertainty to emerge from statistical research.
Fisher applied his innovative techniques to answer such practical questions as the effects of manures and other fertilizers on crop yields from various strains of wheat and potatoes. What this new mode of experimentation accomplished was to provide ways to assess the effects of various treatments, such as different plant nutrients, simultaneously, and thereby to determine an optimal mix, as well as to identify causes and relationships. Fisher's ideas, exposited in two books, Statistical Methods for Research Workers (1926) and The Design of Experiments (1935), came rapidly to dominate the theory and practice of agricultural experimentation. They had a phenomenal effect on virtually all of the experimental sciences, not only by providing useful methodology but by providing a paradigm for comparative studies.
There can be little doubt that the cross-disciplinary research of the statisticians and agricultural research workers at Rothamsted and elsewhere was a major factor in the development of agricultural science and a triumph in the development of theoretical and applied statistics. Today, developments in the multi-billion dollar food industry are in part based on well-designed controlled experimentation together with technological advances in the fields of biology and chemistry. Current research, as reported in technical journals such as Food Research, The American Journal of Enologic Viticulture, and The Journal of Scientific Food and Agriculture, attests to the reliance of these fields on statistical design and analysis of experiments.
Apart from the contribution of plant nutrient research, the field of genetics had a major impact on agriculture, leading to the development of hybrid species with traits that maximize crop yields, shelf life, and other valued qualities. Statisticians contributed to the validation, through planned genetic experiments, of Mendel's original stochastic model for the inheritance of conceptual entities called genes. Indeed, the founders of modern statistics, Francis Gallon, Karl Pearson, and R. A. Fisher, were all stimulated by and deeply involved with both genetics and agriculture. The problems and needs of these two fields generated statistical problems whose solution led to many of the methods now in common use.
In addition to its role in agriculture, the genetic model had a major impact on the massive developments in molecular biology in the past three decades. These developments came almost totally from biological and biochemical studies. However, the probabilistic or stochastic elements of modeling in this area remain of critical importance. The formulation of models for genetic change and the accompanying stochastic processes had a large impact on the development of probability theory and represents an area of ongoing interest and research among biologists, probabilists, and statisticians.
In 1662, John Graunt used probabilities to estimate mortality rates. In 1747, James Lind performed his now famous clinical trial (without randomization) on the treatment of scurvy. (A similar trial is narrated in the Bible in the book of Daniel, in which Israelite youths persuaded their Babylonian captors to let them follow a vegetarian diet for two weeks to determine if they remained as healthy as the Babylonian courtiers whose diet included meats forbidden to the Israelites.) In 1760, Daniel Bernoulli used probability theory to analyze the relationship between inoculation and smallpox. concluding that inoculation at birth produced immunity and thereby increased life expectancy.
A physician of the early 19th century, Pierre Louis (1787-1822), is now considered by many to have been the father of modern epidemiological science. He carried out observational studies to determine the causes of typhoid fever and emphasized the need to discover laws on which the practice of medicine could rest. Louis had enormous influence in making statistics an integral part of epidemiological research.
During the past thirty years, clinical research on chronic diseases, epidemiological investigations, and clinical trials stimulated an extraordinary volume of interest and activity in statistics. The interaction between statistics and medicine has proven extremely successful, and, today, many statistical developments are widely accepted in the medical research world.
One of the foremost problems confronting statisticians working with medical researchers was that of estimating relative risk - that is, the proportion of persons who will contract a disease among those exposed to a risk factor compared with the proportion among those not exposed to the risk factor. An expensive and time-consuming method of estimating relative risk is to track cohorts of exposed and nonexposed persons over time. A more cost-effective method is to use a case-control design that compares diseased with healthy persons. Jerome Cornfield, working at the National Institutes of Health, showed that the odds ratio in a case-control design that is, the proportion exposed to the risk factor among the diseased group compared with the proportion exposed among the healthy group-will, under certain conditions, estimate relative risk. Subsequently, statisticians explored aspects of designs (matching, multiple matched controls, etc.) and engaged in complex analyses that took account of covariates in analysis of risk.
A second area of collaboration between the medical and statistical sciences dealt with methodology for evaluating the usefulness of drugs for treating disease. The initial call for a randomized controlled clinical trial came from statisticians, notably Bradford Hill in England. The claim that this approach was the best way to assess the efficacy of a treatment at first met with resistance from clinicians. Today, most clinicians in the medical disciplines recognize the merits of the clinical trial, and its use has become mandatory to satisfy regulatory requirements, notably by the Food and Drug Administration.
Although the use of clinical trials has raised many ethical and statistical issues of great importance, the major statistical development resulted from the need to analyze survival times before observing death for all subjects. Hence, methodology had to be developed to analyze clinical trial data in the presence of incomplete or censored observations. Estimates of a treatment or control survival probability must take into account each patient's preexisting conditions, the variable times at which patients begin treatment, the variable lengths of observations, and time-dependent covariates. The result is a very complex methodology - far more complex than simply comparing the proportions surviving at the end of a three-year or five-year study. Work on this topic during the last 20 to 25 years has provided an immense stimulus to statistical research and generated results which have had a profound effect on ongoing clinical investigations.
World War II was an opportunity for the statistical community to mobilize its efforts to help solve a number of military problems. As is often the case, the problems and their solutions had widespread application to many fields.
A Statistical Research Group (SRG) was set up at Columbia University during the Second World War. W. Allen Wallis served as research director, and the staff included many statisticians and economists who subsequently became leaders in their field (Wallis, 1980). The charge to the SRG was to serve the Armed Forces and the Office of Scientific Research and Development in the resolution of problems confronting military tacticians and the designers and producers of weapons.
SRG tackled a wide range of projects. Some of SRG's assignments were to specify the optimal mix of types of ammunition in fighter plane machine guns, to develop sampling-inspection plans for rocket propellants, to determine optimal settings on proximity fuses for air bursts of artillery shells, to evaluate the comparative effectiveness of four 20-millimeter guns versus eight 50-caliber guns, and to develop the geometry of aerial combat. Through these specific applications, the SRG staff made methodological advances of far-reaching theoretical and practical importance. Specifically, the group pioneered in the development of quality control procedures for manufacturing processes and the development of sequential analysis. The idea of sequential sampling-inspection plans that achieved economy by determining earlier stopping points arose from an application, but the later development of sequential analysis theory and methods had profound effects on the entire field of statistical science and proved of great practical importance for industrial production and other substantive fields.
Modern transportation and communication systems have benefited greatly from advances in statistical methodology. particularly from practical applications of queuing theory. The Danish engineer and mathematician A. K. Erlang pioneered the development of queuing theory in the early part of this century. An acquaintance, the chief engineer of the Copenhagen Telephone Company, introduced Erlang to F. Johannsen, the company's managing director. Johannsen himself had made efforts in the early 1900s to explain telephone waiting time and congestion behavior through probability models. He not only encouraged Erlang to become involved in these problems but went further and in 1908 set up a research laboratory within the company with Erlang as its director. From then on, Erlang pursued studies of telephone traffic patterns, using stochastic methods to model the phenomena. At the same time, Erlang maintained close contact with the company's engineers and with the circuitry and other telecommunication problems that they faced, even to the point of climbing into manholes to examine underground cable connections.
The deep involvement by Erlang in both theory and practice led to a series of seminal articles in telephone traffic and queuing theory. Scientists at the British Post Office and the American Telephone & Telegraph Company quickly recognized their value. The probabilistic models developed by Erlang, which were related to the birth and death processes found in population genetics, attracted many probabilists and stimulated the study and application of Markov processes and fluctuation theory. The extensive interaction between application and theory is evident in the literature, which, by 1960, contained about 250 papers on the theory of queues and about 500 papers on applications to telecommunications, transportation systems, maintenance and service systems, inventory systems, health care, hydrostorage (dams), etc.
With the ever-increasing complexity of applications and with the surge in the study of computer systems, the post-1965 developments in queuing theory have been largely oriented toward queuing network problems. These networks lead to very complicated models and have stimulated and employed research in areas such as point processes and simulation methods. The need for further research continues unabated.
Today, a large number of specific research areas raise statistical issues that should be addressed via cross-disciplinary investigations with the goal of advancing knowledge and practice in both the substantive field and statistics. In this appendix, we provide greater detail on selected areas where there are opportunities for important cross-disciplinary work. The topics singled out below in chemometrics, computer science, financial analysis for business decision making, food production, industrial production, and the use of highly complex materials and processes (random media) are only illustrative of a much wider class of areas that could benefit from collaboration between statisticians and other scientists. Advances in these areas also call for greater interaction within the statistics profession itself between statisticians working in theory, including probability theory and statistical inference, on the one hand, and those working in applied methods, including computation and modeling, on the other.
Chemometrics is the use of mathematical and statistical methodologies to analyze chemical data by extracting information from chemical measurements. Because chemistry is an experimental science, it interacts with statistics in a great many ways, including experimental design, optimization, simulation, and graphical data presentation.
An important driving force for joint work in chemistry and statistics is the existence of modern chemical instrumentation that provides large quantities of data, often at an extremely rapid rate. The data are frequently complex with a large number of dimensions, may sometimes have a time element, and can be further complicated because of missing values. In some instances, standard multivariate or time series methods may suffice for analysis, but, more commonly, novel developments are required, for example, to handle the problem of multivariate calibration.
The existence of large quantities of multivariate data provides an opportunity to develop and use exploratory data analytic methods for large data sets. The methods of pattern recognition and allied approaches, for example, could fruitfully be applied to many chemical problems. Potentially useful pattern recognition methods include mapping and display, discriminant analysis, clustering, and modeling. All of these methods involve using indirect approaches to study high-dimensional data that cannot be viewed or analyzed directly.
Cross-disciplinary research in this area must address related concerns such as the effect of different scalings and transformations on the behavior of exploratory data analytic methods when working with real data sets. Multivariate nonparametric methods must be explored, given that the data sets encountered in chemistry often are not Gaussian. To date, statisticians have not studied such nonparametric methods for handling large multivariate data sets in sufficient detail to be useful to chemists. Additionally, problems of errors, missing values, deviant observations and other anomalies in the data sets must be investigated. Although modern statistical researchers have addressed these and other complexities, their work has generally been confined to modest size data sets. Any good methodology developed on the basis of research on large complex data sets will have immediate application in chemistry.
Another needed area for cross-disciplinary work between statistics and chemistry lies in the application of experimental design methods to several types of chemical experiments. Examples are the search for optimum experimental conditions (e.g., temperature, pH, time) for organic synthetic reactions, and for optimum settings on complex instruments such as triple quadrupole mass spectrometers. Modern chemical instruments are too complex to permit manual tuning; they require more sophisticated optimization methods. Advances here would have important effects on chemical instrumentation.
Many chemical events cannot be studied using equations based on first principles. Examples include the scattering of ions off surfaces (studies of semiconductor materials), the retention of compounds by chromatographic columns (separation of biological molecules), and the modeling of complex chemical equilibria (e.g., chemical reactions in the atmosphere). A potentially useful approach in such instances is to employ simulation techniques. The general area of simulation provides many opportunities for the marriage of chemistry and statistical methods. Issues such as the propagation of error that arise in the application of simulation techniques to chemical models deserve increased attention.
In sum, chemistry presents an extensive array of opportunities for the useful collaboration of statisticians in cross-disciplinary research that can help to make fundamental advances that are profitable to both disciplines.
The field of computer science provides a particularly rich area for cross-disciplinary statistical research. This is due in part to the fact that computer science has developed as an engineering activity with emphasis on the deterministic rather than the stochastic nature of the systems being studied. Furthermore, the newness of the discipline and the systems in its purview present a variety of novel, hardly-touched problems. We outline several areas that are ripe for cross-disciplinary activities.
Scheduling and Communication in Parallel Processing. Three standard methods can be used to implement parallel programs: (1) multiprogramming where the processes share the memory of a single processor; (2) multiprocessing where the processes share a single memory but are executed on different processors, and (3) distributed processing where the processes are executed on separate processors each having its own memory. Two basic issues in parallel processing are synchronization of the several processes and communication among them. Synchronization and communication can be effected in several different ways depending on the method chosen for implementing parallelism.
Ideally, any system for implementing parallel programs behaves in a deterministic way, and the optimal allocation of work among the processors can be predetermined so that a minimum amount of time is taken to execute the task at hand. In actuality, every system behaves in a stochastic way. For example, the parts of a parallel system that are in use for standard timesharing services impose a stochastic load on at least part of the system. Consequently, it is not possible to predetermine the allocation of work to the processors, and an "optimal" allocation will require the analysis of a stochastic model of the behavior of the system.
A variant of this same problem is the behavior of an algorithm running on a parallel system rather than the behavior of the system itself. In some examples of solving linear systems of equations. each of several processors may be sequentially assigned one univariate minimization step of an iterative algorithm. However, while one processor is determining the displacement at a point x, other processors may complete their tasks and displace the point x to some other point. Thus, the displacement calculated by the one processor will not be applied at x but rather at some other unknown point. As the calculations continue, the processors will eventually fall completely out of phase with each other, but, remarkably, in certain cases the algorithm will converge nonetheless. Indeed, experimental evidence suggests that under certain conditions such chaotic iterative methods converge more rapidly than synchronous or partially synchronous iterative methods. This situation clearly calls for a stochastic model of the iterative process! A particularly useful by-product of a stochastic model is very likely to be a much more detailed understanding of the convergence of the method.
Probabilistic Analysis of Algorithms. A decade ago it was common to invent an algorithm to solve some particular problem and then to analyze its behavior in terms of the "worst case" performance. More recently, accepted practice is to assume a probability distribution for the inputs and then to calculate the expected performance under the assumed distribution. This is often extremely difficult because the behavior of complex algorithms is almost always conditional on previous actions. There are a number of opportunities for cross-disciplinary research in this area. They range from determinations of the expected behavior of specific algorithms under specific assumptions to the more general development of mapping of probability distributions on inputs to probability distributions on outputs, thereby generating the propagation of error and an error analysis.
Uncertainty in Rule-Based Expert Systems. Rule-based expert systems have moved from a select type of research activity to a growing commercial endeavor. At the same time, the artificial intelligence research community has come to recognize that, in most decision-making situations, the data (namely the rules supplied by the "expert" and the initial evidence supplied by the "user" to start the inferential process) are not known with certainty. Hence, the inference procedures used in traditional rule-based systems are not appropriate. Over the last decade, researchers have developed a number of numerical representations of uncertainty together with related inference procedures for use in rule-based systems. However, none of these schemes has proved widely successful, and there is currently no generally accepted method for incorporating uncertainty into a rule-based expert system. Research and development of useful principles and measures of uncertainty in expert systems are badly needed.
The pressures on American business firms to improve their competitive posture in the world economy put a premium on effective decision making. Yet there is growing concern that the accounting and auditing systems used in business firms are not appropriate instruments upon which to base sound management decisions (see Kaplan, 1986, and Frosch, 1987). This concern stems from at least two sources.
First, accounting systems are generally designed for the purpose of calculating federal and state taxes, satisfying Securities and Exchange Commission requirements, and systematizing annual reports. They do not suitably address risk aspects inherent in management decisions. Techniques that incorporate sophisticated stochastic models to quantify some risk aspects of decision making are only now being developed and used to compute an expected value of those decisions which preserve future action options. Such "option theory" methods are useful in quantifying the value of flexible manufacturing capability, of establishing a pilot plant, of investing in automated equipment, etc. Many statistical issues are fundamental to the development of decision-making frameworks that extend beyond the usual "discounted cash flow" analysis.
Second, there is concern that accounting systems need to make more effective use of appropriate statistical methodology in order to provide an adequate information base for business decisions. Adoption of appropriate statistical techniques can generate large savings in many "routine" accounting procedures, such as auditing (e.g., for inventory or tax purposes), allocating overhead expenses, and estimating future business expenses attributable to current sales (e.g., warranty expense). Applicable statistical techniques in this context include the use of complex sampling schemes, methods for analysis of censored data, forecasting methodology, Bayesian methods, and the application of non-standard mixture models (see the recent Guthrie Report, 1988).
Managers, regulators, and statistical researchers in business, academia, and government need to combine efforts to address the issues involved in improving the capabilities and efficiencies of accounting and auditing systems to support effective business decision making. The statistical community can make major contributions in two areas: first, in developing improved methods for quantifying important stochastic aspects of specific classes of business decisions, and, second, in bringing to the attention of the accounting and auditing community advances in statistical methodology that may enhance their current practices.
Global influences on food production and the quality of the environment are now generally recognized. Researchers are developing very large complex computer simulation models to study the possible consequences of problems such as deforestation, pollution, acid rain, overgrazing, and the greenhouse effect. Using large-scale simulations to develop international policies for adequate global food production entails problems similar to those found in using complex system simulations to determine policies for the manufacturing process. However, the former task is further complicated because the input factors themselves are complex systems driven by random effects.
For example, simulating the interaction among the various processes by which chemicals enter the ecosystem, diffuse into the water supply, and affect food production and health involves a variety of models in which different disciplines have a role, glued together by statistical formulations and interpretations. In particular, geophysical and hydrological models describe the diffusion of chemicals through the water supply, plant-growth models describe crop development, moisture models incorporate climate factors, and agro-economic models describe the value of alternative policies.
The complexity of these models is evident by considering the following components:
There has not been sufficient study of the inadequacies and uncertainties in complex models of global food production to permit the adoption of available statistical methodology. Additionally, novel methods must be developed to cope with the complexity of the models and their applications. Statistical analyses of these systems are essential for analysis of alternative food production policies and of the costs and risks involved. Any evaluations will require large teams of scientists with complementary knowledge in diverse disciplines, ranging from hydrology to economics, and the capability to deal with vast data sets embedded in highly complex systems. The need to conduct large-scale cross-disciplinary research projects is compelling the problems of food production and climatology, by their very nature, are immediate and primary, with ramifications extending from the individual farm to the future of the planet.
Current developments in industry are generating important problems whose solutions require statistical research. Manufacturers are producing an increasing variety of products composed of a larger number of components of greater complexity than was true in the past. This proliferation of product lines and the increasing complexity of manufactured products have given rise to new problems in design and scheduling, process control, and quality assurance. New methods are needed to maintain order and to obtain high yields, as well as to avoid increases in fixed costs, warranty and service costs, recall campaigns, and liability claims.
The systems approach has evolved in manufacturing in response to these problems. A parallel development of statistical tools should accompany this development, so that appropriate and more powerful statistical procedures are used in manufacturing. Historically, industrial firms have used statistical tools to determine efficient inventory and replacement policies, manage queues productively, and estimate and contain the risk of defective parts. These tools need to be extended in scope and complexity to meet the challenges of the systems approach. Methods must be developed to evaluate the applicability of various statistical techniques for analyzing the outputs of simulations of manufacturing systems. These methods must consider multiple inputs and outputs, as well as multiple system uses. System-level statistics are needed for hierarchical control, reliability, quality control, scheduling, and computer-integrated manufacturing. Advances in computational mathematics, subjective inference, and exploratory data analysis make it feasible to move statistical process control to continuous product improvement.
Software products, as well as hardware, have experienced tremendous growth in volume and diversity. Manufacturers of computer software confront difficult and at times unusual and unique problems in development and production. Statistical approaches to providing quality assurance in software manufacturing have not yet evolved. However, it is clear that poor quality implicates software design as well as implementation. There is an urgent need to explore the role of statistics in improving all aspects of software manufacturing.
There has been much recent interest, and important methodological advances, in understanding the properties of materials and processes so complex that they can only be described statistically. This activity promises to affect a great number of scientific and technological fields and involve numerous theoretical and experimental investigators from government, industry, and academia.
Effective medium theory, or the prediction of bulk properties of composite materials from the known statistics of their microstructure, is gaining increased importance in materials science, chemical engineering, and other technological disciplines. The results can be strikingly counter-intuitive as, for example, in the case where the introduction of a small volume of bubbles into a liquid produces a sound velocity considerably below that of either the liquid or the gas. The mathematics of homogenization theory have been applied to problems in diffusion, electrical conductivity, and other aspects of the behavior of composite materials, with important practical results. New methods, such as differential effective medium theory and techniques of compensated compactness, make this a rapidly advancing field.
Waves in a randomly nonhomogeneous medium can be described by effective medium theory, if the wavelength is sufficiently large and the propagation distance is not too long. For long distance propagation, random effects can predominate so that no description in terms of propagation through an effectively homogeneous medium is applicable, and, instead, the wave behavior is understandable only through random scattering mechanisms. There are many applications of work on such mechanisms, including exploring the optical properties and electronic density of states of amorphous semiconductors.
Theories of random scattering of classical waves describe diverse phenomena ranging from the twinkling of stars to the corruption of se