Click for next page ( 458


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 457
457 Alexander Levis "introduction by Session Chair" Transcript of Presentation Summary of Presentation Power Point Slides Video Presentation Alexander Levis is chief scientist of the U.S. Air Force, Washington, D.C. He serves as chief scientific adviser to the Chief of Staff and Secretary of the Air Force and provides assessments on a wide range of scientific and technical issues affecting the Air Force mission. Dr. Levis received his professional education at the Massachusetts Institute of Technology. Prior to his current position, he was University Professor of Electrical, Computer and Systems Engineering at George Mason University in Fairfax, Virginia, and head of the System Architectures Laboratory of the C31 Center. For the last 20 years, Dr. Levis has conducted basic and applied research in and taught many aspects of command and control, from organization design for command centers, to operational and system architectures, to decision support systems. Dr. Levis has served as senior officer in national and international professional societies, is on the editorial board of several professional journals, and on the Board of Directors of the AFCEA Education Foundation. He has held two appointments to the Air Force Scientific Advisory Board, where he participated in several summer studies and several ad hoc studies. 457

OCR for page 457
458 DR. LEVIS: Just on a serious note on that, the term stegonography was mentioned in the first session, and people asked what is it. As far as I know, Al-Queda has used it already. So they are fairly sophisticated. Good afternoon. My name is Alex Levis, and I will be chairing this session. My current position -- I have never worn a uniform in my life, but my current position is with the Air Force. I am chief scientist. It is the best job in the world. It has no specification. What I am supposed to do is give advice to the chief of the Air Force and the Secretary of the Air Force. The Secretary of the Air Force did his Ph.D. on decision analysis at Harvard. He does not need any advice from me on matters technical. The chief has a war to fight, he needs no advice from me. So I am having a great time coming to workshops, and I really appreciate the invitation. Somewhere in my distant path actually, I do have a degree in mathematics, but it is an undergraduate one. I noted when David earlier -- there was a question asked yesterday, how many people have a degree in mathematics, and four or five people raised their hands. The question today that David asked is, how many mathematicians, and 458

OCR for page 457
459 two-thirds of the people raised their hands. I wonder what the significance of this is. I would like to make some comments first. I have learned a lot of interesting things yesterday and today. Information security I am a little more familiar with. But the thing that hasn't come up very much is the problem. Those who want to contribute in mathematics, and I believe strongly in that, and I'll try to show a couple of examples, mathematics to the problem. I am an engineer, and I need to understand the problem before I apply solutions to it. The problem has not been defined in this workshop. We all took it for granted. From the discussion it is apparent that our knowledge is what we all hear from the newspapers, et cetera. I don't think even the White House has defined homeland security, let alone -- I don't know the difference between homeland defense and homeland security, by the way, but different people use them in different ways. So let me try to define in non-technical terms the problem. We don't really know who the adversaries are. We know some individuals, we know some organizations, but we don't have a complete knowledge of the adversaries. We don't know where they are. You see every night on TV that we really don't know where they are, and we don't know 459

OCR for page 457
where they are going to attack, whether it is going to be in the U.S. The call just went out again, after being prepared for a year and a half. We don't know when they will attack. We do not know what they will attack, and we don't know how they are going to attack. We know bits and pieces of that, but this is the classic journalism, the five questions. So when we talk about homeland security, we have to look at all those aspects. You can characterize the uncertainty. We talked a lot about probability yesterday and a little bit today, but there are different kinds of uncertainty that are associated with this. You have temporal uncertainties, you have locational uncertainties, you have all sorts of things that somehow have to mix and match together. This session, remember, is about integration and fusion. Now, the strategies that you can consider fall in three categories: reactive strategies, -- PARTICIPANT: By the way, you forgot one question. We don't know why they are attacking us. DR. LEVIS: No, I 'm sorry, that we know. They don't like us. We can start a debate, but I come from that part of the world, and I can tell you in great length why they don't like us. 460

OCR for page 457
461 We have concentrated mostly in terms of what people have been discussing and the approaches they are indicating. We are looking primarily at the reactive problem, what happens if somebody does that. Much of the work that was presented in epidemiology would go into the reactive part: once there is an attack, what can we do about it. There is also the anticipatory part. Some of the work yesterday relates to that. We know that somebody is crossing the broader. We don't know where, but we are going to look for it, we are expecting an attack on the bridges of San Francisco during that week. This is anticipatory. The other one is the proactive; get them before they get a chance to get us, which is what we are doing right now, what the Air Force is actually doing. The term that the Air Force uses for this kind of a notion, that is, the last one, is predictive battle space awareness. It doesn't really mean anything. This is a bumper sticker to put at the end of the car. It tells you it is great to know what they have done. It is great to have all the sensors to tell me what happened, but that really is of very limited value. What I really want to know is what 461

OCR for page 457
462 they are going to do next, so I can go and get them before they do it. That is the hard problem. With the expansion of sensors, information technology, et cetera, as many of you indicated, we have a wealth of data, but our difficulty is to take that data and start determining intent, which takes us away from just projection of the data. We have to understand what is going on, because you need to know intent, to understand intent, to make the predictions. The first part I have already mentioned. Yesterday we talked a lot about searching for a needle in a haystack. I am the last person to tell you how to do that. I have no idea now to do that. But there are alternative approaches, how to think about the problem. First of all, somebody mentioned yesterday getting a magnet or changing the properties of the needle so that you can find it more easily, given the kind of sensors that you have. A lot of our intelligence community is worrying about things like that. There are ways of trying to mark things that we eventually may want to find. That is one way of approaching it. The last one is, find the needle before it gets in the haystack. As an engineer, this appeals to me very much. I am lazy. I want to solve the easy problem, I 462

OCR for page 457
463 don't want to solve the hard one. We need to keep this in mind. There is a whole spectrum of activities over here. To get to this predictive integration notion, given that there are large gaps in knowledge, because you can get to intent -- intent is human, and you can't really model all that stuff. You cannot go to databases to find those things. You need to do some modeling and simulation. This is now a case study to respond to a question that was raised later, you have all those good solutions, how do you go and give them for people to use. That is a very hard problem. You cannot just say, I have a wonderful solution. I think it was mentioned, you have to understand the science of the problem, we have to understand the application of the problem, because nobody is going to pay attention. You expect somebody else to understand the solution and then convert it and apply it. Not anymore; nobody has the patience for that. One needs to go out. One example over here very quickly which is relevant to the presentations today, that is why I chose it, has to do with influence nets. After 9/11, actually a couple of weeks later in September, I went to the Air Force Studies and Analysis. These are the people who do the actual math and do modeling and so forth for the Air Force. Originally they were the whiz kids. Some of you who are my 463

OCR for page 457
464 age will remember McNamara; this is the whiz kids group of McNamara's. They still exist, with a fancy name. I gave them some software from the lab, research software, to start modeling Al-Queda. They did it, and it has been used since then to do a number of analyses. That software was certainly not for prime time. It was homemade by my master's level students. Since then, we have got people from the Air Force Research Laboratory, this is from Rome Lab up in upstate New York, the information directorate, and they were developing software over there which had some relation to mathematics, but it was not very explicit what the relationship between the software and the mathematics was. It was very heuristic in a way, but it did approximately the same things, but it is much more user friendly. Once we established from the laboratory the validity of the approach using more rigorous tools, a heuristic approach was being used and is used currently. Now we need the data. Some of the data came from databases, as described yesterday, but some of the data had to do with judgments. We brought in the analysts from many of the three-letter agencies that were discussed yesterday to provide that information. 464

OCR for page 457
465 The moment the analysts got that stuff, after they learned within two or three days how to use the software and how to model in that way, immediately they started asking for more. All this does is, you wiggle the inputs to see what the outputs are. This is good for planning, but how do I go to the execution? This is what we showed to the generals. This way they understood the influence nets and Bayesian networks, except for the Secretary, who does know Bayesian networks. Fortunately, I did my homework, and before I discussed those things I knew that he knew Bayesian networks, and I didn't say silly things to him. But the idea of how to use them is a little bit different here. You have to put yourself in the mind of the adversary and define a set of effects that you want to achieve. We want Milosovich to change his mind, we want Saddam Hussein to stop bothering us and go on vacation or something like this. You put that stuff over here, and then you start looking, what are the things that influence his decisions. This blob here is where the modeling takes place. You want to bring them to the point that you have what is called actional events, the things that we can do -- we are 465

OCR for page 457
466 the good guys, the blue -- that will eventually have an effect here. This is the kind of models -- now we have made a bunch of models of those things. Actually we have been using this kind of stuff in DoD since 1994, in the intelligence part of DoD. They use it that way. But now the question came up -- and I wish I had mathematicians working with me to solve the problem, because I don't know -- and my students don't know the answer, and this is not the field that we work in. Suppose now that they start the engagement. Bombs start dropping or things are happening in Afghanistan. I start having observations that come over here from various intermediate nodes; this thing occurred, that thing occurred. Can I propagate forward? Sure, I can propagate forward except I have to update first and update the priors and do all those things, and do it. That is brute force, one can do it. It takes a few weeks, implement. But then they are a question. If I look at this node happened well am interest sensors you get it done, you more sophisticated and I see what has and that improves my information over here, how I doing in reaching the outcomes, can I solve the problem? Can you tell me where I should put to see what is happening, so I can get a better 466

OCR for page 457
467 understanding of whether I am achieving my goals or not. That is a lot harder problem, but from what I know, mathematicians will know how to do them correctly, as opposed to trying to ad hoc them, the way it is occurring right now. But you cannot do it in two years. You need to be ready to go and work and do it in a couple of weeks and put it in. I hope it will be over before it is actually needed. This is a real transition, but it took a lot of people. That is one of the good things when you are chief scientist; I could call people and cash in all my IOUs. This is how it has been established now, studies and analysis, now has the capability to do modeling, using Bayesian nets. We never had that capability before. The research laboratory, Air Force Office of Scientific Research, indeed, some of you in the audience probably know it, that is the basic research component of the Air Force research establishment. This is the applied research, the Air Force Research Laboratory. They are providing the algorithms and so forth. Here is the alphabet soup of the intelligence community, providing the data and the updates. We are using it, responding to decision makers. What we need here 467

OCR for page 457
556 So the problem is very complex, it is very difficult, it is a large country. There is no way that you can border it totally, disallow anybody to enter it and so forth, the way of life that has been discussed. We have a huge intelligence establishment for which we pay a lot of taxpayers' dollars. Part of their job is to do the anticipatory and not even suggest, just bring the data up front for the appropriate decision makers to decide whether to be proactive or not. I am speaking as an individual, but I don't think we are doing that badly, when you consider how many real instances of terrorism we have had in this country. It is the most open country in the world that I know. PARTICIPANT: I wasn't suggesting that we have done badly. It was just a concern of mine towards this sort of strategy around a lot of what we heard today. There was a sense that we could not -- PARTICIPANT: You are not by the microphone. PARTICIPANT: There was just a very defensive kind of mindset, and I was just concerned that this feeling of vulnerability could itself -- I was just concerned about the strategy part. DR. DEMPSTER: It seems to me to be the new thing that there are events of extremely low probability that 556

OCR for page 457
557 have extremely high costs. focusing on a bit. DR. LEVITT: Alex mentioned something in his opening. He said, I could tell you, but then I'd have to kill you. I think the problem with the offensive stuff is that it tends to be handled by the intelligence community. I think there is an enormous amount going on, but I don't know much of it, and therefore I wouldn't know, as far as germane to this workshop goes, where the mathematical challenges lay in offensive activity, which I think are highly focused. However, I completely agree with the scope of homeland defense. There is the newspaper version, or what I consider the public political version. On the other hand, where I see the real payoff is that there is enormous amount, and there has been for man years, of data in automation, collection, and critical things going on, that really can be exploited. I think that we don't know what the limits of automation might be, we don't have to have people, you are talking PCs and memory. If we could deal with the inferential issues, so that we didn't generate false alarms all the time, that there might be a tremendous payoff. That is what maybe we should be 557

OCR for page 457
558 DR. LASKEY: I wanted to comment on that, and say that I agree thoroughly with the points you made about not sitting around and just sorry if my talk came how it was intended. passively observing. In fact, I'm across that way, because it wasn't Inference can be used to passively observe, but you notice that level five of the five levels of fusion that I talked about was placing your sensors. Part of what you are doing is, you are not just passively observing, but you are watching and anticipating what is going to happen and preparing yourself to respond to it, but also at the same time, if we discover that if there is an Al-Queda cell that is planning something, I'm sure that our military people would go in and stop them before they had the chance to do anything about it. But it is not just waiting until the horse is gone to close the barn door. The technologies that I was talking about would apply much more broadly. DR. LEVIS: Other comments? Questions to the speakers? The timetable for the airplanes? I'd like to make one comment and a question, and then we can adjourn, if you like. Earlier today there was a question about funding. This is Washington, so I would like to address it very briefly. 558

OCR for page 457
559 For mathematics, the Air Force has the Office of Scientific Research. There is the Mathematics and Computer Science Division within it. I think one person was present here today. They are looking into those problems, and it is basic research. However, things have changed a little bit, and you have to write more than one paragraph or one opening sentence. The old joke, everything looks like a worm. You have to write a little more and show some understanding of how that work will have an impact. Of course, good mathematics has a good impact, but that is not sufficient. There is a lot of scrutiny for relevance. So understanding the problem makes also a much better proposal, even though you are doing the mathematics that you want. People mentioned DARPA. I have known DARPA for many, many years. For DARPA you have to know the problem. There is a very small part of the DARPA budget that is in basic research. Most of it is in applied research and demonstrations. In the process of using those large three to five year demonstrations, a lot of research is being done, and I am sure some of you have participated. But those programs are fairly focused in trying to solve a particular problem and make a demonstration. So in order 559

OCR for page 457
560 to play, you don't have to know every aspect of the problem. They are usually large themes, and you spend an awful lot of time going to meetings and coordinating on those things. But there is a substantial amount of funding that allow the basic research to be done in the context of the applied research. But you have to know -- the quick kill three program is, if you are going to have a new algorithm that is going to do it faster, cheaper, better and all the other things. There is a substantial amount of funding. The Defense Information Systems Agency together with DARPA, they have a joint program office called JPO. That office has kicked off a major homeland security initiative. It is not exactly new money yet, there will be new money in the future, but right now they are earmarking existing money everywhere that could be pointed in that direction. I have been through that exercise, I write a lot of proposals. I can see how we can take some of the work that we are doing, dress it up appropriately, and look at that problem. But in order to get funding for that, I will have to be credible that at least I understand some aspects that are relevant to homeland security, not leave it to the reviewer to make the connection. I have seen enough from 560

OCR for page 457
561 the inside, the paperwork, there is somebody that will have to write a paragraph that is that long on a form, explaining why your work is relevant to the problem. So you are passing the buck to somebody else. When you are doing that, things don't work very well. It is much better if you write it yourself, if I write it myself and submit it and discuss it. It helps the proposal along. But there is funding that is developing within the various -- I haven't checked with the Office of Naval Research, but I am sure they are looking at that problem in the mathematics section, but one has to be a little more closely aligned to the problem than was discussed in here, at a fairly abstract and high level. Thank you. I don't have any money, by the way. Whoever would like to close -- DR. KELLER-McNULTY: I guess I was voted in on this. I am the only other board member besides you two here. I just want to first of all thank everybody for coming to the workshop. I think it has been pretty exciting. I know that I have learned a lot, and I have tried to listen carefully. I think that there are clearly great challenges mathematically for us to try to address that can be very 561

OCR for page 457
562 supportive to homeland security. I also think that there is a fair amount of what we already do that can almost be directly applied to some of the immediate problems, and it behooves us to seek out those opportunities and try to do that. We heard a lot of things in the last couple of days. Some of the things that resonated with me were the following. First of all, we have to keep the human in the loop. No one is talking about -- and some of our colleagues when we talk to them, some of our scientific colleagues, they get really nervous that we are trying to develop methods that somehow take them out of the decision making loop. I think it was pretty clear from everything we heard that the human has to be in the loop, the science has got to be incorporated into the problem, the domain knowledge has got to be included in the solutions and in the frameworks we build to solve these problems. Which then of course points to one of the very first things that Alex said when he opened the final session, which was that you have got to know the problem. Clearly, if we want to get collective funding to work in this area, we had better darn well be willing to admit to knowing that we are trying to solve a real problem, and try to figure out how to pull the pieces together. 562

OCR for page 457
563 The other thing that came out really clear to me is -- and something that we sometimes forget to talk about when we are talking to people about the problems in the work we are doing, is that quantification underlies almost everything we are doing here. We tend to forget about it, because it is part of our being as mathematical scientists and as computer scientists. But everything is about uncertain quantification. The question that came up about the nuclear weapons certification, it is not just about predicting a point value, whether or not we think these systems will work. It is all about uncertainty quantification and knowing when we have to go back and start testing again, or collect other types of information. That is true throughout all the homeland security problems. A lot of people talked about how we are in a stage and in an era of being swamped with data, we just have so much of it. What was it that Art said? That data is dumb, which is true. Kathy commented that we have all this data, data, data, we don't have time to think. That is true on a certain level, but as soon as we ratchet these problems up to the incredible dimensions that they are -- and Tod showed us what the dimensionality of the space is, there actually is a lot of data sparsity. 563

OCR for page 457
564 In the last session, I think that Val's talk tried to show how we can try to put things together in the presence of a lot of data when you take some slices through the problem, but not a lot of data or information when you take other slices through that problem space. So that is really important that we remember as well, that it is not just about massive data mining, but it is about how to integrate all of this information. The final thing I want to say is that what we are talking about is modeling complex systems. Perhaps we do have to put hats on a bit like engineers and take a systems approach to looking at these problems and figuring out how we can build and develop the mathematical frameworks to actually make progress. I don't think the problems are impossible. I actually disagree with one of the last comments that was made from the floor, that a lot of what I heard at this workshop is about prediction and forecasting, how do we take what we have learned, what we are seeing, and project ahead and forecast ahead, to try to understand what the next thing is that needs to be done or where the next vulnerability is. These are not easy problems. We have to come together and work on them. I hope that we do. I have been 564

OCR for page 457
565 pretty energized by this and I look forward to continuing to interact with many of you on these. That's it , . Safe travel. 565

OCR for page 457
566 Remarks on Data Integration and Fusion Alberio Grunbaum Mathematics has anmazing, almost poetic, ability to look at a problem in a certain area and then take a step back and sucicten~y realize that maybe we can use these tools for some other problem in a different area. For example, there are a number of people who have worked first in geophysics and then in mectica] imaging! He cautioned, however, that as he and his colleagues movect in this poetic fashion from one area of application to a different one, it was very important to remind themselves that they tract to go back and start from scratch and talk to the experts in this new fieict. The problems might look the same from the mathematical point of view, but they have all very different features. 566