National Academies Press: OpenBook

Alternative Technologies to Replace Antipersonnel Landmines (2001)

Chapter: 4 Evaluation Methodology

« Previous: 3 Current Uses of Antipersonnel Landmines
Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×

Page 30

4

Evaluation Methodology

There is no formula or specific set of criteria on which to base a determination of the point at which the putative military utility of AP mines is exceeded by the humanitarian costs of their employment. This is a complex problem involving subjective judgment; for a nation to deny its military the use of a weapon that could assist in prevailing against opposing forces while reducing friendly casualties, there must be a compelling case that the weapon is not indispensable to legitimate military operations and that its utility is clearly outweighed by the resultant killing and wounding of innocent civilians. (Gard, 1999)

The committee considered a large number of potential alternatives to APL. Some are under development as part of the U.S. government's three-track initiative, and some are being developed by other nations. Still others were suggested by nongovernmental organizations and independent scientists committed to the elimination of APL. Members of the committee also suggested several possible alternatives; some are improvements to existing systems and some totally new. This chapter describes the methodology, baseline systems, criteria, and performance factors the committee used to rank potential alternatives.

METHODOLOGY

Information about the alternatives described in this report was gathered over a 10-month period. The sponsor, in meetings with the committee, stressed that all types of alternatives should be considered: materiel and nonmateriel, as well as existing, developmental, conceptual, and notional technologies. Although the committee cast as wide a net as possible, some potential alternatives may have been missed. At the time the data-gathering phase of the study ended, the Track III initiative had reduced the number of concepts under consideration to seven finalists. Three of these were undergoing concept evaluation by industry and were considered competition sensitive; therefore, these were not considered in this study.

Although independent modeling or simulations may have assisted the committee in its evaluations, this was not feasible within the committee's constraints of time and budget. Therefore, the committee developed a score sheet to measure the effectiveness of alternatives. Although the analysis is admittedly subjective, and the results are qualitative rather than quantitative, the scoring was helpful for ranking the alternatives.

The criteria for the score sheet were developed by the entire committee. Committee members with technical expertise provided detailed write-ups of each of the candidates. A subcommittee of technical and military operational experts then met and scored the alternatives against the criteria. The results were presented to the full committee, and adjustments were made, as necessary. The score sheets resulting from these assessments are provided in Chapter 5, Chapter 6, and Chapter 7.

The committee adopted the terminology used in previous studies by DOD to categorize types of alternatives. Materiel alternatives are weapon-like systems; nonmateriel alternatives are nonweapon solutions, such as changes in doctrine, organization, concepts, and training or weapons designed for another purpose. Only the materiel alternatives were scored.

BASELINE SYSTEMS

There are many very different types of mines. Mines used solely against a dismounted enemy are so different from the mine systems used against mounted targets that the committee divided its alternatives into two categories: those to be used against a dismounted threat and those to be used against a mounted threat. It then selected two baseline systems against which candidate systems could be measured for all criteria: the M14/16 nonself-destructing mines for use against a dismounted threat and the Volcano system (M87), a mixed system, for use against a mounted threat.

CRITERIA

The committee developed the criteria as a qualitative guide for assessing how well alternatives would fulfill the requirements for APL. These are availability, military effects, humanitarian concerns, overall technical risk, tactics and doctrine, and economic factors. The first three sets of criteria were most important to the committee's overall assessments and recommendations. The committee reiterates that the scores are qualitative and, therefore, somewhat subjective. They were used to help organize the committee's thinking, but not as a basis for pass/fail judgments.

Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×

Page 31

Availability for Implementation

The first and most important criterion was the availability of the system, in keeping with the requirement in the Statement of Task that the committee identify technologies that would be available by 2006. Although 2006 was the critical milestone, many emerging technologies that most likely would not be available until after that date showed considerable promise. Ultimately, the committee divided the potential alternatives into three groups: systems available now (see Appendix C and Chapter 5 for detailed descriptions); systems currently under development that have a reasonable likelihood of being available by 2006 ( Chapter 6); and technologies unlikely to be available until after 2006 ( Chapter 7).

Military Effects

The second criterion was whether the alternative system could fulfill DOD's requirements (i.e., be militarily effective). The committee's choice of the criterion was neither arbitrary nor capricious. After several attempts to design military criteria for landmines, the committee determined that the capabilities described in the mission need statements1 were the most complete description available.

A mission need statement is one of the first documents to describe the requirement for a new piece of military equipment. The mission need statements used in this study were validated by the Joint Requirements Oversight Council, a high-level element of the Office of the Joint Chiefs of Staff, and were considered adequate by DOD for industry to use as a basis for designing alternatives. Therefore, the committee considered them to be an accurate reflection of military service requirements. Although the performance standards in these documents are at a higher level than the requirements for technologies described in the study's Statement of Task (i.e., they would perform better than current APL, not similarly), they provide the most complete description of the requirements for mine alternatives known to the committee; an approved operational requirements document was not available. The standards can be read in their entirety in Appendix G.

The numbering of each criterion coincides with the numbers on the score sheets. A score of zero shows that the alternative performed at the same level as the baseline system. A positive number indicates that the alternative performed better than the baseline system. A negative number shows that the alternative did not perform as well as the baseline system.

Against Mounted Threats

The Mixed Landmine System Alternatives mission need statement describes requirements for a mixed system for use against a mounted threat (see Appendix G). The committee used the Volcano (M87) as the baseline system against which alternatives were judged.

A1 Enhance effects of close and deep friendly fires

0 = same as Volcano

1 = greater coverage than Volcano and/or technological improvements

2 = man-in-the-loop or improved sensors

3 = sensors with battlefield awareness

−1 = nonlethal component or easily detectable

−2 = longer sensor-to-shooter cycle

−3 = longer sensor-to-shooter cycle and a small footprint

−4 = longer shooter-to-sensor cycle, a small footprint, and may require a cease fire from other weapons to protect the delivery platform

A2 Has multiple methods of delivery

0 = at least two methods of delivery (as many or more than Volcano)

−1 = only one method of delivery

A3 Provides a range of effects that inhibit mounted and dismounted maneuvers

0 = similar to Volcano, has both AT and APL components

1 = sensor that enables use of something other than an APL mine to counter dismounted targets

2 = multiple sensors/highly sophisticated sensors

−1 = inhibits only personnel or only tanks; nonlethal against personnel

−2 = nothing in place on ground to stop or slow enemy movement

A4 Resists full spectrum of enemy breach methods, including dismounted methods

0 = has AP and AT components

1 = has man-in-the-loop to react to visual observation of breach

2 = unbreachable without great risk to troops; very difficult to breach

3 = multiple sensors, allowing other weapons to be brought to bear; larger area covered; automatically counters breach attempts

−1 = nonlethal AP components; some AT components have antihandling devices

−2 = has only AT or only AP

N/A=no minefield to be breached

A5 Provides early warning of ground attack

0 = might provide early warning within normal observation range

1 = will provide early warning because of man-in-the-loop (observes ground attack)



1 The two current mission need statements that relate to APL are Battlefield Shaping and Force Protection against Personnel Threats (alternatives to APL when used alone) and Mixed Landmine Systems Alternatives (alternatives for current mixed systems, combining both APL and AT mines) (see Appendix G for full text).

Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×

Page 32

2 = will provide reliable early warning via sensors at greater distance to observe attack

−1 = will not provide early warning within normal observation range

A6 Is safe for friendly forces

All alternatives approved in the mission need statements will be as safe as Volcano by doctrine. DOD will not develop a weapon that poses a threat to its own forces.

0 = as safe as Volcano

A7 Is effective in all types of terrain and weather

0 = terrain and weather have no effect

−1 = cannot see target because of weather/terrain; reaction requires sensor input that can be affected by weather/terrain

A8 Poses minimal residual hazard to own forces and noncombatants after military conflicts

0 = can self-destruct/self-deactivate; no ground munitions to explode (munitions have exploded in the air or on contact with the ground)

1 = can be command detonated; is strictly nonlethal

A9 Is difficult to detect by enemy forces

0 = same as Volcano; may be visible when shot off but are not easily visible on ground

1 = smaller and less easily seen than Volcano

2 = nothing to be detected on the ground

−1 = bigger, easier to see; sends a signal that can be detected

−2 = visible and audible by the enemy before breaching

A10 Poses minimal risk of fratricide

0 = has an APL component

1 = has no APL component; has nonlethal AP component

2 = man-in-the-loop allows identification of friend or foe before detonation; more accurate minefield location

3 = sensors tell own troops where munitions are; munitions can be turned off when own troops approaching; is strictly nonlethal

A11 Effects modifiable for mounted and/or dismounted threat

0 = once in place munitions are either AT or AP

1 = has man-in-the-loop, therefore other systems can be used

2 = more sensors allow several options for bringing other weapons to bear

A12 Has controllable activation/deactivation and duration before and after installation.

0 = set before emplacement, cannot be changed after

1 = set at time of use; man-in-the-loop

2 = can do many things with munitions (turn on/off; command destruct)

A13 Is effective in nuclear, chemical, and biological environments

0 = not affected (same as Volcano)

−1 = sensors or communications may be affected by electromagnetic pulse

A14 Is easy and efficient to distribute

0 = remotely delivered; AT and AP components delivered together

−1 = emplaced by hand; components delivered separately (i.e., requires two rounds)

−2 = more components require hand emplacement than Volcano

Against Dismounted Threats

The Battlefield Shaping and Force Protection Against Personnel Threats mission need statement describes the capabilities required for protection against a dismounted threat (see Appendix G). The committee used the M14/M16 as the baseline system for evaluating these alternatives.

A15 Can delay, disrupt, and/or canalize enemy movement/ maneuvers.

0 = same as M14/M16

1 = greater area covered; greater range; selection of when to detonate; line of sight

2 = detection of enemy forces further out via sensors allow time for use of other weapons

−1 = nonlethal with strong effects that might affect enemy movement

−2 = nonlethal with weaker effects not likely to affect enemy movement

A16 Denies enemy access to terrain/facilities (including short-term/long-term deterrent for boundaries and DMZ areas)

0 = same as M14/M16

1 = greater area covered; greater range; selection of when to detonate; line of sight required

2 = detection of enemy forces further out via sensors, allowing time for use of other weapons

−1 = nonlethal, with strong effects that might affect enemy movement

−2 = nonlethal, with weaker effects that are not likely to affect enemy movement

A17 Enhances effects of friendly force weapons, obstacles, and munitions (including AT mines)

0 = same as M14/M16

1 = greater coverage; man-in-the-loop

2 = better sensors

−1 = nonlethal

A18 Generates exploitable delays and opportunities (fixes or contains enemy)

Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×

Page 33

0 = same as M14/M16

1 = greater area covered; greater range; selection of when to detonate; line of sight required

2 = detection of enemy forces further out via sensors allowing time for use of other weapons

−1 = nonlethal, with strong effects that might affect enemy movement

−2 = nonlethal, with weaker effects that are not likely to affect enemy movement

A19 Generates detection, alert, and/or early warning

0 = makes a noise, alerting friendly troops to enemy intrusion

1 = sensors alert to enemy intrusion

2 = more sophisticated sensors alerting of intrusion at greater distance (sooner)

−1 = no noise is made autonomously; line of sight required; difficult to cover blind spots

A20 Facilitates classification of target

0 = no classification

1 = man-in-the-loop

2 = man-in-the-loop with sophisticated sensor

A21 Produces desired effects on enemy forces (nonlethal to lethal)

0 = solely lethal or nonlethal; no munition

1 = permits choice of munition (lethal or nonlethal)

2 = has inherent selection of effects, from lethal to nonlethal

A22 Reduces casualties/risk for U.S. and/or allied forces

0 = same as M14/M16

1 = self-destruct/self-deactivating or man-in-the-loop

2 = nonlethal; more positive identification at greater ranges

A23 Deters pursuit to facilitate breaking of contact under pressure

0 = no man-in-the-loop, weapon does not require operator

−1 = man-in-the-loop; nonlethal

Humanitarian Concerns

The third criterion was fulfillment of the humanitarian intent of international agreements, such as the CCW and the Ottawa Convention. Although compliance with these agreements was not specifically included in the Statement of Task, the committee considered it implicit in the rationale for this study. Senator Leahy, a strong supporter of detmining, providing aid for victims of mines, and the Ottawa Convention, introduced legislation requesting this study (Rieser, 1999). Because humanitarian concerns were clearly one impetus for this study, the committee used those elements from the CCW and the Ottawa Convention that addressed the post conflict hazard of mines and possible injuries to noncombatants as humanitarian criteria. The five characteristics listed below, derived from the CCW and the Ottawa Convention, were applied to alternatives used to counter mounted and dismounted targets.

Against Mounted Threats

B1 Presents postconflict residual hazard (CCW)

0 = no self-destruct/self-deactivation mechanism; munitions persist on ground

1 = has self-destruct/self-deactivation; easily removable; purely nonlethal; no munitions persisting on ground

B2 Location can be recorded (CCW)

0 = approximate locations are known

1 = more precise location known

2 = will self-record where they are

3 = no munition

B3 Detectable to facilitate removal (CCW)

0 = sufficient metal content for detection

1 = no munitions

B4 Discriminates between combatants and civilians (Ottawa)

0 = has no means of discrimination

1 = less than positive target identification (i.e., with a nonvisual sensor); observed fire

2 = discriminates by man-in-the-loop, visual identification; has nonlethal AP component

B5 Does not explode on presence, proximity, or contact of a person (Ottawa)

0 = explodes on presence, proximity, or contact

1 = does not explode on presence, proximity, or contact; is nonlethal

Against Dismounted Threats

B1 Presents residual hazard (CCW)

0 = no self-destruct/self-deactivation mechanism

1 = is self-removing (via self-destruction/self-deactivation) or nonhazardous after war; is very easily removable; is strictly nonlethal

B2 Location can be recorded (CCW)

0 = is hand emplaced

1 = has man-in-the-loop or sensor

B3 Detectable to facilitate removal (CCW)

0 = sufficient metal content for detection

1 = no munitions

B4 Discriminates between combatants and civilians (Ottawa)

0 = has no means of discrimination

1 = less than positive target identification (i.e., with a nonvisual sensor)

2 = discriminates by man-in-the-loop, visual identification

Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×

Page 34

B5 Does not explode on presence, proximity, or contact of a person (Ottawa)

0 = explodes on presence, proximity, or contact

1 = does not explode on presence, proximity, or contact; is strictly nonlethal

Overall Technical Risk

In assessing overall technical risk, the committee assigned the highest score to systems already in production and progressively lower scores to systems with technologies beyond the state of the art. This very simple scoring was helpful for assessing the more “futuristic” alternatives.

C0 0 = in production

−1 = capability has been demonstrated

−2 = capability not prototyped, but uses technology within the state of the art

−3 = technology is beyond the state of the art

Changes in Tactics and Doctrine

The consideration of tactics and operational concepts was specified in the Statement of Task. This criterion was given a simple yes or no on the score sheet to indicate whether changes in tactics or doctrine would be necessary to implement the alternative (see D0 on score sheet).

Cost

Because cost2 was not included in the Statement of Task and because the determinations by the committee were only estimates, scores for this criterion only indicate the committee's evaluation process. The following costs were considered for each alternative:

E1 Research and development

0 = weapon already exists

−1 = known, funded cost; relatively low cost; prototype exists or is technologically easy to achieve

−2 = no prototype, but elements of the system exist; technology appears to be straightforward

−3 = conceptual stage; requires a technology breakthrough

E2 Procurement cost

0 = production base exists/has been produced

−1 = production capability exists, anticipated product costs are low

−2 = requires complex manufacturing process and/or high product cost

−3 = manufacturing processes are conceptual, costs unknown



2 Despite the sponsor's admonition that cost not be considered a factor, it was included because the committee believed it might be a consideration for the future. However, cost estimates were rudimentary.

Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×
Page 30
Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×
Page 31
Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×
Page 32
Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×
Page 33
Suggested Citation:"4 Evaluation Methodology." National Research Council. 2001. Alternative Technologies to Replace Antipersonnel Landmines. Washington, DC: The National Academies Press. doi: 10.17226/10071.
×
Page 34
Next: 5 Alternatives Available Today »
Alternative Technologies to Replace Antipersonnel Landmines Get This Book
×
Buy Paperback | $57.00 Buy Ebook | $45.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This book examines potential technologies for replacing antipersonnel landmines by 2006, the U.S. target date for signing an international treaty banning these weapons. Alternative Technologies to Replace Antipersonnel Landmines emphasizes the role that technology can play to allow certain weapons to be used more selectively, reducing the danger to uninvolved civilians while improving the effectiveness of the U.S. military. Landmines are an important weapon in the U.S. military’s arsenal but the persistent variety can cause unintended casualties, to both civilians and friendly forces. New technologies could replace some, but not all, of the U.S. military’s antipersonnel landmines by 2006. In the period following 2006, emerging technologies might eliminate the landmine totally, while retaining the necessary functionalities that today’s mines provide to the military.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!