Trust, Confidence, and Organizational Decisions about AI Adoption: The Impact for US Defense
Speaker: Horowitz, M. (University of Pennsylvania)
Date: 20 October 2020
Speaker Session Summary
SMA hosted a speaker session as a part of its SMA General Speaker Series, featuring Dr. Michael Horowitz (University of Pennsylvania).
Dr. Horowitz started the presentation by laying out three key findings derived from his research. First, the key challenges in adopting military innovations are often organizational, rather than financial or technological. Second, great power competitors and non-state actors view artificial intelligence (AI) and robotics as technologies with the potential to disrupt US military and economic superiority. Lastly, the gap between innovation rhetoric and action within the US defense complex generates risk for US conventional military superiority in the coming decades. Dr. Horowitz then described how to think about key technologies from a military power perspective today and discussed how they have developed and might develop across the globe. He then used armed drone proliferation as an example of a recent up-and-coming, cutting-edge technology that numerous states have developed.
Next, Dr. Horowitz explained what AI is. He described it as something that can direct physical objects, process data, and perform overall information management. It is an umbrella of technologies, and its impact is much broader than that of traditional military technologies. Much of AI is developed by the private sector, which demonstrates that technology that is driven by markets spreads faster. As a result, countries around the world can invest in general purpose technologies’ capacities that otherwise wouldn’t be available to them. General purpose technologies consist of technology umbrellas with broad applications across many areas of the world, such as the combustion engine, electricity, or computing, Dr. Horowitz explained. They also have significant consequences for global power, directly, indirectly, and over time. He added that technological change impacts short-term military power and a state’s economic capacity, while simultaneously causing social and political disruption, which ultimately leads to long-term economic and military power.
According to Dr. Horowitz, there are three stages that govern the way that adoption of these technologies occurs. The first stage is referred to as “Technology Hype,” or when a new innovation causes inflated expectations of how a given technology will change the world. The second stage is known as the “Trust Gap,” or the inability to trust machines to do the work of people, in addition to the unwillingness to deploy or properly use these systems. The third stage is referred to as “Over Confidence,” or the inverse of the trust gap. In this stage, over confidence leads to risky bets on technology, delegation of cognitive judgment to a machine, and a failure to question algorithms if they make mistakes. Examples of this stage include the Air France crash of 2009 and the 2003 Patriot Missile fratricides. This framework is of particular importance because nearly every country has military incentives to rapidly develop AI and robotics, Dr. Horowitz argued. This is why there is ongoing international competition between nations with respect to AI. Furthermore, in the military innovation process, there are four periods: invention, incubation, implementation, and diffusion. Innovation has a “debut” point in which there is a key shift from the incubation period, where there is a lot of experimentation and testing, to the implementation period.
Dr. Horowitz concluded by discussing two pathways, or futures, in the development and implementation of AI. The first Dr. Horowitz referred to as “Dual Use AI/Robotics,” in which commercial incentives for technological invention in a relatively multilateral competition cause more rapid diffusion of technology. In this scenario, first mover advantages are relatively limited, and fast followers could plausibly catch up. The second pathway is “Militarily-exclusive AI/Robotics,” in which commercial spillovers could be relevant, but it is ultimately unclear. In this scenario, there is relatively bilateral competition with less rapid diffusion. Furthermore, first mover advantages are much larger, and it is more plausible to “lock in” advantages.
Speaker Session Recording
Note: We are aware that many government IT providers have blocked access to YouTube from government machines during the pandemic in response to bandwidth limitations. We recommend viewing the recording on YouTube from a non-government computer or listening to the audio file (below), if you are in this position.
Comments