February 25, 2025
I recently attended the NATIONAL TRAINING & SIMULATION ASSOCIATION Congressional Modeling and Simulation (M&S) Caucus Conference. The event helps shape legislative priorities for the coming year as they relate to modeling and simulation. While the focus is largely on the Department of Defense (DOD), many of the training concepts discussed connect well with the fire service. AI was a major topic throughout, with policymakers, military leaders, and industry experts debating how it fits into simulation, training, and national security.
Congressional leaders made it clear that AI is no longer something distant. It is happening now, and how it is used will shape the future. VADM (Ret) Sean Buck spoke about the need to direct AI in ways that benefit people. Representative John Rutherford pointed out Florida’s role in simulation and its effect on multiple industries, from national defense to disaster response. He also stressed the need to bring the AI and M&S caucuses together, as they are currently separate. He warned that foreign competitors are not waiting around for the United States to catch up. AI-driven systems like DeepSeek, while powerful, are also being used as data-harvesting tools by adversarial nations. Rutherford has a particular interest in disaster management, given the frequency of hurricanes in his state, and believes AI and simulation can help improve responses to large-scale emergencies. He also sees digital twins becoming standard practice rather than an occasional tool. He raised the question of how to move past AI generating false information.
Representative Jack Bergman, speaking via video message, pointed out that AI brings challenges and opportunities. He believes Congress has a responsibility to guide its development in a responsible way.
The keynote was by, Yevgeniya (Jane) Pinelis, Ph.D., an expert in AI testing and evaluation for the DOD She raised important questions, such as whether AI needs to be better than a human at a task or simply fast enough to be useful. In high-risk environments, AI that is slightly less reliable than a person might still be the best option. But AI is also vulnerable to cyber threats. The process of measuring its effectiveness must be defined, and it needs to be built in a way that people can trust.
Dr. Pinelis explained that there is no single approach that works for every situation. The expectations for an AI pet are obviously different than those for an AI system managing nuclear weapons. Testing and validation depend on available time and budget, which are always limited. Those developing AI must decide which risks they are willing to take and make sure those risks are understood.
A common misconception point out by Dr. Pinelis, is that AI will cut human workload to the point where human input becomes unnecessary. In reality, people still have to correct AI’s mistakes, make final decisions, and verify that the information provided makes sense. AI must give people the right details at the right time, in a way that makes sense to them. Transparency is essential so that operators understand AI’s strengths and weaknesses. This is especially relevant in emergency response, where AI-driven tools can support decisions but should not replace human judgment. Ethical concerns were raised as well. AI must be built in a way that supports decision-making rather than overriding it. Trust comes from thorough testing, clear oversight, and a way to shut down AI systems when they fail.
Dr. Pinelis then talked about what I think is the key takeaway from the conference, modeling and simulation can play a larger role in testing AI. Since AI systems must be evaluated in countless situations, simulation offers a way to test them without the risks and costs of real-world failures. Running AI through simulated environments helps refine decision-making and allows for adjustments before deployment. AI-driven simulations could help train firefighters in incident command, predict fire behavior in real time, and fine-tune response strategies.
During a panel discussion on AI policy, Bharat Patel of Project Linchpin stressed the need to separate AI from traditional software. He suggested placing data collection devices on personnel, units, and vehicles to feed AI training models, but said that legislation would be needed to make this possible. Mark Gombo from Microsoft Federal highlighted the importance of public-private partnerships in AI development and pointed out that bad AI outcomes make headlines, while good AI applications are often ignored. He emphasized that AI should be tested in as many environments as possible and that knowing what level of risk is acceptable depends on the situation. Jaimie Weber from Tampa General Hospital spoke about AI’s role in healthcare, stressing that human oversight is essential. She noted that AI struggles with unusual cases and that medical professionals need to be trained to work alongside it rather than just rely on it.
Final discussions reinforced the idea that AI is rapidly expanding into all areas of national security, emergency response, and public safety. The fire service should be thinking about how to use AI while maintaining control over decision-making. AI has the potential to improve training and real-world responses, but it must be tested and understood before it is fully integrated. AI-powered simulations could change the way firefighters train, helping them practice for complex emergencies before they happen. AI tools for dispatch and incident command could speed up response times and improve resource management. But over-reliance could lead to a loss of traditional skills, making it harder for responders to understand why a recommendation is correct. Human oversight will always be necessary, especially in life-or-death situations.
The NTSA Congressional M&S Caucus Conference made it clear that AI’s role in defense, emergency management, and training is growing rapidly. Policymakers and industry leaders are pushing for stronger testing, smarter integration, and ethical guidelines to prevent misuse. The fire service can learn from these discussions by thinking ahead about how AI will fit into operations. By using modeling and simulation to refine AI applications, emergency responders can build trust in new technology and make sure it is an asset rather than a liability.
Speaker presentations can be found here: