Report Details Government鈥檚 Ability to Analyze Massive Aerial Surveillance Video Streams
Yesterday I wrote about Dayton Ohio鈥檚 plan for an aerial surveillance system similar to the 鈥渘ightmare scenario鈥 ARGUS wide-area surveillance technology. Actually, ARGUS is just the most advanced of a number of such 鈥減ersistent wide-area surveillance鈥 systems in existence and development. They include , , (used on blimps in Afghanistan), and .
One of the problems created by these systems鈥攚hich have heretofore been used primarily in war zones鈥攊s that they tend to generate a deluge of video footage. A 2010 article that American UAVs in Iraq and Afghanistan produced 24 years鈥 worth of video in 2009, and that that number was expected to increase 30-fold (which would be 720 years鈥 worth) in 2011. Who knows what that鈥檚 up to this year, or where it will be by, say, 2025. The human beings who operate these systems can't possibly analyze all that footage.
In an attempt to solve this problem, Lawrence Livermore Labs has created a system for the military called 鈥淧ersistics.鈥 It can be used in conjunction with drone (or manned) camera systems such as ARGUS to help manage the vast oceans of video data that are now being generated. The system is
designed to help the Department of Defense and other agencies monitor tens of square kilometers of terrain from the skies, with sufficiently high resolution for tracking people and vehicles for many hours at a time.
That鈥檚 from a May 2011 that I recently came across with the faintly ominous title 鈥淔rom Video to Knowledge.鈥 Produced by Livermore Labs, it contains a lot of interesting detail about Persistics and the problems and solutions involved in massive aerial video surveillance.
The Persistics system consists of algorithms that 鈥渁nalyze the streaming video content to automatically extract items of interest.鈥
Its analysis algorithms permit surveillance systems to 鈥渟tare鈥 at key people, vehicles, locations, and events for hours and even days at a time while automatically searching with unsurpassed detail for anomalies or preselected targets.
With Persistics, the report boasts, 鈥渁nalysts can determine the relationships between vehicles, people, buildings, and events.鈥 Among the capabilities touted in the report are:
- 鈥淪eamless stitching" together of images from multiple cameras to create 鈥渁 virtual large-format camera.鈥
- Stabilizing video (鈥渆ssential for accurate and high-resolution object identification and tracking鈥).
- Eliminating parallax (the difference in how an object appears when viewed from slightly different angles).
- Differentiating moving objects from the background.
- The ability to automatically follow moving objects such as vehicles.
- Creating a 鈥渉eat map鈥 representation of traffic density in order to 鈥渁utomatically discern if the traffic pattern changes.鈥
- Comparing images taken at different times and automatically detecting any changes that have taken place.
- Super-high 鈥1,000-times鈥 video compression.
- The ability to provide all the locations a particular vehicle was spotted within a given time frame.
- The ability to provide all the vehicles that were spotted at a particular location within a given time frame.
Technologically, according to the report, the Persistics program relies heavily on the explosion in the power of consumer Graphics Processing Units (GPUs) used in video games and the like.
The report also says that the system 鈥渋s being further enhanced鈥 to work with ARGUS, and includes new details about that system:
Persistics can simultaneously and continuously detect and track the motion of thousands of targets over the ARGUS-IS coverage area of 100 square kilometers. ARGUS-IS can generate several terabytes of data per minute, hundreds of times greater than previous-generation sensors.
Previous reports said that ARGUS could cover 15 square miles; here it reports 100 square kilometers, which is 38.6 square miles. (I suppose we should expect Moore鈥檚 Law-like expansion in the capabilities of these systems.)
Of course, the system is designed to store and retrieve all the records and data about everything that it surveils:
Persistics supports forensic analyses. Should an event such as a terrorist attack occur, the archival imagery of the public space could be reviewed to determine important details such as the moment a bomb was placed or when a suspect cased the targeted area. With sufficiently high-resolution imagery, a law-enforcement or military user could one day zoom in on an individual face in a heavily populated urban environment, thus identifying the attacker.
As with every privacy-invading technology designed and/or sold as helping foil terrorists, we have to wonder how long it will be before it鈥檚 applied to tracking peace activists.
Future work on Persistics is focused on the kind of behavioral analytics that have been discussed in the context of programs such as 鈥Trapwire.鈥 Livermore scientists, according to the report, are now working on automated methods for identifying 鈥減atterns of behavior鈥 that could indicate 鈥渄eviations from normal social and cultural patterns鈥 and 鈥渘etworks of subversive activity.鈥
Also under development are efforts to allow the three-dimensional viewing of targets, as well as 鈥渕ethods to overlay multiple sensor inputs鈥攊ncluding infrared, radar, and visual data鈥攁nd then merge data to obtain a multilayered assessment.鈥
Of course, much of this is unobjectionable from a domestic civil liberties point of view when it鈥檚 used as originally intended: on foreign battlefields. The problem comes when the government brings the technology home and turns it inward upon the American people. In fact, at the close of the report, Livermore contemplates exactly that:
Unmanned aircraft have demonstrated their ISR [intelligence, surveillance, and reconnaissance] value for years in Afghanistan and Iraq. As U.S. soldiers return home, the role of overhead video imagery aided by Persistics technology is expected to increase. Persistics could also support missions at home, such as monitoring security at U.S. borders or guarding ports and energy production facilities. Clearly, with Persistics, video means knowledge鈥攁nd strengthened national security.
Among the federal agencies most interested in the technology, the report says, is DHS.