Drones – the flying robots with paired propellers you can fly at home – seem to be capable of more and more things by the day. Scientists have recently published work that shows that they can be used to provide a model for the true flying car; not only that, but they can also demonstrate how these hypothetical vehicles can navigate built-up areas and conform to hybrid air/surface traffic management systems. Drone research and development can also be put to good use in aerial photography. The robots can carry high-end cameras that can deliver sweeping vistas and scenery in cinematic shots. In fact, their work is already evident in films such as Mad Max: Fury Road and the Jurassic Park sequel.
However, camera drones are somewhat resistant to detailed directorial requirements, such as tracking a specific subject and adjusting its shooting angle to do so. But now, the limitations of drone photography have been challenged by a recent project completed at MIT. This work may increase the range and depth of drone programming so as to improve the content of the footage the robots capture from the air. This may ultimately lead to new dimensions in video recording quality for people who work with professional-grade camera-drones.
Drones, particularly high-end prosumer or consumer versions, are capable of recording singular aerial video sequences of high resolution and moderate length, which are often a boon to vloggers posting to YouTube and other platforms. With sufficient scaling and funding, these capabilities can be translated to Hollywood-level cinematics. Directors can use high-end drones to capture impressive vistas, or at least partial action scenes from above. However, cinematographers are often limited as to what they can programme the drones to do; the robots may only point their cameras in one direction as they fly along.
One solution may be to use multiple drones that work in tandem to get more extensive coverage or footage from an increased number of angles; however, the drones often lack the ability to avoid collisions or synchronise their movements with respect to other equipment and personnel. A team of researchers from the Computer Science and Artificial Intelligence Laboratory (CSAIL, a group that studies robotics and improvements in robot-human interfaces within MIT) and ETH Zurich decided to address these problems with innovations in drone controls, algorithms and sensors. Led by Professor Daniela Rus, they enhanced the ability of a test drone to cover one subject and follow them with greater accuracy and precision – without crashing into anything in the process.
Drones and AI
This process involved developing a form of artificial intelligence (AI) for the drone to respond to an image of a certain subject, at a specific angle, taking up a certain area of the screen. For example, if a director wanted a shot of an actor, in profile, standing in the centre of the screen.
The drone was thus programmed to use its native speed and agility to keep the subject in the frame under those conditions at all times. Therefore, the AI-enhanced drone could achieve improved accuracy and quality in the shots it was supposed to get. As the team perfected this code, they could then move on to fine-tuning this method of recording. This meant that the drone could be ‘taught’ to prioritise one condition, for example keeping the subject centred on screen, over another condition, such as keeping the subject in profile. The aim of this was to get even more realistic and high-quality footage. Rus and her team also devised algorithms that helped the drone stay ‘aware’ of the obstacles around it, to prevent collisions and enhance seamless shooting. This may help the drone work around crew, other aerial vehicles and equipment working in its environment, or even link up with a team of similar drones to contribute to a completed scene.
Technology with much potential
This new development in camera drone technology may have additional applications. It may enhance search and rescue operations as they assess large-scale, complex and elevated terrain to locate missing climbers or disaster survivors. Camera drones with this technology could also track criminals identified through facial recognition. Admittedly, this prospect may invite applications with potential implications for individual privacy, such as widespread technological surveillance. However, the most obvious and immediate directions are new-generation, enhanced drones with improved control interfaces and recording capabilities.
It must be noted here that some current consumer models already have some of these functions, in some form. For example, the DJI™ Spark is programmed to recognise a number of gestures made by the user and move in response to them. It can also move independently in relation to said user, giving the impression that it can follow someone as it flies. However, some feedback in the media suggests that this interface method has a certain degree of awkwardness, is complicated in terms of instructions and may also be a little unreliable. The contributions of this team may one day be used to improve this hands-free drone interface and enhance camera-drone operation in general. Dr. Rus – who also led the project that produced the ‘flying car drone’ as described above – presented these findings at the IEEE conference on robotics and automation, which took place earlier this year.
This new development in drone technology allows a flying robot to orbit around someone to keep them in shot, rather than just automatically recording them in one direction even if they turn suddenly. Cinematographers may hope to integrate the work done at CSAIL and ETH Zurich before too long. The new style of drone-enabled filming may help more directors rival sequences such as those found in modern-day masterpieces such as Game of Thrones.
Top image: System directs camera-equipped drones to maintain framing of an aerial shot. (Larry Hardesty | MIT News Office)
References:
Hardesty, L. Cinematography On The Fly. CSAIL News. 2017. May 17. Available at: http://www.csail.mit.edu/cinematography_on_the_fly
Froats, L. 7 Movies And Tv Shows You Didn’t Know Used Drones. Drone 360. 2017. Available at: http://blogs.drone360mag.com/drone360/2016/06/28/7-movies-and-tv-shows-you-didnt-know-used-drones-to-set-the-scene/
LTT. DJI's "fun-sized" drone.. isn't that fun. Linus Tech Tips. 2017. Available at: https://www.youtube.com/watch?v=v5z-F7BpBrw
No comment