Beyond the pixels
The digital revolution has radically changed the way people interact. Today almost half of the world鈥檚 population are connected to the internet whether it鈥檚 watching or sending videos, looking at images on social media, reading or watching the news. We now have 24/7 access to the rest of the world online. But not everything we see is reality.
Pam Johnston, a developing lecturer at 亚博体育官网鈥檚 School of Computing Science and Digital Media, has been looking beyond the pixels to detect when an image or a video has been manipulated 鈥� ultimately preventing digital tampering.
Pam initially started her journey with 亚博体育官网 as a KTP associate working with Dr Eyad Elyan and a local technology company and when that came to an end, she applied for a fully-funded PhD which focussed on learning and utilising video compression features for localisation of digital tampering.
鈥淰ideo compression is pervasive in digital society,鈥� Pam comments. 鈥淲ith rising usage of deep convolutional neural networks (CNNs) in the fields of computer vision, video analysis and video tampering detection, it is important to investigate how patterns invisible to human eyes, like compression, may be influencing modern computer vision techniques.
鈥淚鈥檝e been looking at how compression affects learning in neural networks. I knew all data would be compressed to some degree and that compression algorithms were specifically designed to fool human eyes but no one seemed to be looking at whether they also affected 鈥� or even fooled 鈥� machine vision algorithms. I look at videos and see that they are compressed, which means they鈥檙e not perfect representations of reality. The compression patterns reveal a little bit about the history of a video and I started investigating whether computers could detect that 鈥� whether CNNs could be trained to estimate the level of compression right from the pixels themselves.鈥�
Pam explains that around the time of her research 鈥楧eep Fakes鈥� hit the headlines, also known as 鈥楧igital Puppetry鈥� which is a form of video tampering that involves altering people鈥檚 faces, or putting someone鈥檚 face over an actors.
Pam explained: 鈥淭he thing that made my research a little bit more unique is that I looked at trying to detect multiple different tampering techniques with one algorithm. We reviewed some of the current video manipulation techniques and it鈥檚 frightening to see how realistic some of them are. The whole point of tampering a video is to make it look like it hasn鈥檛 been tampered with. Humans can鈥檛 even see some video tampering, let alone guess at the type of tampering, so new, computer-based detection techniques are required and they should be broadly applicable to multiple tampering types.
鈥淥f course, you need quite a lot of data and a few powerful computers to train a CNN to detect compression straight from the pixels, but we managed it to some degree in the end.
鈥淚 am working towards a detection technique that works on different kinds of video manipulation 鈥� I have one algorithm that detects two different kinds of tampering using compression estimates from my trained CNN.
鈥淯ltimately, I want to be able to detect images that have been entirely manufactured and reliably detect images that have been manipulated but then re-compressed. So right now my research is focussing more on detecting recompression from pixels, or the number of times a given image patch has been compressed. Again, that鈥檚 mostly invisible to human eyes, but research shows that computers are quite good at it.
鈥淚f we can detect it reliably, then we鈥檒l be able to detect when a video has been manipulated and help to prevent digital puppetry.鈥�
