Motion Saliency

Published: October 06, 2022

Another ancient publication in computer science terms, but the main paper still gets cited, so I continue to provide my old code and publications on the topic. And who know, with the advent of IoT smart cameras, who need to be light on hardware, maybe there is new life in the near future for the old computer vision approaches?

The inital work was done by a research group at the Faculty of Technical Sciences and focused on Video Quality Assesment, as a part of the resarchers was involved on project for a local company with interests in that domain. I had an interest in visual saliency since my PhD days, fueled by the work of Laurent Itti and Cristoph Koch, to which my attention was drawn to by my advisor, colleague and friend Prof. Oge Marques and I thought it that saliency has to play a significant part in how users experience degradation in the quality of video introduced by coding. Plus saliency was a far more fun topic for me to study.

The work on visual saliency in images could not be transferred to video directly and I felt that motion is acutally key in this case, so I combined what I knew about moving object segmentation and saliency and created this code. As usual, it's C++ and OpenCV. I haven't built it in a while. Hopefully it still works.

The VQA stuff was then just a bunch of previously-proposed metrics extracted separately for salient and non-salient regions. To get the estimate of the quality as percieved by the users I used Wakaito Evironment for Knowledge Analysis (WEKA), another tech that I encountered in my PhD days, to train different ML models to do it for me. If you're interested in trying it out for yourself, you can download my old data set in WEKA format.

More recently, but still a long time ago, the saliency model went on to be featured in more publications on which I collaborated with my friend and colleague Matei Mancas, and his gruop at the University of Mons, Belgium.

If you want to read up about it a little more, these publications should help:

If you would just like to see the code in action, take a look at the videos below.

"Ant" sequence MPEG2 coded at 0.5 Mbps: