Computer Vision & Graphics Machine Learning & Data Science

Virginia Tech & Facebook Video Completion Algorithm Achieves SOTA Results

Researchers introduced a novel flow-based video completion algorithm that compares favourably with the state-of-the-art in the field.

Video completion is a challenging computer vision task that involves filling a given space-time region with newly synthesized content — in effect, revealing the unseen. Video completion has been widely applied in applications such as video restoration, editing, watermark/logo removal, etc. Most advanced video completion methods are flow-based: synthesizing colour and flow jointly and propagating colour along flow trajectories to improve temporal coherence. Now, researchers from Virginia Tech and Facebook have introduced a novel flow-based video completion algorithm that compares favourably with the state-of-the-art in the field.

Existing flow-based video completion methods have the following three limitations: they are unable to synthesize sharp flow edges and so tend to produce over-smoothed results; the chained flow vectors between adjacent frames can only form continuous temporal constraints, which prevents constraining and propagating to many parts of a video; and they propagate colour values directly without considering factors such as lighting changes, shadows and so on.

Screen Shot 2020-09-08 at 1.55.13 AM.png
Screen Shot 2020-09-08 at 12.52.08 AM.png

The proposed method addresses these limitations in four ways:

  • Flow edges: It obtains piecewise-smooth flow completion by explicitly completing flow edges and utilizing the completed flow edges to guide the flow completion.
  • Non-local flow: It introduces additional flow constraints to a set of non-local frames such as those that are temporally distant, creating shortcuts across flow barriers and propagating colour to more parts of the video.
  • Seamless blending: By operating in the gradient domain, it avoids visible seams in the results.
  • Memory efficiency: It can deal with up to 4K resolution videos (where previous methods fail due to excessive GPU memory requirements).
Screen Shot 2020-09-08 at 1.41.26 AM.png
Screen Shot 2020-09-08 at 1.44.23 AM.png

The researchers validated their proposed method on 150 video sequences from the DAVIS dataset, where visual and quantitative results show that the proposed method compares favourably with state-of-the-art algorithms. There are however a couple of limitations. The method still has some failure cases under circumstances such as fast motion, which can result in poorly estimated flow and poor colour completion. Also, the method runs at 0.12 fps, which is slower than the 0.405 fps speed of end-to-end models.

The paper Flow-edge Guided Video Completion is on arXiv. Visit the project page here.

Analyst: Yuqing Li | Editor: Michael Sarazen; Yuan Yuan

Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors

This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon KindleAlong with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.

Click here to find more reports from us.

We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.

13 comments on “Virginia Tech & Facebook Video Completion Algorithm Achieves SOTA Results

  1. digitmgmsa

    It has fully emerged to crown Singapore’s southern shores and undoubtedly placed her on the global map of residential landmarks. I still scored the more points than I ever have in a season for GS. I think you would be hard pressed to find somebody with the same consistency I have had over the years so I am happy with that.

  2. I like your video! Well done

  3. I truly like your style of blogging. I added it to my preferred’s blog webpage list and will return soon…

  4. David Lewis

    Thank you for this post! Why don’t you post your video on tiktok? Now is easy to get tiktok likes. Read this guide how to get tiktok likes

  5. 360DigiTMG

    Good information you shared. keep posting.
    certification of data science

  6. course training

    Just saying thanks will not just be sufficient, for the fantastic lucidity in your writing. I will instantly grab your feed to stay informed of any updates.
    data scientist course

  7. The results must be amazing!

    • William Garrison

      Computer vision is the result of processing images received from a digital camera to make appropriate decisions. For example, we want to design a driver assistance system that automatically detects pedestrians, and if the pedestrian is close enough in front of the car, it starts to brake automatically. To solve such problems, you must first of all understand what the image is, what physical and geometric laws are used to obtain it, and then consider the algorithms and approaches to solving the problems of short circuit, which are known today. I advise you to familiarize yourself with the posts on this topic on instagram, their authors usually use to increase the number of subscribers.

  8. digitmg 360

    Thanks for sharing nice information….
    data analytics training in aurangabad

  9. Hii,
    This is Great Post.. for me, Thanks for sharing with us!!
    Buy Real Facebook Live Stream Views

  10. profitlanguage

    I appreciate you letting me know about this krunker

  11. Larry Martin

    The advancement of flow-based video completion algorithms is a promising development for enhancing video editing and restoration tasks, showcasing the ongoing innovation in computer vision.
    Heavy Item Moving in Lake Martin Alabama

Leave a Reply

Your email address will not be published. Required fields are marked *