{"id":128,"date":"2019-08-02T21:06:55","date_gmt":"2019-08-02T19:06:55","guid":{"rendered":"https:\/\/websites.fraunhofer.de\/video-dev\/?p=128"},"modified":"2019-08-23T11:24:50","modified_gmt":"2019-08-23T09:24:50","slug":"calculating-vmaf-and-psnr-with-ffmpeg","status":"publish","type":"post","link":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/","title":{"rendered":"Calculating VMAF and PSNR with FFmpeg"},"content":{"rendered":"\n<p><a href=\"https:\/\/ffmpeg.org\/\">FFmpeg<\/a> is a great tool for video processing, it basically allows us to manipulate videos any way we like. Depending on the concrete use case, however, it can be challenging to assemble the right command. In this blog post, I will explain how to calculate the <a href=\"https:\/\/github.com\/Netflix\/vmaf\">Video Multi-Method Assessment Fusion<\/a> (VMAF) and <a href=\"https:\/\/en.wikipedia.org\/wiki\/Peak_signal-to-noise_ratio\">Peak signal-to-noise-ratio<\/a> (PSNR) scores in a single FFmpeg command, and produce the results in JSON format or a text file.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The scenario<\/h3>\n\n\n\n<p>Let us assume that we&#8217;ve created multiple encodes from our high quality source video and would like to check if the quality of our encodes is sufficient enough to deliver the video to the end user. As an example, this is a mandatory step for per-title encoding, which I explained in one of my previous <a href=\"https:\/\/websites.fraunhofer.de\/video-dev\/per-title-encoding\/\">blog posts<\/a>. The most straightforward way to determine the quality of a video is by watching it and assigning a subjective quality score. For instance, we can rate the quality of our encodes on a scale from 1 to 5, with 1 being bad and 5 being very good. This simple scoring system is often referred to as the Mean Opinion Score (MOS). However, it requires us to watch all of our encodes multiple times, which obviously only scales to a certain degree. Even Netflix and YouTube can not afford to put thousands of people in front of a TV and let them rate the quality of their encodes. Hence, we need a more sophisticated approach to determine the quality of a video. Ideally, this process is completely automated and does not require any human interaction.  <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Video Quality Metrics<\/h3>\n\n\n\n<p>Video quality metrics such as VMAF and PSNR promise to do exactly that. Based on the source video, and the encoded video a video quality score is derived. Although I will not go into the details of both metrics, I want to highlight some of the key facts and common pitfalls:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>In order to compute a PSNR or a VMAF score, the source video and the encoded video need to have the same resolution.<\/li><li>Both video quality metrics work on a frame basis. This means that videos are compared frame by frame and compute an overall score. Usually, the overall score is based on the arithmetic mean, alternatives like the harmonic mean are also valid and possible. <\/li><li>Some encoders include an additional black frame at the beginning of the video. This can falsify the results as it leads to a frame shift and an unequal number of frames between source- and encoded video.<\/li><li>While PSNR is based on the mean squared error, VMAF is a machine learning-based model that was trained on actual MOS scores. Hence, the VMAF scores are usually linked closer to the actual perceived quality of a video.<\/li><li>The default VMAF model is trained for video consumption on a 1080p TV device. There are two more models, one for mobile devices (trained on Samsung S5 with resolution 1920&#215;1080), and one for 4k devices.<\/li><\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Metric Calculation with FFmpeg<\/h3>\n\n\n\n<p>Now that we know why we need metrics like VMAF and PSNR, we can take a closer look on how to calculate them. Without any further delay, here comes the FFmpeg command:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ffmpeg -i encode.mp4 -i reference.mp4 -filter_complex \"[0:v]scale=1920x1080:flags=bicubic[main]; [1:v]scale=1920x1080:flags=bicubic,format=pix_fmts=yuv420p,fps=fps=25\/1[ref]; [main][ref]libvmaf=psnr=1:phone_model=1:log_fmt=json\" -f null - > out.json;<\/code><\/pre>\n\n\n\n<p>Okay, so that is a lot to take. Let us take a closer look at the individual parts. We start off pretty easily by specifying the paths to our encoded and  reference videos.  Then, we assemble our filter chain. First, we tell FFmpeg to take the video track of our first input (&#8220;[0:v]&#8221;) and scale it to a resolution of 1920&#215;1080 using the bicubic scaler. We pipe the resulting file into a variable called &#8220;main&#8221;. Please note that you can use different upscaling algorithms (e.g lanczos or bilinear) at this point, but they should be the same for the encode and the reference files. After scaling the encode, we add some filter options to the reference file. We scale the reference up to 1080p as well. In addition, we set the framerate to 25fps and the pixel format to yuv420p. The result is piped into a variable called &#8220;ref&#8221;. Depending on your needs you can omit the fps- and the pixel format filter, or add it to the filter chain of the encoded video. Just remember to make sure that your encode and your reference file have the same format and number of frames. <br>Finally we use our two variables, &#8220;main&#8221; and &#8220;ref&#8221;, and feed them into the VMAF library. We tell FFmpeg to determine the PSNR score (&#8220;psnr=1&#8221;) as well and to use the phone model for VMAF calculation. If you want to use the standard model you can just omit the &#8220;phone_model=1&#8221; option. We specify a JSON output format and write the results to &#8220;out.json&#8221;. That is it. Our result will look like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Exec FPS: 9.794471\nVMAF score = 80.45\nPSNR score = 39.6<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Conclusion<\/h3>\n\n\n\n<p>This concludes our dive into the world of FFmpeg and video quality metrics. If you want to find out more about encoding, feel free to check out our <a href=\"https:\/\/www.fokus.fraunhofer.de\/en\/fame\/deep-encode\" onclick=\"wiredminds.trackEvent('TRACK-LINK: Link from:psnr-vmaf-ffmpeg to:deep-encode')\">website<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>FFmpeg is a great tool for video processing, it basically allows us to manipulate videos any way we like. Depending on the concrete use case, however, it can be challenging to assemble the right command. In this blog post, I&#8230;<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13,14],"tags":[],"coauthors":[6],"class_list":["post-128","post","type-post","status-publish","format-standard","hentry","category-encoding","category-video-quality-metrics"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.9 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Calculating VMAF and PSNR with FFmpeg - Video-Dev<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Calculating VMAF and PSNR with FFmpeg - Video-Dev\" \/>\n<meta property=\"og:description\" content=\"FFmpeg is a great tool for video processing, it basically allows us to manipulate videos any way we like. Depending on the concrete use case, however, it can be challenging to assemble the right command. In this blog post, I...\" \/>\n<meta property=\"og:url\" content=\"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/\" \/>\n<meta property=\"og:site_name\" content=\"Video-Dev\" \/>\n<meta property=\"article:published_time\" content=\"2019-08-02T19:06:55+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-08-23T09:24:50+00:00\" \/>\n<meta name=\"author\" content=\"Daniel Silhavy\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@dsilhavy\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Daniel Silhavy\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/\"},\"author\":{\"name\":\"Daniel Silhavy\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#\\\/schema\\\/person\\\/f7e1eee3cb4eae87a59648195014a809\"},\"headline\":\"Calculating VMAF and PSNR with FFmpeg\",\"datePublished\":\"2019-08-02T19:06:55+00:00\",\"dateModified\":\"2019-08-23T09:24:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/\"},\"wordCount\":833,\"commentCount\":5,\"articleSection\":[\"Encoding\",\"Video Quality Metrics\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/\",\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/\",\"name\":\"Calculating VMAF and PSNR with FFmpeg - Video-Dev\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#website\"},\"datePublished\":\"2019-08-02T19:06:55+00:00\",\"dateModified\":\"2019-08-23T09:24:50+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#\\\/schema\\\/person\\\/f7e1eee3cb4eae87a59648195014a809\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/calculating-vmaf-and-psnr-with-ffmpeg\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Calculating VMAF and PSNR with FFmpeg\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#website\",\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/\",\"name\":\"Video-Dev\",\"description\":\"Future Applications and Media - Video Development Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#\\\/schema\\\/person\\\/f7e1eee3cb4eae87a59648195014a809\",\"name\":\"Daniel Silhavy\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/0-1-150x150.jpegccb13c1b60303228bf3c575f3345fe29\",\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/0-1-150x150.jpeg\",\"contentUrl\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2019\\\/11\\\/0-1-150x150.jpeg\",\"caption\":\"Daniel Silhavy\"},\"description\":\"Daniel Silhavy studied Computer Science at the Technical University of Berlin (TUB). He received his Masters degree with the completion of his thesis \u201cAd Insertion in MPEG-DASH\u201d at Fraunhofer Institute for Open Communication Systems (FOKUS) in 2015. Currently, he is employed as a scientific assistant and project manager at the Business Unit Future Applications and Media (FAME). He specializes in the R&amp;D of topics dealing with IPTV and adaptive media streaming.\",\"sameAs\":[\"https:\\\/\\\/www.fokus.fraunhofer.de\\\/fame\\\/team\\\/silhavy\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/daniel-silhavy-21650a129\\\/\",\"https:\\\/\\\/x.com\\\/dsilhavy\"],\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/author\\\/silhavy\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Calculating VMAF and PSNR with FFmpeg - Video-Dev","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/","og_locale":"en_US","og_type":"article","og_title":"Calculating VMAF and PSNR with FFmpeg - Video-Dev","og_description":"FFmpeg is a great tool for video processing, it basically allows us to manipulate videos any way we like. Depending on the concrete use case, however, it can be challenging to assemble the right command. In this blog post, I...","og_url":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/","og_site_name":"Video-Dev","article_published_time":"2019-08-02T19:06:55+00:00","article_modified_time":"2019-08-23T09:24:50+00:00","author":"Daniel Silhavy","twitter_card":"summary_large_image","twitter_creator":"@dsilhavy","twitter_misc":{"Written by":"Daniel Silhavy","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/#article","isPartOf":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/"},"author":{"name":"Daniel Silhavy","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#\/schema\/person\/f7e1eee3cb4eae87a59648195014a809"},"headline":"Calculating VMAF and PSNR with FFmpeg","datePublished":"2019-08-02T19:06:55+00:00","dateModified":"2019-08-23T09:24:50+00:00","mainEntityOfPage":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/"},"wordCount":833,"commentCount":5,"articleSection":["Encoding","Video Quality Metrics"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/","url":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/","name":"Calculating VMAF and PSNR with FFmpeg - Video-Dev","isPartOf":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#website"},"datePublished":"2019-08-02T19:06:55+00:00","dateModified":"2019-08-23T09:24:50+00:00","author":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#\/schema\/person\/f7e1eee3cb4eae87a59648195014a809"},"breadcrumb":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/websites.fraunhofer.de\/video-dev\/"},{"@type":"ListItem","position":2,"name":"Calculating VMAF and PSNR with FFmpeg"}]},{"@type":"WebSite","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#website","url":"https:\/\/websites.fraunhofer.de\/video-dev\/","name":"Video-Dev","description":"Future Applications and Media - Video Development Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/websites.fraunhofer.de\/video-dev\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#\/schema\/person\/f7e1eee3cb4eae87a59648195014a809","name":"Daniel Silhavy","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2019\/11\/0-1-150x150.jpegccb13c1b60303228bf3c575f3345fe29","url":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2019\/11\/0-1-150x150.jpeg","contentUrl":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2019\/11\/0-1-150x150.jpeg","caption":"Daniel Silhavy"},"description":"Daniel Silhavy studied Computer Science at the Technical University of Berlin (TUB). He received his Masters degree with the completion of his thesis \u201cAd Insertion in MPEG-DASH\u201d at Fraunhofer Institute for Open Communication Systems (FOKUS) in 2015. Currently, he is employed as a scientific assistant and project manager at the Business Unit Future Applications and Media (FAME). He specializes in the R&amp;D of topics dealing with IPTV and adaptive media streaming.","sameAs":["https:\/\/www.fokus.fraunhofer.de\/fame\/team\/silhavy","https:\/\/www.linkedin.com\/in\/daniel-silhavy-21650a129\/","https:\/\/x.com\/dsilhavy"],"url":"https:\/\/websites.fraunhofer.de\/video-dev\/author\/silhavy\/"}]}},"_links":{"self":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts\/128","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/comments?post=128"}],"version-history":[{"count":17,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts\/128\/revisions"}],"predecessor-version":[{"id":175,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts\/128\/revisions\/175"}],"wp:attachment":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/media?parent=128"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/categories?post=128"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/tags?post=128"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/coauthors?post=128"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}