{"id":769,"date":"2020-11-16T17:31:54","date_gmt":"2020-11-16T16:31:54","guid":{"rendered":"https:\/\/websites.fraunhofer.de\/video-dev\/?p=769"},"modified":"2020-11-17T09:35:20","modified_gmt":"2020-11-17T08:35:20","slug":"deep-encode-part-i","status":"publish","type":"post","link":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/","title":{"rendered":"Deep Encode: Part I"},"content":{"rendered":"\n<p>Back in August 2019, we published a post called \u2018<a href=\"https:\/\/websites.fraunhofer.de\/video-dev\/per-title-encoding\/\">Per-Title-Encoding<\/a>\u2019, shortly followed by \u2018<a href=\"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/\">Calculating VMAF and PSNR with FFmpeg<\/a>\u2019.&nbsp;<\/p>\n\n\n\n<p>To pick up where we left off, we\u2019d like to begin a &#8220;Deep Encode&#8221; series &#8211; a sequence of blog posts covering a project called \u2018<a href=\"https:\/\/www.fokus.fraunhofer.de\/en\/fame\/deep-encode\">FAMIUM Deep Encode<\/a>\u2019. FAMIUM Deep Encode is a long-term research and development project in which we&#8217;ve developed an AI-supported per-title encoding architecture. In other words, improve the per-title encoding process with a bunch of machine learning models.&nbsp;<\/p>\n\n\n\n<p>In this series, we will cover different aspects of the project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we&#8217;ve faced and concepts we\u2019d like to explore in the future.<\/p>\n\n\n\n<p>For <em>this <\/em>particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Overview<\/h2>\n\n\n\n<p>So, why an AI-supported solution for per-title encoding?<\/p>\n\n\n\n<p>Well, as you\u2019ve seen from the <a href=\"https:\/\/websites.fraunhofer.de\/video-dev\/per-title-encoding\/\">first<\/a> and <a href=\"https:\/\/websites.fraunhofer.de\/video-dev\/calculating-vmaf-and-psnr-with-ffmpeg\/\">second<\/a> blog posts, the entire per-title encoding workflow can become complex in producing a sufficient amount of test encodes needed in order to produce sufficient data for an optimal encoding ladder.&nbsp;As a result, this process becomes time consuming, a high amount of storage is used, etc.<\/p>\n\n\n\n<p>As you can see, not an ideal solution for a team of engineers, much less a video encoding enthusiast.<\/p>\n\n\n\n<p><strong>What do we do?<\/strong><\/p>\n\n\n\n<p>Two words: MACHINE. LEARNING. (Which will be covered in more detail in the next two posts!)<\/p>\n\n\n\n<p>What if: we developed a solution that can produce virtual encodes (total time-saver), send those encodes to a few machine learning models (since machine learning is all the rage these days), and spit out an optimal encoding ladder. And to put a cherry on top, generate production encodes from the encoding ladder. Neat, right?<\/p>\n\n\n\n<p>Admittedly, this project consisted a lot of trial-and-error, from finetuning our models, to increasing the speed of the video analysis. Nevertheless, the ongoing process has been quite fascinating, and proved to be full of potential (a.k.a, a long laundry list of ideas).<\/p>\n\n\n\n<p><strong>What\u2019s been happening?<\/strong><\/p>\n\n\n\n<p>Up until now, we\u2019ve developed that aforementioned \u2018what if\u2019, and knee-deep in optimizing the workflow architecture:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"698\" height=\"287\" src=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-698x287.png\" alt=\"\" class=\"wp-image-784\" srcset=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-698x287.png 698w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-400x165.png 400w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-768x316.png 768w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title.png 1516w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption><sub>AI-Supported Per-title Encoding architecture<\/sub><\/figcaption><\/figure>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>We have a handy and interactive UI that supports the architecture, so that interested customers can log in and encode their own videos. This technology remains codec-agnostic, although we are exploring each codec individually in order to create THE technology.<\/p>\n<\/div><\/div>\n\n\n\n<div class=\"wp-block-group\"><div class=\"wp-block-group__inner-container is-layout-flow wp-block-group-is-layout-flow\">\n<p>There\u2019s an entire complexity analysis component made up of workers and API\u2019s (developed in JS and Python) that analyzes the source video in order to create the virtual encodes. This part was a cool way to really dive deep into the field of computer vision and explore different algorithms that can best represent the complexity of a video. As part of the complexity analysis, we have several workers running hot with: scene detection, general metadata extraction, SI\/TI algorithms, classification, etc. This will be covered in more detail in the next post.<\/p>\n<\/div><\/div>\n\n\n\n<p>Then, we have five machine learning models: Feed Forward Fast Connect, XGBoost, Convolutional Neural Networks, and 2 types of Stacked Models. And yes, they all predict an optimal encoding ladder. However, we&#8217;ve discovered that due to the architecture of each machine learning model, the results were quite different from one another. For purposes of comparison, we&#8217;ve continuously optimized all five models in order to find the &#8216;ideal&#8217; algorithm. Our machine learning component will be covered in more detail in part 3 of this series. <\/p>\n\n\n\n<p>Finally, we have a production encoding feature &#8211; where you can select the type of encoding you want, fiddle around with the encoding settings, check its actual quality (VMAF), and download the encode to be played in any media player.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What\u2019s in store for the future? Per-scene encoding!<\/h2>\n\n\n\n<p>Per-scene encoding is where individual scenes <em>within<\/em> an entire video are encoded, rather than encoding the entire video with an optimal encoding ladder.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"698\" height=\"231\" src=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/compare-698x231.png\" alt=\"\" class=\"wp-image-783\" srcset=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/compare-698x231.png 698w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/compare-400x132.png 400w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/compare-768x254.png 768w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/compare-1536x508.png 1536w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/compare.png 1822w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption>Per-title vs. Per-scene Encoding (Copyright: JanBaby)<\/figcaption><\/figure>\n\n\n\n<p>With this process, bitrates can be saved even further, and multiple encoding ladders would be produced <em>per scene<\/em>, meaning, individual scenes that require lower (or sometimes higher) bitrates than others would be encoded as such, thereby improving the overall quality of the video. <\/p>\n\n\n\n<p>How is this possible? As with the concept of per-title encoding, certain scenes within a video are more\/less complex than others, and would require different bitrate values for sufficient quality. <\/p>\n\n\n\n<p>To test this concept, we took a 30-second sports clip at 1080p (high amount of movement with high redundancy), ran it through our scene change detector, complexity analysis, and machine learning model. In the table below, one scene with high redundancy and low movement (the opening scene) only required 1141 kb\/s, while another scene with low redundancy and a high amount of movement required a higher bitrate of of 3316 kb\/s at 1080p, both achieving the same <em>predicted<\/em> VMAF quality of 90. <\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"698\" height=\"118\" src=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-results-2-698x118.png\" alt=\"\" class=\"wp-image-792\" srcset=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-results-2-698x118.png 698w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-results-2-400x68.png 400w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-results-2-768x130.png 768w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-results-2-1536x260.png 1536w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-results-2-2048x347.png 2048w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption>Per-Scene Encoding: test results<\/figcaption><\/figure>\n\n\n\n<p>As part of the per-scene encoding workflow, the scenes were stitched together to analyze the overall file size, bitrate, and VMAF quality, which resulted in 11mb, 2427kbps and 90, respectively. In comparison to the per-title encoding method, at 1080p, the overall file size, bitrate and VMAF resulted in 13.72mb,  3111kbps and 90, respectively. Based on these preliminary results, it can be assumed that both file size and bitrate values can be saved using the per-scene encoding method instead.<\/p>\n\n\n\n<p>To make this process \u2018AI-supported\u2019, the entire architecture would be updated so that multiple virtual encodes <em>per scene<\/em> are fed to the machine learning models, producing several several encoding ladders for one video:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"698\" height=\"301\" src=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-2-698x301.png\" alt=\"\" class=\"wp-image-782\" srcset=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-2-698x301.png 698w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-2-400x173.png 400w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-2-768x331.png 768w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-2-1536x663.png 1536w, https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-scene-2.png 1576w\" sizes=\"auto, (max-width: 698px) 100vw, 698px\" \/><figcaption>AI-Supported Per-scene Encoding Architecture<\/figcaption><\/figure>\n\n\n\n<p>As you can see in the workflow diagram, in comparison to the previous per-title workflow, there is an additional step prior to the &#8216;Complexity Analysis&#8217;. Here, a scene change detector extracts scenes from the source video based on scene changes (black outs, frame fades, etc.) and a predetermined threshold. (As a side note: the &#8216;threshold&#8217; is a factor that influences the number of scenes that are extracted.) The extracted scenes are then analyzed in terms of complexity, and its results are sent to the machine learning model.<\/p>\n\n\n\n<p>In other words, it\u2019s as if per-title-encoding got bit by a radioactive spider. #notsponsoredbymarvel<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p>In this post, we&#8217;ve introduced our Deep Encode project, touched upon vital components that we&#8217;ve developed, and discussed the concept of per-scene encoding. Special thanks to my fellow colleagues, Neel and Tu, for their contributions to this post.<\/p>\n\n\n\n<p>Stay tuned for Part 2 of this series, where we dive deep into the development of video complexity!<\/p>\n\n\n\n<p>If you have any questions in regards to our Deep Encode activities, please check out our <a href=\"https:\/\/www.fokus.fraunhofer.de\/en\/fame\/deep-encode\">website<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Back in August 2019, we published a post called \u2018Per-Title-Encoding\u2019, shortly followed by \u2018Calculating VMAF and PSNR with FFmpeg\u2019.&nbsp; To pick up where we left off, we\u2019d like to begin a &#8220;Deep Encode&#8221; series &#8211; a sequence of blog posts&#8230;<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34,33,32,35],"tags":[38,37,36],"coauthors":[24],"class_list":["post-769","post","type-post","status-publish","format-standard","hentry","category-machine-learning","category-per-scene-encoding","category-per-title-encoding","category-video-encoding","tag-machine-learning","tag-per-scene-encoding","tag-per-title-encoding"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Deep Encode: Part I - Video-Dev<\/title>\n<meta name=\"description\" content=\"In this series, we will cover different aspects of the FAMIUM Deep Encode project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we&#039;ve faced and concepts we\u2019d like to explore in the future. For this particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Deep Encode: Part I - Video-Dev\" \/>\n<meta property=\"og:description\" content=\"In this series, we will cover different aspects of the FAMIUM Deep Encode project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we&#039;ve faced and concepts we\u2019d like to explore in the future. For this particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/\" \/>\n<meta property=\"og:site_name\" content=\"Video-Dev\" \/>\n<meta property=\"article:published_time\" content=\"2020-11-16T16:31:54+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-11-17T08:35:20+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-698x287.png\" \/>\n<meta name=\"author\" content=\"Anita Chen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Anita Chen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/\"},\"author\":{\"name\":\"Anita Chen\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#\\\/schema\\\/person\\\/66a433660483086bca16d8f94bc0e5c1\"},\"headline\":\"Deep Encode: Part I\",\"datePublished\":\"2020-11-16T16:31:54+00:00\",\"dateModified\":\"2020-11-17T08:35:20+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/\"},\"wordCount\":1128,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2020\\\/10\\\/per-title-698x287.png\",\"keywords\":[\"machine learning\",\"per-scene encoding\",\"per-title encoding\"],\"articleSection\":[\"machine learning\",\"Per-Scene Encoding\",\"Per-Title Encoding\",\"video encoding\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/\",\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/\",\"name\":\"Deep Encode: Part I - Video-Dev\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2020\\\/10\\\/per-title-698x287.png\",\"datePublished\":\"2020-11-16T16:31:54+00:00\",\"dateModified\":\"2020-11-17T08:35:20+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#\\\/schema\\\/person\\\/66a433660483086bca16d8f94bc0e5c1\"},\"description\":\"In this series, we will cover different aspects of the FAMIUM Deep Encode project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we've faced and concepts we\u2019d like to explore in the future. For this particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#primaryimage\",\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2020\\\/10\\\/per-title.png\",\"contentUrl\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/wp-content\\\/uploads\\\/2020\\\/10\\\/per-title.png\",\"width\":1516,\"height\":624},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/deep-encode-part-i\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Deep Encode: Part I\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#website\",\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/\",\"name\":\"Video-Dev\",\"description\":\"Future Applications and Media - Video Development Blog\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/#\\\/schema\\\/person\\\/66a433660483086bca16d8f94bc0e5c1\",\"name\":\"Anita Chen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ec50fa1c488b53b24ac247a1a683afc2d2c7a34b2bab56ccbb2e7bc956449778?s=96&d=mm&r=g08ee5d73485bba7d6a61df5e1da32ff1\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ec50fa1c488b53b24ac247a1a683afc2d2c7a34b2bab56ccbb2e7bc956449778?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ec50fa1c488b53b24ac247a1a683afc2d2c7a34b2bab56ccbb2e7bc956449778?s=96&d=mm&r=g\",\"caption\":\"Anita Chen\"},\"url\":\"https:\\\/\\\/websites.fraunhofer.de\\\/video-dev\\\/author\\\/chb\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Deep Encode: Part I - Video-Dev","description":"In this series, we will cover different aspects of the FAMIUM Deep Encode project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we've faced and concepts we\u2019d like to explore in the future. For this particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/","og_locale":"en_US","og_type":"article","og_title":"Deep Encode: Part I - Video-Dev","og_description":"In this series, we will cover different aspects of the FAMIUM Deep Encode project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we've faced and concepts we\u2019d like to explore in the future. For this particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.","og_url":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/","og_site_name":"Video-Dev","article_published_time":"2020-11-16T16:31:54+00:00","article_modified_time":"2020-11-17T08:35:20+00:00","og_image":[{"url":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-698x287.png","type":"","width":"","height":""}],"author":"Anita Chen","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Anita Chen","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#article","isPartOf":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/"},"author":{"name":"Anita Chen","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#\/schema\/person\/66a433660483086bca16d8f94bc0e5c1"},"headline":"Deep Encode: Part I","datePublished":"2020-11-16T16:31:54+00:00","dateModified":"2020-11-17T08:35:20+00:00","mainEntityOfPage":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/"},"wordCount":1128,"commentCount":0,"image":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-698x287.png","keywords":["machine learning","per-scene encoding","per-title encoding"],"articleSection":["machine learning","Per-Scene Encoding","Per-Title Encoding","video encoding"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/","url":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/","name":"Deep Encode: Part I - Video-Dev","isPartOf":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#website"},"primaryImageOfPage":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#primaryimage"},"image":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#primaryimage"},"thumbnailUrl":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title-698x287.png","datePublished":"2020-11-16T16:31:54+00:00","dateModified":"2020-11-17T08:35:20+00:00","author":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#\/schema\/person\/66a433660483086bca16d8f94bc0e5c1"},"description":"In this series, we will cover different aspects of the FAMIUM Deep Encode project: architecture components (part 2), machine learning (part 3) and production encoding (part 4). For each aspect, we will discuss its current development, issues we've faced and concepts we\u2019d like to explore in the future. For this particular article, we will discuss how this project came to be, provide an overview of our project components, and per-scene encoding.","breadcrumb":{"@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#primaryimage","url":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title.png","contentUrl":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-content\/uploads\/2020\/10\/per-title.png","width":1516,"height":624},{"@type":"BreadcrumbList","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/deep-encode-part-i\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/websites.fraunhofer.de\/video-dev\/"},{"@type":"ListItem","position":2,"name":"Deep Encode: Part I"}]},{"@type":"WebSite","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#website","url":"https:\/\/websites.fraunhofer.de\/video-dev\/","name":"Video-Dev","description":"Future Applications and Media - Video Development Blog","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/websites.fraunhofer.de\/video-dev\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/websites.fraunhofer.de\/video-dev\/#\/schema\/person\/66a433660483086bca16d8f94bc0e5c1","name":"Anita Chen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/ec50fa1c488b53b24ac247a1a683afc2d2c7a34b2bab56ccbb2e7bc956449778?s=96&d=mm&r=g08ee5d73485bba7d6a61df5e1da32ff1","url":"https:\/\/secure.gravatar.com\/avatar\/ec50fa1c488b53b24ac247a1a683afc2d2c7a34b2bab56ccbb2e7bc956449778?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ec50fa1c488b53b24ac247a1a683afc2d2c7a34b2bab56ccbb2e7bc956449778?s=96&d=mm&r=g","caption":"Anita Chen"},"url":"https:\/\/websites.fraunhofer.de\/video-dev\/author\/chb\/"}]}},"_links":{"self":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts\/769","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/comments?post=769"}],"version-history":[{"count":16,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts\/769\/revisions"}],"predecessor-version":[{"id":823,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/posts\/769\/revisions\/823"}],"wp:attachment":[{"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/media?parent=769"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/categories?post=769"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/tags?post=769"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/websites.fraunhofer.de\/video-dev\/wp-json\/wp\/v2\/coauthors?post=769"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}