{"id":3010942,"date":"2026-01-28T09:00:00","date_gmt":"2026-01-28T17:00:00","guid":{"rendered":"urn:uuid:e39cf68c-24dc-46f9-ad7d-74ad0f15d14b"},"modified":"2026-01-27T08:51:18","modified_gmt":"2026-01-27T16:51:18","slug":"content-adaptive-encoding-for-interactive-game-streaming","status":"publish","type":"research-post","link":"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/","title":{"rendered":"Content Adaptive Encoding for Interactive Game Streaming"},"content":{"rendered":"\n<div class=\"wp-block-columns is-layout-flex wp-container-core-columns-is-layout-9d6595d7 wp-block-columns-is-layout-flex\">\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\" style=\"flex-basis:70%\">\t\t<div\n\t\t\tclass=\"wp-block-sie-social-share social-share--raw\"\n\t\t\tdata-aa-modulename=\"social-share\"\n\t\t\tdata-url=\"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/\"\n\t\t\tdata-title=\"Content Adaptive Encoding for Interactive Game Streaming\"\n\t\t\tdata-heading=\"Share this\"\n\t\t\tdata-bluesky-title=\"\"\n\t\t\tdata-twitter-title=\"\"\n\t\t\tdata-twitter-hashtags=\"\"\n\t\t\tdata-reddit-title=\"\"\n\t\t\tdata-email-subject=\"\"\n\t\t\tdata-email-body=\"\"\n\t\t\t>\n\t\t<\/div>\n\t\n\n<div class=\"post-author\" data-no-of-bylines=\"6\"><ul><li>Shakarim Soltanayev<span class=\"sie-style-body-small\">Senior Machine Learning Engineer, Sony Interactive Entertainment<\/span><\/li><li>Odysseas Zisimopoulos<span class=\"sie-style-body-small\">Senior Machine Learning Engineer, Sony Interactive Entertainment<\/span><\/li><li>Mohammad Ashraful Anam<span class=\"sie-style-body-small\">Senior Software Engineer, Sony Interactive Entertainment<\/span><\/li><li>Man Cheung Kung<span class=\"sie-style-body-small\">Senior Software Engineer, Sony Interactive Entertainment<\/span><\/li><li>Angeliki Katsenou<span class=\"sie-style-body-small\">Manager, Machine Learning, Sony Interactive Entertainment<\/span><\/li><li>Yiannis Andreopoulos<span class=\"sie-style-body-small\">Director, Machine Learning, Sony Interactive Entertainment<\/span><\/li><\/ul><\/div>\n\n\n<p class=\"sie-paragraph sie-paragraph-49f17606-462a-4b22-b4d2-0927647986cd\">Video-on-demand streaming has benefitted from Content-Adaptive Encoding (CAE), i.e., adaptation of resolution and\/or quantization parameters for each scene based on convex hull optimization. Unlike traditional video-on-demand (VoD) services, where the use of buffering can ensure a smooth playback experience with high coding efficiency, interactive game streaming (IGS) requires real-time encoding and delivery to support the interaction between the user controls and the cloud server running the game.<\/p>\n\n\n\n<p class=\"sie-paragraph sie-paragraph-6f0f9ba4-2bae-448e-a35a-2f9695a73612\">We propose the first CAE approach for resolution adaptation in IGS based on compact encoding metadata from past frames, CAE-IGS. Specifically, we train a convolutional neural network (CNN) to infer the best resolution from the options available for the upcoming scene based on a running window of aggregated coding block statistics from the current scene. Our proposal makes the following contributions:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li class=\"sie-paragraph sie-paragraph-a743aa3e-606c-43e4-b9ff-ef2750fffbb2\"><strong class=\"sie-paragraph sie-paragraph-a743aa3e-606c-43e4-b9ff-ef2750fffbb2\">Lightweight model that runs on CPU within 1ms: <\/strong>CNN training to infer the best resolution for the upcoming scene based on the optimal resolution per bitrate (selected offline from the available ones). Minimal computation overhead is added as inference time is within 1ms on a single CPU core, thereby imposing no latency overhead.<\/li>\n\n\n\n<li class=\"sie-paragraph sie-paragraph-b706f78f-1065-4384-8a62-2bc2d97f61cd\"><strong class=\"sie-paragraph sie-paragraph-b706f78f-1065-4384-8a62-2bc2d97f61cd\">Utilizing Past Scene Encoding Statistics: <\/strong>Instead of utilizing video frames as input (which is computationally infeasible in IGS), the proposed CNN infers the resolution to use for the next scenes by ingesting macroblock line-aggregated statistics, i.e., coding tree block (CTB) stats for High Efficiency Video Codec (HEVC), that are produced from past&nbsp; frames by the encoder without incurring additional overhead.<\/li>\n\n\n\n<li class=\"sie-paragraph sie-paragraph-daec7113-7b16-4367-ab73-de0087a9faa0\"><strong class=\"sie-paragraph sie-paragraph-daec7113-7b16-4367-ab73-de0087a9faa0\">Preserving temporal smoothness:<\/strong> Resolution adaptation takes place at scene changes, where an IDR (instantaneous decoder refresh) frame is de-facto used, thereby avoiding any extra IDRs that could cause adverse effects in Rate-Quality (RQ) control (such as frame drops or re-buffering).&nbsp;<\/li>\n<\/ul>\n\n\n\n<p class=\"sie-paragraph sie-paragraph-9aaf3447-36ee-4cd5-9150-0a8f41bd3fae\">CAE-IGS is evaluated with challenging top-tier 1080p-60fps gaming content and HEVC encoding. Our test conditions incorporate the occurrence of frame drops due to the strict congestion control conditions of ultra-low latency game streaming. This scenario corresponds to IGS for portable devices, such as the Sony PlayStation Portal Remote Player, which experience a range of RQ conditions when users move to different environments.<\/p>\n\n\n\n<p class=\"sie-paragraph sie-paragraph-f4416cce-ebc3-47bb-b2cd-800abf41ff6d\">Under the same bitrate targets, the results demonstrate 2.3-point improvement in visual quality measure in VMAF versus the static ladder method, without detrimenting (and even slightly improving) frame drop statistics. This improvement is predominantly in the most active bitrate zones. Finally, we note that this methodology can be customised to different streaming use cases and generalises to any hybrid video codec with small adjustments.<\/p>\n\n\n\n<p class=\"sie-paragraph sie-paragraph-24a27460-6e56-425e-a32c-4daaffa42f24\">For more details on the methodology and results, check out the paper presented at PCS2025: <a class=\"sie-paragraph sie-paragraph-24a27460-6e56-425e-a32c-4daaffa42f24\" href=\"https:\/\/www.arxiv.org\/abs\/2511.22327\">https:\/\/www.arxiv.org\/abs\/2511.22327<\/a><\/p>\n<\/div>\n\n\n\n<div class=\"wp-block-column is-layout-flow wp-block-column-is-layout-flow\">\n\t\t<div class=\"related-posts__container\" data-selection-type=\"latest\" data-post-type=\"research-post\">\n\t\t\t<h2 class=\"related-posts__heading sie-style-h5\">Latest Research Posts<\/h2>\n\t\t\t<div class=\"related-posts related-posts--vertical\">\n\t\t\t\t<article class=\"related-post related-post--research-post\"><div class=\"related-post__content\"><h3 class=\"related-post__title sie-style-body-small-v2\">\n\t\t<a href=\"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/vision-language-models-for-quality-assurance\/\">\n\t\t\tVideoGameQA-Bench: Evaluating Vision-Language Models for Video Game Quality Assurance\n\t\t<\/a>\n\t<\/h3><\/div><\/article><article class=\"related-post related-post--research-post\"><div class=\"related-post__content\"><h3 class=\"related-post__title sie-style-body-small-v2\">\n\t\t<a href=\"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/learning-representations-in-video-game-agents\/\">\n\t\t\tLearning Representations in Video Game Agents with Supervised Contrastive Imitation Learning\n\t\t<\/a>\n\t<\/h3><\/div><\/article><article class=\"related-post related-post--research-post\"><div class=\"related-post__content\"><h3 class=\"related-post__title sie-style-body-small-v2\">\n\t\t<a href=\"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/generating-6dof-object-manipulation-trajectories\/\">\n\t\t\tGenerating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Vision\n\t\t<\/a>\n\t<\/h3><\/div><\/article>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/div>\n\n\n\n<div class=\"wp-block-sie-scroll-to-top\" data-aa-modulename=\"sie-scroll-to-top\"><button class=\"sie-btn sie-btn--action\" type=\"button\"><span>Back to top<\/span><\/button><\/div>\n","protected":false},"author":15,"parent":0,"template":"","byline":[401,402,403,404,405,312],"research-post-category":[],"class_list":["post-3010942","research-post","type-research-post","status-publish","hentry","post-content-adaptive-encoding-for-interactive-game-streaming"],"ab_tests":{},"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.4 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Content Adaptive Encoding for Interactive Game Streaming - Sony Interactive Entertainment<\/title>\n<meta name=\"description\" content=\"AI-driven content-adaptive encoding for interactive game streaming, delivering higher visual quality under ultra-low-latency and bandwidth-constrained conditions.\" \/>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Content Adaptive Encoding for Interactive Game Streaming\" \/>\n<meta property=\"og:description\" content=\"AI-driven content-adaptive encoding for interactive game streaming, delivering higher visual quality under ultra-low-latency and bandwidth-constrained conditions.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/\" \/>\n<meta property=\"og:site_name\" content=\"Sony Interactive Entertainment\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"2 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Shakarim Soltanayev\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/innovation\\\/research-academia\\\/research\\\/content-adaptive-encoding-for-interactive-game-streaming\\\/\",\"url\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/innovation\\\/research-academia\\\/research\\\/content-adaptive-encoding-for-interactive-game-streaming\\\/\",\"name\":\"Content Adaptive Encoding for Interactive Game Streaming - Sony Interactive Entertainment\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/#website\"},\"datePublished\":\"2026-01-28T17:00:00+00:00\",\"description\":\"AI-driven content-adaptive encoding for interactive game streaming, delivering higher visual quality under ultra-low-latency and bandwidth-constrained conditions.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/innovation\\\/research-academia\\\/research\\\/content-adaptive-encoding-for-interactive-game-streaming\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/innovation\\\/research-academia\\\/research\\\/content-adaptive-encoding-for-interactive-game-streaming\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/innovation\\\/research-academia\\\/research\\\/content-adaptive-encoding-for-interactive-game-streaming\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Content Adaptive Encoding for Interactive Game Streaming\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/#website\",\"url\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/\",\"name\":\"Sony Interactive Entertainment\",\"description\":\"Pushing the Boundaries of Play\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/sie-dev.altis.cloud\\\/en\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Content Adaptive Encoding for Interactive Game Streaming - Sony Interactive Entertainment","description":"AI-driven content-adaptive encoding for interactive game streaming, delivering higher visual quality under ultra-low-latency and bandwidth-constrained conditions.","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Content Adaptive Encoding for Interactive Game Streaming","og_description":"AI-driven content-adaptive encoding for interactive game streaming, delivering higher visual quality under ultra-low-latency and bandwidth-constrained conditions.","og_url":"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/","og_site_name":"Sony Interactive Entertainment","twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"2 minutes","Written by":"Shakarim Soltanayev"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/","url":"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/","name":"Content Adaptive Encoding for Interactive Game Streaming - Sony Interactive Entertainment","isPartOf":{"@id":"https:\/\/sie-dev.altis.cloud\/en\/#website"},"datePublished":"2026-01-28T17:00:00+00:00","description":"AI-driven content-adaptive encoding for interactive game streaming, delivering higher visual quality under ultra-low-latency and bandwidth-constrained conditions.","breadcrumb":{"@id":"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/sie-dev.altis.cloud\/en\/innovation\/research-academia\/research\/content-adaptive-encoding-for-interactive-game-streaming\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/sie-dev.altis.cloud\/en\/"},{"@type":"ListItem","position":2,"name":"Content Adaptive Encoding for Interactive Game Streaming"}]},{"@type":"WebSite","@id":"https:\/\/sie-dev.altis.cloud\/en\/#website","url":"https:\/\/sie-dev.altis.cloud\/en\/","name":"Sony Interactive Entertainment","description":"Pushing the Boundaries of Play","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/sie-dev.altis.cloud\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/research-post\/3010942","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/research-post"}],"about":[{"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/types\/research-post"}],"author":[{"embeddable":true,"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/users\/15"}],"version-history":[{"count":3,"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/research-post\/3010942\/revisions"}],"predecessor-version":[{"id":3010949,"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/research-post\/3010942\/revisions\/3010949"}],"wp:attachment":[{"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/media?parent=3010942"}],"wp:term":[{"taxonomy":"byline","embeddable":true,"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/byline?post=3010942"},{"taxonomy":"research-post-category","embeddable":true,"href":"https:\/\/sie-dev.altis.cloud\/en\/wp-json\/wp\/v2\/research-post-category?post=3010942"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}