Sony patent uses AI to cut holes in cloud game frames

A patent application from Sony Interactive Entertainment Europe, published on October 29, 2026, as US 2026/0340791 A1, describes an artificial intelligence technique to reduce the data sent in cloud games: cut parts of the frame before transmission and rebuild them on the player's device. The exact status matters: this is a published patent application, not a feature announced for PS5, PlayStation Portal or PS6, and Sony has not announced that it will implement this technology.

The idea works in two stages. On the sending side, a “decimation” network divides each frame into a grid and assigns an importance score from 0 to 1 to each part of the image. Areas below the application's example threshold — 0.4 — are removed, creating “holes” in the frame. Before erasing everything, the system can insert small anchor elements, real fragments of the original image, inside those holes. On the device, a second, reconstruction network uses the remaining image and the anchors to fill the gaps.

Why the anchors matter

The application's central point is to stop the AI from “inventing” details that were never in the original scene. Without a reference, a model filling in pixels can create plausible but false textures. The anchors act as faithful clues to what was really there, allowing larger areas — such as sky or ground — to be erased without relying only on guesswork. According to Patentlyze, the two networks are trained together on video game frames, and the loss function evaluates both the quality of the rebuilt image and the compactness of the stripped image. After that, the data still passes through a conventional codec before travelling over the network.

The application also allows control of the trade-off: the user or the system can specify a target quality or compression level, and that determines how many holes — and of what size — are opened. The document's main example is cloud game streaming, in which the image is rendered on a server and sent to the player; it also names virtual reality and mixed reality devices as possible receivers.

What is still unproven

The application gives no bandwidth-saving or latency figures, so the real gain remains an untested promise, not a measured result. There is also no confirmed link to the price of a future console: speculation tying the patent to a cheaper PS6 is editorial interpretation by other outlets, not a Sony statement. As editorial reading, the most obvious risk is in fast-paced genres, where an imprecisely rebuilt detail could be annoying — precisely the kind of error the anchors try to reduce, without eliminating it entirely.

In short: Sony has documented an ingenious way to send “less picture” so the cloud weighs less on the connection. Whether it becomes a product, in which service, and with what real gain, are questions only an announcement — or a public test — can answer.

Sources

Image credit: drawing from patent application US 2026/0340791 A1, via Patentlyze.

Transparency: this article was prepared with the assistance of artificial intelligence, cross-checking public posts on X and the internet sources listed above. Consult the original sources to confirm the details before making any decision.

— Muse