I figured there is a need for generating a lot of samples and building a predictive model per game for best results. Documentation confirms:
> Most of these corner cases can be resolved by providing the model with enough training data without increase the complexity and cost of the technique. This
also enables game developers to train the neural upscalers with their
content, resulting in a completely customized solution fine-tuned for the
gameplay, performance, or art direction needs of a particular title.
It sounds like this a geared towards games. However, I like the idea of exposing all of the ML features through Vulkan extensions rather than some proprietary API. Though I think exposing them through OpenCL extensions would work for me as well.
armchairhacker 2 hours ago [-]
I think ML has lots of potential in this area specifically.
Imagine a game with bare-bones graphics and lighting, and a NN that converts it into something pretty. Indie developers can make AA-looking games and all game developers can devote more effort into design and logic. Artists will still be needed for art direction and possibly fine-tuning, although there will be less needed for each game (also less developers needed with AI agents and better tools).
Related, ML also has potential for AI enemies (and allies). Lots of players still prefer multiplayer, in part because humans are more realistic enemies (but also because they want to beat real humans); but multiplayer games struggle because good netcode is nontrivial, servers are expensive, some players are obnoxious, and most games don’t have a consistent enough playerbase.
N_Lens 2 hours ago [-]
This article 2 links deep had better technical details -
Upscaling solution mainly targeted at mobile gaming, with an 'AI pipeline' for upscaling graphics (They claim 540p upscaled to 1080p at 4ms per frame). I'm a bit skeptical because this is a press release for chips that are in the works and claim to be releasing in DEC-26, and then on actual devices after that. So sounds more like a strategic/political move (Perhaps stock price related manoeuvring).
Unreal Engine 5 plugin will allow previewing the upscaled effects using the though, which will be nice for game developers.
wmf 1 hours ago [-]
It's a copy of DLSS/FSR4 which are pretty well understood by now. As for the schedule, Arm always announces IP ahead of time.
> Most of these corner cases can be resolved by providing the model with enough training data without increase the complexity and cost of the technique. This also enables game developers to train the neural upscalers with their content, resulting in a completely customized solution fine-tuned for the gameplay, performance, or art direction needs of a particular title.
Source: https://developer.arm.com/documentation/111019/latest/
Imagine a game with bare-bones graphics and lighting, and a NN that converts it into something pretty. Indie developers can make AA-looking games and all game developers can devote more effort into design and logic. Artists will still be needed for art direction and possibly fine-tuning, although there will be less needed for each game (also less developers needed with AI agents and better tools).
Related, ML also has potential for AI enemies (and allies). Lots of players still prefer multiplayer, in part because humans are more realistic enemies (but also because they want to beat real humans); but multiplayer games struggle because good netcode is nontrivial, servers are expensive, some players are obnoxious, and most games don’t have a consistent enough playerbase.
https://community.arm.com/arm-community-blogs/b/mobile-graph...
Upscaling solution mainly targeted at mobile gaming, with an 'AI pipeline' for upscaling graphics (They claim 540p upscaled to 1080p at 4ms per frame). I'm a bit skeptical because this is a press release for chips that are in the works and claim to be releasing in DEC-26, and then on actual devices after that. So sounds more like a strategic/political move (Perhaps stock price related manoeuvring).
Unreal Engine 5 plugin will allow previewing the upscaled effects using the though, which will be nice for game developers.