[Hard core Rikki]

I just found this very interesting new algorithm for depixelizing 'pixel art' (commonly used in retro consoles and sprite-based 2D games). It's supposedly gives better graphics than typically performed through the past established approaches (edge-smoothed upsampling and vectorization).

Thoughts on this? It could be interesting to see how this would fare performance-wise in emulators like Snes9x, VBA-M and MAME.

- Awesome new algorithm for depixelizing pixelart (scribd mirror, original in comments) : gaming
- Depixelizing Pixel Art (text-only cached copy)

 

[Hard core Rikki]

Found a mirror with the pictures
Photo Album - Imgur

Samples

 

[cottonvibes]

that is pretty awesome.
seems it even looks a bit better than hq4x.

not sure if that document contains enough information to reproduce the algorithm though; although i haven't fully read it yet.
they might even patent the algorithm or something ><

 

[shalma]

Johannes Kopf (Microsoft Research) + Dani Lischinski (Hebrew University)
> Don't think they can try to patent a SIGGRAPH-quality paper
> Besides, it looks similar to hq4x (souped-up with some 5-gallon techniques that can't be patented by themselves)
> No mention of hardware devices (like the Ricoh s-dd1 / spc7110 papers)


Very impressive though! A comparable solution could probably be cooked up by someone who is clever (+skilled x2) - hq4x did come before this and does surprisingly well.

From what else I've heard, you need to be a mathematics heavyweight to actually do the heuristics and adaptive intelligence of weights.
> Besides, they take so many 'potshots' at the competition that someone's going to try 1-upping them now.
> But they are descriptive in their presentation
> They have -videos- (and explicitly mention wanting to optimize the math to run real-time in an emu)


The rumor mill spreads fast.
> Seems like someone leaked this paper early :O

 

[lagunareturns]

from what i can tell is this method cant be done in real time and it wont work easily on fullscreen images with multiple layers of 2d moving around as the filter will get confused and render the same character slightly differently when its moving around as it cant distinguish it from the background... still i can be wrong but not about the fact it can be done real time anywhere as efficient as hqx4

 

[shalma]

(hearsay)
That's what I first thought. But they claim

2.4 GHZ single-core (54 images of x sprites)
- 0.62s median, 0.79s average, 3.06s max

Isn't there 4+ GHz Sandy bridges now? And it may be a few years before something like this comes to reality (better multi-core techniques?).


Note that they are upsampling to higher 4x+ size.
- They limit to snes-era colors since they are more 'limited' in scope
- They haven't tested checkerboard images (high dither)
- Can scale to any size you want without artifact problem (they single out hqx for this)
- They specifically mention hqx, scale2x, 2xsai in their analysis D)
- They said that their technique is not well-optimized and has room for much improvement
- They express reasonable interest in emulators (basis for their research)


True- there's no way this can be as efficient as hq4x. The method is way too complex.
- Funny enough, they have an in-game shot of Super Mario World (cropped though to 4x) - the gradients are some smoother than hq4x
- They also claim that their method can help improve general image techniques though
- And they point out that magnifying a tiny input to HD-size will result in jumpy animations
- Anti-aliased inputs barf the algorithm (ex. Doom face)


But a hq4x++ may be possible..?
- They seem reasonably confident that it can be used in emulators (50% document focus)
- They use an interesting 3-heuristic method to separate the pixels/sprites around (seems robust theory-wise)
- We don't have to do -exactly- what they say is the 'proper' or 'perfect' method - just some simplified math to cut some corners (if it doesn't produce super-rounded or gradients, will most notice when it's animated?)


I too would like to see moving animations though.
- At least they've pushed the envelope on a new way about this age-old problem
- There is recent research 2009-2011 about moving the smaller pieces into real-time (ex. diffusion solvers)
- At least some uber-smart people are still interested in old hand-drawn art


Summary- yeah. No arguments with you on those points.



edit:
Coming out of my 'biased' mode, I always seem to forget that some of the original 'texture' always get lost using these methods. I dislike over-smoothing and over-sharp.

Real-time though? I should be doubtful too.
- Stupid me.

 

[omegadox]

I want to see what N64 would look like this these filters

 

[DinJerr]

Agree with what Laguna said. Unless it's filtering on a per-sprite basis, pretty sure edge detection would be messed up once it is superimposed over other layers.

 

[Phil]

Couldn't it be modified to flatten the image on the fly?