Next Generation Emulation banner

Using Hardware Antialiasing in PCSX2 !

61K views 73 replies 28 participants last post by  monkeygenie  
#1 · (Edited)
Tutorial: Using Hardware Antialiasing in PCSX2 !

Now i finally found out how it works :innocent:

It only works with GSdx in DX9 Fullscreen mode.
It does not work with ZeroGS, GSdx windowed or DX10 mode.
It may work with ATI cards, but i don't know a tool for that.

here is a small tutorial for Nvidia cards:

1.) Make sure you're using the latest Nvidia Forceware driver
(it may work with beta drivers too, but i didn't test it.)
2.) Download nHancer and install it.
3.) Get the latest GSdx version, link in my signature
4.) run nHancer
5.) enable options/Allow override of predefined settings,
6.) create a new profile, name it PCSX2.
7.) select the PCSX2 profile and add all pcsx2 exe's you're currently using.
8.) under the enhancements tab, select an Anti-Aliasing mode, i used Multisampling/4X that looks fine.
be warned that an higher antialiasing mode can slow down a game from 60 to 5fps!
9.) go to the compatiblity tab and check Anti-Aliasing compatiblity, select the "Stalker" or "Vengard" Compatiblity mode.
10.) run pcsx2 and configure GSdx
11.) under resolution,select your native display resolution
(example: for 22" widescreens its 1680x1050, 19" 4/3 would be 1280x1024,
you can select lower resolutions like 1024x768 or 800x600, but you must use the same resolution for the internal d3d resolution!)
12.) The D3D internal res have to match with the fullscreen resolution, otherwise it will not work.
If you selected 1680x1050, then type 1680x1050, 1280x1024 then 1280x1024... and so on...
13.) run a game and hope it'll work.

Important!
If you notice crashes and instability problems of your System use "Vengard" as Compatiblity mode instead Stalker!

i tested it with FFX-2 and it works fine.
FFX works fine too, but there is one thin horizontal line at the lower screen in movies.

here some compare shots:

Image

Image


http://img3.imagebanana.com/img/sy6mwqm0/compare1.jpg
http://img3.imagebanana.com/img/25ii0586/compare2.jpg
http://img3.imagebanana.com/img/lr5ktec6/compare3.jpg
http://img3.imagebanana.com/img/0rv2849d/compare4.jpg

FFX
http://img3.imagebanana.com/img/mo7seeci/FFX_1_8XAA.jpg
http://img3.imagebanana.com/img/qa2e5j0w/FFX_1_noAA.jpg
http://img3.imagebanana.com/img/ffg0j91/FFX_2_8XAA.jpg
http://img3.imagebanana.com/img/k4yh8iso/FFX_2_noAA.jpg
http://img3.imagebanana.com/img/l961af2t/FFX_3_8XAA.jpg
http://img3.imagebanana.com/img/tjzs3v7i/FFX_3_noAA.jpg

FFX-2
http://img3.imagebanana.com/img/kj073lra/FFX2_1_8XAA.jpg
http://img3.imagebanana.com/img/vsf80z1/FFX2_1_noAA.jpg
http://img3.imagebanana.com/img/0fl0kqg7/FFX2_2_8XAA.jpg
http://img3.imagebanana.com/img/o7mx7ftz/FFX2_2_noAA.jpg
http://img3.imagebanana.com/img/e39wfqb/FFX2_3_8XAA.jpg
http://img3.imagebanana.com/img/cuuji0m/FFX2_3_noAA.jpg
http://img3.imagebanana.com/img/l5kxmxl/FFX2_4_8XAA.jpg
http://img3.imagebanana.com/img/0dlu4th/FFX2_4_noAA.jpg
http://img3.imagebanana.com/img/crmenx/FFX2_5_8XAA.jpg
http://img3.imagebanana.com/img/olvntnbg/FFX2_5_noAA.jpg
http://img3.imagebanana.com/img/tg6zz1k/FFX2_6_8XAA.jpg
http://img3.imagebanana.com/img/u1kzwzh/FFX2_6_noAA.jpg
 
#2 ·
can you post some comparison pics?

and does it look better than just using double internal resolution?
 
#3 ·
Godlike!
 
#4 ·
Wow, we already have enough trouble running these games at super high resolution and now AA is introduced...we're going to need 2-3 future generation cards to do 16x AA Super-Sampling...even though, with that enabled, Native would be godlike, but imagine like 1920 x 1200 @ 16x AA, with Super-Sampling and Trin filtering...FFX would look next Gen :p
 
#6 ·
Yep, I already noticed this way to use AA in PCSX2 some while ago. I can confirm that it works, at least with graphics cards of the GeForce 8xxx/9xxx series.

Sadly it's barely faster than increasing the internal rendering resolution in the games i tried. So it's relatively useless to me.


Btw. I remember some other compatibility settings than "STALKER" were also working, giving slightly different results in image quality or speed.
 
#7 ·
First is without AA and the second is with 4x MSAA.
Look especially at the ground!
Internal and external resolution are of course the same!
 
#8 ·
very hard to notice, but on the ground i can see it does fix those jaggy lines.

maybe it'll look more noticeable in other games.
 
#9 ·
Yeah...I was hoping AA would help alot more...but it's not really at all.
 
#10 ·
i think if you use native resolution, with AA, it might be more noticible.

since native resolution usually has alot of jaggy edges.

and it would be good for the games that don't work with high internal resolutions.
 
#12 ·
how does it work in gran turismo 4?

doubling the res still leaves gt4 looking pretty rough :/
For me GT4 mostly suffers from a sort of blurry image, just like some other games (for example Grandia 3 or Dragon Quest 8). For these games I find it better to use a higher internal resolution because it sharpens the whole image a bit.

Generally with 4x MSAA the image quality concerning the polygon edges is a bit better than using a quadrupled internal res. On the other hand a higher internal res treats the whole image while MSAA works on polygon edges exclusively.
Other than in PC games where MSAA is usually much faster than using a higher resolution, the difference doesn't seem to bit that high in PCSX2.


To put it shortly: Imo it depends on the game whether it is better to use MSAA or a higher internal resolution.


I've taken a few screenshots of Grandia 3 in order to demonstrate what I stated. From left to right it is: 1280*960 without AA / 1280*960 with 4xAA / 2560*1920 (without AA). Especially pay attention to the pillars on the left.
 
#13 ·
I'm pretty sure it should work to force AA from cpanel using driver 174.74 or newer using the new AA force method setting "override any application settings". At least pre this driver lots of pc games wouldn't use AA if forced it from cpanel so RivaTuner or nHancer etc was required but with driver version 174.74 or newer it works with all pc games I've tested and I'm pretty that goes for pcsx2 too but I haven't tested yet.
 
#14 ·
I'm pretty sure it should work to force AA from cpanel using driver 174.74 or newer using the new AA force method setting "override any application settings". At least pre this driver lots of pc games wouldn't use AA if forced it from cpanel so RivaTuner or nHancer etc was required but with driver version 174.74 or newer it works with all pc games I've tested and I'm pretty that goes for pcsx2 too but I haven't tested yet.
Nope, it doesn't work that way in PCSX2.
 
#18 ·
OK, just for Quake3k:

BTW: Do you really like to have fat fighters?? I always use nvidia's scalling...
 
#22 ·
Look at my screenshots and compare them with yours.
I have black borders left and right.
You could press F6 (or F5?) while ingame.

Delete all your pcsx2 settings in your nvidia driver.
Then try it just with nHancer.
Make sure you use the same external and internal resolution
 
#24 ·
I never have problems when I use v-sync.
I thought there are only big slow downs if frame skipping is on.
However, I don't need it.
 
#26 ·
External: Just the resolution which your monitor shows (the output)
Internal is more complicated.
I think it is how good a game will be rendered by the plugin.
(The real game resolution gets increased).

The external should depends on the internal.
That means an external resolution of 1680x1050 never would help you
if the internal will be bad (example: 800x600)

Ok, I am now tired and going sleeping :D
If I told bull****, it was because of the time. haha