[Elb]

In this message I'd like a propose a method to semi-systematically improve the correctness of PCSX2 (but which also applies to any other emulator of a user-programmable system).

The idea is to run a special program (compiled for the PS2) on a devkit or modded PS2 that runs a game on the console and allows to examine the state of the game and the operations it performs.

Using such a program (which I'll call a "monitor"), an emulator developer would run the same game on both a PC using the emulator and on the PS2 using the monitor for a certain number of "steps" (corresponding to CPU instructions) . The state of the emulated hardware and the real hardware are then compared (over some kind of connection between the PC and the PS2): if they are the same, the emulator is correct and otherwise an emulator bug has been exposed.

By either comparing regularly during the execution or comparing at the end and varying the number of steps like in a binary search, it would then be possible to determine the exact point of the divergence between the emulator and the real hardware: the code emulating the operation in question can then be analyzed and fixed.

The practical implementation of all this is actually more difficult than it may seem at first glance: first of all, the presence of the monitor will alter the PS2 environment (most visibly, it will significantly slow down the game), so several parts of the PS2 may need to be virtualized (e.g. timers, instruction counters and whatever facility is used to communicate with the PC, if the game also uses it) so that the game doesn't malfunction due to the alterations; furthermore, the game must be prevented from taking actions that would cripple the monitor (such as writing to a RAM region used by the monitor), which must be also virtualized.

Furthermore, some hardware components behave non-deterministically: to accommodate this, the effects seen on the PS2 need to be recorded and communicated to the emulator, which must use them (an obvious example of this is the CD/DVD drive, whose response time is certainly not constant).

Finally, the PS2 is a multiprocessor system: handling this requires to run a monitor on each processor, and synchronize them all so that instructions are interleaved in a predictable way; alternatively, auxiliary processors could be emulated on the main one, which would however require prior perfect emulation of their instruction set, since one would not be able to "just run" the instructions (along with register save/restore around them).

Fortunately official developers probably perform something similar to the process discussed here to debug their games, which means that games are probably at least partially resistant to being run under debugger-like programs; furthermore, the hardware may have debugging capabilities built-in, which would ease the monitor implementation or even allow to observe and single-step games externally without a monitor; such capabilities may however only be present on devkit PS2 consoles or boards.

Summing up, while the implementation of all this is likely time consuming, it should provide an almost straightforward way to improve the emulator and make specific games work.

 

[egoneo]

easy to say now u try doing it. hehe

 

[General Plot]

That's the thing about some newbies, they think they know it all. Communication for comparison between a real PS2 and a PC running an emulator is not feesable. Your "theroy" would only apply in an imaginary dream world, where all things are possible. But in the real world, it's unrealistic and downright ridiculous. If you knew anything about the PS2's architecture, you would understand that it would not "slow itself down to the speed of an emulator for the purpose of analysis". If you think I'm wrong, give your theory a shot, and see just how unsuccessful you'll be.

 

[AkumaKnight]

i dont even think you know what you just said with all do respect....

 

[Betamax]

Correct me if I'm wrong here, but to sum up this proposal, you are suggesting that:

a) Create a monitor program
b) Create a virtual layer to run the games on so that the monitor doesn't affect it

c) communicate the results to the emu running the monitor.

The problem with this concept is that b) is effectively suggesting that you create a ps2 emulator for the ps2. (Virtualisation being in essence a pseudonym for emulation or simulation).

Also from this paragraph it is obvious that you haven't really thought this through completely:

Finally, the PS2 is a multiprocessor system: handling this requires to run a monitor on each processor, and synchronize them all so that instructions are interleaved in a predictable way; alternatively, auxiliary processors could be emulated on the main one, which would however require prior perfect emulation of their instruction set, since one would not be able to "just run" the instructions (along with register save/restore around them).

IMHO, such a sync process would use a significant amount of bandwidth, respecially as it would have to act in real time under this proposal. As for the alternative, well emulating the vector units, etc on the ps2's main cpu and expecting the games to run with any degree of accuracy/performance is just crazy (those chips are there for a reason you know )

Finally, in order for any of this to be of any use, you would require that the states of both the virtual machine on the ps2, and the emu be in perfect synchronisation. If not the data sent is usless for comparision. Such a synchronisation is a near impossible task to accomplish.

 

[nahteecirp]

Im not sure totally sure what your trying to do here but your biggest problem is there is no way in hell a ps2 is going to run a program that moniters its I/O and a game, as the only realistic way to do that is to use ps2 linux to run it, but then good luck getting it to play a game. Even if that works you have to sort through (literally) millons of line of code and figure out whats doing what. And I havent even started on the weeks it will take to program all this.

 

[Elb]

The problem with this concept is that b) is effectively suggesting that you create a ps2 emulator for the ps2. (Virtualisation being in essence a pseudonym for emulation or simulation).

Yes, however it is possible to start from a port to the PS2 of PCSX2 and many things can be done by "just doing them" on the hardware.

IMHO, such a sync process would use a significant amount of bandwidth, especially as it would have to act in real time under this proposal. As for the alternative, well emulating the vector units, etc on the ps2's main cpu and expecting the games to run with any degree of accuracy/performance is just crazy (those chips are there for a reason you know )

Finally, in order for any of this to be of any use, you would require that the states of both the virtual machine on the ps2, and the emu be in perfect synchronisation. If not the data sent is useless for comparison. Such a synchronisation is a near impossible task to accomplish.

The sync process only happens between the PS2 processors (R5900, R3000, VUs, etc.) using memory synchronization primitives (such as spinlocks).

The idea is to be able to measure the number of execution steps such that an execution of a certain number of steps will always produce the same results (where the results are RAM contents, hardware registers and perhaps a movie of the digital outputs and a stream of vibration commands - the former would be reported by the monitor while the latter by linking the outputs to a PC in some way).

On a single processor system without external hardware, the number of instructions executed fulfills the requirement.

On a multiple processor system, the instructions executed by the various processors must be interleaved in a deterministic fashion, so that the sequence is always the same. The measure can then be the length of the sequence of instructions executed by all the processors.

Of course, normally the sequence is non-deterministic, so the processors must not run directly the code but they must run a monitor that can run an instruction at a time, and enforce the sequence. This can be performed by having a single global variable indicating the processor number that is currently allowed to execute. Every processor waits its turn, runs a constant number of instructions and increments the variable. For example, you could first run an R5900 instruction, then an R3000 one, then a VU0 one (if it can be used independently, not sure), then a VU1 one and repeat (but for correct results, you should set the number of instruction for each processors so their relative speeds are roughly the normal ones, rather than one per processor).
Of course this assumes that all processors have instruction sets powerful enough to do the synchronization: I don't know if this is the case for VUs.

The addition of external hardware is a further complication that needs special handling: for instance interrupts for the completion of a CD/DVD request will be non-deterministic, and thus may arrive on instruction A on the PS2 and instruction B on the PC with A != B which may screw up successive comparisons.

A possible solution is to virtualize them and always simulate them a certain number of instructions after the request is started, waiting for the real interrupt if it hasn't already happened.

Another possibility is to communicate the instruction numbers that one gets on the PS2 to the PC and use them. However, this means that different runs will produce different results, so finding exactly the first faulty instruction will be harder.

As another example, input devices will need to be virtualized according to a "scripted" sequence of input events, suitable for starting and testing the game.

Once you have a measure of the number of execution steps that leads to a unique sequence of operations for each number of steps (i.e. it is deterministic and predictable), you simply run the emulator until you reach it and tell the PS2 to reach the same number. Once the PS2 reaches it, it sends the whole RAM and hardware register contents (should be less than 50 MB, thus taking 5 seconds to send over 100mbps Ethernet) to the PC. Once the emulator is finished, it compares the PS2 message contents with the data it computed and sees whether the first bug is before or after that number of execution steps.

So there is no need of continuously synchronizing the PC and PS2, the only need is to have them always execute in the same way and stop at the same point. Then, using a binary search, you can find out the first step the the emulator gets wrong.

Also performance is not a problem, since if necessary we can virtualize all timers so that the game doesn't notice that it running slowly (after all, PCSX2 is currently very slow compared to the PS2 but still runs some code). The only constraint is to be able to find problems in a reasonable amount of time, which should be doable.

 

[Elb]

Im not sure totally sure what your trying to do here but your biggest problem is there is no way in hell a ps2 is going to run a program that moniters its I/O and a game, as the only realistic way to do that is to use ps2 linux to run it, but then good luck getting it to play a game. Even if that works you have to sort through (literally) millons of line of code and figure out whats doing what. And I havent even started on the weeks it will take to program all this.

I actually don't know how this would work exactly, but I assume that there are modchips (or ps2 linux kits, or devkits if any developers have them) that allow to run data coming from either a memory card, a cable or the hard disk. Then, starting a game should be doable just by initializing the hardware like a reboot would do (and PCSX2 must do it so it is known how to do it) and jumping to the start address of the game/BIOS. Monitoring I/O is done by doing the same, but virtually on an instruction interpeter running on the PS2.

 

[AkumaKnight]

all i really have to say is instead of telling us all this why dont you try it your self and let us know . ill give you props on your idea though but think on how much time it would take some one just to make half of that stuff happen.

 

[ChaosCode]

Isnt this just another way of saying lets put a ps2 on a pci card and use the mips chip from it on the emulator? if so, it was stated that we want full emulation and not part. I meen whats the point. besides the main thing holding a computer back from emulating the ps2 fully is the speed and by doing whats your saying, you would have to transfure a h3ll of alot of data between the computer and ps2. Its just not going to work.

 

[General Plot]

And it seems to me that even with the progress that the team is making, people just still can't be satisfied. As if they suggest a theory, that the devs can use their "suggestions" to get the emu running better. With all those guys have figured out on their own already, don't you think that they've already considered their own possible ways of working out the bugs? And I would assume that they are more apt to succeed in their manner, rather than try and use some pie in the sky idea. They are constantly making progress. If it ain't broke, don't fix it. Their methods are producing some good results, so they're already on the right path. If you have some urge to test your nutty professor expeirments, then by all means, do so. Just don't fill the forum with suggestions on a field of technology that is obviously out of your scope of education.

 

[asmod666]

btw if u found a way to transfer 50 mo in 5 sec from a PS2 i would gladly take ur hints, i would badly need it according that all network operation is done in almost all development projet i know by the IOP , i would really love to understand how u can reach such speed... i am actually using an ethernet file system when i work on a debug station, and trust me, the ps2 network transfer rate is way slower than this. And if we speak about synchro between real time PS2 running code an PCSX2 emulator... well, it's sadly a dream. IOP usually crash when u try to send too much data. PS2 dev kit is working a bit like u are saying, there are specials hardwares that monitors all the PS2 processors ( well, not all sadly ). So u can stop your code when u want ( sometimes...) and watch variables,register values.

According to me, the best way to improve and to speed up the compatibility development process of PCSX2 would be that a lot of coder start to write technicals demos concerning specificals part of the PS2, test them on a devkit or a debug station, and try to find why ( if ) they don't run well on the PCSX2 emulator