Error Detection in IPS

[Europia79]

For SNES, my usual method to verify whether or not a patch actually requires a Header or not (despite what the documentation may say because that information is often unreliable) would be to patch against BOTH versions: Headered & Unheadered ROMs.

Fortunately/Unfortunately (depending on perspective), I came across several Chrono Trigger patches that worked on BOTH Headered & Unheadered ROMs. So, it wasn't immediately apparent which one was the correct Parent/Baserom to use for patching.

Hence, I had to conduct binary fingerprint analysis of the patches against the ROMs:

One pattern that I noticed was that "shared bytes" often time signify the correct ROM: Altho, this is with the exception of cases where the patch has been created against an Unheadered Source ROM with a Headered Target ROM (or vice versa).

Something else that I noticed was IPS data being written regardless of the target area: Notably with respect to the FIRST BYTE.

Question: Does it make sense that the first byte of IPS data is the exact same as the first byte of the target ?

I don't see how that is a logically justifiable position: In other words, will an IPS implementation ever create a data record where the first byte of the source and the first byte of the target are the same ?

If not, then THIS is finally an excellent opportunity to introduce error detection to such a terrible format (as stated in #89 ): And at the same time, wouldn't break compatibility at all.

Also, I would like to re-iterate my support of #88 which would be a Header (or Footer) with the necessary hash information to perform error checking: Add it on patch creation, and remove it before applying. Easy peasy.

[Alcaro]

Yes, that is possible. The obvious one is if the first changed byte is at offset 0x454F46; the IPS format can't represent that, so Flips pretends 0x454F45 is changed as well and writes an unchanged byte. (I've seen other IPS creators do the same, throw errors, or simply emit corrupt patches; there's no real standard there.)

It's also possible for the IPS creator to be poorly optimized; I've seen one that implements AAAAAAAAAAAAAAAA -> CABBBBBBBBBBBBBB as 'write CA, then switch to RLE for the Bs' (Flips only writes the C). Sure, that's not the first byte, but last byte being wasteful is just as fine heuristic as first, right? (And there's probably a patcher out there that'd be wasteful on the mirrored input.)

And, as mentioned, unheadered source ROM and headered target will, of course, write huge numbers of unchanged bytes if applied to something headered.

And I've seen several other kinds of IPS weirdness, like not having the records in ascending order; it wouldn't surprise me if wasteful bytes show up in one of the 999999 IPS creators in existence. Actually, it would surprise me if it does not.

Sounds like a fine heuristic, though. If it writes pointless bytes on a headered ROM and everything looks sensible if unheadered, there's a 90% chance the unheadered is correct. (And if it's not wasteful on either, then the heuristic will just return 'no idea'.)

No clue how that'd look to users, though. A --ips-guess-header command line flag would be reasonable, but I don't know how that'd look in the GUI.

[Europia79]

For the IPS Format "Specification", I went by https://zerosoft.zophar.net/ips.php

As far as "EOF" (0x454F46): My interpretation is that it's just an address (like any other) and should be treated as such: With the special case where there's no size and no data after it. I am guessing that the original author just used it as a visual marker to ensure that all data was written to the file ? I don't know.

I have already written a script, but based on what you're saying, I will add a special case for address 0x454F45.

And I'll also have to devise a method for cases where only ONE Header is present for Target/Source ? Not sure what that'll be tho. I'll have to think about it.

As far as how it looks to the User, it sounds akin to a "Compiler Warning" message ? (or "Patch Warning" message).

[Alcaro]

That is one possible interpretation, yes. The other interpretation is that EOF means end of patch and anything beyond it is to be ignored.

A third interpretation is that the three bytes after EOF (if any) are the size to shrink the ROM to, and anything after that is to be ignored.

Naturally, different IPS patchers pick different interpretations. That's where the true complexity of IPS lies - the format is simple, but it's also a long-term victim of Postel's law. There are 999999 different patchers, and they all behave subtly differently; no matter how unlikely a specific construct is, it's very likely used by something, and it's impossible to say anything certain. (You can certainly make educated guesses, though.)