Superchip (SARA) detection

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

JetSetIlly

I'm looking to improve the superchip detection employed in Gopher2600.

The current method is very simple and only considers whether the first byte of the dump is repeated 255 times. This catches dumps where the superchip area is all $00 or all $ff, for example.

However, Trebor has noted that an NTSC dump of Dig Dug does not pass this test. https://github.com/JetSetIlly/Gopher2600/issues/40

Looking at the data of this dump, the superchip area looks random. I'm therefore looking for a process that can detect a superchip even in those circumstances.

I think there's always going to be the chance for false positives (and false negatives) but I've come up with a process that seems to work. At least, it works better than the simple process I've used up to now.


The new rules are:

1) If the program entry address is in the superchip area then there is no superchip

2) If all bytes in the superchip area are the same then there is a superchip

3) If the most frequent byte and the second most frequent byte appear more than all the other bytes combined, then there is a superchip

Rule 3 comes from inspection of the Dig Dug dump.

00000000: c369 ca09 ff00 fb00 f3c3 a3b7 ff00 fb00
00000010: ff00 ff6a ff00 3100 ff00 ff00 ff00 ff00
00000020: ff00 ff00 ff00 ff00 f3c3 2710 ff00 ff00
00000030: c347 ca65 1f00 ff00 ff00 ff00 ff00 ff00
00000040: ff00 ff00 ff00 ff00 ff00 ff00 ff00 ff00
00000050: ff00 ff00 ff00 ff00 ff00 ff00 ff00 ff00
00000060: ff00 ff00 ff00 c30b ca00 ff00 ff00 fb00
00000070: ff00 ff00 ff00 fb00 ff00 ff00 ff00 ff22
00000080: c369 ca09 ff00 fb00 f3c3 a3b7 ff00 fb00
00000090: ff00 ff6a ff00 3100 ff00 ff00 ff00 ff00
000000a0: ff00 ff00 ff00 ff00 f3c3 2710 ff00 ff00
000000b0: c347 ca65 1f00 ff00 ff00 ff00 ff00 ff00
000000c0: ff00 ff00 ff00 ff00 ff00 ff00 ff00 ff00
000000d0: ff00 ff00 ff00 ff00 ff00 ff00 ff00 ff00
000000e0: ff00 ff00 ff00 c30b ca00 ff00 ff00 fb00
000000f0: ff00 ff00 ff00 fb00 ff00 ff00 ff00 ff22

There's likely a good reason why $ff and $00 appear more often during dumping but I don't know what that might be. But whatever the reason, that characteristic seems significant.


To eliminate false positives (ie. saying there is a superchip when there isn't) I've added some additional rules that try and determine if the data in the superchip area is just ROM data.

To do this I scan the dump just beyond the superchip area (128 bytes or up to the entry address, whichever comes first). If the data in this extended area is similar to the data in the superchip area then I say that there is no superchip.

So, inserted into the rules table above:

2b) If all the bytes in the extended area are the same as the superchip area then there is no superchip

and
 
3b) If the most frequent byte and the second most frequent byte (in the extended area) appear more than all the other bytes combined (in the extended area), then there is no superchip


The new rules pass all my regression tests and also correctly catch superchip ROMs that it didn't before. But there's always the possibility that I've made wrong assumptions. For example, the extension area is defined to be a maximum of 128 bytes for no particular good reason. It could easily be 256 bytes I suppose.


I'd be interested in hearing any insight from the people on this forum.


Here's the link to the actual code in Gopher2600. That might be clearer than my explanation above.

https://github.com/JetSetIlly/Gopher2600/blob/a25caa7994bf3c7033aeb1638871c6b9584cffa8/hardware/memory/cartridge/fingerprint.go#L514-L608
https://github.com/JetSetIlly/Gopher2600
@JetSetIlly@mastodon.gamedev.place
@jetsetilly.bsky.social

Thomas Jentzsch

Stella's check is simpler:

For each 4K, check if the 1st 128 bytes are identical to the 2nd 128 bytes.

alex_79

#2
Quote from: JetSetIlly on 21 Jul 2025, 06:22 AMThere's likely a good reason why $ff and $00 appear more often during dumping but I don't know what that might be. But whatever the reason, that characteristic seems significant.
I believe the reason is that those are the most likely values that developers used to fill the ROM area where the SARA chip is mapped, but it might also happen to find initialized or random data there.

Despite the fact that such area is not accessible by the console, the SARA chip needs some time to decode the address after the bus is stabilizied and in this short interval the ROM actually gets selected and (briefly) drives the data bus. If the address is in the "write port" range, when the SARA chip is finished decoding it, it deselects the rom and leaves the data bus into high-Z state waiting to latch the value to be written to RAM.
Because of the capacitance of the bus lines on the console board (many of these roms were obtained using a 7800 console with the "DevOS" bios), that value (or part of it) from the ROM might persist enough to be latched.

I did test a few cartridges in the past and found that this persistence effect is not as reliable as with the known "zeropage instead of immediate" bug that many games have, likely because of the much shorter time the bus is driven in this case, so some of the bits might give unstable/different results each time and conditions like temperature and specific console also play a role.

In any case, whatever actual value is latched, it will be written to the SARA RAM and so it will appear again in the second 128 bytes area, which is what the detection method that Thomas mentioned is checking.

The only case where that method could fail is in case of dumps of prototypes made by reading the eproms directly (so without the SARA chip in place), if the first 256 bytes were not initialized.
I don't know if there's any example of roms like that floating around.

JetSetIlly

Quote from: Thomas Jentzsch on 21 Jul 2025, 03:30 PMStella's check is simpler:

For each 4K, check if the 1st 128 bytes are identical to the 2nd 128 bytes.
Haha. I hadn't noticed that about the DigDug dump.

A classic case of overthinking the problem. Although I think checking whether the entry point is inside the superchip area is a good addition. It precludes the (very unlikely) possibility that it is real ROM data.

Quote from: alex_79 on 21 Jul 2025, 07:07 PMIn any case, whatever actual value is latched, it will be written to the SARA RAM and so it will appear again in the second 128 bytes area, which is what the detection method that Thomas mentioned is checking.

I see. The dumper is reading the addresses linearly and therefore accesses the write addresses first. That makes sense. Now that you've mentioned it, I think I knew that but had forgotten it.

I'll change the frequency check to a duplicate check.

In my original idea, checking the area immediately after the superchip addresses, was an attempt to defend against ROM data that just happens to look like data from a superchip dump. If it had similar characteristics then it suggests the data is not from a superchip dump.

But on reflection, that seems highly unlikely to happen. I'm unduly worried about false-positives and false-negatives I think.
https://github.com/JetSetIlly/Gopher2600
@JetSetIlly@mastodon.gamedev.place
@jetsetilly.bsky.social