Nintendo Switch Cartridges - Key Area

General No-Intro related discussions.
Post Reply
Whovian9369
Datter
Posts: 72
Joined: 09 Sep 2016 18:36

Nintendo Switch Cartridges - Key Area

Post by Whovian9369 »

There has recently (as in the past month or so) been a discovery that you could get more data (0x200, if I recall?) out of a Switch cartridge.
SciresM, (one of the public devs in the community with very deep access into the Switch,) has said previously in the VGPC Discord server that:
SciresM: wrote: tl;dr for switch is that what people call xci doesn't match actual format
there's an 0x200 sector that goes before the header
which contains cartridge communication keydata
(it's title-specific, not cart-specific)
...
whether can be done properly without custom dumper sysmodule is an open question imo
it can be done that way easily since in that model you take over all lotus comms directly
you could probably patch FS to output the sector to you.
(Source, in VGPC Discord server)
It has since been figured out that you can just dump it directly from FS:
SciresM wrote: tl;dr is that the the asic handler object in nn::gc:: keeps a copy of the cartridge header, a copy of the cartridge cert, and a copy of the cartridge key area in its members
So you can debug FS and read the key area out of memory.
(Source, in VGPC Discord server)
SciresM wrote: official XCI files produced by Nintendo tools start with the key area
The key area is at offset 0 and the signature/header is at offset 4096
And the key area is shared for all cartridges for a game
"Redumped" is a yes.
...
Your dumps are bad, and you should fix it.
Aren't you supposed to be about preservation?
...
The key area contains the communication titlekey, which is fixed per game.
...
In official files produced by Nintendo tools
the key area is the first part of the file
and what your dumps call "XCI" is at offset 0x1000.
If you care about preserving games, dumps should be re-done.
(First message source, in VGPC Discord server)

What I'm personally thinking is that we should mark all the previous Switch dumps without this data as "Bad", put this data into the main XCI dumps via a currently-WIP rewrite of the Switch cart dumping tool NXDumpTool (which we already use for Switch cart dumps before this new information), and then get people to redump these as from what I'm aware - This is how we would normally do it.
My main hang up here at the moment is that as this tool does not have any public, fully compiled, "Stable" releases of this updated version app out there yet from the official developer. What *is* out there however, is unofficial (and basically "Alpha") builds that have been compiled and put into the VGPC Discord Server.

I have talked to Hiccup about this in private, and he feels that we should put this data in its own file (that could be optional for submission and optionally excluded from the DATs) alongside the XCI files, keeping the old dumps in the database as "Good."
He has told me that he will put his argument as its own post, so please refer there for his view on this.
One of my issues with this is that dumpers and datters will likely get confused in the long run (how we are to add the data if it's its own file, users submitting only the Key Area file and not the XCI, or the XCI and not that extra data, etc) more than this would help.

Thanks for your time.

Edit: There seems to be a good thread going of hashes for the XCI and this data over at viewtopic.php?f=7&t=4036
Last edited by Whovian9369 on 20 Sep 2020 00:10, edited 1 time in total.
Hiccup
Datter
Posts: 1732
Joined: 09 Oct 2015 11:29

Re: Nintendo Switch Cartridges - Key Area

Post by Hiccup »

Okay, my argument was that datting marking all the ROMs as bad would create confusion/difficulty as even though the main part of the XCI is fine, all the file hashes will have to be replaced. If the redumps include the key area at the top of the main XCI data, then the datter would have to obtain (large, rare, pirated, possibly unavailable) XCI files to check the new dump against the old one, and the dumper would have to upload the (copyrighted) key data to the forum for the dumper to append to the XCI so that the hash can be checked. Alternatively, the datter would have to hash the XCI+key data as a whole, plus the the XCI seperately, which would take double the time. And if you are doing that, you might as well just dat the key area as a separate file (using the "Label" field of the file set to "Key Area"). This would make things easier for both the dumpers and datters. E.g. that's how I've done it for this redump. https://datomatic.no-intro.org/index.ph ... 102&n=0014
And if somebody wants the whole/proper XCI, they can just append the key area to the top. That's all that NXDumpTool will be doing anyway. Its not a complicated operation as Whovian implies.
Whovian9369
Datter
Posts: 72
Joined: 09 Sep 2016 18:36

Re: Nintendo Switch Cartridges - Key Area

Post by Whovian9369 »

Hiccup wrote: If the redumps include the key area at the top of the main XCI data, then the datter would have to obtain (large, rare, pirated, possibly unavailable) XCI files to check the new dump against the old one, and the dumper would have to upload the (copyrighted) key data to the forum for the dumper to append to the XCI so that the hash can be checked.
Part of my issue with Hiccup's argument is that we would have to, no matter what, pirate an old copy of a dump and the new copy to compare them.
Instead we can do what we normally would do for brand new dumps or replacing bad dumps and simply trust the dumper, like we would for basically any other system's bad dump fix or any brand new dumps where we don't need to compare that each old dump is "the same."
Hiccup wrote: Alternatively, the datter would have to hash the XCI+key data as a whole, plus the the XCI seperately, which would take double the time.
and I don't understand why we would need to hash the two versions twice, either -- To check what revision they go with?
Why not just trust the filename that NXDumpTool provides? Or any pictures or info of the cartridge that the dumper provides? ... Like we would for any other system's new or bad-dump-repair dumps?

Honestly, I say that this shouldn't be different than "business as usual" just due to the set size, and instead that we should do it like we normally would despite the set size being as big as it is.
As far as I'm aware this data isn't "extra data" to the rom like a save file would be, as SciresM said himself that this data appears to be part of the actual rom data itself. Plus, we dat *roms* and not normally extraneous data alongside said roms, with a few exceptions if it has extra data on a brand new cart (save files that have "starting data" on it right from the factory, for example).
Shiranui
Posts: 104
Joined: 25 Aug 2019 20:43

Re: Nintendo Switch Cartridges - Key Area

Post by Shiranui »

IMO all games should be redumped including "everything" into one single file, but once there is a tool that makes that possible (manually padding the .bin and adding it to .xci is really tedious).

Also, Hiccup pointed in the other thread that maybe there are other "areas" in the cart that cannot be accessed at this point, so making new DAT right now could be pointless if this ends being true.
SciresM wrote: The key area contains the communication titlekey, which is fixed per game.
I would better say "fixed per cartridge revision".
Zelda Breath of the Wild Rev1 and Rev3 have different card_key_area.
Same for Torna the Golden Country Rev0 and Rev1.
gamecaptor
Dumper
Posts: 220
Joined: 16 Jul 2010 06:27

Re: Nintendo Switch Cartridges - Key Area

Post by gamecaptor »

I just realized this was a thing and o-boy am I confused.

Looking through my set, I see some of the ROMs are marked as bad as they are missing the initial area .bin files.

Yet I see a recent dump, Mario Golf, that is good but it does not have the initial area .bin file being asked for whereas Metriod Dreed is marked as bad as it is requiring the initial area .bin file.

Two questions:

1. Does this mean that ALL previous cart dumps are bad as they lack this initial area information?
2. Does this mean that some of the current cart dumps have been dumped 'correctly' and contain this information?
Hiccup
Datter
Posts: 1732
Joined: 09 Oct 2015 11:29

Re: Nintendo Switch Cartridges - Key Area

Post by Hiccup »

The initial area data is datted as separate file, so that older dumps would not be "invalidated".

When you say some dumps without the initial area are marked as bad, do you mean your ROM manager is listing the initial area as absent (e.g. the archive as incomplete?). There shouldn't be (and I don't think there are) any XCIs marked as bad in the dat due to lack of initial area.
gamecaptor
Dumper
Posts: 220
Joined: 16 Jul 2010 06:27

Re: Nintendo Switch Cartridges - Key Area

Post by gamecaptor »

Hiccup wrote: 27 Mar 2022 00:51 The initial area data is datted as separate file, so that older dumps would not be "invalidated".

When you say some dumps without the initial area are marked as bad, do you mean your ROM manager is listing the initial area as absent (e.g. the archive as incomplete?). There shouldn't be (and I don't think there are) any XCIs marked as bad in the dat due to lack of initial area.
You know what, you're right Hiccup. They are not 'bad', just 'incomplete'. My brain sees RED = BAD, but really it should be RED = SOMETHING IS MISSING BUT IT'S STILL TECHNICALLY OK! :)

So what about the cart dumps that do not have the initial area but are marked as ok. Does those carts not have it, or has the DAT not been updated to reflect that it's missing that piece (i.e. the archive is incomplete)?
Hiccup
Datter
Posts: 1732
Joined: 09 Oct 2015 11:29

Re: Nintendo Switch Cartridges - Key Area

Post by Hiccup »

Well if there is no dump that includes the initial area in datomatic, then there will be no initial area in the dat.
gamecaptor
Dumper
Posts: 220
Joined: 16 Jul 2010 06:27

Re: Nintendo Switch Cartridges - Key Area

Post by gamecaptor »

Ah, I see. My confusion was coming from reading through this thread. I had the impression that is something that all Switch carts should have, but I suppose not.

Thanks again Hiccup.
Hiccup
Datter
Posts: 1732
Joined: 09 Oct 2015 11:29

Re: Nintendo Switch Cartridges - Key Area

Post by Hiccup »

Well all carts do have the initial area, and its good to dump it, but only somewhat recently did nxdumptool get support for dumping it, plus a lot of the dumps are from the warez scene and they don't care about that sort of stuff. So that's why only some entries have the initial area.
User avatar
xuom2
High Council
Posts: 927
Joined: 22 May 2008 18:45

Re: Nintendo Switch Cartridges - Key Area

Post by xuom2 »

we marked as BAD the archives with no initial area?
Hiccup
Datter
Posts: 1732
Joined: 09 Oct 2015 11:29

Re: Nintendo Switch Cartridges - Key Area

Post by Hiccup »

Nah, they aren't "bad" as such, just missing not-so-important data. We could mark them all as bad, but I don't know if that would achieve much.
User avatar
xuom2
High Council
Posts: 927
Joined: 22 May 2008 18:45

Re: Nintendo Switch Cartridges - Key Area

Post by xuom2 »

Lol nointro is all about useless data!

I just prefer the approach we had with NDS.

Now you have good dumps in two formats (with 1 or 2 files) and incomplete dumps not marked as such.
Hiccup
Datter
Posts: 1732
Joined: 09 Oct 2015 11:29

Re: Nintendo Switch Cartridges - Key Area

Post by Hiccup »

Well I don't really mind which way its done. I guess its true, this is a a very similar situation with the "underdumped headers" on NDS.
User avatar
InternalLoss
Datter
Posts: 52
Joined: 23 May 2018 00:03

Re: Nintendo Switch Cartridges - Key Area

Post by InternalLoss »

xuom2 wrote: 05 Apr 2022 16:34 Lol nointro is all about useless data!

I just prefer the approach we had with NDS.

Now you have good dumps in two formats (with 1 or 2 files) and incomplete dumps not marked as such.
I'm inclined to agree here - we should have accurate dumps with the key area, and then the prior dumps are marked as "incomplete" because of it - nxdumptool should also ideally be updated accordingly so people are able to dump the XCIs properly instead of having to concatenate it by hand.
Post Reply