Dumping Guides

General No-Intro related discussions.
hking0036
Datter
Posts: 109
Joined: 01 Oct 2016 18:33

Re: Dumping Guides

Post by hking0036 »

Whovian9369 wrote: 31 May 2018 07:12
hking0036 wrote: 31 May 2018 00:00 Switch has an XCI Dumping script public you can run if you have access to pegaswitch (3.0.0 and very recently 4.1.0 only),

You can read this on how to set up pegaswitch: https://gbatemp.net/threads/tutorial-se ... ew.489299/

XCI Dumper is available here: https://gist.github.com/AcK77/56b86469f ... 2cb962ee30

This only applies to switch carts, currently there's no good way to dump digital (and nintendo is b&ing people from pulling from NUS this time around). As the public CFWs come closer there will presumably be more (and easier) tools for this.
I'd argue that decrypting NCAs from the SD Card is the way to go for installed titles, since they're the same as the CDN copies in all of the instances I've seen.

Requires removing the SD Card encryption, of course. Thanksfully it's just double encrypted.

Original NCA > Title Key encryption (which is the "original" version) > SD Card encrypted

How to get the SD Seed is something I'll get into later...

Then compare the SHA256 of the newly SD-Decrypted file to the folder name, and you're golden!

... I'll probably write that into Hiccup's wiki later.
Follow-up on this, NCAs are the game content which is well and good, .NSP files are akin to .CIA files from 3DS, and contain nca, cnmt, tik, cert. tik/cert (should be) console-specific so if we come down to that then they should be blanked anyways, but I don't know what the cnmt entails, I'll try and dig up more about that.
Whovian9369
Datter
Posts: 72
Joined: 09 Sep 2016 18:36

Re: Dumping Guides

Post by Whovian9369 »

hking0036 wrote: 16 Jun 2018 16:23
Whovian9369 wrote: 31 May 2018 07:12
hking0036 wrote: 31 May 2018 00:00 Switch has an XCI Dumping script public you can run if you have access to pegaswitch (3.0.0 and very recently 4.1.0 only),

You can read this on how to set up pegaswitch: https://gbatemp.net/threads/tutorial-se ... ew.489299/

XCI Dumper is available here: https://gist.github.com/AcK77/56b86469f ... 2cb962ee30

This only applies to switch carts, currently there's no good way to dump digital (and nintendo is b&ing people from pulling from NUS this time around). As the public CFWs come closer there will presumably be more (and easier) tools for this.
I'd argue that decrypting NCAs from the SD Card is the way to go for installed titles, since they're the same as the CDN copies in all of the instances I've seen.

Requires removing the SD Card encryption, of course. Thanksfully it's just double encrypted.

Original NCA > Title Key encryption (which is the "original" version) > SD Card encrypted

How to get the SD Seed is something I'll get into later...

Then compare the SHA256 of the newly SD-Decrypted file to the folder name, and you're golden!

... I'll probably write that into Hiccup's wiki later.
Follow-up on this, NCAs are the game content which is well and good, .NSP files are akin to .CIA files from 3DS, and contain nca, cnmt, tik, cert. tik/cert (should be) console-specific so if we come down to that then they should be blanked anyways, but I don't know what the cnmt entails, I'll try and dig up more about that.
(I never did make that guide... I'll have to get to that a little later, haha! Thanks for reminding me, hking!)

I don't know if we should dat NSP files quite yet, especially until we figure out "what" they are, how they're "properly" made, etc... Until then, it might be a good idea to DAT those yet, for hopefully obvious reasons :P

CNMT is basically the 3DS TMD but changed somewhat, I believe. http://switchbrew.org/index.php?title=NCA has some info, I believe.
hking0036
Datter
Posts: 109
Joined: 01 Oct 2016 18:33

Re: Dumping Guides

Post by hking0036 »

Whovian9369 wrote: 17 Jun 2018 02:52
hking0036 wrote: 16 Jun 2018 16:23
Whovian9369 wrote: 31 May 2018 07:12

I'd argue that decrypting NCAs from the SD Card is the way to go for installed titles, since they're the same as the CDN copies in all of the instances I've seen.

Requires removing the SD Card encryption, of course. Thanksfully it's just double encrypted.

Original NCA > Title Key encryption (which is the "original" version) > SD Card encrypted

How to get the SD Seed is something I'll get into later...

Then compare the SHA256 of the newly SD-Decrypted file to the folder name, and you're golden!

... I'll probably write that into Hiccup's wiki later.
Follow-up on this, NCAs are the game content which is well and good, .NSP files are akin to .CIA files from 3DS, and contain nca, cnmt, tik, cert. tik/cert (should be) console-specific so if we come down to that then they should be blanked anyways, but I don't know what the cnmt entails, I'll try and dig up more about that.
(I never did make that guide... I'll have to get to that a little later, haha! Thanks for reminding me, hking!)

I don't know if we should dat NSP files quite yet, especially until we figure out "what" they are, how they're "properly" made, etc... Until then, it might be a good idea to DAT those yet, for hopefully obvious reasons :P

CNMT is basically the 3DS TMD but changed somewhat, I believe. http://switchbrew.org/index.php?title=NCA has some info, I believe.
Yeah, I get that. I think there's more to come with that soon though. There seems to be a title installer/dumper in the work that may end up making .nsp soon, but ultimately I think we just need to sit on this some more (there's only one entry so far after all).

A guide would be nice to have around! :)
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

The Nintendo Switch guide could use some updating. I'd be more than glad to help if you let me contribute.
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

https://wiki.no-intro.org/index.php?tit ... ping_Guide

It's done. Feel free to check the guide and add more info or correct it if you spot any mistakes! :D

I'm DarkMatterCore, btw. I'm the guy behind NXDumpTool. This is my *very* old nickname.
Hiccup
Datter
Posts: 1722
Joined: 09 Oct 2015 11:29

Re: Dumping Guides

Post by Hiccup »

PabloACZ wrote: 19 Nov 2019 02:46 https://wiki.no-intro.org/index.php?tit ... ping_Guide

It's done. Feel free to check the guide and add more info or correct it if you spot any mistakes! :D

I'm DarkMatterCore, btw. I'm the guy behind NXDumpTool. This is my *very* old nickname.
Thanks, that's a big improvement.
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

I migrated the digital title dumping steps to its appropiate wiki page: https://wiki.no-intro.org/index.php?tit ... ping_Guide

I'd love to fill what's missing there. Have you already defined a standard rule set to dat NCA files?
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

I just updated both Nintendo Switch cartridge and digital title dumping guides.

The cartridge guide just suffered minor changes. The digital title guide, on the other hand, now features a full section on how to gather NCA/Ticket information from output NSP dumps.

Any changes and suggestions are well received. :D
Hiccup
Datter
Posts: 1722
Joined: 09 Oct 2015 11:29

Re: Dumping Guides

Post by Hiccup »

DarkMatterCore wrote: 21 Nov 2019 18:24 I just updated both Nintendo Switch cartridge and digital title dumping guides.

The cartridge guide just suffered minor changes. The digital title guide, on the other hand, now features a full section on how to gather NCA/Ticket information from output NSP dumps.

Any changes and suggestions are well received. :D
Nice work again.

I've just improved this, so I you could use that as a basis for creating the "what info to submit" section:
https://wiki.no-intro.org/index.php?tit ... ting_guide

After that I can remove the stuff you have carried across from the datting guide, to just leave the stuff pertinent to datters processing submission info.
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

Hiccup wrote: 22 Nov 2019 21:15
DarkMatterCore wrote: 21 Nov 2019 18:24 I just updated both Nintendo Switch cartridge and digital title dumping guides.

The cartridge guide just suffered minor changes. The digital title guide, on the other hand, now features a full section on how to gather NCA/Ticket information from output NSP dumps.

Any changes and suggestions are well received. :D
Nice work again.

I've just improved this, so I you could use that as a basis for creating the "what info to submit" section:
https://wiki.no-intro.org/index.php?tit ... ting_guide

After that I can remove the stuff you have carried across from the datting guide, to just leave the stuff pertinent to datters processing submission info.
Excellent, thanks. I'll get on it as soon as I get back home.

There's some info that can already be retrieved from NSP dumps:
  • NXDumpTool generates .cnmt.xml files straight from .cnmt.nca files during the dump process. The Title ID can be easily retrieved from them instead of decrypting .cnmt.nca files and manually extracting the CNMT from the PFS0 section.
  • If the steps from the guide are followed properly, tickets from NSP dumps are already stripped and follow the necessary naming scheme (if available, of course). These are the exact modifications made to every ticket:
    • The RSA signature (0x100 bytes @ 0x004) is replaced with 0xFF bytes.
    • The RSA signature issuer (0x40 bytes @ 0x140) is replaced with "Root-CA00000003-XS00000020".
    • The titlekey data block (0x100 bytes @ 0x180) is zeroed-out. The first 16 bytes from it are then replaced with the encrypted titlekey.
    • The titlekey type field (1 byte @ 0x281) is set to 0x00 (common).
    • The Ticket ID field (8 bytes @ 0x290) is zeroed-out.
    • The Device ID field (8 bytes @ 0x298) is zeroed-out.
    • The Account ID field (4 bytes @ 0x2B0) is zeroed-out.
    This essentially converts a personalized ticket (titlekey type field set to 0x01) to a common one. Bear in mind that these modifications *only* take place if the titlekey type isn't common. Otherwise, it is left untouched.
I'd like to know more about CDN requests, though. Do they really use different IDs? I can probably dig through the installed data to see if I can find anything useful.
Hiccup
Datter
Posts: 1722
Joined: 09 Oct 2015 11:29

Re: Dumping Guides

Post by Hiccup »

DarkMatterCore wrote: 22 Nov 2019 21:48
Hiccup wrote: 22 Nov 2019 21:15
DarkMatterCore wrote: 21 Nov 2019 18:24 I just updated both Nintendo Switch cartridge and digital title dumping guides.

The cartridge guide just suffered minor changes. The digital title guide, on the other hand, now features a full section on how to gather NCA/Ticket information from output NSP dumps.

Any changes and suggestions are well received. :D
Nice work again.

I've just improved this, so I you could use that as a basis for creating the "what info to submit" section:
https://wiki.no-intro.org/index.php?tit ... ting_guide

After that I can remove the stuff you have carried across from the datting guide, to just leave the stuff pertinent to datters processing submission info.
Excellent, thanks. I'll get on it as soon as I get back home.

There's some info that can already be retrieved from NSP dumps:

...

I'd like to know more about CDN requests, though. Do they really use different IDs? I can probably dig through the installed data to see if I can find anything useful.
Thanks for the info.

What I wrote on that page was really just me being pedantic. The switch likely just records the title id it gets from the eShop, and sends the same title id to the CDN. There is no known case of the title ids in different locations (eshop, cnmt) being different. Simply because its easiest. I've updated the datting guide regarding this.

However, as you have a modded switch, there is something you could do research on - logging the network activity of a Switch when downloading from the eShop - either on the Switch, or using a PC.
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

Hiccup wrote: 23 Nov 2019 18:38Thanks for the info.

What I wrote on that page was really just me being pedantic. The switch likely just records the title id it gets from the eShop, and sends the same title id to the CDN. There is no known case of the title ids in different locations (eshop, cnmt) being different. Simply because its easiest. I've updated the datting guide regarding this.

However, as you have a modded switch, there is something you could do research on - logging the network activity of a Switch when downloading from the eShop - either on the Switch, or using a PC.
That makes sense. Thanks for clarifying.

I'll try to sniff some packages using Wireshark whenever I have the chance.
Hiccup
Datter
Posts: 1722
Joined: 09 Oct 2015 11:29

Re: Dumping Guides

Post by Hiccup »

DarkMatterCore wrote: 25 Nov 2019 03:08
Hiccup wrote: 23 Nov 2019 18:38Thanks for the info.

What I wrote on that page was really just me being pedantic. The switch likely just records the title id it gets from the eShop, and sends the same title id to the CDN. There is no known case of the title ids in different locations (eshop, cnmt) being different. Simply because its easiest. I've updated the datting guide regarding this.

However, as you have a modded switch, there is something you could do research on - logging the network activity of a Switch when downloading from the eShop - either on the Switch, or using a PC.
That makes sense. Thanks for clarifying.

I'll try to sniff some packages using Wireshark whenever I have the chance.
Thanks
To do that you'll probably need to do some some sort of mod to the Switch to account for the HTTPS encryption (HTTPS encrypts the HTTP headers too). E.g. patching sysmodule to disable SSL or installing dummy certificates (like one used to allow the Fiddler program to read HTTPS traffic).
User avatar
DarkMatterCore
Posts: 35
Joined: 06 Apr 2011 16:34

Re: Dumping Guides

Post by DarkMatterCore »

Both cartridge and digital software dumping guides (Nintendo Switch) have been updated to reflect the changes from NXDumpTool v1.1.8 (including dump verification against No-Intro and delta fragment dumping).

I wanted to thank you guys for the HTTPS endpoint! I hope more and more people find it useful as well.
Hiccup wrote: 25 Nov 2019 22:55 Thanks
To do that you'll probably need to do some some sort of mod to the Switch to account for the HTTPS encryption (HTTPS encrypts the HTTP headers too). E.g. patching sysmodule to disable SSL or installing dummy certificates (like one used to allow the Fiddler program to read HTTPS traffic).
NXDumpTool kept me busy (just look at the changelog - there are other minor fixes I forgot to mention there). I'll try to put up a quick mitm sysmodule and see what can I gather.

Btw, regarding the CNMT modifications, they'll now *only* take place if the SHA-256 checksum for any of the NCAs is different than the one stored in the CNMT (which only happens if "Generate ticket-less dump" is enabled). This means that generating valid dumps without touching the CNMT at all is a possibility - no configurable option required. Let me know if this meets your criteria.
Shiranui
Posts: 104
Joined: 25 Aug 2019 20:43

Re: Dumping Guides

Post by Shiranui »

Hi, any guides to dump Nintendo Switch updates & DLC?
Post Reply