Xbox 360 digital title preservation

General No-Intro related discussions.
Qubits
Posts: 3
Joined: 06 Aug 2021 10:28

Re: Xbox 360 digital title preservation

Post by Qubits »

Resistiv wrote: 22 Jan 2020 02:07 Yes, it looks like you can indeed just copy over files from a hard drive using something like FATXplorer, however the files are often signed differently in their headers, which produces differing hashes. Here's looking at one dump, comparing my dump (left) and the scene dump in the dat (right) of N+ (World) (XBLA). Everything after the highlighted byte is matching in the attached image. Of course, more samples will be needed for decisive action, but it seems that the first 568 bytes (0x238) can be skipped when hashing.
I can explain this in general. Your dump is a PIRS type STFS file, which is a dump from a disc. The other is a LIVE type STFS which is a downloaded dump from XBOX live. The signatures at the beginning are different because the key for the digital signature of PIRS and LIVE files differ.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

Nice work. It sounds like that this signature should be kept (maybe even if its contains console specific info) as it can br used to verify contents, but it should be split from the file so that consistent hashes for the main file can be achieved, and so that these stripped third party dumps can be included in the dat.
Qubits
Posts: 3
Joined: 06 Aug 2021 10:28

Re: Xbox 360 digital title preservation

Post by Qubits »

The signature does not contain console info.
The signature is generated from 280 bytes of a certain area in the header.
These 280 bytes include console specific info as well as the "top" hash of all files (bit difficult to explain, hashes are generated for blocks, then like a pyramid there are hashes of hashes created, resulting in a "top" hash).

It's difficult to strip something. the STFS files are what really is transfered from xbox live.
(not for example like nsps, where the ncas is what's actually downloaded).
So stripping something would mean creating more or less your own file format. If you cut the header, the file would become unusable on original hardware.

There's nothing like what you call a "main file". STFS is a container format (like e.g. ncas are too).
So you could extract that container. These often contain 100s or 1000s of files. You *could* dat those, but again extracted files are not usable.

Datting XBLA and XBLIG games is unproblematic. As written, trials always exist so everyone downloading the trial of the game gets the exact same file and when using dashlaunch it's the full game.
Datting and preserving these also makes sense because they are fully funktional as trials on unmodified hardware.

The question is how to dat the DLCs. I think the best option, considering everything, would be to go with your initial idea and zero out parts of the header that are unique.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

Well I was just referring to the DLC when I mentioned console-unique data in the last post. Yeah I guess that third party dumps with removed signature could just be marked as such, and marked as bad. For DLCs, as well as zeroing out the unique data, maybe it could be stored in a seperate ("made-up") file (as long as the dumper in question is okay with that).
Vigi
Dumper
Posts: 175
Joined: 18 Jun 2008 10:16

Re: Xbox 360 digital title preservation

Post by Vigi »

I did some work on the Title Updates datfile, but it's a huge mess and will take many more hours to get it sorted. Not to mention that there are around 3448 title updates in total, and the datfile (which was discontinued) only has 1170 entries, so that's only 1/3 of all the known updates.

If someone wants to continue the work, let us know.

Basically we have worksheets with all the known updates for every game. But that doesn't include the filenames, hashes, etc.
When you have the media id of a game, you can check on redump (the database and the undumped wiki's) for region and languages.
Then on places like http://xboxunity.net/ you can find most of the decrypted updates. Or you download them on your Xbox 360 if you have a particular game and extract them from the HDD.

As for the encrypted title updates. We also have a (hopefully) full set of links of encrypted .XCP packages straight from the MS servers that could be downloaded and hashed, but getting the proper title/region/languages will be a huge task also and there's currently also no public method for decrypting them. It's also unknown currently if there are separate decryption keys that need to be included along with the packages.
User avatar
xuom2
High Council
Posts: 926
Joined: 22 May 2008 18:45

Re: Xbox 360 digital title preservation

Post by xuom2 »

just contact me, I can do any necessary bulk import into database, skipping the GUI.
emoose
Posts: 1
Joined: 28 Sep 2021 09:25

Re: Xbox 360 digital title preservation

Post by emoose »

Vigi wrote: 25 Sep 2021 12:04 As for the encrypted title updates. We also have a (hopefully) full set of links of encrypted .XCP packages straight from the MS servers that could be downloaded and hashed, but getting the proper title/region/languages will be a huge task also and there's currently also no public method for decrypting them. It's also unknown currently if there are separate decryption keys that need to be included along with the packages.
There is xcp_dumper.py that was posted relatively recently, haven't tried it myself but seems to work similar to an old tool I wrote for XCPs years ago.

Big problem is actually getting the keys to decrypt it though, from what I know those can only be retrieved via a 360 console connected to LIVE. :/

Also IIRC, decrypted XCP packages would result in a .cab file containing pretty much an STFS package inside, but with the first 0x32C bytes nulled out (signature & per-box/per-user license entries), only after download it'd request LIVE to update the licenses/signature for the package - maybe copying their example could help with hashing STFS packages?
Temple Of Horus
Posts: 2
Joined: 06 May 2022 09:33

Re: Xbox 360 digital title preservation

Post by Temple Of Horus »

I have written two programs to help preserve xbox 360 digital content. I know xcp files are encrypted and currently unusable, but I hope one day we will be able to decrypt them.
Xbox 360 Content Preservation program is for downloading files of a specified xbox 360 game using hex title id. I have implemented few tricks to obtain contentId for delisted/unreleased content, which mostly works for older titles.
Xbox 360 Content Preservation Database program is to make backup of marketplace-xb.xboxlive.com site.
Backup of marketplace-xb.xboxlive.com [19.5.2022]: https://archive.org/details/database-of ... 5.2022.-7z
Note: All delisted/unreleased content before backup date doesn't contain contentId. Windows games contentIds are in store 3.

There is also one trick to obtain contentId of delisted/unreleased content using hexOfferId by guessing hexOfferId, but you need to do it manually and it mostly works for older titles. For example for Tomb Raider Underworld (JP):
1. http://marketplace-xb.xboxlive.com/mark ... ec0bbf0001
hexoffers=0x534307ec0bbf0001
2. http://marketplace-xb.xboxlive.com/mark ... ec0bbf0005
hexoffers=0x534307ec0bbf0005
You can see hexoffers are same except last number in the first string. What you can do is to try change this number (for example to number 4):
hexoffers=0x534307ec0bbf0004
You get link: http://marketplace-xb.xboxlive.com/mark ... ec0bbf0004
And this is the trick how you can get contentId for some delisted/unreleased content by guessing hexOfferId.
Another example, same game:
0x534407fa0ccf0001
0x534407fa0ccf0002
0x534407fa0ccf0005
Last edited by Temple Of Horus on 20 May 2022 08:57, edited 2 times in total.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

Temple Of Horus wrote: 09 May 2022 09:05 ...
Nice work. I haven't been following Xbox 360 digital preservation much, but I'll add this to the wiki so people know to look at this in the future.
Post Reply