Xbox 360 digital title preservation

General No-Intro related discussions.
Post Reply
pelago

Xbox 360 digital title preservation

Post by pelago »

Is Xbox 360 digital (e.g. XBLA, Games On Demand, DLC, updates) preservation within scope for no-intro?
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

Yes and there's already a dat for it. But I don't think it has been updated for a while.
pelago

Re: Xbox 360 digital title preservation

Post by pelago »

Thanks, I found the DAT, that's great.

I don't see a dumping guide on the wiki, unless I've overlooked one. It'd be good if "we" could dump/verify ourselves, rather than rely on the scene releases.

I believe the console ID and user ID are embedded in the downloads, so everyone who downloads a title will get a slightly different file.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

Indeed it would be good to get some dumps made by trusted users. It would be good to test how the "same" file on different consoles differs. If there are just some ID fields that differ, they could be 00'd or FF'd out and the differing data can optionally be added to the comment of the dump. That's how its done in most other dats.

I don't really know much about the Xbox 360, but I think it may just be matter of copying the relevant file(s) from the filesystem using homebrew tools.
Resistiv
Posts: 8
Joined: 21 Jan 2020 22:26

Re: Xbox 360 digital title preservation

Post by Resistiv »

Yes, it looks like you can indeed just copy over files from a hard drive using something like FATXplorer, however the files are often signed differently in their headers, which produces differing hashes. Here's looking at one dump, comparing my dump (left) and the scene dump in the dat (right) of N+ (World) (XBLA). Everything after the highlighted byte is matching in the attached image. Of course, more samples will be needed for decisive action, but it seems that the first 568 bytes (0x238) can be skipped when hashing.
Image
You do not have the required permissions to view the files attached to this post.
User avatar
RetroGamer
Dumper
Posts: 56
Joined: 08 Oct 2014 22:53

Re: Xbox 360 digital title preservation

Post by RetroGamer »

Back in 2012 and 2013 i dowloaded from Marketplace 100's of demos and dumped them to add the verifications to DoM. I Also discovered some new dumps...

It was many years ago but, if i recall well, the X-Box 360 scene releases (and the 360 no-intro dat) were a mess because some groups released betas (PNET) as final releases, the early scene releases were intact and in later releases they zeroed the "Console ID" bytes in the beggining of the file)... So that means that DB is inconsitent and it should have a standard (i talked about this with xuom2 back in the day...)

I never understood the "Console ID" concept but i think everyone who downloaded from Marketplace get the same file.

It's a shame that there weren't more dedicated dumpers of this digital games because it was very easy to dump and now, maybe, they are all gone...
pelago

Re: Xbox 360 digital title preservation

Post by pelago »

Well, the Indie games are no longer there, but the Arcade and Games on Demand stuff still is. Except for a few things that have been delisted. So it's not too late yet for the majority of stuff to be properly preserved.
User avatar
layzee
Posts: 23
Joined: 25 May 2008 12:14

Re: Xbox 360 digital title preservation

Post by layzee »

I notice that most of the Xbox 360 DLC is marked as (World). For example, Lost Odyssey has three DLC:

Lost Odyssey - Double Bonus Pack (World) (Addon)
Lost Odyssey - Dungeon Pack - Seeker of the Deep! (World) (Addon)
Lost Odyssey - Triple Bonus Pack (World) (Addon)

"World" implies that the DLC is compatible with both the English versions of Lost Odyssey (PAL and NTSC-US/Canada) and Asian versions (Japan and Asia). Is that the case?

A quick search elsewhere suggests that Xbox 360 DLC can often be region-free but not always. In some cases, even game save files can be region-free which I think is unusual.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

Someone posted a dump of "Artoon" on archive.org (linked on a reddit thread):
I'm going to contact the user to see if they have more info/dumps.

So I datted it:
https://datomatic.no-intro.org/index.ph ... =57&n=5515
The file path wasn't provided so I couldn't add that (if what appears in all/most of the file fields in the dat is file path, that is)

This is what rpcli (rom-properties command line) has to say on the file:

Code: Select all

== Reading file 'C08C09E0CA9098FFA26B173CAFBE8A3AC6BD3D1F58'...
-- Microsoft Xbox 360 Application Package detected
Name:         'Artoon'
Title:        'Indie Games'
Package Type: 'Xbox Live Package'
Content Type: 'Marketplace Content'
Media ID:     '00000000'
Title ID:     '584E07D2 (XN-2002)'
Version:      '0.0.0.0'
Base Version: '0.0.0.0'
-- Internal icon is present (use -x0 to extract)
   Format : ARGB32
   Size   : 64 x 64
I'm not sure if this is added correctly - I haven't added to this dat before.
Might be a good idea if RetroGamer or someone writes down how the existing dat has "worked" (naming conventions etc).
rwebster
Posts: 25
Joined: 01 Mar 2021 18:15

Re: Xbox 360 digital title preservation

Post by rwebster »

I think I have a copy of Artoon on my 360 if needed.

Edit

In fact I have a handful of Indie games installed if they need to be dumped and preserved.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

rwebster wrote: 25 Apr 2021 10:08 I think I have a copy of Artoon on my 360 if needed.

Edit

In fact I have a handful of Indie games installed if they need to be dumped and preserved.
It would be good if you could dump those, yeah.
rwebster
Posts: 25
Joined: 01 Mar 2021 18:15

Re: Xbox 360 digital title preservation

Post by rwebster »

Hiccup wrote: 25 Apr 2021 15:23
rwebster wrote: 25 Apr 2021 10:08 I think I have a copy of Artoon on my 360 if needed.

Edit

In fact I have a handful of Indie games installed if they need to be dumped and preserved.
It would be good if you could dump those, yeah.
I'm just waiting on an RGB Scart cable to use with my Elite on my CRT, where all the indie games I own are safely installed. Should be here next week.

Recently picked up a 360 Slim which is hooked up now with HDMI. Went through my download history when setting it up and not all the indie games could be redownloaded, even from purchase history. So it seems we might be relying on people still having some games installed in order to get them dumped and archived.
GateGuy
Posts: 1
Joined: 08 Oct 2020 20:13

Re: Xbox 360 digital title preservation

Post by GateGuy »

For what it's worth, I just compared my dump of UNO Rush to the scene dump and found that every byte starting from 0x258 was the same; however, 0x32C was the first non-zero byte where everything started to line up. I assume the area before one of these two bytes is a unique header for whatever account and/or console dumped the game (with the exception of the first four bytes, which spell "LIVE" and must be present or the game won't boot). I zeroed out every byte from 0x004 through 0x257 (again, 0x258 through 0x32B were all zero for both dumps, so they may or may not be significant) and Xenia was able to boot it.

More testing is required to see if this number of bytes is constant between games, but it looks like the best way to preserve XBLA games without relying on scene dumps is to zero out bytes 0x004 through... wherever the cutoff is.

EDIT: Or simply ignore that first group of bytes when hashing; that would probably be better.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Xbox 360 digital title preservation

Post by Hiccup »

I think a good start would be for someone with a lot of disk space to download a fullset of the current dat, then run a script that will hash the files without the unique areas, and also make copies of the unique areas. Then that data can be uploaded to DoM. That way it'll be possible to compare with scene releases in the future without having to download them.

I can make the script, but I don't have the disk space for an XB360 digital fullset.
Qubits
Posts: 3
Joined: 06 Aug 2021 10:28

Re: Xbox 360 digital title preservation

Post by Qubits »

I studied the file structure of x360 digital titles (STFS files) quite thoroughly about a year ago.
Documentation available on the net is really shitty. I learned a lot by studying code others wrote.

First (and this might already be known), most LIVE games floating around the net and also most, scene releases have the bytes 0x6 to 0xB zeroed.
The reason for this is a tool called "YarisSwap" that was and maybe still is a popular tool to patch downloaded trials to run as the full game.
The author of this tool falsely assumed that in this area console specific data was stored. However, this is not the case.
The bytes 0x4 to 0x103 contain a digital signature, signed by Microsoft. With this signature you can verify that the data if the STFS file is unaltered or, to put it in another way, cryptographically prove it is a "good dump". This is largely different to checksums found in other roms. Checksums can be modified after altering data. You can't do that with a digital signature.
A good (and widely unknown tool) to test the integrity of a stfs file is stfschk

To my knowledge all XBLA and XBLIG games have/had Trial versions. These are identical to the full versions. There's just a single bit in the header that needs to be flipped to make it a full version. Nowadays with tools like dashlaunch for the x360 this happens upon load. So there no need for manual patching at all.
The trial versions of course have a good signature and are not console bound. So from a preservationist's standpoint these are without doubt the files that should be archived.

Unfortunately there's no trials for any DLC, so it's much more difficult there. If you buy a title or DLC these stfs files will contain console specific information when you make a purchase. The file is bound to the console and only works if the signature and console info match. So it's understandable you might want to erase this data. Unfortunately, this will corrupt the signature as well.
You can still check the file hashes with stfschk, but you can't be sure that data was not altered if the signature is invalid.

For datting the "trials" with good signatures, I can assist. Admins can PM me if wanted.
Post Reply