Satellaview Database Talk

General No-Intro related discussions.
Post Reply
User avatar
LuigiBlood
Posts: 7
Joined: 23 Mar 2019 23:34

Satellaview Database Talk

Post by LuigiBlood »

Okay so... About Satellaview, I'm dealing with a huge problem.
Satellaview Memory Pack dumps are like dumping a PS1 Memory Card with all its saves, and to be frank, I'm not entirely satisfied with this database.

While it is not No-Intro's thing to keep user content, when it comes to rest, most of the time, we're dealing with Memory Pack dumps, some of them having several files in it.
I've been entertaining the idea of several databases for Satellaview where I keep every single RAW Memory Pack dump, then split files from it, and then modified content to render them playable.
I assume the last one is not to No-Intro's interests, as it is purely changes to stuff like checksum and other things.

I have been able to figure out what's the file's state (corrupted, processed, unprocessed, etc).
Files can also have the exact same content but the header can be different (date info), as it is not calculated in the checksum.

I'm interested for ideas, and I'm also interested to give ideas on how Satellaview could be managed on No-Intro.
It's for me especially annoying that I'm basically one of the few people who truly understands the Satellaview.
Screwtape
Posts: 34
Joined: 29 Aug 2017 08:46

Re: Satellaview Database Talk

Post by Screwtape »

As I understand it, the No-Intro model is "dump the actual content of the cartridge ROM", which is a good and noble goal, but is not sufficient for game preservation when important data is not actually stored in a ROM (NES games), or some ROMs cannot be easily dumped (SNES DSP chips), or when pristine originals are not available (Satellaview, also FDS until the lot-check leak).

From a preservationist point of view, there's at least three categories of preservable content here:

- the raw memory pack dumps, including duplicates and corrupted dumps
- cleaned, repaired and reconstructed dumps, as close to the originals as we can figure out how to make them
- a description (or better yet, automated scripts) of how the first collection was turned into the second collection

The idea is that if somebody just wants to *play* a game, or get a feel for the kinds of games released on the Satellaview, they can look at the "reconstructed" set. However, if somebody has a concern with one of reconstructed games (maybe the credits are different to non-Satellaview games in the same series, maybe somebody finds a bug, or it's just weirdly easy or difficult), they can go look at the description for how that game was reconstructed to see if there's anything that would affect the game in question, and ultimately they can go back to the raw memory pack dumps and judge for themselves how that game should be reconstructed.

For example, in my role as self-appointed bsnes historian, I've collected all the historical bsnes source-archives I could find, and converted them into a single Git repository with tags and commit messages and all the rest, making it easier for people to explore and understand how bsnes was developed. However, I didn't just reconstruct a git repository and call it a day, I archived all the original tarballs (even the corrupted ones) and wrote a bunch of scripts to build them into a git repository. Those scripts make a lot of editorial decisions, including ignoring some sources and renaming others, and I made sure each of those decisions was documented and justified, so that future historians can come along later and see *why* I made the choices I did, and make those choices differently.

I don't know if No-Intro is the right place for *any* of those three categories, and it's almost certainly the wrong place for archiving conversion scripts, but I think that's the best way to do it.
User avatar
LuigiBlood
Posts: 7
Joined: 23 Mar 2019 23:34

Re: Satellaview Database Talk

Post by LuigiBlood »

You have put in better words what I want to do with Satellaview, RAW dumps, then cleaned, basically.

It's really overall that I think No-Intro is not best suited for Satellaview, but maybe something could be done.
But I wanted to see if No-Intro is interested doing something in their own way, because as much as I don't mind the idea of preserving RAW dumps of Memory Packs, it is not enough because it's the problem of digital content.

I'm also wondering, because IIRC No-Intro had a history of removing some unique data in the past for Satellaview, is the Satellaview database a good base to start another one? Or should I look up the origin of every dump first (which will take a lot more time)?
omonim2007
Datter
Posts: 437
Joined: 20 Jul 2016 12:20

Re: Satellaview Database Talk

Post by omonim2007 »

Hello everyone and thank you for your interest in this rare gaming system.

I filled the main body of the database as it is about two years ago.
Information from the website https://superfamicom.org/blog/ was taken as a basis.

All newer dumps from the same site were added to the database by other datters. Therefore, you can use this database as your main reliable source. The only caveat is that all statuses (Bad, Hacked, Trusted) are given to dumps in accordance with the way the original sources (ikari_01, kiddocabbusses and the others) did it and the information has not been double-checked.

It would be nice if you could remove unnecessary (garbage and so on) information from bad dumps so that we enter the results into the database as "Trusted modifications". If there is bad data in the dumps that have the "Verified" status and you can modify them to their normal correct form, then this will be just great.

Systems that are quite similar in the method of cataloging dumps have a different form of dats:
1). FDS. Only one good dump is selected out of many others good dumps (all the others are discarded from the date file, but physically present in the database at Dat-o-Matic, they are simply not exported).
2). Satellaview. The database is made according to the principle when each dump described separately.

The first method is simplified, it pursues the goal of having just one working version of the game without many similar digital copies (even though they are all correct). The second method (as it seems to me) is more correct, and we have all known dumps at the output, regardless of their status.

Since historically the Satellaview database develops along the second path, we can continue to work with it in the same way. If everything suits you and you still want to contribute to correcting the existing sources in the database, then I will be glad to help you at the final stage (I will make all the necessary edits in the database).

But in any case, we need to understand that we will still represent and store information about the game/magazine in its original form (if the game takes up less space than fits on the storage medium, we will still represent it as part of the storage medium without cutting off unused data ).
User avatar
LuigiBlood
Posts: 7
Joined: 23 Mar 2019 23:34

Re: Satellaview Database Talk

Post by LuigiBlood »

omonim2007 wrote: 21 Dec 2020 09:35 All newer dumps from the same site were added to the database by other datters. Therefore, you can use this database as your main reliable source. The only caveat is that all statuses (Bad, Hacked, Trusted) are given to dumps in accordance with the way the original sources (ikari_01, kiddocabbusses and the others) did it and the information has not been double-checked.
That's one thing I intend to do for my own database work, that's to recheck every content. Some things weren't understood back then.
omonim2007 wrote: 21 Dec 2020 09:35 It would be nice if you could remove unnecessary (garbage and so on) information from bad dumps so that we enter the results into the database as "Trusted modifications". If there is bad data in the dumps that have the "Verified" status and you can modify them to their normal correct form, then this will be just great.
I preferably want to leave the dumps as is, while extracting each content to list in a seperate (but linked if possible) database + making a playable modification of my own if necessary.
omonim2007 wrote: 21 Dec 2020 09:35 But in any case, we need to understand that we will still represent and store information about the game/magazine in its original form (if the game takes up less space than fits on the storage medium, we will still represent it as part of the storage medium without cutting off unused data ).
In this case that means the dumps shouldn't be modified in any way and should be a good base for me. I am fully fine with a collaboration of sorts if I give proper information about each dump.
I had actually started a spreadsheet of my own with information but I was not sure whether No-Intro could be a good base afterwards. Glad that it is and shouldn't have to search for everything.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Satellaview Database Talk

Post by Hiccup »

It seems like some (maybe lots) of dumps that were posted on Satellablog are incorrectly credited to the blogposter (MatthewCallis or kiddo) rather than the stated dumper. Also, sometimes its not explicitly stated on the blogpost who the dumper is (maybe Callis and/or kiddo should be contacted). LuigiBlood says that most of the Satellaview dumps are by MatthewCallis, but I think it'd be good to find out for certain.
Post Reply