[GBC] Inaccurate database updates?

[BigFred]

It has been some time since I last checked the recent dats with my set but I thought it might be a good time now since there has been a lot of activity lately. Thanks to anyone involved in all the updates. I know this is very hard and sometimes stupid work.

However I noticed some issues we need to discuss:

Monsters, Inc. (Europe) (Rev A) --> Monsters, Inc. (USA, Europe) (Rev 1)

There is no proof this revision was released in the US and even the internal spreadsheet only lists the UK Rev 1. We have searched for an US shaped cart but even the USA-1 cart version still contained a rev 0. When you check the bootscreen the US version shows 2001, all European editions 2002. Possibly the UK version was just updated to match the European release date. Not all versions are always released in both regions and we shouldn't make such assumptions without proper verification.

Mortal Kombat 4 (Germany)... --> Mortal Kombat 4 (Europe) (No Fatality Version)...

The verified entry clearly shows a "D" serial so this should definitely be Germany. It's also very common that such censored versions are produced for Germany (I know the comments in the game header but this is not relevant for the region assignment). What also bugs me: In DoM there is a dump with a "P" serial DMG-A4KP-EUR that supposedly matches this ROM. This seems to be impossible, P and D never match - especially since the other "uncensored" version is confirmed from the same serial.

Which brings me to the most serious issue I found: The "P" dump entry for the No Fatality version is supposed to be produced by Paradox. In general a lot of scene releases have been added to DoM as "Trusted" dumps and even serials are listed. We cannot seriously define scene groups as trustworthy sources. We only trust dumpers we know. I cannot even think of anything I would trust less than these. And the serials are not real cart serials, but simply guesswork. We only list proven cart serials, not ones reproduced from ROM headers or similar.

Let's look at an example:

Hercules - The Legendary Journeys (Europe) (En,Fr,De,Es,It,Nl) --> Hercules - The Legendary Journeys (USA, Europe) (En,Fr,De,Es,It,Nl)

The game has been assigned to USA and Europe now and DoM lists CGB-BHCE-USA. In fact this game was never released in the US. The game has been searched for by collectors but was never found. It's just that the European version uses BHCE in it's header so you might believe it's an US dump. This is why it's so important to rely on cart serials. If a cart is found indeed we can rename it everytime.

We should remove all serials from scene releases and list them as Third Party only - unless you really know the guy who did the dump.

Shanghai Pocket (USA) --> Shanghai Pocket (USA, Europe)

The 1.0 was not found on an European cart yet, was it? Was the game released at the same time in both regions?

VIP (USA, Europe) ---> VIP (Europe)

Up for debate. One batch of the US release (US shaped box) officialy comes with an European cart inside (proven from sealed copies). Another contains a real US cart. Any opinions how we should handle this?

Shaun Palmer's Pro Snowboarder (USA, Australia) --> Shaun Palmer's Pro Snowboarder (USA)

Australien dump is bad indeed. However the header checksums match the US dump so it's actually "confirmed" they match. What should we do?

Magical Tetris Challenge (Europe) (En,Fr,De,Es,It,Nl,Sv) (Rev 1) (ver.07.12.99)
Magical Tetris Challenge (Europe) (En,Fr,De,Es,It,Nl,Sv) (ver.12.10.99)

What's the use of these dates? They are revision 0 and 1 so why make it more complicated?

[Tauwasser]

Most of these are acts of one individual, omonim2007.
I believe he acted in good faith, see here.

Still, there are discrepancies in scene serials, as you showed.

The Rev A --> Rev 1 etc. switch was triggered by me. I know I was the one that caused a reversal last time it happened, but now I see this as correct, too. Revisions were a mess for way too long in GB(C) set.

The Mortal Kombat issue seems peculiar, because the Release Lister V2 Page does indeed state Language as English, but CRC32 is definitely for the German ROM.

For regions where no Revision was found, I would want to not include that region. So if Rev 0 is USA, Europe, but Rev 1 was only found in Europe, don't add USA to Region field. Wait until an actual -USA cartridge was found to contain Rev 1.

Australian releases are a pain in the butt, because traditionally people haven't bothered to add Australia as a Region, because of the high overlap with US releases. I'd personally -- especially in light of the recent master list leak -- want to wait for a -AUS cart to be dumped for each affected release.

VIP (USA, Europe) --> VIP (Europe) case is similar to the Berenstain Bears.
There was no European ROM release of Berenstain Bears in the beginning. Instead, they relabeled the cartridges as CGB-BESP-EUR on top of the existing CGB-BESE-USA sticker! Only later was a European ROM released (P serial) with the identical contents to the US (E serial) release. But the sticker changed from CGB-BESP-EUR to CGB-BESP-EUR-1 to reflect this. Still, it's listed as USA, Europe.

[hydr0x]

It, unfortunately, seems as if it isn't deemed necessary anymore to explain and communicate the basic No-Intro rules (e.g. that this is a cart-based database) to new datters. I don't think giving full privileges without a full "No-Intro 101" by an established datter is a good idea.

I don't remember exactly, but that Shaun Palmer dump was probably made by me?! I own the Australian exclusive "PAL" release, serial is CGB-B3SP-AUS

[omonim2007]

Offensive remarks here and in other topics without comment.

When you make several hundred (or even thousands) of edits in the database, some small things can get lost. I will try to answer what I was guided by when making changes for these games listed above in first post.

Monsters, Inc. -> Here we need to roll back, everything is right about the regions.

Mortal Kombat 4 -> Yes, a very difficult situation. Even if you do not take into account the release of the Paradox group. My opinion is that you cannot just leave the region to Germany, since the game could be sold (or not - this is no longer known) in the Netherlands. In any case, it is in English. I deleted the serial number, it may be false. Make your own decisions.

Hercules - The Legendary Journeys -> Here we need to roll back, I have now re-checked the situation with this game. Also deleted the serial number so that there will be no more doubts about regional affiliation. But I definitely found the information that the game was imported in small batches to the USA (all that cart - European versions), the situation is almost the similar to Berenstain Bears., only as a result the independent release in the USA was never implemented. Probably, the hackers in the Capital group were able to get some kind of overview version of the game (that is, European), otherwise they would hardly have identified it as the United States. Although, now you can only guess about the logic that they had in those early years...

Shanghai Pocket -> As above, let's roll back.

VIP -> Yes, I got acquainted with the history of these publications. I think we need to leave it as it is now, because there is a full-fledged version of the game for the US region. Well, or do everything in a similar way with the games already listed above.

Shaun Palmer's Pro Snowboarder -> Here I am guided by a comment: "Dump was bad and actually we have no proper verification of the data. The crc in the database is taken from the original US dump". How can we trust a source who is not sure whether this is the correct dump or not? I removed the Australia region to the next redump, which will confirm that the checksum is the same. The serial number xxx-xxx-AUS is also not specified. I described the situation on the part of the person who saw the post and comments in the database and made the appropriate conclusions. I think you would do the same in my place.

Magical Tetris Challenge -> I found data about the version of the game and the exact time of the build inside the game data and decided to share it (this information is rarely found). Yes, I agree, let's remove it from the description in the database.

The topic with the correctness of the old dumps has already risen above (there is a link above). My opinion is: if a modern dump confirms an old scene dump, then the scene dump must have the "Trusted" status. If the serial numbers also match, then we can leave this information in database.

Information from my own experience after entering information about several hundred/thousand old scene (I mean ONLY Game Boy Color system now) dumps: about 99% of the information can be trusted. This was confirmed by our subsequent redumps through all past years. In any case, this is only a proposal for universal consideration. If the higher council considers that it is not, then I am ready to accept it.

[BigFred]

Hello omonim2007,

thank you for your comments and in general for all the work you have done. I hope there wasn't anything in my post offending you? If so I apologize.

I completely agree it can be difficult for new datters and dumpers to understand what they are supposed to do. One big problem is the fact that several well-known and experienced datters just disappeared and left a pile of work. I myself have been absent for a long time now with just a few comments here and there in the forums but no real activity. Just too many things killed motivation - it felt like all responsibility for all systems was focusing on just 2-3 people and this was the point where the project didn't feel like "fun" anymore but just hard work. So it's mainly the fault of the former team if there are a few misconceptions.

I guess we need a proper SOP like guideline of the rules - for dumping and datting - available as a PDF or something similar you can download anytime.

Just a few comments/hints:

About Mortal Kombat 4 and the region tag:

There is no strict rule about the region - in fact this attribute is bit arbitrary. A real proper and unique way to describe a dump would be to list the serials only or use NTSC, NTSC-J, PAL or maybe "PAL-region" etc. However this would be "unwieldy" for daily use. The naming scheme is also meant as an easy way to manage your library and to give you an idea where to look for the game in the wild. So we decided to use a region flag that aims for the most important markets which are Japan, USA and Europe. Those are the most important regions games are usually produced for. Most other regions just "benefit" from these markets and receive relabeled copies of such versions. A few rules to remember:

Regions are not meant to accurately describe where a game was released. USA versions are usually also available in Canada, Mexico and several other Aian and American states. PAL is also released in Australia, Hong Kong and many others. Listing all these would be madness. The system is more meant to give the user a rough idea of the area the game was originally produced for. We usually use the serials as a reference:

The last letter or the middle part in Nintendo's serial system defines the region:

"P"(PAL) and X, Y etc. usually are codes used for releases that aim at several different regions in Europe. They are normally variants of the same game with different multilanguage options or similar. So these use "Europe" in most cases.

"E" is the standard for USA. In most cases the flag is just that, but you will notice a few USA, Australia versions in the dat. This normally happens when Australia got a unique serial that matches an US release. Australia in fact got it's own identifier "U" that is however just rarely used. They normally get "P" version from Europe. However "U" versions must be dumped and documented since they could be different from USA or European versions.

This is where it's becoming a bit confusing. On home systems, P, E and U versions normally are different dumps. However on handheld systems it's like a gambling game. They could be different or they might match. In some cases E and P are different ROMs, in other cases they match. Same for U: Most appear to match the E versions, but there are exceptions. You need to dump and find out. This is why we can't ignore those but we can ignore all regions that receive E or P serials because for a dump you can ignore the part following the region letter.

D, F, S, I etc. are country specific releases that are automatically assigned to "Germany", "France", "Spain" or "Italy". It doesn't matter if a "D" version is released anywhere else. Probably also Switzerland or Austria, but this is beyond the point - those are releases that are specifically modified for the German/French etc. market. These carts will always be different dumps than E, P or U.

In addition it doesn't matter if the game has been translated. What counts is that the code has been altered in some way for the market. This also includes censorship or similar changes. An example is the game Gift that was renamed to Gifty for the German market because of the association Gift = poison.

The system can never be foolproof however. There are always exceptions and some choices are just a matter of taste.

Regarding Shaun Palmer:

The game is a bit of a special case for instance. It's a "P" release, but it's not available in Europe so it wouldn't be smart to call it Europe. Since it's not an "E" version it needs to be dumped and documented however. Hydr0x tried to dump it but the data had a few faulty bytes. Still you could compare it to the US version and it's almost a match. The header is identical and since this includes the serial and global checksums which are unique it seems safe to say this is the same as US. Still I can see why you wouldn't want to assign the flag without a real proper dump.

And finally about the scene dumps:

Serials often can be reproduced from the ROM header. The middle part is stored there in many cases. Advanscene usually did this for the database - they just added AGB- and -XXX at the end and that's it. Or you could just do a google search for the cart and use that one. We have no idea where those serials come from. Hardly any group ever included cart pictures in their releases so anything is just an assumption.

"Trusted" dump means that you consider a dump to be proper even if it's the first dump and there is no verification from a second source. There is nothing like "probably" good. Usually this includes proper documentation of cart and pcb by a dumper that is known by us for years. We still need a verification but have no reason to question the integrity of the data for now.
Of course most scene releases are correct, but in general they could be anything: Many groups had access to Beta versions, added hacks or trainers or even misdumped a cart. We have seen many of such cases already. Dumping in those days wasn't as comfortable as today. Besides they lack proper documentation. A verification doesn't make those "trusted" in retrospective since you only know it's good because you redumped it. It's simply a verification of a Third Party source. The only scene releases I trust are those I did myself or someone I know I can rely on.

[BigFred]

Another one I missed:

DSF Carl Lewis Athletics (Europe) (En,Fr,De,Es,It,Nl) was added to the DB implying this might be the release with the "DSF" box in Germany. I believe this is not true for a few reasons:

The dat entry refers to the same serial "BCIP" as the regular Carl Lewis dump. The -NOE part doesn't matter.

After a quick check on ebay the cartridge that is included in the "DSF" box appears to be identical to the normal release. I didn't even find a P-NOE version, it appears to be just P-EUR.

It looks more like a Beta to me than an actual retail version with the strange internal name and no serial in the header at all. This is exactly what I mean. It's a scene dump again and no one knows what it is. As I already mentioned, it's not unusual scene groups had access to early builds they released as retail. It happened a lot of times. For now I'd relabel it Carl Lewis Athletics (Europe) (En,Fr,De,Es,It,Nl) (Beta).

[xuom2]

Thanks to Omonim2007 for the endless hours of work on the databases, adding data to our generic "initial entry" sources (do like me: ignore trolls and lurkers lol).
Thanks to Bigfred for finding out the wrong decisions. Edits are logged and maintainers can approve them or not. Of course I agree with you: we are missing serious and detailed conventions and rules. I am missing the days where Kazumi was our lighthouse
I agree this hobby is not very fun, lol: during this month I will upload gbx, snes, n64 and genesis "scene" data to the appropriate structures, moving Omonim's data from "source" to "release" where present. I have to unpack the releases, scan them and do some "csv" work to match them with actual data.

[Hiccup]

I have always been unsure about the regions, and I guess the "base it around the three main regions" makes sense, but we really need fully thought-out rules for this.

Also, is it actually possible to tell if a game is "released" in a region and not just imported, if the packaging is the same? (Unless the barcode or something will differ)

Thirdly, I think the proposed datting guidelines should go on the wiki. They could be locked to only be editable by admins.

[hydr0x]

Also, is it actually possible to tell if a game is "released" in a region and not just imported, if the packaging is the same? (Unless the barcode or something will differ)

It is not, at least not if you mean countries by "regions". It's usually possible to tell when it comes to the big "regions" like US (=North America actually) or PAL (= Europe + Australia). That's exactly why No-Intro has never tried to accurately tag the countries a release was released in, the exception being games with serials which already show the correct country (but still, even in those cases, we tag a game as "Germany" when it's in fact Germany + Austria + Switzerland), or games which have clearly been translated/altered for a specific market. Another exception are games where it's known that the PAL release was only available in Australia, although those cases are hit and miss, we have plenty of games tagged as Europe which are in fact Australia-only releases. We also tag games as US despite them only being released in Mexico, so that rule is a bit fluffy.

Hydr0x tried to dump it but the data had a few faulty bytes

Just to be clear: You dumped this from my cart back when we met at the trade fair. You never told me you knew/suspected my dump to be bad, otherwise I could have redumped it once I acquired my own Retrode. Luckily I haven't sold this cart yet so I can redump it once I stumble upon the cart again

[BigFred]

Just to be clear: You dumped this from my cart back when we met at the trade fair. You never told me you knew/suspected my dump to be bad, otherwise I could have redumped it once I acquired my own Retrode. Luckily I haven't sold this cart yet so I can redump it once I stumble upon the cart again

Yeah, it seems I tried dumping it at the trade fair. I found an old post:

viewtopic.php?f=7&t=2219&start=30

and I comment

without any option to clean the contacts we only got bad dumps

Seems we checked the dumps but couldn't get this one right after several attempts. So you should search for contact cleaner now and try again

[buffaloj0e]

I just bought a copy of this game for comparison.

[Tauwasser]

Shanghai Pocket (USA) --> Shanghai Pocket (USA, Europe)

The 1.0 was not found on an European cart yet, was it? Was the game released at the same time in both regions?

I found and dumped one with a EUU serial, see DoM.
Not sure if EUU is Europe proper or what the U at the end means exactly.

[hydr0x]

EUU is a regular European serial used alongside EUR serials if there were, usually, two different boxes for Europe (usually one for northern Europe with English, German, etc. and one for southern Europe with Italian, Spanish, etc.)

[BigFred]

Went through GBC again and also finished GB Mono. Two more questions:

Game Boy Wars 2 (Japan) (Alt 1) (SGB Enhanced) (GB Compatible)
Game Boy Wars 2 (Japan) (SGB Enhanced) (GB Compatible)

Seems to be the same 4 MiBit game data supposedly burned to 8 MiBit ROM chips with different padding bytes - one being filled with 00, one with FF. It's possible this really happened, but the (Alt1) is a scene release. As long as no one else verifies the 00 padded dump I wouldn't want to keep this in the dat. It's arguable anyway if in such cases we should use the whole chip "data" or only the game data itself.

Then there has been a name change in the mono set:

The Chessmaster dumps all had the serial in the filename removed. Afair we added the info to avoid confusion since there appear to be different Chessmaster games:

viewtopic.php?f=6&t=2798

I found and dumped one with a EUU serial, see DoM.
Not sure if EUU is Europe proper or what the U at the end means exactly.

That's pretty cool. I left it as "USA, Europe".

[Tauwasser]

The Chessmaster dumps all had the serial in the filename removed. Afair we added the info to avoid confusion since there appear to be different Chessmaster games:

viewtopic.php?f=6&t=2798

I did that, because while matching the whole DoM to the master sheet (Github), I noticed that the serials were wrong.

The Chessmaster (Japan) published by Altron (DMG-EMA) [old tag DMG-EM] [Rom DMG-EMA-0]
The Chessmaster (Europe) published by Nintendo (DMG-EM-NOE, -FAH etc.) [old tag DMG-EM] [Rom DMG-EMX-0]
The Chessmaster (USA) published by Hi-Tech (DMG-EM-USA) [old tag DMG-N5] [Rom DMG-EME-0]
The Chessmaster (USA) (Rev 1) published by Mindscape (DMG-EM-USA-1) [old tag DMG-N5] [Rom DMG-EME-1]

The New Chessmaster (Japan) published by Altron (DMG-ACHJ-JPN) [Rom DMG-ACHJ-0)
The New Chessmaster (USA) published by Hi-Tech (DMG-N5-USA, -FAH etc.) [Rom DMG-N5E-0]









It seems the whole confusion stemmed from the USA version and then propagated from there.