SNES P/C feature

[kazumi213]

I don't approve most of P/C changes. Specially:

- using earliest release as prefered pick, instead of corresponding latest version available
- many BS games declared as clones, like "Kirby no Omochabako" ones, or generally the sequentially released ones. This is "custom merging".

Both alone completely break 1G1R concept for SNES

Region re-labeling should be discussed. I agree that mine is maybe too arbitrary, but I'd prefer to keep 3 letters for regions, 2 letters for languages

The correct implementation of alternative release titles is ("Tintin in Tibet" example, size and hashes omitted)

<game name="Tintin in Tibet (Europe) (En,Fr,De,Nl)">
<description>Tintin in Tibet (Europe) (En,Fr,De,Nl)</description>
<release name="Tintin in Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="En" />
<release name="Tintin au Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="Fr" />
<release name="Tim in Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="De" />
<release name="Kuifje in Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="Nl" />
<rom name="Tintin in Tibet (Europe) (En,Fr,De,Nl).sfc" size="" crc="" md5="" sha1="" status="verified" />
</game>

Otherwise you declare more regions than required (in example imagine that there are no (Netherlands) games in the DAT, like there are no (Poland), (Hungary) etc for NDS), while the "language" DAT feature remains unused. Also this game would we listed on a P/C list as:

Tintin in Tibet (Europe) (En,Fr,De,Nl)|(EU PARENT)

and not

Tintin in Tibet (Europe) (En,Fr,De,Nl)|(EU PARENT) (FR PARENT) (DE PARENT) (NL PARENT)

Note that "release name" use full title because this game has no clones.

[NGEfreak]

- using earliest release as prefered pick, instead of corresponding latest version available

I don't see why this even matters. Ultimately the merged set name is named after one of the release titles. I could declare any random ROM image to be the parent. The end effect would be the same. To be consistent I always chose the earliest non-beta release to be the parent. (Note: Of course there are also other priorities like US > EU, etc)

- many BS games declared as clones, like "Kirby no Omochabako" ones, or generally the sequentially released ones. This is "custom merging".

Yeah, this needs to be revised. Mind you if we split them all we will end up with over 400 game entries of Game Tora magazines. I guess it's better to merge some of them.

Both alone completely break 1G1R concept for SNES

What is 1G1R?

The correct implementation of alternative release titles is ("Tintin in Tibet" example, size and hashes omitted)

<game name="Tintin in Tibet (Europe) (En,Fr,De,Nl)">
<description>Tintin in Tibet (Europe) (En,Fr,De,Nl)</description>
<release name="Tintin in Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="En" />
<release name="Tintin au Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="Fr" />
<release name="Tim in Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="De" />
<release name="Kuifje in Tibet (Europe) (En,Fr,De,Nl)" region="EU" language="Nl" />
<rom name="Tintin in Tibet (Europe) (En,Fr,De,Nl).sfc" size="" crc="" md5="" sha1="" status="verified" />
</game>

Otherwise you declare more regions than required (in example imagine that there are no (Netherlands) games in the DAT, like there are no (Poland), (Hungary) etc for NDS), while the "language" DAT feature remains unused.

At least for SNES it's not necessary to use the language field. You only need to use it if one release has multiple titles. For Tintin in Tibet there exists at least 4 different boxarts (with 4 different titles). This means there are at least 4 different entries added to the releases table of the backend.

Also the region information from the release list is not the same as the region tag from the filename. The filename tag is a merged combination of the release regions. In fact the backend doesn't store the filename region tag at all instead it's automatically created during the DAT export.

A long term goal is to remove any EU entries from the release list and replace them with a detailed list of the real regions.

Also this game would we listed on a P/C list as:

Tintin in Tibet (Europe) (En,Fr,De,Nl)|(EU PARENT)

and not

Tintin in Tibet (Europe) (En,Fr,De,Nl)|(EU PARENT) (FR PARENT) (DE PARENT) (NL PARENT)

I don't understand why you need to declare so many parents. All child entries can only have one parent anyway.

Note that "release name" use full title because this game has no clones.

I prefer to name all merged sets with only the title. Imho, this looks more consistent.

I think it's better to make such behavior an option in the software instead of hard coding it into the DAT. I don't think it would be hard to add an switchable option to the software like if file count == 1 use filename as set name.

[kazumi213]

I don't see why this even matters. Ultimately the merged set name is named after one of the release titles. I could declare any random ROM image to be the parent. The end effect would be the same. To be consistent I always chose the earliest non-beta release to be the parent. (Note: Of course there are also other priorities like US > EU, etc)

Yeah, this needs to be revised. Mind you if we split them all we will end up with over 400 game entries of Game Tora magazines. I guess it's better to merge some of them.

What is 1G1R?

Well your last question explains it all. Some users are not interested in collecting ALL region dupes/devstatus of a given game (all members in each family), but just in having one member, 1 ROM(set) from each game, hence 1 game, 1 ROM (1G1R). This prefered member is of course the "best" that the DAT can offer, based on region/language preferences and latest possible devstatus. The 1G1R list generation algorithm in CMPro allows to filter out all undesired members, thus offering Scanning/Rebuilding operation limited only to desired ROM sets.

With your new parent selection approach, now all users using 1G1R algorithm in CMPro are forced to get earliest devstatus family members (even at least a couple of betas where final releases exist).

Also, by arbitrarily collapsing some BS titles, you are effectively removing unique games from the 1G1R lists and its user's collections.

Other P/C changes are completely acceptable, like some previous clones becoming parents. I humbly admit that I don't know SNES very well. I any case, adding entries to the resulting 1G1R lists is less critical than removing from it. Think that you are assuming part of the responsability of deciding what to collect (from the 1G1R user point of view)

Please give it the 1G1R list a try to see what I mean. It won't force you to modify anything if you don't want.

- Load the DAT you posted on CMPro 3126a (latest "b" build requires a fix on P/C DAT info for 1G1R to properly work, already applied to DoM generated DATs).
- Go to "Settings -> Regions". All DAT declared regions appear. Click "Select All".
- Now rearrange the list as you prefer by using the tiny "up/down" arrows. Top one has the highest priority.
- Set to "1G1R Mode" below.

Now if you scan, only a fraction of games will be analyzed and displayed, those matching your region preferences. They "should" be the best possible picks too, but this depends on how the datter choses to assign the "release" info.

At least for SNES it's not necessary to use the language field. You only need to use it if one release has multiple titles. For Tintin in Tibet there exists at least 4 different boxarts (with 4 different titles). This means there are at least 4 different entries added to the releases table of the backend.

That's exactly the scenario and goal for using "language" declarations, because there is only 1 possible hash for all of them, not 4. You only declare additional/more accurate regions when you want to specify which hash goes to which region.

Also the region information from the release list is not the same as the region tag from the filename. The filename tag is a merged combination of the release regions. In fact the backend doesn't store the filename region tag at all instead it's automatically created during the DAT export.

The language tag is not the same as the region information in release list either. Pokemon Platinum eng only is sold in Netherlands. Only one region needs to be declared as default for all users not getting localizations for this game. We probably won't ever be able to accurately list which hash goes to every possible region. So you only declare regions that you are able to confirm somehow according to system, and the minimum required ones.

Please note that the whole release feature was not designed to include "database" info on DAT. We don't include PCB revision or serial in example. Goals when designing the release DAT specification were:

- Allow 1G1R concept. This required a way to assign prefered hash to each region. I personally decided to limit region declaration for a given system in my dats to acknowledged regions for the system. But of course it is possible to declare as many regions as you want to be completely accurate when assigning hashes (you can clutter the "Settings->Region" list as much as you want), just not in that specific Tintin example (unneeded).

- Allow custom *archive* names when *merging* based on region (where required to assign a hash) and/or language preference. No alternative ROM names, more deep changes were required to DAT format and ROM manager behaviour, and only when merging because family archives can always be argued to have an arbitrary name. In unmerged mode, NI convention *always* sets naming. Pretending to maintaining a naming database in 10+ languages is unrealistic

A long term goal is to remove any EU entries from the release list and replace them with a detailed list of the real regions.

Very well, but until them show must go on and current system works.

I don't understand why you need to declare so many parents. All child entries can only have one parent anyway.

That's how current DoM generation routines would require Tintin to be store in database in order to create your release structure. It is true that mine cannot be currently generated either. A way to declare region/languages-based titles should be designed and implemented on DoM.

I prefer to name all merged sets with only the title. Imho, this looks more consistent.

It may "look" better to you (and other users, I know) but there is no technical reason to remove proper tags from the filename of an archive containing only 1 member. Those tags are to be applied according to NI convention in unmerged mode, and they still have to be applied when in merged mode since merging results are designed by NI.

I think it's better to make such behavior an option in the software instead of hard coding it into the DAT. I don't think it would be hard to add an switchable option to the software like if file count == 1 use filename as set name.

Sure but you need a way to declare in DAT where the "tag" part starts and tweak the rom manager to properly parse the new DAT format.

[alexian]

I don't understand why you need to declare so many parents. All child entries can only have one parent anyway.

Hmm. It seems like there is a base misunderstanding. And perhaps this is because the verbiage on the syntax is not the clearest.

In this situation:

Tintin in Tibet (Europe) (En,Fr,De,Nl)|(EU PARENT) (FR PARENT) (DE PARENT) (NL PARENT)

The "PARENT" tags here would not indicate that the game has 4 different parents, but that this version of this game is the preferred version for the European, French, German, and Dutch regions. So, with these tags, read "Parent" as "Preferred", if that helps make things clearer. And, as Kazumi213 said, these tags should be kept as simple as possible. If there was only one set of language choices that covered the European region, tag it as just a European rom. Only in later systems did we start to see, say one European version that would cover English, German, and French, and a 2nd that would cover English, Spanish, and Italian. In such cases, extra tags are needed for one of the roms.

Complicating matters, the "Parent" for a given P/C grouping is something completely different. In a true P/C list, the parent is not indented, and all clones are indented under it. All this grouping does is to make sure that all the roms that are considered to be the same "game" are kept together. Essentially, all of the games in the grouping are clones, and any one of the roms could easily be the non-indented one. But, under the 1G1R settings, clrmamepro will only pick one game from each of these sets to include in the users set. Other games in the same group will be completely ignored. This is where the PARENT tags come in.

Clrmamepro uses the PARENT tags, and the users choice and order of regions to determine what will be in the custom set that it looks for. So, there must be only one of each PARENT tag in each grouping, or clrmamepro will get confused. (There can be one "USA PARENT", and one "EUR PARENT", but two "USA PARENT"s in the same group will cause problems) The "PARENT" tags should be placed only on the "best" game for the region in that grouping.

So, we kinda use the word Parent in a couple different ways here. But I hope I've cleared things up a bit.

[alexian]

Note: In the P/C dats generated from DoM, the PARENT tags have the effect of adding in the "release" information on the cmp dat. "Release" is only added in for those games that have a PARENT tag. On the cmp data that you posted, "release" is listed on every single dat. And therein lies the problem.

When cmp sees two games listed in the same P/C grouping as the preferred game for a given region (via the "release" syntax), it will pick one of these. I'm assuming it probably picks the first one it finds. All other "release" roms for that region for that P/C group will be dropped.

This is why, when you turn on the 1G1R functionality in cmp with the dat you posted, things don't work right. If you only list "release" tags on the preferred rom for each region for each group, everything should be back to good.

[alexian]

I have posted a P/C list, based primarily off of kazumi213's prior work in the custom dats. Frankly, I'd be happy if someone else (kazumi213?) wanted to own this P/C list. Consider mine more of a seed to get things rolling.

My intent has been to make the upgrade to the DoM list as non-intrusive as possible. Where new roms have been added, one may see changes in a 1G1R collection. But my intent was to *not* make any unnecessary changes to anyone's 1G1R set.

It's been a work in progress through the day. My apologies to anyone who downloaded earlier versions. The latest version out there should be fairly accurate to kazumi213's original P/C list.

[kazumi213]

I prefer to wait for NGEfreak to decide. I know you just wanted to help alexian, thank you but please note that he's the SNES maintainer.

[NGEfreak]

Well your last question explains it all. Some users are not interested in collecting ALL region dupes/devstatus of a given game (all members in each family), but just in having one member, 1 ROM(set) from each game, hence 1 game, 1 ROM (1G1R). This prefered member is of course the "best" that the DAT can offer, based on region/language preferences and latest possible devstatus. The 1G1R list generation algorithm in CMPro allows to filter out all undesired members, thus offering Scanning/Rebuilding operation limited only to desired ROM sets.

I don't get why anyone would want to use this feature. It's not possible to make a personal list by using such simple rules. There are many special cases. Sure, I would want the US version most of the time, but in case of Final Fantasy II/IV for example I would prefer the uncut Japanese version over the US version.

Also, by arbitrarily collapsing some BS titles, you are effectively removing unique games from the 1G1R lists and its user's collections.

Well, their problem. They are also going to lose re-release clones like Final Fantasy IV - Easy Type, Ys V – Expert, International Sensible Soccer - World Champions Limited Edition and many more.

That's how current DoM generation routines would require Tintin to be store in database in order to create your release structure. It is true that mine cannot be currently generated either. A way to declare region/languages-based titles should be designed and implemented on DoM.

Well, then it's about time to change the DoM structure. All this stuff can be created on the fly if you have a proper database design. There is no need to manage this stuff with circuitous file lists.

Sure but you need a way to declare in DAT where the "tag" part starts and tweak the rom manager to properly parse the new DAT format.

No, you don't. All required information is already in the DAT:

Code: Select all

if (set_has_no_clones == true && user_prefers_set_name == true) {
    set_name = set_name
} else {
    set_name = get_preferred_release_name()
}

I don't understand why you need to declare so many parents. All child entries can only have one parent anyway.

Hmm. It seems like there is a base misunderstanding. And perhaps this is because the verbiage on the syntax is not the clearest.

In this situation:

Tintin in Tibet (Europe) (En,Fr,De,Nl)|(EU PARENT) (FR PARENT) (DE PARENT) (NL PARENT)

The "PARENT" tags here would not indicate that the game has 4 different parents, but that this version of this game is the preferred version for the European, French, German, and Dutch regions. So, with these tags, read "Parent" as "Preferred", if that helps make things clearer. And, as Kazumi213 said, these tags should be kept as simple as possible. If there was only one set of language choices that covered the European region, tag it as just a European rom. Only in later systems did we start to see, say one European version that would cover English, German, and French, and a 2nd that would cover English, Spanish, and Italian. In such cases, extra tags are needed for one of the roms.

Complicating matters, the "Parent" for a given P/C grouping is something completely different. In a true P/C list, the parent is not indented, and all clones are indented under it. All this grouping does is to make sure that all the roms that are considered to be the same "game" are kept together. Essentially, all of the games in the grouping are clones, and any one of the roms could easily be the non-indented one. But, under the 1G1R settings, clrmamepro will only pick one game from each of these sets to include in the users set. Other games in the same group will be completely ignored. This is where the PARENT tags come in.

Clrmamepro uses the PARENT tags, and the users choice and order of regions to determine what will be in the custom set that it looks for. So, there must be only one of each PARENT tag in each grouping, or clrmamepro will get confused. (There can be one "USA PARENT", and one "EUR PARENT", but two "USA PARENT"s in the same group will cause problems) The "PARENT" tags should be placed only on the "best" game for the region in that grouping.

So, we kinda use the word Parent in a couple different ways here. But I hope I've cleared things up a bit.

I've changed the export script, so each region "parent" is now only defined once per game (see attachment). I guess now the 1G1R problem is fixed?


I still have some questions:

In Settings -> Regions it's possible to select various regions. What for? Apparently it doesn't make a difference if I select all or just a few. Maybe to filter certain regions?

The release element has the attribute date. What influence has this information on the P/C process? How is it possible to configure the related behavior?

It's not possible to load the DAT in the latest ClrMamePro version. Why? The DAT content is valid according to the DTD.

[alexian]

1G1R -- 1 Game / 1 Rom. The system is only going to look for 1 Rom for each Game. The P/C list maintainers are responsible for a) grouping all the roms for a single game into one group, and b) picking the "best" rom for each region. According to No-Intro tradition, the "best" rom for each region is typically the last version released (except in certain cases, the maintainer may choose to ignore special versions).

Let's take a look at the last example in the dat:

Code: Select all

3436 - Zootto Mahjong! (Japan) (NP)|(JPN PARENT)
	3437 - Zootto Mahjong! - Event Version (Japan) (BS)
	3438 - Zootto Mahjong! - Preview Ban (Japan) (BS)

All three roms are seen as being the same game. However, two of these are specific to the Broadcast Satellaview system, the other was released through Nintendo Power magazine.

While NP magazine carts play through a standard SFC system like any other store-bought cart, the BS system had restrictions built into the games so they would only play on certain days, for certain times of the day. In this case, the P/C maintainer (myself, I guess, but much of my work for this system was based on kazumi213's work) chose the NP version as the "best" version.

This one case shows how 3 roms become just 1 when you switch on 1G1R. Similar situations would likely account for the other 261 "missing" roms.

[kazumi213]

I don't get why anyone would want to use this feature. It's not possible to make a personal list by using such simple rules. There are many special cases. Sure, I would want the US version most of the time, but in case of Final Fantasy II/IV for example I would prefer the uncut Japanese version over the US version.

Nobody has ever claimed here about 1G1R being the ultimate selection for *all* users. It just contains what it says it will contain: best picks from prefered regions. If you have different region preferences depending on specific games, well you'll have to hadle them yourself.

Well, their problem. They are also going to lose re-release clones like Final Fantasy IV - Easy Type, Ys V – Expert, International Sensible Soccer - World Champions Limited Edition and many more.

No, it's a bad decision from the datter if he chooses to perform technically incorrect mergings. On my previous post I metioned the BS Kirby. They are different games. You just can't merge them. And you can't merge "episodes" 2, 3 and 4 to episode 1 of some BS titles because you are supposed to want *all* of them. You *could* merge the 4 episodes to a non BS (standard), complete edition (if any exists), but it should be previously judged whether the BS edition has a special value, because I consider that kind of merging a bit "forced". Merging of different editions for the *same* region should always be carefully evaluated by datter.

Regarding your loses examples: see above, careful selection by datter. IMO 1G1R users can live without "Easy Type". For "Expert" and "Limited Edition" it depends, if they are "more complete" than standard, not just increased difficulty or a pesonalized titlescreen, then I'd separate them, this is always wiser than merging because having the standard and the newer release was possible and intended in that region after all.

Well, then it's about time to change the DoM structure. All this stuff can be created on the fly if you have a proper database design. There is no need to manage this stuff with circuitous file lists.

Fell free to discuss your ideas with xuom2. xuom2 and me have put quite some time in designing current P/C system and generation routines. P/C lists work extremely well for quick and easy maintenance of DoM P/C info.

No, you don't. All required information is already in the DAT:

Code: Select all

if (set_has_no_clones == true && user_prefers_set_name == true) {
    set_name = set_name
} else {
    set_name = get_preferred_release_name()
}

Yes you are right. This means that we have to always use untagged titles in "release name". It should be easy to implement on CMPro and DoM generation routine. I will try to contact Roman on this idea. I like it.

I still have some questions:

In Settings -> Regions it's possible to select various regions. What for? Apparently it doesn't make a difference if I select all or just a few. Maybe to filter certain regions?

It's not possible to load the DAT in the latest ClrMamePro version. Why? The DAT content is valid according to the DTD.

Fixed on latest CMPro 3126b. Now it makes a difference (and the expected one).

However our XML DATs required a fix which is already applied on DoM: before 3126b we didn't declare region for games without clones (they were the only option for that "family"). Now region is declared for them too in XML DATs because the CMPro 1G1R selection algorithm checks for it in a different, more restrictive way. If they don't specify region they are directly filtered out (so all games without clones will not appear on latest CMPro 1G1R unless you declare regions for them too).

The release element has the attribute date. What influence has this information on the P/C process? How is it possible to configure the related behavior?

No influence. What's your idea?

I've changed the export script, so each region "parent" is now only defined once per game (see attachment). I guess now the 1G1R problem is fixed?

I will check it.

In this case, the P/C maintainer (myself, I guess, but much of my work for this system was based on kazumi213's work) chose the NP version as the "best" version.

Again, the P/C maintainer is the system maintainer by default. If NGEfreak has his own P/C preferences, he decides. We can only try to explain the current conventions applied to *all* other NI systems and its functional reasons.

And please note that I've not necessarily stopped my work on SNES P/C info yet.

[kazumi213]

I've been checking the posted BETA DAT. Extreme improvements. Thank you NGEfreak. It is working properly on latest CMPro 3126b.

There's still room for improvement, but I'm sure we can reach an agreement. Main suggested changes:

- Most of the "episode" BS games should all be parents (as described before). Another option is creating multifile archives with them, but this won't be completely "correct" because each episode can be played independently. Multifile archives are reserved for interdependent roms (MAME romset approach, disk-spanned games, etc).

- A few "better" picks are possible, since "more complete"/later revision exists (I'm not using rigorous naming):
Game Genie
Pro Action Replay
JRA
Puyo Tsuu Remix (I think the remix has added features)
Super Fire Pro Wrestling X Premium (maybe not if the "premium" features were just in the game package)
"World Cup Final Data" (could be considered a later revision?)

- Consider you comment about prefering JPN Uncut FF. Why forcing 1G1R users to get a non-public, unfinished release where a final, commercial one exist? I think the latter should always be provided in 1G1R mode, no matter if region preference should lead to existing proto/beta:
Arcus Spirits
Iron Commando
Little Magic
Speedy Gonzalez
Virtual Soccer
(maybe more)

- You've done a great job fixing some missing/wrong P/C relations. However I think the following were actually correct:
Exertainment Mountain Bike prefered vs Cannondale. According to this, Cannondale was the "alternate" version, not the other way.
Street Combat is the western port of Ranma 1-2 - Chounai Gekitou Hen (see here). This is one of those cases where "more than text" was replaced when localizing, but the P/C relation is legit.
Same as above for The Jetsons and Youkai Buster. Compare yourself: here and here

[hydr0x]

Exertainment Mountain Bike prefered vs Cannondale. According to this, Cannondale was the "alternate" version, not the other way.

Exertainment Mountain Bike Rally is not playable without the necessary hardware. Cannondale Cup is the standard controller version. So, the latter is the only one useful for emulation gamers and should be the parent.