Hello everyone,
kazumi123 and I have been working on parent clone relationships and merging for some time now. Besides this i have been searching for a good system to create "1 game 1 rom" sets.
Through a lot of testing with the snes dat, and before that the gba and nds dat we have found a system that seems to be working for all scenarios.
Would you please have a look at this system? We both hope this system can be included in all the official dats. (lately many people have been offering help to create the list with parent-clone relationships, at least for the merging part) updating a dat that already has the relations should not be too hard.
(Please be adviced that roman is working a cleaner system than no-dumps for merging.)
Below is the suggested system:
Merging and Parent-Clone selection in DATS
To add support for merging and all kinds of parent-clone relationships to the official dats we have come up with the following system:
Selection of Default parent-clones for merging
Merging selection criteria:
1)
First USA, Second PAL-English, Third Japanese, Fourth PAL-nonEnglish
2)
First Highest Revision of rom, Second Beta, Third Alpha, Fourth Demo/Sample
If the highest ranking region does not have a "1.0 or higher rom revision" the next region is select untill one is found, if non are found the same process goes for the highert beta revsion and so forth
This selection should result in a textfile list resembling this example:
Code: Select all
Parent1
Parent2
clone1
clone2
clone3
parent3
clone4
parent4
Example:
Code: Select all
game (
name "2020 Super Baseball"
description "2020 Super Baseball"
rom ( name "2020 Super Baseball" size 0 flags nodump )
First Highest Revision of rom, Second Beta, Third Alpha, Fourth Demo/Sample
The selected parent is tagged with "(region PARENT)" and "(region PARENT SET)" (where "region" is either NTSC-U, PAL or NTSC-J.)
Example:
Code: Select all
"2020 Super Baseball (Japan) (NTSC-J PARENT) (NTSC-J PARENT SET)"Example:
Code: Select all
"3 Jigen Kakutou Ballz (Japan) (UNIQUE NTSC-J PARENT) (NTSC-J PARENT SET)"Example:
Code: Select all
"2020 Super Baseball (USA) (NTSC-U PARENT) (PAL PARENT) (NTSC-U PARENT SET)" << game is not available in pal regions(The following text has been written bij kazumi123)
Code: Select all
===================
ROM MERGING SUPPORT
===================
Tested on: RomCenter 2.71 and clrmame pro 3.105a
Example:
·
·
·
ACME Animation Factory
ACME Animation Factory (Europe)
ACME Animation Factory (USA)
Acrobat Mission (Japan)
Action Pachio (Japan)
ActRaiser
ActRaiser (Europe)
ActRaiser (France)
ActRaiser (Germany)
ActRaiser (Japan)
ActRaiser (USA)
·
·
·
The supplied DATs can be used to merge a SNES ROM collection. In the example above, listed ROMs will be merged
into 4 archives named:
ACME Animation Factory
Acrobat Mission (Japan)
Action Pachio (Japan)
ActRaiser
Note the "family" archives (with more than 1 ROM) use a generic name without tags. Standard archives (single ROM)
use full name with tags as usual. ROMs inside the archives will remain named as listed in the example (no changes).
To merge:
In RC:
1. Load DAT.
2. Go to File->Preferences (Romsets tab) and set "Roms structure" to "Merged (All ROMs in the Main Set)".
3. "Fix" your ROMs.
If you set "ROMs structure" to "Un-merged (All ROMs in both sets)", the DAT works as official No-Intro DAT (default).
In CM:
1. Load DAT.
2. Go to "Scanner" and be sure that under "NoDump Handling" options group, "Create missing" is UNCHECKED and
"Hide missing" radio button is SELECTED.
3. Go to "Merger", select "Merged sets" and "(Un)Merge" your ROMs.
If you operate the "Scanner" using "Non-Merged sets", the DAT works as official No-Intro DAT.
NOTE: In CM,
No. Parents = No. fake parents = No. families
No. Clones = No. ROMs in families
No. Others = No. unique parents
No. Parents + No. Others = Total parents
No. Clones + No. Others = Total ROMs
CM reports all fake parents (NoDump ROMs) as missing ROMs. This is normal.
========================
REGIONAL PARENTS SUPPORT
========================
The region tags have been used on the DESCRIPTION field only, so they are NOT used to rename. They were suggested
by tetsuo55 and replace my "(p)" tag, allowing for a more versatile DAT.
If you want to create a one game, one rom (1G1R) list which is region oriented, you simply do the following. In all
cases you get a list containing 1990 matches.
In RC:
1. Activate "Show clones" mode
2. Type the following in the "Search" box:
(UNIQUE*PARENT)|(NTSC-U PARENT) ---> for a NTSC-U 1G1R list
(UNIQUE*PARENT)|(PAL PARENT) ---> for a PAL 1G1R list
(UNIQUE*PARENT)|(NTSC-J PARENT) ---> for a NTSC-J 1G1R list
In CM:
1. Go to Scanner tool
2. On "Set Info" window: uncheck "Parent/Clones" and type the following on the "Select Sets" box:
%d=*(UNIQUE*PARENT)*;%d=*(NTSC-U PARENT)* ---> for a NTSC-U 1G1R list
%d=*(UNIQUE*PARENT)*;%d=*(PAL PARENT)* ---> for a PAL 1G1R list
%d=*(UNIQUE*PARENT)*;%d=*(NTSC-J PARENT)* ---> for a NTSC-J 1G1R list
For getting a list which only shows parents from a given region:
In RC:
1. Activate "Show clones" mode
2. Type the following in the "Search" box:
(region PARENT SET)
In CM:
1. Go to Scanner tool
2. On Set Info window: uncheck "Parent/Clones" and type the following in the "Select Sets" box:
%d=*(region PARENT SET)*
where "region" is either NTSC-U (748), PAL (549) or NTSC-J (1614).
=======================
REGION TAGS DESCRIPTION
=======================
1. (UNIQUE region PARENT), where "region" is either NTSC-U, PAL or NTSC-J.
Indicates those ROMs which only exist in one region (they can have clones,
but all clones exist in that region only)
2. (region PARENT): used on those ROMs which exist in the three regions
3. For ROMs which only exist in 2 regions:
- (NTSC-U PARENT) (PAL PARENT): used on those ROMs which exist on NTSC-U and
NTSC-J regions but not PAL. They are NTSC-U ROMs.
- (NTSC-U PARENT) (NTSC-J PARENT): used on those ROMs which exist on NTSC-U
and PAL regions but not NTSC-J. They are NTSC-U ROMs.
- (PAL PARENT) (NTSC-U PARENT): used on those ROMs which exist on PAL and
NTSC-J regions but not NTSC-U. They are PAL ROMs.
To summarize: first region tag indicates actual region of the ROM, second
is the missing region.
4. There are 6 ROMs which have a different tagging (can be considered examples
of the priority rules):
- 3 of them have (NTSC-J PARENT) (NTSC-U PARENT) because they are (Japan, USA).
- 2 of them also have (NTSC-J PARENT) (NTSC-U PARENT) because they are ROM which
don't exist in NTSC-U but the existing PAL ROMs (which should be chosen vs the
NTSC-J ones) are (Proto)
- 1 of them has (PAL PARENT) (NTSC-J PARENT) because it is a ROM which don't
exist in NTSC-J but the existing NTSC-U ROM (which should be chosen vs the
PAL one) is (Proto)
5. In addition to the above tags there are:
(region PARENT SET): used to be able to filter the parents from a given region.
Code: Select all
game (
name "2020 Super Baseball"
description "2020 Super Baseball"
rom ( name "2020 Super Baseball" size 0 flags nodump )
)
game (
name "2020 Super Baseball (Japan)"
description "2020 Super Baseball (Japan) (NTSC-J PARENT) (NTSC-J PARENT SET)"
cloneof "2020 Super Baseball"
romof "2020 Super Baseball"
rom ( name "2020 Super Baseball (Japan).sfc" size 1572864 crc e95a3dd7 sha1 fe448ae2c065dff8b0c2aacc35f9d9ee9432b04f )
)
game (
name "2020 Super Baseball (USA)"
description "2020 Super Baseball (USA) (NTSC-U PARENT) (PAL PARENT) (NTSC-U PARENT SET)"
cloneof "2020 Super Baseball"
romof "2020 Super Baseball"
rom ( name "2020 Super Baseball (USA).sfc" size 1572864 crc 0d77933e sha1 f285d6b3717a571002a5e9e0b732f08f100d2514 )
)
game (
name "3 Jigen Kakutou Ballz (Japan)"
description "3 Jigen Kakutou Ballz (Japan) (UNIQUE NTSC-J PARENT) (NTSC-J PARENT SET)"
rom ( name "3 Jigen Kakutou Ballz (Japan).sfc" size 1048576 crc f0810694 sha1 0caca71401b96b3ab42ec89e9bcf7af82b1bd3d4 )
)
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ prince albert on 11th October 2007, 11:15 wrote:
just curious, would it be possible to post or pm me the snes p-c dat, as i made up an xmdb and merged my snes roms, and i would be interested to see if i missed any relationships, and also to see how well this works?
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ tetsuo55 on 11th October 2007, 11:18 wrote:
Its quite possible that i made mistakes myself, it would be a good idea to compare the results.
A good thing about this system is that xmdb are not needed anymore.
kazumi123 will be posting the snes dat asap.
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ neoforma on 11th October 2007, 11:36 wrote:
No XMDB and no .NET? That's just great.
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ kazumi213 on 11th October 2007, 13:30 wrote:
Just a few additional comments.
The generic name used for the "family" archives is the untagged title of the ROM from the region with higher priority in the convention. In this SNES P-C DAT it is the (USA) title when both (USA) and (Europe) exists. However in the GBA and NDS P-C DATs the (Europe) title is used. I think it is the US one, but I would like to confirm which one should be used according to No-Intro Convention because it is an important change to implement on the GBA P-C DAT and NDS P-C lists.
Also, on this SNES P-C DAT the "regional tags" use NTSC-U, PAL and NTSC-J terms. These seem appropiate for systems like the SNES, but not for others like GBA or NDS. The following terms are suggested: USA, EUR and JPN instead of the above, maybe to be used even in the SNES P-C DAT (uniformity). Remember the regional tags are not used as part of the naming, but to add the regional parent support, so they just need to be functional and intuitive.
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ tetsuo55 on 11th October 2007, 13:57 wrote:
yes its true that the tags have to be universal for all dats, it would still only be the 3 regions though, as WORLD roms can be classified by adding all 3, also the simpler the chosen tags the easier it will be to maintain/use
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ tetsuo55 on 11th October 2007, 14:46 wrote:
The first post has been updated with tag definitions thanks to kazumi123.
Edit:
The formatting has also been adjusted for better readability
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ prince albert on 13th November 2007, 12:19 wrote:
Not sure if this is of much help, but I just stumbled across a handy little tool called CloneSpy, that compares all roms with each other and produces an output of the roms with similarities, including a percentage matrix to show what ones are most similar. If this is old, forgive me, but it could be useful to find unknown parent-clone relationships. Only problem is the time factor, I think it might take a while when operating on a larger set (Took ~8 minutes on ~280 sms games), but nonetheless thought I'd mention it.
NOTE: (I think) The roms need to be renamed to *.bin for this to actually scan them, at least thats what I had to do.
It can be found at:
http://www.gamereset.de/
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ kazumi213 on 13th November 2007, 14:26 wrote:
Didn't know it, so big thanks
I have to play with it a little. As you've said, maybe very time consuming for a whole set, but seems really, really interesting for specific cases.
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ tetsuo55 on 15th January 2008, 08:00 wrote:
Important update!!!
clrmamepro was updated to support "virtual parents"
This finally means that we have official support in clrmamepro for these sets
From the changelog:
clrmamepro 3.109a
misc: main button bar remembers focus
misc: set information -> 'unload' (set list) button added
misc: set information -> you can use '%h=1' in 'select sets' to select sets with chds
misc: set information -> you can use '%p=1' in 'select sets' to select sets with samples
misc: set information -> split 'mark parent/clones' to two separated checkboxes
misc: set information -> mark parent/clones checkboxes also affect loading of set lists
misc: don't show warning for clones where all not-parent roms are nodumps
misc: introducing datfile header switch "forcenodump" which can be 'obsolete', 'ignore' and 'required': obsolete = default (nodumps are marked as unneeded, missing ones are not shown); ignore = existing nodumps are not marked as unneeded, missing ones are ignored; required = existing nodumps are not marked as unneeded, missing ones are shown
misc: dir2dat->nodump header options available
misc: allowing empty set definitions (theoretically this allows using of pseudo parents)
Now all we need are parent-clone lists for all sets, and an update to the Dat-O-Matic to support all dats
Did the tool help any over the past weeks?Didn't know it, so big thanks
I have to play with it a little. As you've said, maybe very time consuming for a whole set, but seems really, really interesting for specific cases.
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ kazumi213 on 15th January 2008, 15:19 wrote:
Regarding current CM P/C DATs, the new header switch has no effect. Default mode doesn't interfere. It is of use for those who want CM to work as before 3.108a. By using (I'm not sure of the exact syntax)
forcenodump required
on header (below "name", "version", "author" and the like) this should be preventing dummy files from being deleted. Missing nodumps will be shown again.
Regarding the "empty set definitions are allowed", I have yet to check what this really means. More on this later.
clonespy: damn, I had totally forgotten this. I've made a quick test using OTU and EBA (Spain) (attached picture)
OTU scores a surprisingly 94% with EBA (Spain), and EBA (Spain) a 71% with OTU. Note that OTU is 512 Mbit while EBA is 1024, so OTU is "contained" in EBA but not the other way. Renaming the ROMs to .bin is required
This time I will perform my planned GBA checks (parent-clone doubts) which I forgot to accomplish.
However I've noticed that the program sometimes work, sometimes not (dos box appear, but nothing else happens). ListSpy (the results viewer) doesn't fully show the zipped CLONE.TXT example (just first family). Compatibility problems? Undocumented usage keys?
===============================================
Re: Adding Official support for Merging, regional and 1game1rom sets \ kazumi213 on 15th January 2008, 20:39 wrote:
This is currently working as I expected: nothing really useful. Now we can remove the line in red (see example), but fake parents still require declaration. Otherwise Profiler destroys the P/C relations. As suggested by NGEfreak, a new keyword ("fakeparent" or "pseudoparent") instead of "cloneof" and "romof" should be coded on CM, allowing to avoid explicit current fake parent declaration. I'm not going to delete the unneeded line yet.Regarding the "empty set definitions are allowed", I have yet to check what this really means. More on this later.
Code: Select all
Working as of 3.109a
game (
name "Sonic Advance 2"
description "Sonic Advance 2"
rom ( name "Sonic Advance 2" size 0 flags nodump ) <==== REMOVE
)
game (
name "Sonic Advance 2 (Europe) (En,Ja,Fr,De,Es,It)"
description "Sonic Advance 2 (Europe) (En,Ja,Fr,De,Es,It) (EUR PARENT) (EUR PARENT SET)"
cloneof "Sonic Advance 2"
romof "Sonic Advance 2"
rom ( name "Sonic Advance 2 (Europe) (En,Ja,Fr,De,Es,It).gba" size 16777216 crc 89509891 sha1 b0f64bdca097f2de8f05ac4c8caea2b80c5faeb1 )
)
game (
name "Sonic Advance 2 (Japan) (En,Ja,Fr,De,Es,It)"
description "Sonic Advance 2 (Japan) (En,Ja,Fr,De,Es,It) (JPN PARENT) (JPN PARENT SET)"
cloneof "Sonic Advance 2"
romof "Sonic Advance 2"
rom ( name "Sonic Advance 2 (Japan) (En,Ja,Fr,De,Es,It).gba" size 16777216 crc 513804ff sha1 dffd0188fc78154b42b401398a224ae0713edf23 )
)
game (
name "Sonic Advance 2 (USA) (En,Ja,Fr,De,Es,It)"
description "Sonic Advance 2 (USA) (En,Ja,Fr,De,Es,It) (USA PARENT) (USA PARENT SET)"
cloneof "Sonic Advance 2"
romof "Sonic Advance 2"
rom ( name "Sonic Advance 2 (USA) (En,Ja,Fr,De,Es,It).gba" size 16777216 crc 7efee7f7 sha1 7bcd6a07af7c894746fa28073fe0c0e34408022d )
)
It would be better:
game ( <==== REMOVE
name "Sonic Advance 2" <==== REMOVE
description "Sonic Advance 2" <==== REMOVE
rom ( name "Sonic Advance 2" size 0 flags nodump ) <==== REMOVE
) <==== REMOVE
game (
name "Sonic Advance 2 (Europe) (En,Ja,Fr,De,Es,It)"
description "Sonic Advance 2 (Europe) (En,Ja,Fr,De,Es,It) (EUR PARENT) (EUR PARENT SET)"
cloneof "Sonic Advance 2" <==== REMOVE
romof "Sonic Advance 2" <==== REMOVE
fakeparent "Sonic Advance 2"
rom ( name "Sonic Advance 2 (Europe) (En,Ja,Fr,De,Es,It).gba" size 16777216 crc 89509891 sha1 b0f64bdca097f2de8f05ac4c8caea2b80c5faeb1 )
)
game (
name "Sonic Advance 2 (Japan) (En,Ja,Fr,De,Es,It)"
description "Sonic Advance 2 (Japan) (En,Ja,Fr,De,Es,It) (JPN PARENT) (JPN PARENT SET)"
cloneof "Sonic Advance 2" <==== REMOVE
romof "Sonic Advance 2" <==== REMOVE
fakeparent "Sonic Advance 2"
rom ( name "Sonic Advance 2 (Japan) (En,Ja,Fr,De,Es,It).gba" size 16777216 crc 513804ff sha1 dffd0188fc78154b42b401398a224ae0713edf23 )
)
game (
name "Sonic Advance 2 (USA) (En,Ja,Fr,De,Es,It)"
description "Sonic Advance 2 (USA) (En,Ja,Fr,De,Es,It) (USA PARENT) (USA PARENT SET)"
cloneof "Sonic Advance 2" <==== REMOVE
romof "Sonic Advance 2" <==== REMOVE
fakeparent "Sonic Advance 2"
rom ( name "Sonic Advance 2 (USA) (En,Ja,Fr,De,Es,It).gba" size 16777216 crc 7efee7f7 sha1 7bcd6a07af7c894746fa28073fe0c0e34408022d )
)
I've solved my problems as follows:However I've noticed that the program sometimes work, sometimes not (dos box appear, but nothing else happens). ListSpy (the results viewer) doesn't fully show the zipped CLONE.TXT example (just first family). Compatibility problems? Undocumented usage keys?
CloneSpy.exe: right-click -> Properties -> Compatibility. I set it to "Windows 2000" and checked "Disable visual themes" and "Turn off advanced text services...". Also, be sure to disable antivirus for max speed comparison.
ListSpy: Runs flawless through DOSBox emulator.
I've checked the "families" in next example. Doubtful members are in bold.This time I will perform my planned GBA checks (parent-clone doubts) which I forgot to accomplish.
According to results both "Duel Monsters 6" and "Desing & Destroy" are "poor" clones. And "Duel Monster 5" has no similarities with "Eternal Duelist".==BattleBots - Beyond the BattleBox
BattleBots - Beyond the BattleBox (Europe) (En,Fr,De)
BattleBots - Beyond the BattleBox (USA)
BattleBots - Design & Destroy (USA)
==Yu-Gi-Oh! - The Eternal Duelist Soul
Yu-Gi-Oh! - The Eternal Duelist Soul (USA)
Yu-Gi-Oh! Duel Monsters 5 Expert 1 (Japan)
==Yu-Gi-Oh! - Worldwide Edition - Stairway to the Destined Duel
Yu-Gi-Oh! - Worldwide Edition - Stairway to the Destined Duel (Europe) (En,Ja,Fr,De,Es,It)
Yu-Gi-Oh! - Worldwide Edition - Stairway to the Destined Duel (USA) (En,Ja,Fr,De,Es,It)
Yu-Gi-Oh! Duel Monsters 6 Expert 2 (Japan)
Yu-Gi-Oh! Duel Monsters International - Worldwide Edition (Japan) (En,Ja,Fr,De,Es,It)
Yu-Gi-Oh! Duel Monsters International - Worldwide Edition (Japan) (En,Ja,Fr,De,Es,It) (Rev A)
However I maintain the current relations. Please, try them on emulator.
In addition to in-game similarities, "Duel Monsters 5" and "Eternal Duelist" share serial and boxart. Regarding "Duel Monsters 6" and "Design & Destroy", I remember reading somewhere that "Duel Monsters 6" was re-released in Japan as the "Worldwide Edition", while "Design & Destroy" was a weird re-release to fix the crappy AI in "Beyond the Battlebox".