Issues with recent DAT updates

[scorp256]

I do not know, if these are known issues, in any case better to tell, I think. I will mention only ones I noticed with DATs I recently updated, most likely there are more similar issues in other DATs.

Nintendo - eReader

Just a mess of overlapping <rom tags, just open in text editor and you will see. Unusable.

Nintendo - FDC

Lot of cases which got repeated rom tag after status="baddump" like below.
Of course all rom records should be closed with />
But they all are bad dumps and all have same name but different CRC.
Seems it should be fixed somehow with different filename (containing crc of dump for example) or with only one bad dump or whatever.

Code: Select all

		<game name="0053 - Golf - US Course (Japan) (DV 1) [b]">
		<description>0053 - Golf - US Course (Japan) (DV 1) [b]</description>
		<rom name="0053 - Golf - US Course (Japan) (DV 1) [b].fds" size="131000" crc="9944329A" md5="ED62CEB4EC492FD3C19634E041B9B3F5" sha1="B7726762EAC903FA1450F7828EA8063E562E4689" status="baddump"		<rom name="0053 - Golf - US Course (Japan) (DV 1) [b].fds" size="131000" crc="8DFBD636" md5="2C89A5EEBF235BF2ADE3C708887D1E73" sha1="1EB005876CEAA99C83C91090B907F85CBE1E3DA2" status="baddump"		<rom name="0053 - Golf - US Course (Japan) (DV 1) [b].fds" size="131000" crc="206C5C6C" md5="BE4EC4568EE43218A2FF627ACF76BA23" sha1="723D44B250109A31EFCFB85D8EA388C7B4030948" status="baddump"/>
	</game>

Nintendo - Nintendo DS

Similar issue, but in this case roms have different names, but in one case it have "b" suffix, in other case - no? Really strange, as both are bad dumps.

Code: Select all

	<game name="3498 - Princess Maker 4 - Special Edition (Korea) [b]">
		<description>3498 - Princess Maker 4 - Special Edition (Korea) [b]</description>
		<rom name="3498 - Princess Maker 4 - Special Edition (Korea) [b].nds" size="134217728" crc="B24C9668" md5="79E07D095B67B3053861454316B5B82A" sha1="D3E0A75DEA59C43BADF942BC94571A018FB00AC5" status="baddump" serial="CP4K"		<rom name="3498 - Princess Maker 4 - Special Edition (Korea).nds" size="111404512" crc="0A42995A" md5="D6C785E70DD8B604EBB8F1C1D56E8F92" sha1="BA55871F6C35386739274FC5B8393CC7EABD314D" status="baddump" serial="CP4K"/>
	</game>

And also it has another issue with disappeared closing tag for serial_media2 (I suppose it have something to do with character encoding, probably these are utf characters or like that):

Code: Select all

	<game name="0759 - Ryuusei no Rockman - Leo (Japan)">
		<description>0759 - Ryuusei no Rockman - Leo (Japan)</description>
		<rom name="0759 - Ryuusei no Rockman - Leo (Japan).nds" size="33554432" crc="EA4F4898" md5="AFE887B51844366FD40DA2D538C011D2" sha1="29E6EBD1E8E4D3E65E2917B3F5C7A869BE867B64" status="verified" serial="A6BJ"/>
		<serial_media1>A6BJN0J12</serial_media1>
		<serial_media2>DA
	</game>

Nintendo - Nintendo DSI DLC

Same issue with non-closed first rom tag:

Code: Select all

	<game name="1068 - Clubhouse Games Express - Card Classics (USA, Australia)">
		<description>1068 - Clubhouse Games Express - Card Classics (USA, Australia)</description>
		<rom name="00000000" size="4041728" crc="5FC69CDE" md5="10D51A63DB848C7896D1323F44E77AA8" sha1="7362904CFE778B24A20396ABEB3C07800A084766" status="verified" serial="KTRT"		<rom name="tmd" size="520" crc="255F0730" md5="BE55BB41A72FB42524F36035424E758F" sha1="1DE53C1212ABE18D7F7E1A1247CF716F091B0A71" status="verified"/>
	</game>

Nintendo - Satellaview

Same issue with non-closed first rom tag:

Code: Select all

	<game name="3593 - Digital Magazine Busters BS - 6-21 Gou (Japan) [b]">
		<description>3593 - Digital Magazine Busters BS - 6-21 Gou (Japan) [b]</description>
		<rom name="3593 - Digital Magazine Busters BS - 6-21 Gou (Japan) [b].bs" size="1048576" crc="396FAD58" md5="16AE85EAAD3DED5F9B56BE9FB703E74C" sha1="497647379E28FC4C6544FEFA1CEFB23C949F7E3C" status="baddump"		<rom name="3593 - Digital Magazine Busters BS - 6-21 Gou (Japan) [b].bs" size="1048576" crc="AAA71F34" md5="09C5872608BED82A85EC334D92350E57" sha1="646A7E6AFD44AC475B0823E79213F352C4BC34B7" status="baddump"/>
	</game>

Sega - Mega Drive - Genesis

Bad value for & in serial_media2 tag:

Code: Select all

	<game name="1852 - Death and Return of Superman, The (Europe)">
		<description>1852 - Death and Return of Superman, The (Europe)</description>
		<rom name="1852 - Death and Return of Superman, The (Europe).md" size="2097152" crc="CFC440E8" md5="D14F262F3D9D46DF0ED1D9F2BB6C3442" sha1="BC23B7C2FC720371A8B7CAED1ACB3F457315E0A6" status="verified"/>
		<serial_media1>670128 REV 1</serial_media1>
		<serial_media2>DEATH & RETURN OF SUPERMAN (9519)</serial_media2>
	</game>

Should be <serial_media2>DEATH &amp; RETURN OF SUPERMAN (9519)</serial_media2>


Hope somebody who can fix it will see this post and perform mentioned fixes

[Gefflon]

Sony - PlayStation 4 (PSN) (Encrypted):

Code: Select all

<rom name="UP1012-CUSA01003_00-AIRCONFLICTSVIET_0.pkg" size="4294967296" crc="232D4C91" md5="5E79CC1741A18C22C8F7E97F6628EFFF" sha1="7727BB068F281C49891FC873AECFE43EC2B980B1"          <rom name="UP1012-CUSA01003_00-AIRCONFLICTSVIET_1.pkg" size="4294967296" crc="CE6B1C2F" md5="678D7D259A3C4A7EEF7F32D365A09A7C" sha1="E360D0B96794B6F3F50EAEB8DFBE49ED5889EF38"          <rom name="UP1012-CUSA01003_00-AIRCONFLICTSVIET_2.pkg" size="2379808768" crc="2EE4A368" md5="FA66A79CE2BDE357E5E3F053017B23BE" sha1="1A9366A0E83843009448C5F02113F8B6CE1DD23F"/>

or

Code: Select all

<rom name="UP0177-CUSA00363_00-ALIENISOLATIONP4_0.pkg" size="4294967296" crc="56076891" md5="F1A14BE504E0D5DB4CA01327E12EB025" sha1="E5137AAF0752850E81C4AFB2F5F7EF4D74F26F42"          <rom name="UP0177-CUSA00363_00-ALIENISOLATIONP4_1.pkg" size="4294967296" crc="2F6209EA" md5="0F480F2A42E010F657F8AC0D69D7FA9A" sha1="2A4E438B8F8FCC3287A695F1A341C7EBAEED43AE"          <rom name="UP0177-CUSA00363_00-ALIENISOLATIONP4_2.pkg" size="4294967296" crc="58A84BF5" md5="0EA4EAA4A02A2EF742117E2F8B9FFF61" sha1="3685B2164ECAB4835F745992848EB68306A9D105"          <rom name="UP0177-CUSA00363_00-ALIENISOLATIONP4_3.pkg" size="4294967296" crc="4BEE2B49" md5="A77650CA1CE4F5065866E9DC5CB74D07" sha1="DEC1ED84C9D5CBEAC65676DF492189A165014F49"          <rom name="UP0177-CUSA00363_00-ALIENISOLATIONP4_4.pkg" size="1374027776" crc="281FDE5F" md5="9F0FF9A13789FBFD16C5C577BDBDAFD3" sha1="410F54CFAEE167B533C9221C3AB3A0B97B48780A"/>

Nintendo - Nintendo Switch:

Code: Select all

                <Original Title>ベヨネッタ</Original Title>

This is not how XML works. It sees "<Original ", so it wants "</Original>", and "Title" should have a value, like "Title=something".
You could rename it to "Original_Title", that'd probably work.

[xuom2]

changes:
- excluded strings with special chars like Microïds
- removed space char from items: "Original Title" -> "Original_Title"
- changed & to &amp; in serials and media data
- closed "rom" item when needed.

[scorp256]

@xuom2 I think it would be good to test all existing sets generated to comply with XML. I think even basic simple_xml_load could do it, so you can perform it automatically.

Issue with Nintendo DS <serial_media2> still there (you told you excluded special chars, seems not entirely):

Code: Select all

	<game name="0759 - Ryuusei no Rockman - Leo (Japan)">
		<description>0759 - Ryuusei no Rockman - Leo (Japan)</description>
		<rom name="0759 - Ryuusei no Rockman - Leo (Japan).nds" size="33554432" crc="EA4F4898" md5="AFE887B51844366FD40DA2D538C011D2" sha1="29E6EBD1E8E4D3E65E2917B3F5C7A869BE867B64" status="verified" serial="A6BJ"/>
		<serial_media1>A6BJN0J12</serial_media1>
		<serial_media2>DA
	</game>

and it sill have duplicates and bad dumps named without "b"

Code: Select all

	<game name="3659 - Mabeopcheonjamun DS (Korea) [b]">
		<description>3659 - Mabeopcheonjamun DS (Korea) [b]</description>
		<rom name="3659 - Mabeopcheonjamun DS (Korea) [b].nds" size="134217728" crc="1E7B2E5D" md5="569DA2B8B71C1C2C32E5362D73C04D88" sha1="66A123B9CB603219EE336A58745C77B9BBC61C5B" status="baddump" serial="YNOK"/>
		<rom name="3659 - Mabeopcheonjamun DS (Korea) [b].nds" size="134217728" crc="1E7B2E5D" md5="569DA2B8B71C1C2C32E5362D73C04D88" sha1="66A123B9CB603219EE336A58745C77B9BBC61C5B" status="baddump" serial="YNOK"/>
		<rom name="3659 - Mabeopcheonjamun DS (Korea).nds" size="134217728" crc="0C312FBF" md5="7CAFB428DD43F7BB538474D6BE640775" sha1="7549B97562195927749D9C52581472D4A980F9A7" status="baddump" serial="YNOK"/>
	</game>

Nintendo eReader became even messier, there are TONS of duplicates, so DAT file is 11MB instead on 1MB...

FDS also have issues with duplicated rom names and even roms themselves (see 2 duplicated lines and 3rd one having same name as previous 2):

Code: Select all

	<game name="0101 - Patlabor - The Mobile Police - Dai 2 Shoutai Shutsudou Seyo! (Japan) [b]">
		<description>0101 - Patlabor - The Mobile Police - Dai 2 Shoutai Shutsudou Seyo! (Japan) [b]</description>
		<rom name="0101 - Patlabor - The Mobile Police - Dai 2 Shoutai Shutsudou Seyo! (Japan) [b].fds" size="131000" crc="05FC472B" md5="C8ECDF04D5D1EB1855CA466A45681223" sha1="CF818C6EE3D869FAFB60223408180B71A3495A59" status="baddump"/>
		<rom name="0101 - Patlabor - The Mobile Police - Dai 2 Shoutai Shutsudou Seyo! (Japan) [b].fds" size="131000" crc="05FC472B" md5="C8ECDF04D5D1EB1855CA466A45681223" sha1="CF818C6EE3D869FAFB60223408180B71A3495A59" status="baddump"/>
		<rom name="0101 - Patlabor - The Mobile Police - Dai 2 Shoutai Shutsudou Seyo! (Japan) [b].fds" size="131000" crc="1EB0CE56" md5="90698DDEDB1B97F0946CD856B9F47D25" sha1="5DB5E943132D74515FE1DBC51F7DF10C91B7808C" status="baddump"/>
	</game>

[xuom2]

I will disable the captcha at least during these crazy days.

(you told you excluded special chars, seems not entirely)

online version of datfiles won't include media/serial data until I discover a better way to exclude special chars or to include them in the correct way.

https://datomatic.no-intro.org/index.ph ... =28&n=3498
-> crc 0A42995A should not be flagged with the gold key (selected). this now has been fixed, but I have to rewrite all caches first.

Animal Crossing-e - Series 2 - Mr. Resetti &amp; Don (USA)

duplicated (Strip 1): another bug that should be gone.

NDS 3659: release Magic_Cheonjamun_DS_KOR_NDS-NEREiD

It's marked Bad, but not the related files.
Today I will scan all similar cases: if they are marked as bad, their filenames get the [bad] flag.

NDS 3659, SATELLAVIEW 3593, ...

I have to scan all items that have only >1 bad sources.
In these cases the only way to exclude all files from datfile is to select the "valid" bad files, flagging them as "forced".

[scorp256]

I have to scan all items that have only >1 bad sources.
In these cases the only way to exclude all files from datfile is to select the "valid" bad files, flagging them as "forced".
Example: NDS 3659, SATELLAVIEW 3593.

I think it would be easier to do it automatically (get latest dat before your changes and use CRC in it to flag valid bad file), as there was only 1 bad file before.

[xuom2]

Sure, to choose which bad file has to be the exported one, I will check the previous datfile.

[jimmsu]

Sony - PlayStation Portable (UMD Video) dat

multiple iso are not seen , see only the first one.
But they are in the file ??

[xuom2]

I think that after today updates, UMD Video contains all discs.

[xuom2]

Updated: NDS, FDS, SNES, 3DS, SATELLA forcing some files to appear (and automatically hiding the others in same archive).

[scorp256]

@xuom2 are you sure you chose the right ones as bad dumps for Satellaview? As I suddenly got 8 missing bad dumps.

Also lot of verified dumps disappeared from 3DS romset like 0120 - Sims 3, The (was present before, not present anymore in dat) http://datomatic.no-intro.org/index.php ... =64&n=0120

Code: Select all

	<game name="0119 - Madden NFL Football (USA)">
		<description>0119 - Madden NFL Football (USA)</description>
		<rom name="0119 - Madden NFL Football (USA).3ds" size="1073741824" crc="7F2C56CE" md5="11595F137A3E8B7093BF67A51B2CAB9F" sha1="4965822F27719E459B5062B35F7102F09535170A" serial="CTR-P-AMDE"/>
	</game>
	<game name="0121 - Monster Hunter 3G (Japan)">
		<description>0121 - Monster Hunter 3G (Japan)</description>
		<rom name="0121 - Monster Hunter 3G (Japan).3ds" size="2147483648" crc="C85F9E63" md5="4F10874EEADA1B8E80A295C7BA1F9B64" sha1="3E60CBC82E36A9533A2CF52F5505D7C0B12881E4" status="verified" serial="CTR-P-AMHJ"/>
	</game>

[scorp256]

Fixdat attached for satella.
For 3ds will do tomorrow, but if there is an encrypted 3ds, there definitely should be decrypted counterpart...
Unfortunately I do not have a full romset for 3ds, so would not be able to post all missing ones, but I think I can just post old 3ds dat file and you can compare it with current set and see missing ones....

[Gefflon]

The "Nintendo - Nintendo DSi (DLC) (CDN)" dat lost its only entry, Cave Story.

[xuom2]

From the log I see it was intentionally deleted.

[scorp256]

So could you please just check if these 8 roms in recent DAT from datomatic match the ones of 5 august (they dont match for sure) and fix it to match?

Code: Select all

0307 - BS Dragon Quest I (Japan) (SoundLink) [b]
3465 - AK LIVE Hit Gang (Japan) (5-30) [b]
3498 - Bokujou Monogatari (Japan) (Demo) [b]
3542 - Machi Magazine (Japan) [b]
3554 - New Yatterman - Nandai Kandai Yajirobee (Japan) [b]
3591 - Star Soldier - 2 Minute Mode (Japan) (En) [b]
3592 - Dai-1-Kai BS Satellaview SateSupo DX - Dai-2-gou (Japan) [b]
3611 - AK LIVE Hit Gang (4-28) + Chrono Trigger Unknown Magazine (Japan) [b]