iNES Header Repair

[B00daW]

Regarding NES/Famicom:

So what kind of resources do we have on our hands for databasing? I'm more of a dumper than a databaser.

We have our dumps and our database. We could do this a couple ways:

1.) We could database the known games with headers on them iNES/UNIF headers on them. And database games with NES 2.0 headers on them.

2.) We could create a database of headers and then have a tool to add them onto the combined ROM images.

I say option 1 seems the easiest for everyone.

Let's say we chose option 1, where and when do we start? I worked on an unfinished NES/Famicom databaser with a friend Captain_N, who is no longer with us, peace be with him, so I understand the process and the amount of time that it takes. Previously when I was databasing I didn't have a full-time job nor a family life. Now I have both of them. I will not be able to assist in the databasing process other than testing and error checking from time to time.

xuom2, I'm pleased to hear that you can help!

Regarding FDS:

I'd be interested to see a full disk image at some point...

[BigFred]

So how do we continue?

We require an option to choose from

A) iNES-format ignoring headers using the XML (current method)
B) iNES-format - headers are included in hash-calculation
C) split ROM-images

for creating the dat.

For seperated PRG/CHR we just need additional fields to store their individual hashes, size and labels. Afaik there are a few select game using more than one CHR or PRG so there must be an option to add more ROM-chips. The dat then needs to store all seperate ROM-images together in one archive similar to arcade-sets. The zip-file itself would use the game title but I'm not sure how the CHR/PRG have to be named. I need to ask bootgod about this once again.

For the headered format the DoM cannot do anything about your files of course. If this option is chosen, game size needs to be set 16 bytes larger for the dat plus a different crc which needs to go into the db as well. For a reference we could still offer a sheet which displays the correct header settings. This way it would be easier to spot mistakes. Besides this information could be used for a download-option which would write all this data into a text file which could then be used with some (not yet available) tool to fix the headers.

The tool would just need to

-calculate the crc without header bytes
-compare crc to db
-overwrite header with data stored in text file linked to the specific crc

I think this is a good idea for a feature in the Game Header successor.

[xuom2]

yeah im searching a "new gameheader" coder. job is paid

[BigFred]

So who wants to be a millionaire?

[xuom2]

LOL

[BigFred]

And the rest will be implemented I guess?

[xuom2]

you speak with me?

[B00daW]

Unfortunately I believe a split ROM format would only be useful for data preservation sake. It is not very good for game preservation sake if they are not playable by any emulators; and especially hardware. Flash carts will not keep a large database full of headers and people will not want to nor will most people even try to slap headers onto their ROM set.

I liked the idea of option 2, but I don't think we should move forward with iNES. I believe NES 2.0 should be used. As for some good news, the author of Nintendulator is back in public. He will likely support NES 2.0 in his emulator if a set of NES 2.0 ROM images exist. kevtris used to work a lot with Quietust on Nintendulator helping it become hardware accurate.

[KingHanco]

When this going to start?

[BigFred]

you speak with me?

I thought so... If the changes are implemented I can start inserting the data immediately. I'd like to make people happy after all the confusion and discussion this system has caused over the years

It is not very good for game preservation sake if they are not playable by any emulators; and especially hardware.

Well, I simply want to archive those games as accurately as possible - this is my main objective and I think that most people agree technically this is the way to go (I think the MAME team is right in every way here). But I don't force anyone to use it. It's just an offer to everyone out there - if such a set is available maybe it will gain more attention by emulator authors. Everyone can make of it what he wants. It's a bit like the hen/egg-problem. Afaik MESS already accepts split SNES-images. As long as anyone has the option no one is hurt

PS: By iNES 2.0 I actually referred to NES 2.0

[xuom2]

DOM already is able to manage archives with more than one file in them.
for this reason, implementing NES which has 2 files instead of one is not a problem.
i can load the 2-file version from a datfile. just please number the archives with same number in DOM.

if i understand correctly, you want a mixed mode, a download option to choose between 2 files (default) and 1 file (custom datfile).
correct?
this needs a bit of code, but nothing complicated.

[BigFred]

There were too many questions lately if this and that ROM image might be bad so I'm voting for removing the current header-ignoring version - hardly anyone appears to understand how the currently used system works. So in the future we use a) NES 2.0-headered ROMs + a table showing the header setting on DoM (as stated above - probably a new table has to be created) and as an official clean version the split CHR/PRG which is more accurate anyway.

For the first option we just need to remove the header-XML and feed DoM as usual. An additional table for the header info is needed but can be added later when we gathered more info about this and discussed how it shout work.

For the second option I don't know. Currently the seperated program and gfx-ROMs are merged. If DoM already can already handle more than one ROM in an archive it's a great start. However it might be nessecary to name the PRG/CHR in a different way than the game itself. And as mentioned before more there are games with more than one CHR Like After Burner. And for sure not every game uses seperated CHR + PRG - many just use one ROM-image. So it must be possible to leave the CHR-field empty or add a 2nd one. Maybe just put two fields for CHR into the db and we are done.

[KingHanco]

I say just do the split files only. Don't do the .prg + .chr = nes. If they want the headless .nes then all they have to do is this.

copy /b *.prg+*.chr *.nes

Be much easier to do. There is two that will be a problem. People will complain about. (Like I was complaining about.)

People who having these problem. Please read.

King Neptune's Adventure (USA) (Unl).nes < Make sure it is 97kb before doing so. Just load on the Nestopia / Edit iNES Header then save it. This will fix what ever the problem is.

Mighty Bomb Jack (Europe).nes < Make sure it is 65kb before doing so. Just load on the Nestopia / Edit iNES Header then re-size to a correct size then save it.

People going have to redo these with the Nestopia / Edit iNES Header in order get the size correct before split those up.

[B00daW]

That again misses the point. Nestopia is not a constantly developed emulator. You are assuming that the author of Nestopia wants to be a part of the dumping/preservation effort and designating him a presumably unwanted job. I have a mound of undumped carts that even after being dumped will not have iNES headers nor numbers because there won't be any to assign by the time I'm finished dumping them.

We also need to get rid of the hacked mappers that include multiple PCB and chip logical configurations that are pushed into one mapper number; instead of and desiredly one configuration for one mapper. If mappers are purified then iNES would already be out of mappers.

For those that don't fully understand what a "mapper" is, it is a pointer system that points an emulator or FPGA/CPLD to an emulated PCB logic configuration that takes care of extended memory mapping and additional audio processing or I/O hardware that could potentially be on a cart. Since configurations are limitless a system is needed to designate configurations by number to emulate ROMs so they can be properly processed by the Nintendo or emulator.

[neoxite]

Just signed up because I'd like to throw in my thoughts on how NES games should be organized in future DAT's.

I agree that for preservation's sake, cataloguing the checksums for the dumped chips is a must. Now for 'header preservation', the situation still looks like a big mess with iNES, NES2.0 UNIF etc floating around, the most widespread format of course being iNES. Now there's the discrepancy between DATing games in a format that will play in emulators vs. DATing games neatly separated into their components (ROM chips and mapper information).

So here's my suggestion:
Please DAT the games in iNES format without an XML header skipper. It's the only one-stop solution to quickly audit games that will work out of the box for the most common purpose, avoiding all the confusion the XML header skipper caused. Then supply a XML file along with the DAT file that is able to tear down those iNES formatted files into their accurate components. This is only one further step to go for people interested in dealing with preservation and EPROMs.

iNES is by far not the most accurate format, but it's still the most common one, working with almost anything. The iNES files to be DATted however should have been generated using the mapper information in the supplied XML file. Also conversion to iNES2.0 or UNIF could be done relatively easily. I am aware certain games are impossible to store in iNES format unless they are padded (Galaxian for example). But given the XML file information, the original ROM chip's contents could be accurately restored (anyone confirm this?).

The quality of bootgod's XML database is excellent! I went and stripped my No-Intro NES 20121027 fullset of its headers and made sure it fully validates in CMPRO without any XML header skippers loaded. Then I used the iNES-header-fix python script supplied earlier in this thread to rebuild iNES headers for all games matching bootgod's XML database. The Database file 'NesCarts (2012-10-22).xml' already matches 2110 games of the 2655 games included in the set!

If bootgod doesn't have any objections, this file could serve as the seed of a future 'No-Intro header database' XML, or the projects could team up on maintaining the file. The mapper information in the XML file could probably also get ported into DoM directly, which while being a chore at first could automate XML creation.

To sum up the benefits:
No-Intro supplies the most accurate preservation information on NES games? Check.
DATted games work out of the box in common emulators? Check.
DATted games can easily be disassembled into their components for EPROM burning? Check.
DATted games can easily be converted into the most suitable format? Check.

Please give this approach a thought and comment on objections or suggestions. Thank you!