iNES Header Repair

[hornpipe2]

My quick stab at an iNES header repair tool, using BootGod's XML database (this thing needs to be updated btw). In Python. Careful as it will trash your existing iNES headers and it doesn't write 2.0 headers, so there's a chance you could lose some info.

http://nwserver.ath.cx/2012/05/30/ines-header-fixer/

[C. V. Reynolds]

This seems really great, though I can't figure out yet how to run it (no experience in Python scripting for me). I was hoping something like this would come along to help with some of the NES's ridiculous header woes. Great job.

The script shows the "8)" as a sunglasses smiley at the time of this writing, though. May want to fix that.

EDIT: Can sort of run it now, but only one file at a time. Can't do batches. Surely I must be doing something wrong, but what?

[hornpipe2]

Just run it with 'find' on a whole folder, as in

find . -type f -name '*.nes' -exec ./ines-fix.py {} \;

[C. V. Reynolds]

I'm truly sorry, but I don't understand any of that at all. My knowledge of Python is nothing and I had never used it before. To show well my lack of knowledge, this is the code that I used to check files one at a time:

(command line ran from Python directory)

python "[directory]\ines-fix.py" "[directory]\[ROM image name].nes"

I do hope that others can understand, though.

[B00daW]

I'm glad someone has started this. It's quite a shame that the way headers are ignored during the DATing process. Without an iNES header an .NES image is basically worthless.

I don't know if any of you remember a tool called "NEStoy"? Back in the day it was the standard for NES preservation, collection, and organization. It also had a huge compressed database of headers associated with CRC that you could slap onto your ROMs and fix them all.

Using the current No-Intro DAT, a header database should be kept with the DAT as reference so that headers can be added/repaired.

[BigFred]

I basically agree but the main goal was a completely headerless format. Just datting PRG/CHR and providing an external XML with header info accessible by emulators. I think it can be done with Nestopia or MESS.

[C. V. Reynolds]

Nestopia can have an external database, yes, but as I've mentioned once before, it unfortunately will not load a headerless NES ROM image (just says "invalid file"). As Nestopia is still considered the best NES emulator by many (including me), yet is not being updated anymore, this is a problem.

Like BigFred, I basically agree, but as I've also said before: I long for the day when headers can be eliminated.

And sadly, I still can't work the topic creator's tool.

[B00daW]

BigFred: First off you pick two of the more inaccurate emulators and then you sort of push these responsibilities on a community that hardly cares about NES emulation, MESS, and then an emulator that hasn't seen an update in ages, Nestopia. First and foremost, you go beyond and over the NESdev community that made formats specifically because they know that NES/Famicom ROM dumps need headers for a more synchronized development effort among coders.

Also most importantly, there is the PowerPak and potentially two more NES/Famicom flash carts that rely on a header so that they can direct their FPGA or CPLDs to switch in the right logic operations. Headers are a necessary "evil" with the NES/Famicom. A ~1.7MHz processor will not run a header database.

Whether we at No-Intro like it or not, true NES/Famicom enthusiasts are stray away from No-Intro NES/Famicom sets because they are more or less "broken". This thread even exemplifies that fact.

If we get the headers fixed it you/we won't have to rely on specific emulator authors NEStopia/MESS to support the burden of emulating constantly increasing mapping logic. If we keep a working standard format, be it iNES or HOPEFULLY move toward NES 2.0, we can reign in a more accurate and lasting emulation of the NES/Famicom consoles.

iNES is running out of mappers and more or less has. Consider iNES to be IPv4. Eventually we're just going to have to switch over to IPv6 or NES 2.0, no matter how reluctant we are.

http://wiki.nesdev.com/w/index.php/NES_2.0

Nintendulator already supports this format. Although the author is not much on the communication side of things, for being anti-social he is very prompt with updates.

[C. V. Reynolds]

First of all, Nestopia isn't inaccurate. I don't know why anyone would think that.

Second, I don't think adding more headers is the answer. NES 2.0 looks terrible. We really do need to move away from attaching headers to the game files. An external database file is definitely the true answer.

Third, are people really going to have a computer so slow they cannot load a header database through an emulator? My computer is 7 years old, and I can load an xml of game headers from Bootgod. Heck, if your computer is all THAT slow, it won't be able to run an accurate NES emulator in the first place. Nevertheless, technology moves forward and we must as well. We cannot remain bound to overly slow old technology forever.

[B00daW]

You misunderstand. I'm talking about the Nintendo itself. A flash cart and the Nintendo would not be able to process an XML file. The point to ROM or image formats is to be a backup playable on hardware. The PowerPak is basically the enthusiast standard for playing NES/Famicom backups at the moment.

Nestopia is not the least accurate but it's also not regularly updated. Nintendulator is the most accurate NES/Famicom emulator.

I'll add some more comments from the NESdev community itself.

http://forums.nesdev.com/viewtopic.php?f=2&t=9328

[C. V. Reynolds]

Yes, so I did. I apologize for that confusion.

I also certainly agree that Nintendulator is the most accurate of the NES emulators as well, by the way (Nestopia is only second).

I ask, however: If one is putting a ROM image into a flash cart, would not the best option be to put them on as split PRG and CHR files? I'm not knowledgeable on flash carts, as I've never used one, but is it impossible to design them without a requirement for headers? If not, then by all means we should use headers and even upgrade to a more compatible format.

Of course, I believe that regardless, No-Intro should continue storing the hashes as it does now. Well, actually I would like the files to be split into, say, "Legend of Zelda, The (USA).chr" and "Legend of Zelda, The (USA).prg", but that's for another debate, I suppose.

[B00daW]

The current NES/Famicom flash cart PowerPak even supports FDS images! It runs FDS and NES/Famicom software by reading the headers and assigning the mapping logic from its database of Verilog coded mappers. A ~1.74MHz processor can easily read 1KB of information and then work out which information to feed to its FPGA that practically emulates mapping hardware on hardware. If there was no header only the first bank of PRG would be loaded. Within that bank there could be an instruction to bankswitch that could never be completed because for all intents and purposes there is no mapping hardware. Something needs to be loaded FIRST in order for the NES/Famicom to understand how to map its memory and expansion registers. There is no other known console other than the NES/Famicom that absolutely needs a header in order to function.

Regarding iNES 1.0. It needs to be deprecated. I am also a dumper and have a CopyNES. Since practically all known mapper numbers are taken, I have stacks of Brazililan, Taiwanese, and Russian unlicensed carts that I am reluctant to dump since they have no mapper number to assign to their PCBs' combination of logic ICs. NES 2.0 is a future-proof format that also takes into consideration the Vs. Unisystem and Vs. Dualsystem. I would be able to reverse engineer mappers and dump new games until there are practically no new carts to dump whatsoever.

Also, unrelated, but if you are in for preservation and dumping. One format you may be interested in getting rid off is the FDS format. Instead of a disk image format it is a disk rip format. It is an archived collection of files instead of an entire image of the disk itself. I haven't come across a disk that uses copy-protection yet, but there may be non-file information written on particular sectors of the disk that is necessary for the disk reader to read.

Getting back on topic. I think No-Intro's NES/Famicom set could be the vehicle needed to launch the NES 2.0 format. It was developed by kevtris (author of the NSF format), Quietust (author of Nintendulator), and supported by tepples (administrator of NESdev forums.) NES 2.0 already works on hardware as well. kevtris has an FPGA console that emulates and runs NES 2.0 formatted backups perfectly fine.

[BigFred]

It is debatable what you consider being the real use of ROM-images. The original hardware will die just as the games so it cannot be the final answer in terms of preservation. Imo an all-software solution is - sadly to be honest - the only way to guarantee an eternal storage for future generations unless you find a way to re-produce the hardware forever. But I do not want to start another point- and endless discussion and I agree the original hardware cannot be ignored as this is the way the games were meant to be played. So my final verdict: DoM already features a lot of options to satisfy individual needs and so I guess it is the best compromise if it will contain all info required to present the right dat for every taste. This means you will have the option to choose between headerless split format or iNES (2.0) or even more obscure formats.

[xuom2]

yeah! im here for the database support.

[KingMike]

From what I have heard is that FDS could only read/write files sequentially.
Which would suggest that the only copy-protection could be to include "hidden files" stored right after the last "non-hidden" file (which was as simple as say, state there's 12 files in the disk header, but put a 13th file after the 12th).
I recall someone on another forum saying that had some pirate disk intended to hack graphics on original FDS disks, but saying it would end up corrupting everything after the edited file on the disk if the edited file was not the last file.