N64 File Format

[Buddybenj]

No-Intro currently uses a byte swapped file format for the N64. However, the developer of CEN64, an emulator which attempts to emulate the N64 as accurately as possible (cycle-accurate), has said that the N64 uses big endian internally, so that's the only ROM format it supports (i.e. big endian ROMs are the most "accurate"). If No-Intro is about preserving ROMs as closely as possible to the actual hardware, shouldn't No-Intro use big endian for N64 ROMs?

[Tauwasser]

Please see this thread.

Basically, it boils down to:

• 16bit-only ROMs don't have Endianness, since all accesses are word-oriented
• mixed 8bit/16bit ROMs do have Endianness

For mixed 8bit/16bit ROMs pinout is mostly compatible between manufacturers (JEDEC), but Endianness not always is. Most manufacturers do not expect you to switch between byte and word mode at runtime. Thus, you will find information regarding Endianness only on rare occasions.

The idea of no-intro is to preserve the ROM, not necessarily the ROM in the Endianness of the processor. Some processors can even switch Endianness at runtime, which diminishes that meaning even further... Also, programming plays a big part, too. You'd mostly want an image that you can program without modifying it, though I personally don't see that as a big hurdle.

cYa,

Tauwasser

[Buddybenj]

There has been a new discussion on the CEN64 forums about the proper N64 byteorder. I'd like to hear what other people think about MarathonMan's (author of CEN64) opinion on No-Intro's choice to use byteswapped ROMs.

http://forums.cen64.com/viewtopic.php?f=9&t=203

[C. V. Reynolds]

What makes byteswapped ROM images more "correct", though? Does the N64 switch the endianness in the processor? When you open a ROM mage in a hex editor, it is immediately clear that the big endian format is the coherent one, anyway. No-Intro seems to use the something equivalent to big endian for the Megadrive/Genesis dat as well. Is that incorrect or does the Genesis actually handle things differently?

At the least, having an option on the Datomatic would be great. Choose one or the other between byteswapped and big endian.

[coraz]

I feel like the format change was rushed and reckless. MAME uses byte-swapped format and it would have been more clever to contact mamedevs about it first, since MAME tends to use 1:1 accurate format. Mooglyguy is the mister n64 of the the mame team, I would have asked his opinion first before doing any change, what if cen64 is wrong and we have to revert the whole set back and forth ? That would be terrible for all no-intro users.

[KingMike]

Trying to follow that discussion immediately gives me a headache, when they refer to files by extension.
Which is arbitrary.
Like Genesis. Doesn't the set currently "md", or was it "gen"? I know "md" was originally used for some really old, obsolete format so it's probably fair enough to reuse since people have probably even forgot about it. Though the "SMD" and "BIN" names remained in constant use (despite that ROMs found on the Internet were regularly renamed anyways). Long after the interleaved ("SMD") format had hopefully disappeared, since it was unnecessary for most users and as I recall, scrambled enough to make it unusable for ROM hacks/translations anyways.

[coraz]

Trying to follow that discussion immediately gives me a headache, when they refer to files by extension.
Which is arbitrary.
Like Genesis. Doesn't the set currently "md", or was it "gen"? I know "md" was originally used for some really old, obsolete format so it's probably fair enough to reuse since people have probably even forgot about it. Though the "SMD" and "BIN" names remained in constant use (despite that ROMs found on the Internet were regularly renamed anyways). Long after the interleaved ("SMD") format had hopefully disappeared, since it was unnecessary for most users and as I recall, scrambled enough to make it unusable for ROM hacks/translations anyways.

This is not a mere extension problem, nointro will always use the .n64 extension regardless of the format.
The issue here is the byte order. cen64 author claim the roms are not byte swapped, fair enough, but I would have tried to get a confirmation before rushing a format change for a whole set, because the mame project which is reputed for its accuracy, still uses byte swapped format. I think mooglyguy, the main author of the cycle accurate n64 driver, is idling on the mamedev irc channel, at freenode. Try to contact him about this issue. I would trust his word over anyone else.

[C. V. Reynolds]

Not everyone reputes MAME for accuracy in all matters.

I think it's worth noting that No-Intro currently uses z64 as the extension for N64 files rather than having the extension be n64. I think we should have the extension be N64 on big endian and byteswapped both, though.

[coraz]

Not everyone reputes MAME for accuracy in all matters.

I think it's worth noting that No-Intro currently uses z64 as the extension for N64 files rather than having the extension be n64. I think we should have the extension be N64 on big endian and byteswapped both, though.

friend, you've done enough rushed changes, it's time to document yourself before doing any more stupid things
using .z64 is a violation of the nointro extension policy
and most of all do as I say and contact the mamedev mooglyguy
he knows more about the n64 hardware than you ever will, stop doing things you don't understand anything about, nointro deserves better standards

sorry to put it so bluntly but this is a serious issue we have here, it's time you understand you need to know what you're doing, especially when it has consequences on the storage of hundreds people

[Buddybenj]

Not everyone reputes MAME for accuracy in all matters.

I think it's worth noting that No-Intro currently uses z64 as the extension for N64 files rather than having the extension be n64. I think we should have the extension be N64 on big endian and byteswapped both, though.

friend, you've done enough rushed changes, it's time to document yourself before doing any more stupid things
using .z64 is a violation of the nointro extension policy
and most of all do as I say and contact the mamedev mooglyguy
he knows more about the n64 hardware than you ever will, stop doing things you don't understand anything about, nointro deserves better standards

sorry to put it so bluntly but this is a serious issue we have here, it's time you understand you need to know what you're doing, especially when it has consequences on the storage of hundreds people

Is there any place where mooglyguy has documented why he believes byteswapped ROMs are more accurate?

[KingMike]

friend, you've done enough rushed changes, it's time to document yourself before doing any more stupid things
using .z64 is a violation of the nointro extension policy
and most of all do as I say and contact the mamedev mooglyguy
he knows more about the n64 hardware than you ever will, stop doing things you don't understand anything about, nointro deserves better standards

sorry to put it so bluntly but this is a serious issue we have here, it's time you understand you need to know what you're doing, especially when it has consequences on the storage of hundreds people

No matter how much of an "expert" one guy is, it seems a rash decision to make based solely on one person's opinion. Especially for a subject that clearly has been so heavily debated for years.

And I've also read things about MAME development in the past and how committed they are to accuracy. I heard of them not making changes to drivers that were already "good enough". (as much as the fact they keep redefining what "good" ROM sets are would suggest a concern for accuracy)

[Tauwasser]

This post would suggest that he doesn't care about the format as long as the contents match.

MAME/MESS recognizes two formats, see n64.cpp line 315: Little Endian and Byteswapped Big Endian.

[KingMike]

So "Big Endian" "Big Endian Byteswapped" and "Little Endian" are basically Big Endian, 16-bit Little Endian and 32-bit Little Endian. That is a bit clearer.

[Tauwasser]

Big Endian and Little Endian are each 32-bit, but the Big Endian Byteswap is 32-bit BE stored as two 16-bit LE values in BE order. Saying 16-bit LE is ambiguous, because you don't know if the high-order bits or the low-order bits of a 32-bit word are first. It's 16-bit LE with high-order bits of a 32-bit word stored at lower addresses.

Code: Select all

32-bit Big Endian:

Address  Data
   A0     D3
   A1     D2
   A2     D1
   A3     D0

32-bit Little Endian:

Address  Data
   A0     D0
   A1     D1
   A2     D2
   A3     D3

32-bit Big Endian stored in 2x 16-bit Little Endian:

Address  Data
   A0     D2
   A1     D3
   A2     D0
   A3     D1

[C. V. Reynolds]

I'm away from the forums for a few days and BOOM. As if I haven't run into enough things to anger me lately in my personal life. But it's okay. I can cope because I keep rolling on, you know?

I want to be perfectly understood, Coraz:

1: I'm not the one who called for z64. I was just saying we SHOULDN'T use z64 as an extension. z64 was just a side effect of the switch to big endian format. We can use both big endian and the n64 extension and it would be delightful.

2: I don't care about the way your expert would supposedly do things and some (like... I don't know... me) don't think MAME is as accurate as you do.

3: I worked out every one of those changes you're complaining about with Xuom over the course of weeks and months. I didn't rush into anything.

4: I'm not your "friend". You can stow the incivility.

EDIT: By the way, Zoinkity (a very knowledgeable user who hacks N64 games) says big endian is native.