Check the higans changelog

General No-Intro related discussions.
Post Reply
Collecter
Posts: 128
Joined: 25 Nov 2016 17:09

Check the higans changelog

Post by Collecter »

This may be interesting there are mentions of no intro and bios

http://www.emucr.com/2017/08/higan-v104r07.html?m=1
User avatar
C. V. Reynolds
Datter
Posts: 267
Joined: 17 Jun 2009 04:42

Re: Check the higans changelog

Post by C. V. Reynolds »

I have been strongly considering packing the DSP files with the games that use them (now that the DOM has such an ability). But I haven't yet decided how to name the files. For example, if DSP1 is packed with Super Mario Kart in one folder (or zip file as the case may be), how would we name it? Should it be named DSP1 still or should it be something like Super Mario Kart (USA).bin or Super Mario Kart (USA).dsp? Regardless of the choice arrived at, I'm not specifically looking for compatibility with higan or any other emulator for that matter.
User avatar
Tauwasser
Datter
Posts: 234
Joined: 04 Oct 2010 06:51

Re: Check the higans changelog

Post by Tauwasser »

The better solution is probably to split the DSP roms in their respective entries into the program and data roms and then link the binlists for the relevant SNES game entries. However, I think right now the interface does not yet allow people redumping games to include only a verification redump of individual files. Also, any edit to the files in the dsp bin list would carry over to all dsp games as well as the stand-alone dsp entry -- which is not desirable.

However, regardless of that, packing the files into a single zip doesn't help right now for higan at lest, because that's exactly the functionality byuu has removed right now to force merged roms. :|
User avatar
C. V. Reynolds
Datter
Posts: 267
Joined: 17 Jun 2009 04:42

Re: Check the higans changelog

Post by C. V. Reynolds »

Your idea of splitting DSP roms into "data" and and "program" is of interest to me. I'm still learning and I'm interested in knowing how these DSPs are constructed.

Certainly it's a problem how we must verify our game data file-by-file. And it's definitely a complex situation that current methods can't handle. We can't even do appended ROMs if we wanted to right now because of it.

It's a shame my proposal won't help higan, though. All I can say is people should use a different emulator (there are some good bsnes forks) or keep different sets. It's Byuu's project and he can and will do what he wants with it. And I'll be doing the same. Merged ROMs won't happen on my watch, I'm sorry to have to say. It's too inaccurate for me to allow and I want to keep the project focused on accuracy. It's bad enough that the NES section is CHR and PRG together. I am not opposed, however, to having an alternate set that contains the "merged" form games.

EDIT: I was overly harsh and frustrated when I wrote this. I'm no longer interested in blocking anything. I think we all could reach consensus and perhaps come up with a solution to benefit everyone.
Last edited by C. V. Reynolds on 20 Sep 2017 14:28, edited 2 times in total.
User avatar
C. V. Reynolds
Datter
Posts: 267
Joined: 17 Jun 2009 04:42

Re: Check the higans changelog

Post by C. V. Reynolds »

Oh, and I think the time may approach that we must discuss Famicom Disk System as well. Currently we have games that are "Side A" and "Side B" together. It'll break probably every emulator to split them, but perhaps we could think about it? If people really wanted to play the things, they could put the sides together themselves. Sorry if that sounded wrong or something.
User avatar
Tauwasser
Datter
Posts: 234
Joined: 04 Oct 2010 06:51

Re: Check the higans changelog

Post by Tauwasser »

The DSPs are actually NEC µPD7720 ICs, datasheet here. They contain a program ROM and a data rom meant for storing constant tables (like sine tables for instance). Their address space is completely separate, no execution of the data rom, no reading data from program rom. Program rom code accesses data rom through an additional register that provides the data rom with the read address.

I'm also against maintaining a DAT with appended firmware, as that's not a practical approach and might get out of hand quickly. What we would need for redump purposes is a general way to mark an entry consisting of N files and redumping/verifying only some of those files. Maybe we could for now use the binlist feature and just reference the correct dsp binlist, but with the drawback that any edit done to a game entry would cascade through to the actual dsp binlist :-/

The real question is if we should transition other sets to split format as well even though there is currently no emulator support. NES is a prime example. I assume that some of the gazillion mapper configurations for NES are precisely only tracking the size of PRG and CHR rom in binary blobs and not doing much else. But NES is in a horrible state as far as documentation of cartridges goes anyway despite bootgod's best efforts. A nice set of schematic documents for each cartridge type would do leaps and bounds there, I feel.
User avatar
C. V. Reynolds
Datter
Posts: 267
Joined: 17 Jun 2009 04:42

Re: Check the higans changelog

Post by C. V. Reynolds »

Thanks. :3 The DSP stuff is fascinating but confounds me. Regardless, I'm pro "split" files, and so the DSP files should be no exception.

As flustered as I can get with the idea that No-Intro is the "evil empire" or such every time we opt to not flip the table over, I must admit the pros of all sides. One thing I think is certainly true is that users can't tell currently which DSP is meant to go with which game. Until the time we can work out the situation, perhaps it would be good to tag any special chip game with a tag describing which special chip it needs. I don't think this was originally my idea, but I can't remember whose idea it was. ;S

Something else I've been considering is removing the "BIOS" bit from the front of the DSP ROMs. They aren't actually BIOS and it would be better to use "DSP" something similar in the "additional" tag instead.
Screwtape
Posts: 34
Joined: 29 Aug 2017 08:46

Re: Check the higans changelog

Post by Screwtape »

> I have been strongly considering packing the DSP files with the games that use them (now that the DOM has such an ability). But I haven't yet decided how to name the files.

I'm not familiar with the technical and policy constraints of such things, but if you have a model with a .zip containing all the relevant parts of a game, it seems like you might as well give each file a descriptive name, instead of limiting yourself to just changing the extension. Feel free to come up with your own naming scheme, but higan's firmware naming is consistent and straight-forward, and may be useful as a starting point.

> It's a shame my proposal won't help higan, though.

byuu said (in the changelog linked in the OP) "If and when No-Intro deploys a method I can actually use, I give you all my word I will give it a fair shot and if it's reasonable, I'll support it in icarus." So I wouldn't worry too much about current or legacy versions of higan.

I believe at the moment byuu is most concerned about finding firmware files at import time: apart from the current firmware files higan expects, there have also been byte-swapped versions, concatenated versions, compressed versions and different filename conventions, which add up to a lot of combinations to check for, especially when you might be looking at a directory with ~4000 ZIP files, any of which might be the firmware archive you want under some new naming convention you haven't seen before.

> One thing I think is certainly true is that users can't tell currently which DSP is meant to go with which game. Until the time we can work out the situation, perhaps it would be good to tag any special chip game with a tag describing which special chip it needs.

Wikipedia lists all the special-chip games, and emulators can guess* which chips a game uses by inspecting the game's internal header, so I'm not sure there's a pressing need for that information, at least for games in the No-Intro set.

*: Emulators can tell when a licensed game uses the DSP1, but there's no marker for DSP1/1A versus DSP1B. Most games work fine with either firmware (and I think some games shipped with different firmware in different manufacturing runs), but Pilotwings is known to misbehave with DSP1B and Ballz 3D is known to misbehave with DSP1A, so a long-term solution needs to record that distinction somewhere.
FitzRoy
Posts: 40
Joined: 30 Aug 2017 09:10

Re: Check the higans changelog

Post by FitzRoy »

C. V. Reynolds wrote:It's a shame my proposal won't help higan, though. All I can say is people should use a different emulator (there are some good bsnes forks) or keep different sets. It's Byuu's project and he can and will do what he wants with it. And I'll be doing the same. Merged ROMs won't happen on my watch, I'm sorry to have to say. It's too inaccurate for me to allow and I want to keep the project focused on accuracy. It's bad enough that the NES section is CHR and PRG together. I am not opposed, however, to having an alternate set that contains the "merged" form games.
There's certainly a lot of chicken-egg situations going on right now. From what I've read, No-Intro very badly wants to split PRG and CHR, but they also want their roms to be playable and NES emulator authors haven't shown much interest in adding support and phasing out ines. A lot of them are developmentally dead, too, including fairly advanced ones like Nestopia.

byuu wants to, but he also doesn't want to use ines. And coming up with a database that doesn't use ines #s takes a lot of work, even for just licensed stuff. byuu got through most of the named boards, but once he got to unnamed weird stuff he burned out of it. And, at the time, he wanted to do romside mapping for everything, though he appears to be recognizing the many pitfalls of that. If old bad dumps persisting in distribution is bad, then old bad mappings floating around is a hundred times worse.

I'm confident that when higan finishes this, we'll be able to convice at least two other emulators to get onboard. And that might be good enough for no-intro to switch because they can tell the angry mobs that they have a few good options for playing those files.
Screwtape
Posts: 34
Joined: 29 Aug 2017 08:46

Re: Check the higans changelog

Post by Screwtape »

I don't want to seem demanding, and I realise No-Intro members are volunteers and the project has a variety of concerns to prioritise, but has there been any consensus on the firmware-bundling question? Is there anything I can do or information I could provide that would help things along?

If not, then fair enough—but I'm keen to see a resolution, and I'd hate to see this issue fall off the radar.
Screwtape
Posts: 34
Joined: 29 Aug 2017 08:46

Re: Check the higans changelog

Post by Screwtape »

I just thought of another game potentially affected by this decision: PowerFest '94. Currently, it's listed as four separate games in the No-Intro SNES DAT, but I don't think the ROMs can be used individually.
User avatar
C. V. Reynolds
Datter
Posts: 267
Joined: 17 Jun 2009 04:42

Re: Check the higans changelog

Post by C. V. Reynolds »

No consensus on the firmware bundling thing. I'd have done it already if I thought it possible. Unfortunately, our verification system would insist on every DSP being dumped right along with the ROM they come with at the moment. Our data system can't handle it now. The only compromise I could think of would be a joined ROM+DSP entry separate from the sole ROM entries. Something akin to the Sonic & Knuckles + other Sonic game setup in the Genesis/Megadrive section.

I never did know what was up with the Powerfest 94 ROMs. I'm too simple to understand that stuff. ;P I'd much like them to be compiled in a sensible way. Whether that's all bundled together or all stacked as one file, I don't know. But something should be done about them regardless. If they can't be booted separately but don't come together, their preservation value is reduced. Does the SNES see each ROM as a different file or as one big chunk of data, by the way?
Screwtape
Posts: 34
Joined: 29 Aug 2017 08:46

Re: Check the higans changelog

Post by Screwtape »

Thanks for the update! Personally, I'm happy to wait for a proper solution rather than a compromise, but again, if there's anything I can do to help, let me know.

According to byuu, the PowerFest '94 cart is or has a memory mapper vaguely like the SPC7110: not all of the data is available all the time, but you can poke at a register to make different data appear at the same locations in the memory map. I suspect the reason that it's been dumped as 4 separate files (rather than one concatenated file like SPC7110 games) is that the cartridge contains 4 separate EPROMs, and it was easier for the dumper to just stick them into an EPROM reader than it was to reverse-engineer the mapping protocol required to read the data out through the cartridge connector.

Some experimentation with the PF94 component ROMs shows that they kinda-sorta work on their own. If you boot up the Super Mario Kart variant for example, you get dumped into Mario Circuit 1 (just like you do when playing the full PF94), but when race ends, the game just freezes. Presumably it's trying to talk to the memory mapper to switch back to the Scoring module, but of course nothing happens.
User avatar
C. V. Reynolds
Datter
Posts: 267
Joined: 17 Jun 2009 04:42

Re: Check the higans changelog

Post by C. V. Reynolds »

I'm working on the chip files some. For the new tags, is Enhancement Chip good or would DSP be better? I'm leaning toward DSP, but I don't know if that would be considered inaccurate (I think all the enhancement chips are digital signal processors?).

Also, if I split the chip files into Program and Data... should I actually call them that? Such as "DSP1 (Program)" and "DSP1 (Data)" for portions of filenames?
Screwtape
Posts: 34
Joined: 29 Aug 2017 08:46

Re: Check the higans changelog

Post by Screwtape »

"Enhancement Chip" would be fine. I'm not sure where the line is drawn between "DSP" and "general-purpose CPU", but the CX4 is a math co-processor, and the ST018 is a complete ARM CPU core so I don't think they're strictly-speaking DSPs.

I can't think of better names than "Program" and "Data", since that's what they are. In particular, the DSPx chips are Harvard-architecture, so their program ROM and data ROM are in completely separate address spaces, and it's not possible to read instructions as data or vice-versa.
Post Reply