Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

General No-Intro related discussions.
Locked
Fugus
Posts: 65
Joined: 11 Oct 2022 17:50

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Fugus »

Arctic Circle System wrote: 21 Nov 2022 07:19
Fugus wrote: 21 Nov 2022 05:57 My personal view is a bit different from this thread. The DATs should be split but not along those lines.

Have 1 DAT that is all the official and unliscenced games released on a system along with any beta's or prototypes on them, basically the stuff that was actually released on the system during it's lifetime.

Then have another DAT which is nothing but Aftermarket and Homebrew stuff.

The first DAT would eventually get completed and marked as such and pinned. The second DAT is one that realistically can never officially be finished as people still release stuff for them for fun.
Is there any particular reason why it should be split, though? We probably won't find every prototype and beta of every non-aftermarket game either, there's always going to be more that are undumped. I doubt the non-aftermarket dats will ever truly be complete. ~Red
The split would basically be one where the first dat would be of games that were physically released on the system within that systems lifespan and any beta's around it. Quite literally, that would be the stuff the average person looked through for the games of their childhood.

The second one would be of the homebrew stuff and games made later that only those who dabble in homebrew and hacks cared about. Basically similar to Hiccup's talk about removing the digital only Homebrew.

And as Hiccup said, not trying to remove anything from the datomatic, just having a split between the games that actually came out for the system or were planned for it during the systems lifetime and the ones that were homebrew or came out after the system had died.
User avatar
Arctic Circle System
Datter
Posts: 64
Joined: 21 May 2020 04:43

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Arctic Circle System »

Fugus wrote: 21 Nov 2022 21:06
Arctic Circle System wrote: 21 Nov 2022 07:19
Fugus wrote: 21 Nov 2022 05:57 My personal view is a bit different from this thread. The DATs should be split but not along those lines.

Have 1 DAT that is all the official and unliscenced games released on a system along with any beta's or prototypes on them, basically the stuff that was actually released on the system during it's lifetime.

Then have another DAT which is nothing but Aftermarket and Homebrew stuff.

The first DAT would eventually get completed and marked as such and pinned. The second DAT is one that realistically can never officially be finished as people still release stuff for them for fun.
Is there any particular reason why it should be split, though? We probably won't find every prototype and beta of every non-aftermarket game either, there's always going to be more that are undumped. I doubt the non-aftermarket dats will ever truly be complete. ~Red
The split would basically be one where the first dat would be of games that were physically released on the system within that systems lifespan and any beta's around it. Quite literally, that would be the stuff the average person looked through for the games of their childhood.

The second one would be of the homebrew stuff and games made later that only those who dabble in homebrew and hacks cared about. Basically similar to Hiccup's talk about removing the digital only Homebrew.

And as Hiccup said, not trying to remove anything from the datomatic, just having a split between the games that actually came out for the system or were planned for it during the systems lifetime and the ones that were homebrew or came out after the system had died.
But is there any particular reason why that's necessary? ~Red
User avatar
Arctic Circle System
Datter
Posts: 64
Joined: 21 May 2020 04:43

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Arctic Circle System »

Aftermarket/non-aftermarket isn't an easy split for some systems either. A system's EoL is dependent on region. The Sega Master System lasted a lot longer in Brazil than in other regions. I'd also like to ask if Star Fox 2 would be considered an aftermarket game for the purpose of this split, as well as why or why not. ~Red
Fugus
Posts: 65
Joined: 11 Oct 2022 17:50

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Fugus »

Arctic Circle System wrote: 22 Nov 2022 03:58 Aftermarket/non-aftermarket isn't an easy split for some systems either. A system's EoL is dependent on region. The Sega Master System lasted a lot longer in Brazil than in other regions. I'd also like to ask if Star Fox 2 would be considered an aftermarket game for the purpose of this split, as well as why or why not. ~Red
Ah, with regard to your reference to Star Fox 2, that game would have been a game that was created by an official company and would be treated similar to the other virtual console games and included with the original games.

I understand your concern about your thing about varying EOL depending on regions, that would be a hard one to crack.

My main way of meaning was that the games that were actually designed by the companies, whether officially or unofficially, to physically be released on the system, whether it be an unfinished beta, prototype or full release would be considered part of the first database. This would be the main collection that most people would use when they wanted to play the games they played or heard about when the system was active.

The second one would be games that had no physical release or intended physical release within the lifetimes of the systems and consisted only of the games that earned the labels of "Homebrew" or "Aftermarket" assuming I am not missing a label off the top of my head.

As for why it was necessary, even splitting them into licensed and unlicensed it not necessary if you wanted to split hairs with it, but it would make them much easier for users to look into it and go.

The only games that were released after the end of life that actually would go with the first database would be stuff officially released like stuff they put on Nintendo's Virtual Console.

The question shouldn't be "Is it necessary", the question is "Does it make it easier to sort and identify for the end user without being harder on those maintaining them.

Using the original Nintendo for instance
- Would have a old DAT of the original games that rarely needs to get updated as it gets completed as most of the work is done with fewer and fewer new things to add. Most people who actually to collect the games of their past would be looking for these.
-Then would have a second DAT that is all the homebrew and aftermarket stuff that still gets created on a regular basis the main people looking for this is the players wanting to see what others have made since and also those active playing hacked games most likely.

From an end-user perspective that would make it much easier to sort and fewer updating every time they see a new updated DAT if they are only focused on one of those sets, for those who are building them, the DAT files gets a smaller database they need to deal with the more active set while the older sets are more stable past naming conventions.


Edit: I am really sleepy and I feel like I am making a million spelling mistakes. So will just put what I am talking about with a short version here

Dat 1: Official and Unofficial games along with its Beta's, Prototypes, Demos, and Virtual Console counterparts. Basically all the parts the industries were putting out that are pretty well set short of stuff like Nintendo's Virtual Console retroactively adding games to their systems themselves.

Dat 2: All the stuff that gets labeled as "HomeBrew" or "Aftermarket". All the stuff that isn't made by the overall industries and is still having new stuff made.
User avatar
xuom2
High Council
Posts: 938
Joined: 22 May 2008 18:45

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by xuom2 »

no-intro after all these years of discussions has still no decent shared dictionary to identify what's homebrew/pirate/aftermarket, but considering the flags used in DB, we have 29 systems that include homebrew/pirate/aftermarket archives.

the split would create 29 new datfiles with a prefix similar to "Homebrew - xxx".

if more dozens of datsets are not a problem for final users, I have no problems with splitting. I think that download filters are the best solution, but I don't care and I will follow the consensus.
I also consider that homebrew stuff has nothing to do with no-intro (but I agree of having them in DB, by default not to be exported to DAT).

details in the attachments.
You do not have the required permissions to view the files attached to this post.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Hiccup »

Flashfire42 wrote: 07 Nov 2022 10:27
Arctic Circle System wrote: 08 Nov 2022 03:59
dreimer wrote: 08 Nov 2022 15:06
Madeline wrote: 08 Nov 2022 23:31
omonim2007 wrote: 10 Nov 2022 13:40
BigFred wrote: 12 Nov 2022 14:23
sCZther wrote: 13 Nov 2022 00:37
Does anyone have an objection to moving all the stuff currently marked as "Homebrew" and/or "Aftermarket" (both types of ROMs that are pretty new to the dat, without any consensus that they should be added) to separate dats like xuom2 said (i.e. back to how it was for many years, without controversy I think. also keep in mind that new dats are added quite often, it is not alarming/a problem to add 30/- just look at the length of the list already), *if* DoM also gives the option to download combined dats? (btw, it is easy to combine them outside of DoM using sabretools or some rom managers)
Fugus
Posts: 65
Joined: 11 Oct 2022 17:50

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Fugus »

Definitely support moving Aftermarket and Homebrew to their own dat files.

Would make the dat with the original much more stable and less frequently updated for those tracking them while the more active stuff would be separate and a much smaller set for the developers here to go through making it a win all the way around.
omonim2007
Datter
Posts: 439
Joined: 20 Jul 2016 12:20

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by omonim2007 »

I am categorically against splitting sets. I still think that it is only necessary to properly adjust the ability to accurately sort records when downloading dat files.

Once again, I'm not writing this from the point of view of an outside observer who just voted and forgot, but from the point of view of someone who edits these dats and keep them in good condition.

In recent years, thousands of add-ons have been released (unlicensed in nature, but not separable from the main Parent records in the databases), and they cannot just be thrown into a separate set. These include unlicensed physical editions, semi-legal re-releases or improvements (distributed through services like GOG.som, piepacker.som, Steam, etc.). I'm not talking about the fact that a lot of old prototypes from the 1990s have been finalized and published just now. We must retain all such publications along with the licensed versions.

The main reason why we can't split sets is that we'll lose the proper Parent-Clone relationship in most dats (this was especially the case with most retro systems from the 1990s).

And, of course, in the event of a split, we will lose the semi-legal games that came out during the life of the consoles, they must definitely remain in the same place with the main licensed games.
User avatar
dreimer
Posts: 270
Joined: 14 Nov 2015 13:26

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by dreimer »

I feel quite honored that I am asked how I think about it. I did some stuff, but waaay less that most others in here. So first a big thanks from me. :D

Of course this would be a way to split at least sort of positive. The problem I see here is as omonim2007 said. We have many ROMs that are in some relation and now that I saw this explaination with Parent-Clone relationships, I think this split would be a MASSIVE task and very likely will make others unhappy again. Especially those who care about more information than just the checksums. (Out of whatever reason they need checksums for ^^). So right now I think this is not a good idea to do any splits. First the argument from xuom2 should be addressed I think. How should a proper split if any is done at all be realized if the needed terms like pirate, homebrew, aftermarket etcetcetc are still not properly defined? This will end in endless discussions aka lost resources the project may need on other locations. If you need help with that task, feel free to ask :D

As I said from the very beginning. I am no fan of any split as I see no reason to separate any ROMs for the identical system in other Dats. I stll hope for a argument for the split I can agree with. Until now I still see more problems than solutions coming up with any splitting.

Just as a lil real life example. I will get a Analogue Pocket very soon. Preparation time :D I know this device can play the originals, but there are other official ways, too. Ways where you don't need to polish the old cart contacts on a daily base ^^ So I dumped them all and threw them in a folder. I added a few homebrews from itch.io and later applied even some hacks from romhacking.net. Then I converted them to .pocket format with even more patches. Half of these steps were verified by using the GB and GBC Dats. So two Dat files. Not 4 or even more due to a split. This is how things should be IMO.

We have to decide if we want to keep things simple for the verifiers and datters out there or for the uploaders and hoarders. Sorry for the black/white split. I know many here fit in both categories, ME included. But still we should not forget the main reason for the whole project here imo.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Hiccup »

omonim2007 wrote: 30 Nov 2022 05:10 I am categorically against splitting sets. I still think that it is only necessary to properly adjust the ability to accurately sort records when downloading dat files.
Its already possible to create customised datfiles, surely you know that? But that doesn't solve the ptoblem of the default datfiles that the majority of people use being bogged down with this homebrew stuff.
omonim2007 wrote: 30 Nov 2022 05:10 Once again, I'm not writing this from the point of view of an outside observer who just voted and forgot, but from the point of view of someone who edits these dats and keep them in good condition.

In recent years, thousands of add-ons have been released (unlicensed in nature, but not separable from the main Parent records in the databases), and they cannot just be thrown into a separate set. These include unlicensed physical editions, semi-legal re-releases or improvements (distributed through services like GOG.som, piepacker.som, Steam, etc.). I'm not talking about the fact that a lot of old prototypes from the 1990s have been finalized and published just now. We must retain all such publications along with the licensed versions.

The main reason why we can't split sets is that we'll lose the proper Parent-Clone relationship in most dats (this was especially the case with most retro systems from the 1990s).

And, of course, in the event of a split, we will lose the semi-legal games that came out during the life of the consoles, they must definitely remain in the same place with the main licensed games.
For completely original homebrew stuff, there is no possible parent-clone relationship with retail stuff.

Semi-legal doesn't mean anything. For one thing this isn't about legality, its about if the platform holder licensed the releases or not. Secondly, that stuff can stay in the main dat. Thirdly, being in a separate dat doesn't "lose" anything.
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Hiccup »

dreimer wrote: 30 Nov 2022 08:59 How should a proper split if any is done at all be realized if the needed terms like pirate, homebrew, aftermarket etcetcetc are still not properly defined?
Original games that are released via small-print run carts and/or digital-only releases can go in the Homebrew dat.

dreimer wrote: 30 Nov 2022 08:59 We have to decide if we want to keep things simple for the verifiers and datters out there or for the uploaders and hoarders. Sorry for the black/white split. I know many here fit in both categories, ME included. But still we should not forget the main reason for the whole project here imo.
Splitting homebrew into a seperate dat will keep things simple for both. Also I think its disingenuous to suggest that the purpose of No-Intro is just to have hashes for people to check their own dumps against. That's one purpose, but the other main purpose is to create high-quality ROM sets. Preservation doesn't happen if all we have is hashes.
relax
High Council
Posts: 941
Joined: 27 May 2008 17:52

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by relax »

I think it's no big problem to split a dat into two parts, unlicensed and licensed ROMs, as long as all unlicensed ROMs are moved (releases from the system's lifespan, aftermarket, pirate, homebrew.) We have the flag for licensed yes/no in the database.

The only issue I see is how to handle modern re-releases of licensed ROMs. The correct flag (unlicensed and licensed) should have been set when adding those ROMs to DoM in the first place, but since the rule for this was not clear, I think this should be agreed on before a split. Virtual console and similiar should be considered as licensed IMO, But what about carts produced by companies like Limited Run Games? They have the permisson from the software copyright holder, but not from the platform owner.

Then comes the question why to to do a split. I might be a purist, but I think lately the official licensed ROMs have almost "drowned" in some sets. One example from the NES dat:

Code: Select all

GunTner (World) (ax1147) (Beta) (Build 157) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 158) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 186) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 194) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 208) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 209) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 211) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 221) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 231) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 235) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 239) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 243) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 249) (Aftermarket) (Homebrew)
GunTner (World) (ax1147) (Beta) (Build 252) (Aftermarket) (Homebrew)
GunTner (World) (Beta) (Build 132) (Aftermarket) (Homebrew)
GunTner (World) (Beta) (Build 47) (Aftermarket) (Homebrew)
GunTner (World) (Beta) (Build 53) (Aftermarket) (Homebrew)
GunTner (World) (Beta) (Build 65) (Aftermarket) (Homebrew)
GunTneR (World) (v1.2) (Demo) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 132) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 141) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 143) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 167) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 190) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 4) (Aftermarket) (Homebrew)
GunTner (World) (v2.73) (Beta) (Build 5) (Aftermarket) (Homebrew)
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Hiccup »

relax wrote: 30 Nov 2022 21:18Virtual console and similiar should be considered as licensed IMO, But what about carts produced by companies like Limited Run Games? They have the permisson from the software copyright holder, but not from the platform owner.
Well the licensed flag has traditionally been used to mean "licensed by the platform holder". Anything that isn't licensed for IP usage would be marked as "pirate" I guess - there's a flag for that, used occasionally.

----

As an aside: unlike what I said before, maybe there is no need for a separate dat, as long as the homebrew/aftermarket stuff is filtered out by default from search results and from the default dat downloads.
Fugus
Posts: 65
Joined: 11 Oct 2022 17:50

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Fugus »

Just thought of a decent compromise on here between the issues.

Just have the Dat-O-Matic start to include preset options at the top which checks the appropriate flags for you and appends that preset to the Dat title so the manager doesn't try to overwrite different versions of it.

Like the NES could have these options at the top

1) Preset: Official/Unofficial/Pirate only.
2) Preset: Aftermarket/Homebrew only.
3) Preset: Complete Collection


Also, I know I brought it up before but if you included the "Superseded" tag could include a "One Game/One Rom per selected Regions" option. But also got ReTool for that for now.


Edit: Nevermind, just started thinking and that is effectively what the Dat-O-Matic already is.
User avatar
Arctic Circle System
Datter
Posts: 64
Joined: 21 May 2020 04:43

Re: Poll: Should No-Intro split unlicensed ROMs into separate DATs from licensed ROMs?

Post by Arctic Circle System »

Hiccup wrote: 01 Dec 2022 04:28
relax wrote: 30 Nov 2022 21:18Virtual console and similiar should be considered as licensed IMO, But what about carts produced by companies like Limited Run Games? They have the permisson from the software copyright holder, but not from the platform owner.
Well the licensed flag has traditionally been used to mean "licensed by the platform holder". Anything that isn't licensed for IP usage would be marked as "pirate" I guess - there's a flag for that, used occasionally.

----

As an aside: unlike what I said before, maybe there is no need for a separate dat, as long as the homebrew/aftermarket stuff is filtered out by default from search results and from the default dat downloads.
As mentioned, weren't there some cases where certain games that started out as unlicensed titles were later licensed by the console manufacturer without creating a new ROM revision? How should that be handled? ~Red
Locked