datfile default options

Post bug reports and suggestions for the website, forums and DOM here.
Post Reply
User avatar
xuom2
High Council
Posts: 926
Joined: 22 May 2008 18:45

datfile default options

Post by xuom2 »

No-Intro was born to list commercial games, and it is good practice adding homebrew and unlincesed as "notable exceptions".

We are now in an opposite situation where we think: "why homebrew XX is not listed when YY is listed?", "why pirate XX is not ok when bootleg YY is ok?"

Separating such archives in different datfiles would be an idea, but are we ready for another dozen of datfiles, named with category "Homebrew and Unlicensed"? I don't think prolification of datfiles is the solution, because this implies that we care how "collectors" keep data on disk (in 1 datfile = 1 directory, I suppose), and I honestly think this job is for rommanagers and something that users have to setup as they want. If you like subdirs with game regions, I can not do a datfile for every region.

I think the filters (those enabled by default for the daily pack) should more respect the initial goal:

Release: "All" , "Fulltitle" , "Proto/Beta/Demo/..." by default "All": won't change, because protos and demos of licensed titles are simply a different "versions". I also suggest a better English flag for the second situation in the GUI ("Unfinished" or "In development"?).

License: "Licensed", "Unlicensed", "Both" by default "Both", should be "Licensed": we should return to our rule of "good archive selectors", and this won't block datters to send any unlicensed or homebrew to db, and users to download datfiles with different options. I can live with a datfile that excludes Action Replay or 200in1 games carts: I think they have some historic reason to be in, but I also think they can be excluded by default.

Life span: "Machine life span", "Aftermarket", "Both" by default "Both", should be "Machine life span", again losing some history around, still data will be in db and available as custom download option.

I hope we will find some common agreement on this because, once done, DOM will have to do tons of batch jobs XD
omonim2007
Datter
Posts: 437
Joined: 20 Jul 2016 12:20

Re: datfile default options

Post by omonim2007 »

As for me, all custom wishes should be implemented at the level of ROM-managers. If some of the collectors are not satisfied with the current look of the databases on the hard disk or in the Cloud, then they can contact the ROM-manager developer and ask them to implement the appropriate features.

There is no need to expand the list of systems and further complicate navigation among them. The current default settings are suitable for most users. And if someone needs special versions of databases, they can go to a certain section and download a database file with a given set of attributes (for example, only licensed games).

The current scheme of work of DoM suits me quite well.
User avatar
xuom2
High Council
Posts: 926
Joined: 22 May 2008 18:45

Re: datfile default options

Post by xuom2 »

True. The best is letting rommanagers do the "datomatic filter" job.
I also think that setting default filters better defines "our project identity" (if we have one).
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: datfile default options

Post by Hiccup »

xuom2 wrote: 03 Oct 2022 07:56 Separating such archives in different datfiles would be an idea, but are we ready for another dozen of datfiles, named with category "Homebrew and Unlicensed"
If they are in are in a separate "DAT grouping" in DoM (like Non-Redump, Source Code, Unofficial, Non-Game), I think it would be fine.
xuom2 wrote: 03 Oct 2022 07:56 I don't think prolification of datfiles is the solution, because this implies that we care how "collectors" keep data on disk (in 1 datfile = 1 directory, I suppose), and I honestly think this job is for rommanagers and something that users have to setup as they want. If you like subdirs with game regions, I can not do a datfile for every region.
While we don't want to restrict how people keep data on the disk, currently ROM managers cannot filter stuff easily, because the fields aren't in the datfiles. Filtering based on datfile names is a hacky way to do it, and comes with quite a few issues (as the author of retool can attest to).
xuom2 wrote: 03 Oct 2022 07:56 I think the filters (those enabled by default for the daily pack) should more respect the initial goal:

Release: "All" , "Fulltitle" , "Proto/Beta/Demo/..." by default "All": won't change, because protos and demos of licensed titles are simply a different "versions". I also suggest a better English flag for the second situation in the GUI ("Unfinished" or "In development"?).

License: "Licensed", "Unlicensed", "Both" by default "Both", should be "Licensed": we should return to our rule of "good archive selectors", and this won't block datters to send any unlicensed or homebrew to db, and users to download datfiles with different options. I can live with a datfile that excludes Action Replay or 200in1 games carts: I think they have some historic reason to be in, but I also think they can be excluded by default.

Life span: "Machine life span", "Aftermarket", "Both" by default "Both", should be "Machine life span", again losing some history around, still data will be in db and available as custom download option.

I hope we will find some common agreement on this because, once done, DOM will have to do tons of batch jobs XD
The issue with excluding this stuff by default is that then most people won't even know these entries exist.

Edit: another problem is its hard to keep track of the edits to the licensed stuff in the database with the News page, with all the unlicensed stuff. So either a filter for news (maybe same filter on the dat download page) or seperate dats would solve that problem.
johnsanc
Posts: 1
Joined: 07 May 2015 00:33

Re: datfile default options

Post by johnsanc »

My 2 pesos:
- Include all dumps in the default DATs
- Work toward a modernized DAT format that includes more structured data (basically most of the info in the DB files). Collaborate with rom manager and other DAT tool authors on this.
- Let rom managers and other tools like retool do client-side filtering, region / language prioritization, etc. No need for DoM to be in this business if all the info required is in the DATs.

Yes I am aware rom manager support for advanced filtering is somewhat limited, but that will eventually change once the dats contain more structured data. Retool was created to fill this gap
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: datfile default options

Post by Hiccup »

Yeah, what johnsanc makes sense to me. Although if we do have unlicensed stuff in the main dats, I want a way to filter it out from the News page. :P
Flashfire42
Posts: 45
Joined: 25 Feb 2020 05:19

Re: datfile default options

Post by Flashfire42 »

Physical Cartridges should take priority and if something is sold then it should be added. Free Homebrews should be added if they are unique games and not patches for hacks. There are more than enough ways to filter stuff out if you don't like it. Pirate/Bootleg stuff should be added if it is dumped from a physical cartridge that was sold. New tags were suggested for No-intro a while ago somewhere in the forum but I can not recall where. We need to clearly define what is a pirate cart, what is a homebrew and what is an aftermarket release. As someone who is a big fan of bootleg cartridges I am happy to consult on these decisions.
User avatar
xuom2
High Council
Posts: 926
Joined: 22 May 2008 18:45

Re: datfile default options

Post by xuom2 »

Hiccup wrote: 03 Oct 2022 19:16 I want a way to filter it out from the News page. :P
I know, this is a bit more difficult than expected
CoreyEMTP
Posts: 2
Joined: 18 Oct 2020 08:40

Re: datfile default options

Post by CoreyEMTP »

I thought I'd throw in a long-time user perspective (I'm old). Hopefully it helps. All content is IMHO.

First, johnsanc's points were spot-on, and something I agree needs to be done across the emulation and preservation scene. The DAT format and TOSEC naming conventions are long-antiquated and in need of serious revision (TOSEC would disagree). I know there have been various attempts to improve them, but I believe nothing will stick until there's more collaboration, not just between the various big groups but also with input from the users.

I myself was the biggest hoarder of crap for decades beginning with my Apple //e days. That version of me absolutely wanted every "[ b ]" ROM you could throw my way. I have since gone to the school of thought that not all data has value. That was hard for me, because I dreaded the loss of any information to the ethos, which I think is a common driver for the behavior. What this verbosely gets me to is that my perspective is one that should be available, because I know for a fact I'm not the only one with the same desires: There must be some solution that makes it possible if not trivial to identify the 1G1R sets from within a ROM manager, without the need for special tools and heavy manual fiddling like is done in the amazing Retool. There are many other perspectives that deserve the same accessibility, and I believe the metadata is the absolute key and foundation.

So, right now I would say I'm incredibly grateful to have all these amazing tools at my disposal, but still frustrated at the extreme difficulty in getting to my desired end result. It's no criticism of the people who give up their time for the benefit of all, not in the slightest, just the lack of standardization.

Thank you all for your incredible work!
Hiccup
Datter
Posts: 1720
Joined: 09 Oct 2015 11:29

Re: datfile default options

Post by Hiccup »

I think the easiest solution is just to add the licensing flags to the datfile itself as xml fields.
Post Reply