About PSN preservation: Some tips for the future

General No-Intro related discussions.
Post Reply
diego-rbb
Datter
Posts: 106
Joined: 27 Mar 2018 10:42

About PSN preservation: Some tips for the future

Post by diego-rbb »

[Decided to post this in the public zone so other users outside No-Intro can read, but, if the moderation believes is better to move into private forums, go ahead]

Hi everyone. The reason I'm writing this post is related to the fact that I recently decided to retire definitively from videogaming preservation. At least, I'll not be active anymore. My partner AlMagno and I are stepping out due to personal reasons and the fact that we already managed to achieve most of the projects we wanted to complete in relation to Redump.org and No-Intro.

With that said, one of the few things that encouraged me to leave recently was accepting the bad management and administration for key projects like Redump or in this case, NoPayStation, projects where I've been mainly active. As no real changes will occur without skilled users able to provide strong alternatives (I've never been one), is time for me to move on as this situation has motivated a lot of internal fights and useless drama from my side. If I can't help anymore, the best is to step aside and causing more drama was just nonsense. I'm happy with all the things we've conquered these years and it's been an awesome time for this hobby, full of wonderful people.

So, all I can say is, keep rocking it and take care. :D

However, I didn't want to leave without giving a few tips in relation to No-Intro's role when it comes to PSN preservation for Sony Playstation 3 and Sony Playstation Vita. The fact that the stuff we've added to No-Intro still needs a few tweaks is still causing me itches. Sadly, a few problems from the original source needs to be addressed... Which is NoPayStation.

NoPayStation is a project that remains as a key database to secure all the existing software library from the PSN's CDN. It's main purpose relies on keeping documentation for each digital item (full game, demos, DLC, avatars, theme, etc) offering access by each Zeus link, which is the direct link to item. It comes packed into PKG format and it needs its own license file (.RAP file for PS3, .Zrif for Vita) which acts as a decryption key to extract the contents/make use of them.

As it stands, NoPayStation operates more as a Scene project rather than a preservation-aimed one. The naming is not precisely the best, which is the reason we I proposed in private to some of their key members and the main administrator to make a rebranding and establish a new name and logo... I think they thought I was trying to steal the project or get some free credit because their answers were kind of weird (better use that word than other ones...). It was a chance to write a good report about the project and promote new users and developers to work into it. Well, that was a "nope".

Up to this day, the platform is in deep bad shape. For what I've been reported from several users (and my own experience), the database is full of bugs that haven't been addressed in a long long time, their "contribution" tool is broken in different ways, preventing us to add new Zeus links in some cases or fixing/adding license files. They couldn't even update the F.A.Q. section in the last year and a half with the new method used retrieve Zeus links and they basically won't change a thing.

As you may know, after a strong fundraiser campaign, I managed to get a lot of volunteers involved to complete as far as we could the PS Vita fullset, adding contents to NoPayStation and No-Intro so joined contents between both could at least deliver the most complete library possible. Lot of Zrifs fixed by Renascene or other users for PS Vita stuff weren't added or fix.

Another problem comes from License file verification. While a batching method to verificate the available files was tasked as a future to-do feature... No real effort has been made by the NPS administration to make a deep analysis around the number of .RAP added to its database that could be fake/wrong.

I say RAPs because luckily, the Renascene admin actually did this for PS Vita and the results were superb. Only a few Zrifs were found to be broken... Inside his file database. Yeah, "his":

https://www.reddit.com/r/emulation/comm ... t_to_rule/

https://renascene.com/psv/info/card/9999

This excel containing what we could call a bible for PS Vita software library, doesn't match NoPayStation at all. While this guy has been working with some NPS mates to add/fix all the Zrifs/Zeus links he received from the community, a good bunch of documentation and files reported in his Excel hasn't been pushed to NPS servers. This leave us a bad perspective as we add all our contents from NPS, when the most correct and verified source is actually the Renascene one.

In relation to our database: We should talk about attaching .RAPs and Zrifs to each PKG file. Hiccup and I received a message from user "norkmetnoil577" which I hope he doesn't mind I share because I found it extremly important:
"Hi Hiccup and Diego,

I hope you're doing well and thanks for all you do for the project!

Ever since I was working a lot on the initial PSN dats, I had wanted to import the .rap files into the same sets as the pkg so they get preserved that way and never lost. This is for PS3 and PSP as well but since PSP security is cracked almost nobody collects the PSP raps only the PS3. For Vita it seems rap isn't used but instead zRIF.

Last time I checked there are thousands of raps which are documented in NoPayStation but not in No-Intro, and none of the zRIFS because we need to come up with a reproducible file format (rap is simply taking the key as hex to a file, but zRIFs aren't hex). It would be great to do a bulk import somehow to automate the insertion of these into the existing sets. My idea is to use the NoPayStation database where the Game ID is in both that and No-Intro Metadata and should be usable for matching.

Anyway I wanted to raise your attention to this and ask your opinion on how it can be done, and while my time is spread thin lately offer to help get it done"
This guy totally gets it and his offering is extremly important to complete the job. While it makes sense to aim for archival preservation to go for the PKG files, these needs to have the license file attached not only for "playable" reasons, but for real preservation matters.

From my side, I can't do anything else to help further on this matter but at least I can warn this community about the current situation so users able to make a difference can step in and help.

While Sony hasn't made an official statement, is clear that we can expect a "total shutdown" date the moment we least expect it. I have no faith for next-gen platforms but at least we can still save whatever we can from these.

And well, that's my final statement. I hope you guys the best as always. Please, don't take this post as a personal attack on NoPayStation. While I have my personal issues with the administrators, these facts tries to reflect problems that end up affecting our own database. I'm pretty sure my statements may not reflect correctly lot of technical aspects related to the NPS project but still, tries to provide a vision about its current status and the need to address them.

Thanks for the atention and thanks for everything guys.
XploitOcelot
Posts: 1
Joined: 30 May 2021 12:57

Re: About PSN preservation: Some tips for the future

Post by XploitOcelot »

And NOW I understand why my attemps to contribute to NoPayStation fell on deaf ears

In one side, is good to see that someone, Renascene in this case, takes the lead, but on the other one, it is quite a shame that No-Intro's DATs are not updated with such important info yet…

My question now… what can I expect, as someone avid for contributing and archiving his own PSN titles, for having all my games properly registered in No-Intro's DATs?

And sorry for my english, mates. I try 😅
Post Reply