A few questions regarding the status of PS3 PSN DLC

[layzee]

1) Are the main groups preserving/cataloging PSN content (game updates, DLC content, custom themes, avatars, etc) No-Intro and NoPayStation?

2) Just to confirm, like how games have different regions (USA, Japan, Europe, etc), the DLC for the games are also region-locked? So each DLC has its own region.

3) How complete are they? There are free DLC and paid DLC so I can't imagine it would be cheap to get everything, especially with multiple regions.

I homebrewed my PS3 and I think I'm ready to start archiving stuff before they disappear so I just want to be clear what we have and don't have. To be clear, I will be specialising in the Japan-region stuff. Please give me a better idea and understanding of where we're at in terms of paid Japanese DLC before I start throwing out the cash. I have a Japanese PSN account and I can download DLC from there without problems (even though I don't live in Japan and I am using homebrew - haven't got banned yet knock on wood). For the paid DLC, I just need to purchase a JP PSN card.

Here are some examples of what looks like gaps to me:

Hard Corps: Uprising (Konami)
https://store.playstation.com/ja-jp/pro ... RPS000GAME

The Japanese DLC is missing. Cost: 945 yen.

Castlevania: Harmony of Despair (Konami)
https://store.playstation.com/ja-jp/gri ... ME00000000

AKA Akumajou Dracula. 10 Japanese DLC missing (4 mission/stage packs, 4 characters, 2 BGM packs). Cost: About 2500 yen.

Drakengard 3 (Square Enix)
https://store.playstation.com/ja-jp/gri ... ip=add-ons

AKA Drag-on Dragoon. No-Intro's English DLC of this game seems complete, the Japanese one is incomplete though.

NieR Replicant (Square Enix)
https://store.playstation.com/ja-jp/pro ... 0000000000

Both the Europe and USA single DLC for this game is preserved. The Japan one is not. Cost: About 880 yen.

Mega Man 10 (Capcom)
https://store.playstation.com/ja-jp/pro ... 0TREBLE000

AKA Rockman. All English DLC available. Japanese DLC missing (stages, avatars, themes, etc). Cost: about 2500 yen.

Various Nihon Falcom game DLCs (Falcom)

A bunch of stuff.

Various Tales of... game DLCs (Bandai Namco)

A bunch of stuff.

These are just a few examples. Is the reason the Japanese DLC is a bit limited due to cost and/or disinterest and/or difficulty/technical issues?

[Landcross]

Are the main groups preserving/cataloging PSN content (game updates, DLC content, custom themes, avatars, etc) No-Intro and NoPayStation?

Yes and no. Kinda. No-Intro is indeed the main group with the goal to actually preserve and catalog (PSN) content. Though in the specific case of PSN stuff, interest has been very low lately and activity was pretty much non-existent for the past year or so. Until I came along

NoPayStation is a lot more active, but their primary goal is not really preserving or cataloging stuff. Sure they do that, but their data-quality is... so-so. Not to say to do bad work (especially in the Vita department), it's just different approaches with different goals.

Just to confirm, like how games have different regions (USA, Japan, Europe, etc), the DLC for the games are also region-locked? So each DLC has its own region.

I'm not entirely sure how region-locked DLC actually is (from a technical standpoint), but in most cases every piece of DLC does at least indeed has a different release in each region yes.

How complete are they? There are free DLC and paid DLC so I can't imagine it would be cheap to get everything, especially with multiple regions.

I'm not sure how complete it is, but it's certainly not complete at all. And it's not cheap (or easy) either. Just because there's just so much... Not to mention content that is not available for sale anymore and has to be traced back by finding people who bought it in the past. Or regions like Korea that are pretty much inaccessible for outsiders.

I am in the process of adding a ton of new stuff to the datfile soon, mostly (though certainly not all) from NPS, so before you go out and buy stuff it's wise to wait a little more or at least check what they have over at NPS. I hope to have the first batch added within a few weeks (though not named properly yet, just using the content-id as name).

I homebrewed my PS3 and I think I'm ready to start archiving stuff before they disappear so I just want to be clear what we have and don't have. To be clear, I will be specialising in the Japan-region stuff. Please give me a better idea and understanding of where we're at in terms of paid Japanese DLC before I start throwing out the cash.

That's cool! Japan is always a lesser active country in terms of (digital) preservation, so all the help is very much appreciated. As I said above, it's difficult to tell exactly what is and is not missing, but you're in luck because I run a (very much work in progress) database over at https://serialstation.com to catalog just that. I could write a query to check which items in my database are not yet in no-intro. Only drawback is that, for that to work properly, I need the content-ids for everything that is already in no-intro and I'm working on that, but again, it's taking time. Not to mention that it's sometimes a bit difficult to weed out what content is actually content and what stuff is 'virtual only'. E.g. a bundle or deluxe edition of a game is virtual only: there's not a 'deluxe edition' pkg file. When buying that item on the PSN store, it just results in multiple normal pkg files (e.g. the base game and DLC packages).

I did start a misslist for PSP Japan a little while ago, but work on that got sidetracked due to other stuff: https://wiki.no-intro.org/index.php?tit ... n_undumped

So, what it often boils down to above is time; I'm the bottleneck. Not because I want to, but because I'm pretty much literally the only one working on all PSN datfiles. And I started doing that just since a month or two ago, before that all those datfiles were dormant for months if not years. So, again, all help is very much appreciated.

Sidenote: just because a pkg file is in the DoM doesn't make it fully 'preserved'. Taking your Nier Replicant DLC as an example. The USA one is complete, with .pkg file and .rap file. The Europe on is not, it only has the .pkg file, not the .rap file. The .rap file is used for decryption. Having the .pkg file is the most important thing of course (who knows what we can do with more powerful computers and old encryption in the future), but having the .rap file too is of course even better.

[layzee]

New to "To Do List": Figure out how to homebrew Xbox 360 and get Lost Odyssey's DLC before it's gone for good.
Edit: Never mind, it's already in No-Intro, and the DLC is classified as (World). Nice to see the Last Remnant stuff there too.

NoPayStation is a lot more active, but their primary goal is not really preserving or cataloging stuff. Sure they do that, but their data-quality is... so-so. Not to say to do bad work (especially in the Vita department), it's just different approaches with different goals.

Could you please briefly explain the difference between No-Intro and NPS and how things are done regarding PSN DLC?

there's just so much

Even if we can't afford to buy every DLC, if we had a missing list at least, that would be a good way of seeing what the DLC "landscape" looks like. I was hoping I could ask relevant communities for donations for grabbing the missing stuff. The ideal process would be like:

1. Game X, Y, Z has missing DLC worth $80.
2. Seeking donations for a $100 PSN card.
3. Purchase PSN card and then the game's DLC.
4. Distribute DLC to relevant parties (e.g. No-Intro).
5. Repeat 1 to 4.

But if as you say, there might not be that much interest in this sort of stuff then it might not be feasible. For now, the best thing I can do with my limited funds is to buy the stuff I'm personally interested in (e.g. see games in previous post) with higher priority placed on significant content (e.g. gameplay-related DLC like extra stages or characters or unique equipment/items) and lower priority on cosmetic-based DLC (e.g. extra costumes, themes and avatars).

Not to mention content that is not available for sale anymore and has to be traced back by finding people who bought it in the past.

Case in point Konami's PT. I guess DLC for Sega Dreamcast would also count: http://dc.dreamcastlive.net/dlc.html

Would you happen to have a list of this sort of thing or could identify some other prominent examples?

Or regions like Korea that are pretty much inaccessible for outsiders.

What is the reason? IP blocking, etc?

I am in the process of adding a ton of new stuff to the datfile soon, mostly (though certainly not all) from NPS, so before you go out and buy stuff it's wise to wait a little more or at least check what they have over at NPS. I hope to have the first batch added within a few weeks (though not named properly yet, just using the content-id as name).

Please do. NPS's own interface is nice and all, but I'm more used to clrmamepro to organise my emulation related stuff.

That's cool! Japan is always a lesser active country in terms of (digital) preservation, so all the help is very much appreciated. As I said above, it's difficult to tell exactly what is and is not missing, but you're in luck because I run a (very much work in progress) database over at https://serialstation.com to catalog just that. I could write a query to check which items in my database are not yet in no-intro. Only drawback is that, for that to work properly, I need the content-ids for everything that is already in no-intro and I'm working on that, but again, it's taking time. Not to mention that it's sometimes a bit difficult to weed out what content is actually content and what stuff is 'virtual only'. E.g. a bundle or deluxe edition of a game is virtual only: there's not a 'deluxe edition' pkg file. When buying that item on the PSN store, it just results in multiple normal pkg files (e.g. the base game and DLC packages).

I'll give your website a look and maybe post back here with comments, if any.

I did start a misslist for PSP Japan a little while ago, but work on that got sidetracked due to other stuff: https://wiki.no-intro.org/index.php?tit ... n_undumped

Yes, I looked into PSP PSN a while ago and also posted here about it (for some reason, my thread got deleted or wiped without warning). I also made a post here: https://gamefaqs.gamespot.com/boards/92 ... p/77883327

If you or anyone else could kindly have a browse of the above link, what are your thoughts on the inclusion of the above files to No-Intro? Technically it's not classified as PSP PSN content but it very much acts as official DLC from an official source (i.e. Capcom). The problem however is that the files may not exist in a "lossless" or in an "untouched default" state so the checksums will be different per user.

Sidenote: just because a pkg file is in the DoM doesn't make it fully 'preserved'. Taking your Nier Replicant DLC as an example. The USA one is complete, with .pkg file and .rap file. The Europe on is not, it only has the .pkg file, not the .rap file. The .rap file is used for decryption. Having the .pkg file is the most important thing of course (who knows what we can do with more powerful computers and old encryption in the future), but having the .rap file too is of course even better.

I'll definitely keep both file formats in mind in the future.

[Landcross]

Hehe, quotes galore in this topic now.

Could you please briefly explain the difference between No-Intro and NPS and how things are done regarding PSN DLC?

Well, as I said it's not so much doing things very differently, it's just that they're very much different... 'target audiences' so to speak. Here at no-intro people can really care about nitpicking details or obscure games that nobody cares about. Cataloging them, datting them, preserving them. NPS's community less so, it's is more about having stuff to play, not necessarily cataloging and preserving everything in a good way. As I said in my previous post, absolutely not everyone at NPS sees it like that of course, NPS certainly does good things and in most ways NPS and NI are doing the same thing (and I believe there's a fair amount of communication between certain people from both communities), it's just mostly different communities with different people having a different goals and takes on it, but at the end of the day both do equal 'important' work. And it has a fancy website as opposed to the (no offense xuom ) slightly old-fashioned DoM haha.

Even if we can't afford to buy every DLC, if we had a missing list at least, that would be a good way of seeing what the DLC "landscape" looks like. I was hoping I could ask relevant communities for donations for grabbing the missing stuff.

Yup, misslists are very useful. It's just difficult and time-consuming to make them. Especially if it concerns tens of thousands of items haha. As I said in my previous post, I want to make them, but first I have a ton of stuff to add to the dats first and then have to figure out a way to make the misslist that takes the least amount of time and is sort-of correct. At the end of the day, especially with PS3 and onwards (PSP is still somewhat limited in terms of content), there's so much stuff out there that at this point in time it's probably easier to just pick random items you're interested in and see if they are already added.

Case in point Konami's PT. Would you happen to have a list of this sort of thing or could identify some other prominent examples?

P.T. is a bit of a bad example, because that one is actually completely wiped from Sony's servers. Usually when something gets delisted (or items that were never visible in the first place, like pre-order bonuses and stuff like that), it's not visible in the PSN store anymore but still available for users that own it. P.T. was just completely removed. But no, I don't have a list of specifically which items are missing and delisted.

What is the reason? IP blocking, etc?

Well, in the case of Korea specificially, you need some kind of Korean citizen id to create a Korean PSN account (or really any Korean online account), for which you need to be living in Korea.

If you or anyone else could kindly have a browse of the above link, what are your thoughts on the inclusion of the above files to No-Intro?

That's is an interesting subject. I heard some people talking about it before. Some (early?) PSP and/or PS3 systems had some kind of custom DLC systems as opposed to using the PSN store. I personally am not familiar with it at all and I haven't spent any time looking into it either because I have already enough on my plate. I don't think it would be a good fit for the current PSN dats, but maybe it's possible to put them in a separate dat? I don't know how much games there are, or even how much interest there is in this stuff in the first place? Knowing it's there is one thing, but having someone actually fetching and hashing it and whatnot is something else (assuming the servers are even still online...).

[norkmetnoil577]

To add to this any help with PSN preservation is great, thank you it is a big task.

Adding your links and RAPs to NoPayStation is a good way to ensure it will make its way here, although there is some lag of course due to human effort to turn it into clean names following our convention and figure out what is what. NPS also allow you to just upload your entire collection of links/licenses and their system will sort out what is new and match up links to their entries for you, I believe.

To any would-be contributors: if you have licenses, please go through the effort to (re) dump your RAPs, and don't just rip the PKG out of laziness/difficulty. We need to save RAPs as well. RAPs for PSP are also seriously underdumped due to the fact that they're not strictly necessary (encryption is crackable I think unlike PS3) but still would be good to have.

To datters: I still don't know a good way to preserve decryption data in file form for Vita and PS4. Maybe Vita could be work.bin?

If you're going to buy things though, finding ones where neither PKG nor RAP is known is probably higher priority since if the PKG is out there then *somebody* should be able to dump the rap or have it...

[layzee]

Argh, I delayed my plans too long... and now everything has turned to shit...

As some of you might have heard, a week or two ago there were rumours that the PSN store as we knew it would be closing soon and the official announcement was made today/yesterday. At the same time, access to the old PSN store (with the PS3/Vita/PSP content) from a desktop computer was blocked and now redirects to the revamped store (with only PS4 and PS5 content). The only way to access, purchase and download the old content is via the PS3 (ending July 2), Vita (ending August 27) and PSP consoles (also ending July 2). The problem: the user interface on the PS3 (which I used) is slow as molasses and unwieldy as hell. Makes browsing the store a massive headache compared to browsing from a PC.

I'm into game preservation so I'm just doing what I can to grab some stuff before they disappear especially gameplay-related DLC (extra stages, scenarios, characters, etc). I believe most or all of the free DLC has been backed up and they are making some progress on the USA region DLC. However, I'm into the paid Japanese-region DLC of which a decent chunk does not appear to have been preserved yet by NoPayStation and No-Intro. I'm not sure whether there are other people are pursuing the same thing as me.

In any case, I've gone ahead and bought a 10,000 yen Japan PSN card and slowly going through the DLC available and purchasing/downloading anything that stands out. I'm just one person with limited funds so I can really only buy DLC from the game franchises I'm interested in (see end of post). First and foremost, I focus on games that has not been ported/remade/remastered to other consoles (i.e. unique).

In terms of what kind of DLC I will buy, it will be prioritised as follows:

1. Major gameplay-related content (e.g. extra stages, characters, scenarios, etc)
2. Minor gameplay-related content (e.g. extra equipment, etc)
3. Cosmetic-related content (e.g. extra BGM, costumes, voices, etc)
4. Themes
5. Avatars

So far, I have bought the Japanese DLC for the following:

Hard Corps: Uprising (Konami) (945 yen)

Extra Characters
追加キャラクタ Leviathan
追加キャラクタ Harley
追加キャラクタ Sayuri

Akumajou Dracula: Harmony of Despair (Konami) (~2500 yen)

Extra Missions
全ての始まり
多くにして、一つなるもの
蝿の王
月風魔伝

Extra Characters
シモン・ベルモンド
リヒター・ベルモンド
マリア・ラーネッド
月風魔

Extra Music
BGMパック1
BGMパック2

I'll update this thread as I buy more stuff. I cannot stress enough how annoying it is to browse the PSN store with a PS3.
Looking at some of the Tales of... (Namco Bandai) and Legend of Heroes (Falcom) games, the total cost of DLC for a game could be more than 10,000 yen... more than the cost of the game itself.
I haven't figured out how to transfer the DLC files from PS3 to PC yet but I'll leave that for later.
List of game franchises I'm focusing on: https://home.exetel.com.au/layzee/temp/ps3gamelist.jpg
Worth mentioning is that some DLC is only obtained directly from the game itself and not the PSN store: https://gamefaqs.gamespot.com/boards/92 ... p/77883327

[layzee]

I went through the DLC list and I wrote down anything that I was even remotely interested in:

PSP
Dissidia 012 Final Fantasy
God Eater
Metal Slug XX
Phantasy Star Portable 2 Infinity
Senjou no Valkyria 2
Senjou no Valkyria E3 Extra Edition

PS3
Akumajou Dracula: Lords of Shadow
Arcana Heart 3
Asura's Wrath
Biohazard 0 / Revelations / 5 / 6
BlazBlue Central Fiction / Chrono Phantasma / Continuum Shift
Dark Souls
Dark Souls 2
Dead or Alive 5
Dead Rising 2
DMC
Drag-on Dragoon 3
Dragon Quest Builders
Dragon's Dogma
Final Fantasy XIII-2 / Lightning Returns
Front Mission Evolved
Guilty Gear Xrd
Kagerou
Kamaitachi no Yoru
King of Fighters 12 / 13
Legend of Heroes series
Lollipop Chainsaw
Metal Gear Rising Revengeance
Metal Gear Solid 5
NieR Replicant
Ninja Gaiden
Ninja Gaiden 3
Ninja Gaiden Sigma 2
Persona 5
Persona Dancing
Psycho Break
Rockman 10
Rockman 9
Senjou no Valkyria
Shirokishi Monogatari
Soul Calibur 5
Star Ocean 5 (13)
Street Fighter III
Street Fighter x Tekken
Tales of Berseria
Tales of Graces f
Tales of Vesperia
Tales of Xillia
Tales of Xillia 2
Tales of Zesteria
Tekken Tag Tournament
Tokyo Jungle
Ultra Street Fighter 4
Vampire Resurrection
Vanquish
Virtua Fighter

Vita
Biohazard Revelations 2
BlazBlue series
Catherine Full Body
Darius Burst
Dragon Quest Builders / Heroes
Dragon's Crown
Exist Archive
God Eater
Grand Kingdom
Itadaki Street Dragon Quest & Final Fantasy
Kagerou
Kamaitachi no Yoru
Legend of Heroes series
Metal Max Xeno (3)
Persona 3 Dancing
Persona 4 Dancing
Persona 5 Dancing
Phantasy Star Nova
Seiken Densetsu 2 Secret of Mana (1)
Senjou no Valkyria
Soul Sacrifice
Tales of Hearts R
Tales of Innocence R
Tokyo Xanadu
World of Final Fantasy
Ys 8

I made another purchase:

NieR Replicant (Square Enix) (880 yen)

Extra Missions
15 Nightmares (the English DLC calls it The World of Recycled Vessel).

I'll probably grab Drag-on Dragoon 3, MGS Revengeance, and Lightning Returns next. Maybe some Dissidia 012 stuff, Asura's Wrath, and Metal Slug XX after.

Any Star Ocean 5 fans here? I noticed how when you browse the SO5 DLC page, for a second or two the number on the left would say "15" but then it turns into "0" so no DLC for SO5. Some brief research shows that SO5 had DLC (e.g. extra items and equipment) at time of release. What's the story here? Are the DLC gone for good or can it still be gotten in-game? And are these items unique?

This DLC bug happens on other games too. Apparently Metal Max Xeno was supposed to have at least 3 DLC and Secret of Mana has at least 1 DLC. Both currently display zero DLC.

Another thing: Tactics Ogre (Square Enix) for PSP. The Japanese version has DLC but only one: "Extra Scenario 3". Apparently you need to beat the game to get the other DLC? How does that work? The English version of the game already has the DLC built in. Same thing with Lord of Arcana (Square Enix) - only one DLC: "Download Quest Pack Vol. 16". Where are all the others?

One last minor thing: when I browse the PSN store for too long, it slows down to a crawl and eventually crashes.

Anyway, that's enough for today.

[Pikachu025]

So I was curious about this and, according to the as-of-today updated no-intro DAT for PS3 content (2021-03-24 19:26:41), there's a total of 8120 items where we have the pkg but not the rap. That's not great! There's also 4 items where we, somehow, have the rap but not the pkg.

Lists attached for reference. I'll see if I can cross-reference this with the nopaystation stuff somehow...

[Pikachu025]

So I cross-ref'd this and even then the list still looks pretty dire -- and that's assuming all the NoPayStation data is even accurate in the first place. Attached is my cross-referenced list of PS3 content.

I'm also pretty sure we're missing even basic knowledge of a lot of JP and other asian country data even existing, and I'm not really sure how to rectify that now that the webstore no longer works...

[norkmetnoil577]

Hi Pikachu025 thanks for your reports. I fixed the missing pkgs files (all were just dupes or incorrect entries).

Yes a lot of RAPs are missing. There are maybe 3000-4000 that are dumped just not matched to pkgs correctly they will be matched up soon hopefully, and many are still undumped.

A huge effort to save all these data before stores close will be needed. Reaching out to potential dumpers on social media to contribute links no NPS, etc.

Once data are in NPS they can be scraped by N-I and datted, but the actual link/file collection is beyond the scope of N-I.

However this cannot be stressed enough - RAPs are harder to dump (more work than just dumping links) but absolutely necessary to have actually usable files.

So, encourage anyone contributing to go through the extra step to get RAPs. CFW PS3 is no longer difficult.

[Aringon]

Can you grab the necessary rif files to make the raps off a PS3 with PS3HEN on the newer superslim 3000+ models that don't support traditional CFW? The details about how to install CFW and what to use where are not very clear for most normal people and could really stand to be simplified.

[Landcross]

So I was curious about this and, according to the as-of-today updated no-intro DAT for PS3 content (2021-03-24 19:26:41), there's a total of 8120 items where we have the pkg but not the rap. That's not great! There's also 4 items where we, somehow, have the rap but not the pkg.

Lists attached for reference. I'll see if I can cross-reference this with the nopaystation stuff somehow...

Note that some pkgs are drm-free and do not require/have a rap file. I can make a list of drm-free items in NI if you want.

I'm also pretty sure we're missing even basic knowledge of a lot of JP and other asian country data even existing, and I'm not really sure how to rectify that now that the webstore no longer works...

Well, you can always reference back to my, very work in progress, database over at https://serialstation.com/

[Pikachu025]

So in case you guys missed the news, Sony has decided to not shut down the PS3 and Vita stores for the time being:

https://blog.playstation.com/2021/04/19 ... perations/

While this is appreciated and gives extra time, I would still recommend not waiting until the inevitable 2nd shutdown to collect this information. Anyway, more importantly, they also say

PSP commerce functionality will retire on July 2, 2021 as planned.

It's unclear what this means exactly, considering that the PSP store on PSPs has been unavailable for years. I personally think what they're saying is that in-game DLC stores in PSP games will shut down, but to be safe we should assume the worst and try to collect all PSP content by that date.

[Pikachu025]

Can you grab the necessary rif files to make the raps off a PS3 with PS3HEN on the newer superslim 3000+ models that don't support traditional CFW? The details about how to install CFW and what to use where are not very clear for most normal people and could really stand to be simplified.

I have been unable to confirm this because I don't have a console with HFW, but apparently this is possible. Supposedly the Hybrid Firmware (or maybe webMAN?) has an option under "Network -> Hybrid Firmware Tools -> Dump tools -> Dump IDPS" to generate the IDPS, which you can select and then fetch /dev_hdd0/tmp/hfw_settings.log via FTP which contains your IDPS. The act.dat and *.rif files can be acquired as normal by fetching the files in /dev_hdd0/home/[userid]/exdata.

[KingMike]

So in case you guys missed the news, Sony has decided to not shut down the PS3 and Vita stores for the time being:

https://blog.playstation.com/2021/04/19 ... perations/

While this is appreciated and gives extra time, I would still recommend not waiting until the inevitable 2nd shutdown to collect this information. Anyway, more importantly, they also say

PSP commerce functionality will retire on July 2, 2021 as planned.

It's unclear what this means exactly, considering that the PSP store on PSPs has been unavailable for years. I personally think what they're saying is that in-game DLC stores in PSP games will shut down, but to be safe we should assume the worst and try to collect all PSP content by that date.

I haven't downloaded on PSP in years, but haven't PSP downloads been available by downloading the game on a supported device (like PC or PS3) and then using a USB cable to transfer to the console? I know I had to do that to get a few games on my PSP, back in the day, after the installations on direct downloads kept failing.