proposal Proposal: "Debloated" Installs (Remove unused files)

[EVERGREEN 19]

Today more than ever, (fast) storage space is expensive. One thing that always makes me mad is the insane amount of unused Localizations, game modes (often dead/closed multiplayer modes) that are installed by default - this is literally dead content. Wasted storage. Wasted money.
Now back in ye old days, it used to be a gigabyte at best. Not the end of the world, and not exactly worth the time investment. But old habits die hard, and I'm still doing it today.

With games becoming larger and larger, storage has become an issue that can thankfully be alleviated. 
I'm going to list a few interesting examples, then propose a solution and finally suggest a way to integrate it to PCGW's structure. I'll also list a couple of issues with my proposal, potential flaws and uses cases etc. If you have a better idea or any suggestion to make this a thing, you're more than welcome.

Please note that all the numbers given are taken from Steam, but GoG, Uplay, EGS & Origin are guilty of the very same thing. Uplay's even worse, as always. 
Any constructive feedback would be much appreciated - I never suggested a feature before, but this one has been on the back of my mind for at least a year. I feel like it could be very useful to many folks out there.

So, let's get to it. Those are easy ones to "clean-up" (more on that later):

• Batman Arkham Origins. Had a multiplayer mode, servers are down. Delete one folder and the install size goes from 27.06Gb to 18.1Gb. 9Gb (33%) saved
• Final Fantasy XIII. Well documented, check the PCGW entry for it, you can remove ~20Gb if you don't want the Japanese audio. 57.6Gb to 37.7Gb. 19.9Gb (52%) saved (!!!)
• Doom 2016. Do you really play the MP or Snapmap modes? That's ~15Gb (11Gb if you only delete the MP) saved. From 69.68Gb to 54.68Gb. 15Gb (21.5%) saved

Here's the problem. I can manually delete all localizations, "deluxe edition content", Readme/Support and redists safely from most MT_Framework, UE3 and Ubi games just fine because they use the same naming conventions. All I have to do is search in the root folder for any file with the _ita. suffix for instance and delete it - but that's because I know what I'm doing and I'm willing to take the time to locate and delete those files. 
Listing that would massively bloat any page of course, and not many users would do it anyways. 

The best way I can think of to implement a reliable and simple method to delete files that we're absolutely sure are safe to delete goes something like this:
 

• Add a "debloatable" boolean to the Other Information infobox, If True, how much can be shaved-off at best.
• Users like myself could build a database of games we know we can "shave" (much like SK/ReShade compat, with a dedicated page)
• The end user would download a batch file, hosted here and verified by members based on a template which would include one option for each localization, and a "clean-up" option (remove Readme, Deluxe content, redists if safe)

So for instance, I can flag all the localization for Resident Evil 6 and write them down in the dedicated page. I don't have any experience making modular batch files like that however, so someone else would have to make a template. I can then edit that batch to point it to all the files we want to delete. The end user launches the batch file, delete all locales but the one he's/she's using and boom. That's money saved right there.
I know there are programs that are much better than Win Explorer's Search feature - if we can feed such a program with a config file it should do the trick too. We'd still need to build a database though. 

I do realize that I make it sound much easier than it may be, or that it may sound overkill if we're talking about a Gb at best. But for extreme cases like Doom 2016, Far Cry 3/4, FF XIII, the Arkham series, The Evil Within - huge games basically, it would be very helpful and hey, I'm already doing it anyways so might as well share it. There's also games like Battlefront 2 (2005) where you can cut the install size in half. It's about 5Gb (vanilla) if memory serves, about 2-3Gb when cleaned. 
With that said, if anything I hope that this thread at least brings more attention to this issue. 

Last but not least, to everyone: Happy holidays! I hope you're all doing well, and ready for more PCGW grunt work for this year to come.
"Keep on keeping on". 

[dave247 9]

I think the logical conclusion for a problem like this would be to create a simple program with a GUI that would be able to detect which games are installed, and present the user with a simple and easy way to select which files/packages they want to remove. Kinda makes me think of those programs where you can edit and create your own WIndows installation ISO's.

Until then though it would be important to start some kind of list/database of games and the related files that can be removed as it would make any possible future improvements (such as a standalone program) that much easier. Considering PCGamingWiki's subject matter and its ability to store and use data it would be very fitting for an idea like this to be essentially pioneered from here.

[EVERGREEN 19]

16 hours ago, Dave247 said:

I think the logical conclusion for a problem like this would be to create a simple program with a GUI that would be able to detect which games are installed, and present the user with a simple and easy way to select which files/packages they want to remove. Kinda makes me think of those programs where you can edit and create your own WIndows installation ISO's.

Until then though it would be important to start some kind of list/database of games and the related files that can be removed as it would make any possible future improvements (such as a standalone program) that much easier. Considering PCGamingWiki's subject matter and its ability to store and use data it would be very fitting for an idea like this to be essentially pioneered from here.

Thanks for the suggestions!
A program with a nice GUI would be better for sure - the way you describe it reminds me of the new-ish ReShade installer actually. I know ReShade itself is open source, idk about the launcher but maybe that could help.
With that said I would assume writing such a program would take more time and effort than a modular batch (correct me if I'm wrong) and this is likely something way too complicated for me with my current (and very limited) programming experience. I'm very interested in any ressource that could help me learn more and do it myself though.
Interestingly, Microsoft released a PowerShell tool called DiskUsage, which can delete stuff based on the extension, file size... This is somewhat similar to what I had in mind (and it might come in handy now that I think of it!) 

As for building a database, that is something I could do relatively quickly since I already know what can be removed for a couple dozen of games. That would probably help to showcase how much "savings" you can expect as well, it could drum up some interest.

In your opinion, what would be the ideal format for such a database? I could make a Google Doc, but that's going to get messy pretty fast. For instance, for most UE3 games it's simpler to just look for any files with the "_LOC_RUS." suffix. These would take too much time to retrieve the filepath of each file, at least with Win Explorer. Perhaps you happen to have a better workflow/program? I would need something that can copy the full name and filepath of multiple files. Pretty sure Windows has some built-in tools for that. 

[dave247 9]

Ah don't worry too much about a program/fancy GUI. That was just me putting some ideas out there when thinking about the possible reason how and why something like Special K for example came into existence.
A batch file can still do a lot including having options for user input just without a fancy GUI and as you said can be a lot simpler to create. Still, if it's something that you are really interested and dedicated to doing then it would serve as a perfect opportunity to learn and get into it! It's the reason why I never was able to commit to learning a programming language, as I never had something in the future that I wanted to do to make use of all the time I spent gain all that knowledge.

Anyway back on topic. By "database" I was actually referring to PCGamingwiki itself, as by using the Semantic MediaWiki extension different variables are able to be stored in predefined parameters using templates, which can then be searched, downloaded, or even referenced on a external website I believe using an API. From my understanding, it's the reason why articles are set up the way they are, with most fields required to follow a strict and specific example or simply just be "true" or "false". I'm only just coming to understand how it works more myself the longer I spend time editing and participating here.

The only problem with this though is that it would take a fair bit of time to set up and get working correctly, not just to work as a database but also still as a typical Wiki article which is made harder by the fact that all of this that PCGamingWIki actually does isn't what the Semantic MediaWiki extension was designed to do in the first place. I'm sure if @Aemony read all of what I was suggesting they would have a heart attack as I can only imagine how complicated and time consuming it would be to set up. 😛

That being said I still think it would be well worth the time and effort to get something like this up and running on the Wiki. Not just because it could be used as an actual database with all of the possibilities that would bring, but at the same time have all that data displayed in a neat and user-friendly way. 

Unfortunately, that's about where my own experience and knowledge ends on the subject. Like I said I'm still learning a lot about how the Wiki works, and only have basic knowledge when it comes to programming due to dabbling in it. I don't know if something like Microsoft Access would be good or even useful to you, but I do know that as an actual database program (abit a simple one), though you'd need to own Microsoft Office to be able to use it.
Aside from that, any kind of spreadsheet program like Excel/Google Sheets would work as a way to store the data you collect and potentially use it in some way, though it would be a lot more difficult as it's not really designed to work that way. Plus you'd still need some other program to be able to mass search/capture path names like you said, which I don't know of any but I'm sure sure a thing would exist.

Edited January 5, 2021 by Dave247

[Aemony 150]

In terms of leveraging PCGW, a couple of questions comes to mind:

• How would this sort of game-specific information be covered? In the game articles themselves or on a separate page?
• How would we ensure that users are aware that things might've changed after an update and these sorts of arguably more volatile changes might break their game?
• What would the benefit be of leveraging PCGW's backend? Even if we were to cover this through the web API query endpoint, something have to be run locally to actually perform said queries.
• If it was decided to leverage PCGW's backend, which one of our various solutions would be best for this sort of things?

 

Typically PCGW actually doesn't really care about the disk space usage of a game all that much because it is prone to differ between users, DLCs, localizations, etc, which means any potential shown "savings" will not be applicable to everyone. Today we, for the most part, merely avoids this headache by merely stating the system requirements, and in the few cases where the system requirements are way wrong we state an approximation of the real value, and then a note about how the system req is wrong.

 

 

In regards to the third one, it wouldn't really necessarily result in much. An arguably easier solution would be to have whatever local program/script that was thrown together automatically download the latest copy of a GitHub hosted INI file that included all detection algorithms (if any were set up) along with all file/folder rules that were to be removed.

In fact, did you know that the 'cleaning tool' CCleaner makes use of an easily configurable "database" of that exact kind? And that an insane custom INI file can be downloaded straight of a GitHub repository that allows users of CCleaner to extend its "cleaning process" to also include a whole ****ton of applications and even games? Take for example its entry on A Hat in Time:

[A Hat in Time *]
Section=Games
Detect=HKCU\Software\Valve\Steam\Apps\253230
FileKey1=%ProgramFiles%\Steam\steamapps\common\HatinTime\HatinTimeGame\Logs|*.*

Detect A Hat in Time by checking for the presence of the registry key HKCU\Software\Valve\Steam\Apps\253230. If it is detected, allow the cleaning of all files below %ProgramFiles%\Steam\steamapps\common\HatinTime\HatinTimeGame\Logs.

 

In regards to the fourth bullet point, if one were to leverage the PCGW backend one would have to avoid Semantic MediaWiki entirely. I once set up e.g. the Windows config paths test property in Semantic MediaWiki in an attempt to store and track entered data/save paths and make them queryable and one issue that I never was able to fix was SMW's inability to handle forward slashes (/) properly. 😖 That's why there's only properties for Windows paths -- Linux and macOS paths uses forward slashes which just broke the whole thing because (Semantic) MediaWiki thought they were HTML code or something... 😐

 

Anyway, a separate program using a separate INI-based database akin to CCleaner currently looks like the better alternative in my eyes.

[EVERGREEN 19]

Big necro, I know...

But here's the good news: a friend of mine turned this idea into a little program. I feed it some "templates", and it deletes localizations, dead multiplayer modes etc. It also tells how much was actually deleted and gives a percentage, so that you know how much you shaved. 

I thought it may be of interest to some of you, so here it is: neatodev/InstallDebloater (github.com)
Thanks a thousand times to Neato for coding this lil' tool!

[magikgimp 4]

This is very cool news. Just as HDD space became not a premium any more, faster access technology came along putting us right back to where the space to cost ratio used to be so this tool is invaluable... until the next tech era I guess! A GUI would be fantastic. It could scan for games on your drives automatically, give options for each profile e.g. yes I still want the other language but no I don't want the redundant multiplayer etc. but as it is it still seems a great project. I hope a lot of people find it and will be recommending it when I see people complaining about this very problem.

[vns6 2]

That's a very good program. But, it's missing some games like Homefront: The Revolution.

[Patrxgt 27]

1 hour ago, vns6 said:

That's a very good program. But, it's missing some games like Homefront: The Revolution.

It's open for contributions, so you can make script for it, if you want. I've recently made one for BloodRayne: Terminal Cut and their respond for pull request was fast and smooth.

[EVERGREEN 19]

On 4/18/2022 at 7:11 PM, magikgimp said:

This is very cool news. Just as HDD space became not a premium any more, faster access technology came along putting us right back to where the space to cost ratio used to be so this tool is invaluable... until the next tech era I guess! A GUI would be fantastic. It could scan for games on your drives automatically, give options for each profile e.g. yes I still want the other language but no I don't want the redundant multiplayer etc. but as it is it still seems a great project. I hope a lot of people find it and will be recommending it when I see people complaining about this very problem.

Thanks a lot! As of now there is no GUI, but we do our best to make templates that are easy enough to edit yourself, so that you can keep some stuff. For instance, for the Arkham games we use the file extension to find a given language, e.g ".ita". We add comments in the template and so that each language is grouped under one category. That way, you can comment all entries related to Italian and keep them.

18 hours ago, vns6 said:

That's a very good program. But, it's missing some games like Homefront: The Revolution.

Ask and you shall receive! I made a template for it just now. Saves about 8.5Gb (~12% of the total size). Thank you for reminding me about this game, because I used to debloat it manually when I was playing it, but I completely forgot. If you know of any "Extreme cases" like FF XIII or Arkham Origins, feel free to suggest them or if you feel like it, contribute. We tried to make it as easy as possible to create templates.  

16 hours ago, Patrxgt said:

It's open for contributions, so you can make script for it, if you want. I've recently made one for BloodRayne: Terminal Cut and their respond for pull request was fast and smooth.

Thanks a lot for your contribution - I recognized your nickname. Happy to hear it was a smooth enough process for you! I was very pleased to see that you isolated languages and everything, I hope it wasn't too confusing or annoying. 
I wanted to tell you to replace my nickname with yours in the .ini In my commit review but I forgot - so I did it myself. Credit where credit is due 😉

[Zorklis 2]

Just to add to the game list. I know Assassin's Creed Brotherhood "Deluxe Edition" includes a ton of wasteful crap: "Assassin’s Creed Lineage - The Complete Movie" (that they should just add on steam videos category, it's a whole 36 minute episode/short movie) + Soundtrack,etc.

Witcher 3 GOTY also includes unneeded stuff can't remember what

[vns6 2]

You can save up to 15+ GB of disk space in Days Gone by deleting its localizations.

[EVERGREEN 19]

On 4/20/2022 at 10:38 PM, Zorklis said:

Just to add to the game list. I know Assassin's Creed Brotherhood "Deluxe Edition" includes a ton of wasteful crap: "Assassin’s Creed Lineage - The Complete Movie" (that they should just add on steam videos category, it's a whole 36 minute episode/short movie) + Soundtrack,etc.

Witcher 3 GOTY also includes unneeded stuff can't remember what

Thanks for the suggestions. I own AC:B and TW3, I'll take a look at them 🙂

On 4/21/2022 at 1:57 AM, vns6 said:

You can save up to 15+ GB of disk space in Days Gone by deleting its localizations.

I do not own Days Gone, but I'll ask around. Perhaps someone can test it for me. If you know exactly which files to delete, feel free to share your findings. I can make a template as long as I know what to delete.

Also, I made a video that goes over why it exists, what it does, and how to use it. I am also working on a second part, a tutorial for template creation. 
If you'd like to try it out but don't anything about using programs in commandline this should be helpful:
InstallDebloater - How to save space on your games! - YouTube

[Zorklis 2]

I thought of messing with AC Unity a bit and checked out it's directory (it has a bunch of noticable junk) and tested out deleting different languages (besides eng and sfx) even french and it seems to run the same (still heard french in game) minus about 4 gb of space saved

[Zorklis 2]

L.A Noire has two folders Redistributables (that has SC/RL -208 MB) and Installers (that has SC/RL + VCRed/DirectX stuff - 298 MB) in total saving 507 MB.

there's also 6 txt readme's that are about 104 KB