More actions
No edit summary |
(not on the list..) |
||
(One intermediate revision by the same user not shown) | |||
Line 4: | Line 4: | ||
|description=NSZ - Homebrew compatible NSP/XCI compressor/decompressor. | |description=NSZ - Homebrew compatible NSP/XCI compressor/decompressor. | ||
|author=nicoboss | |author=nicoboss | ||
|lastupdated=2023/ | |lastupdated=2023/12/19 | ||
|type=PC Utilities | |type=PC Utilities | ||
|version=4. | |version=4.6.1 | ||
|license=MIT | |license=MIT | ||
|download=https://dlhb.gamebrew.org/switchhomebrews/nsznx.7z | |download=https://dlhb.gamebrew.org/switchhomebrews/nsznx.7z | ||
Line 37: | Line 37: | ||
Use the following command to install the GUI version: <br> | Use the following command to install the GUI version: <br> | ||
<code>pip3 install --upgrade nsz[gui]</code> | <code>pip3 install --upgrade nsz[gui]</code> | ||
==== Android ==== | |||
* Install "Pydroid 3" and the "Pydroid repository plugin" from the Play Store. | |||
* Open "Pydroid 3" and navigate to "Pip". | |||
* Enter "nsz" and unselect "use prebuild" then press install. | |||
* Navigate to "Terminal" to use the "nsz" command. | |||
* The first time it will tell you where to copy your prod.keys which you should do using the "cp" command. | |||
* Use any command line arguments you want like "nsz -D file.nsz" to decompress your game. | |||
===Running from source=== | ===Running from source=== | ||
Line 253: | Line 261: | ||
==Changelog== | ==Changelog== | ||
'''v4.6.1 2023/12/19''' | |||
*Fixes MPLUS1p-Medium.ttf not found issue of NSZ 4.6 GUI. | |||
'''v4.6.0 2023/12/18''' | |||
*Major new features | |||
**--verify now uses file-level sha256 NSP hash validation. --verify now guarantees that the sha256 hash of the original file and the decompressed file will match and so the file is bit-identical recreated. | |||
**Major XCI/XCZ rewrite to support multiple XCI partitions: Specify --keep to keep all partitions including their content. By default, the content of all but the secure partition is removed but the empty partition itself are kept for Yuzu Emulator compatibility. | |||
*Bugfixes | |||
**Added missing null terminator at the end of the string table. This fixes #151 | |||
**Fixed issue with PFS0 header overlapping the first file when the --remove-padding option is used | |||
**Fixed some logic responsible to handle too short PFS0 header sizes | |||
**Fixed PFS0 header padding so it follows the PFS0 specification. This fixes #150 | |||
**We don't want to touch the default Kivy font but instead use LabelBase to set DEFAULT_FONT for this specific application. This fixes #154 | |||
**Added the ability to read full xci dumps thanks to @alucryd | |||
**Added master_key_10 key hash thanks to @seiya-git | |||
**Better support deleting source files. This fixes #141 | |||
*Command line argument changes | |||
**Renamed --remove-padding to --fix-padding. This was done as this option now makes NSP padding match the No-Intro standard. | |||
**Renamed --keep-delta to --keep as it's no longer just used to keep NDV0 files but also to keep all XCI partitions | |||
**--verify and --remove-padding are incompatible with each other's. For compatibility reasons --quick-verify will be automatically used if this situation occurs to match the command line argument behavior prior to NSZ v4.6.0. | |||
'''v4.5.0 2023/10/01''' | '''v4.5.0 2023/10/01''' | ||
* Implemented bit-identical FileEntryTable padding recreation | * Implemented bit-identical FileEntryTable padding recreation |
Latest revision as of 03:19, 11 February 2024
NSZ | |
---|---|
General | |
Author | nicoboss |
Type | PC Utilities |
Version | 4.6.1 |
License | MIT License |
Last Updated | 2023/12/19 |
Links | |
Download | |
Website | |
Source | |
A compression/decompresson script (with optional GUI) that allows user to compress/decompress Nintendo Switch ROMs loselessly, thanks to zstd compression algorithm.
The compressed file can be installed directly with supported NSW Homebrew Title Installers.
Installation
There are several ways the install the script. You can find details on installation for all of them below.
You need to have a hactool compatible keys file in a suitable directory to use the script. The keys file must be located as:
prod.keys
file in%USERPROFILE%/.switch/
(Windows)/$HOME/.switch/
(UNIX); orkeys.txt
in the working directory.
It can be dumped using Lockpick_RCM.
Windows Builds
You can also use the Windows binaries. They do not require any external libraries to be installed and can be run without installing anything. You can find the binaries in the release page.
Methods listed below requires you to have Python 3.6+ and pip3 installed.
PIP Package
Use the following command to install the console-only version:
pip3 install --upgrade nsz
Use the following command to install the GUI version:
pip3 install --upgrade nsz[gui]
Android
- Install "Pydroid 3" and the "Pydroid repository plugin" from the Play Store.
- Open "Pydroid 3" and navigate to "Pip".
- Enter "nsz" and unselect "use prebuild" then press install.
- Navigate to "Terminal" to use the "nsz" command.
- The first time it will tell you where to copy your prod.keys which you should do using the "cp" command.
- Use any command line arguments you want like "nsz -D file.nsz" to decompress your game.
Running from source
The tool can also be run by cloning the repository, installing the requirements and then executing nsz using python3 nsz.py
Use the following command to install the console-only versions requirements:
pip3 install -r requirements.txt
Use the following command to install the GUI versions requirements:
pip3 install -r requirements-gui.txt
User guide
Usage
nsz --help usage: nsz.py [-h] [-C] [-D] [-l LEVEL] [-L] [-B] [-S] [-s BS] [-V] [-p] [-P] [-t THREADS] [-m MULTI] [-o [OUTPUT]] [-w] [-r] [--rm-source] [-i] [--depth DEPTH] [-x] [--extractregex EXTRACTREGEX] [--titlekeys] [--undupe] [--undupe-dryrun] [--undupe-rename] [--undupe-hardlink] [--undupe-prioritylist UNDUPE_PRIORITYLIST] [--undupe-whitelist UNDUPE_WHITELIST] [--undupe-blacklist UNDUPE_BLACKLIST] [--undupe-old-versions] [-c CREATE] [file ...] positional arguments: file optional arguments: -h, --help show this help message and exit -C Compress NSP/XCI -D Decompress NSZ/XCZ/NCZ -l LEVEL, --level LEVEL Compression Level: Trade-off between compression speed and compression ratio. Default: 18, Max: 22 -L, --long Enables zStandard long distance mode for even better compression -B, --block Use block compression option. This mode allows highly multi-threaded compression/decompression with random read access allowing compressed games to be played without decompression in the future however this comes with a slightly lower compression ratio cost. This is the default option for XCZ. -S, --solid Use solid compression option. Slightly higher compression ratio but won't allow for random read access. File compressed this way will never be mountable (have to be installed or decompressed first to run). This is the default option for NSZ. -s BS, --bs BS Block Size for random read access 2^x while x between 14 and 32. Default: 20 => 1 MB -V, --verify Verifies files after compression raising an unhandled exception on hash mismatch and verify existing NSP and NSZ files when given as parameter -p, --parseCnmt Extract TitleId/Version from Cnmt if this information cannot be obtained from the filename. Required for skipping/overwriting existing files and --rm-old- version to work properly if some not every file is named properly. Supported filenames: *TitleID*[vVersion]* -P, --alwaysParseCnmt Always extract TitleId/Version from Cnmt and never trust filenames -t THREADS, --threads THREADS Number of threads to compress with. Numbers < 1 corresponds to the number of logical CPU cores for block compression and 3 for solid compression -m MULTI, --multi MULTI Executes multiple compression tasks in parallel. Take a look at available RAM especially if compression level is over 18. -o [OUTPUT], --output [OUTPUT] Directory to save the output NSZ files -w, --overwrite Continues even if there already is a file with the same name or title id inside the output directory -r, --rm-old-version Removes older versions if found --rm-source Deletes source file/s after compressing/decompressing. It's recommended to only use this in combination with --verify -i, --info Show info about title or file --depth DEPTH Max depth for file info and extraction -x, --extract Extract a NSP/XCI/NSZ/XCZ/NSPZ --extractregex EXTRACTREGEX Regex specifying which files inside the container should be extracted. Example: "^.*\.(cert|tik)$" --titlekeys Extracts titlekeys from your NSP/NSZ files and adds missing keys to ./titlekeys.txt and JSON files inside ./titledb/ (obtainable from https://github.com/blawar/titledb). Titlekeys can be used to unlock updates using NUT OG (OG fork obtainable from https://github.com/plato79/nut). There is currently no publicly known way of optioning NSX files. To MitM: Apply disable_ca_verification & disable_browser_ca_verification patches, use your device's nx_tls_client_cert.pfx (Password: switch, Install to OS and import for Fiddler or import into Charles/OWASP ZAP). Use it for aauth- lp1.ndas.srv.nintendo.net:443, dauth- lp1.ndas.srv.nintendo.net:443 and app-b01-lp1.npns.srv.nintendo.net:443. Try with your WiiU first as there you won't get banned if you mess up. --undupe Deleted all duplicates (games with same ID and Version). The Files folder will get parsed in order so the later in the argument list the more likely the file is to be deleted --undupe-dryrun Shows what files would get deleted using --undupe --undupe-rename Renames files to minimal standard: [TitleId][vVersion].nsz --undupe-hardlink Hardlinks files to minimal standard: [TitleId][vVersion].nsz --undupe-prioritylist UNDUPE_PRIORITYLIST Regex specifying which dublicates delegtion should be prioritized before following the normal deletion order. Example: "^.*\.(nsp|xci)$" --undupe-whitelist UNDUPE_WHITELIST Regex specifying which dublicates should under no circumstances be deleted. Example: "^.*\.(nsz|xcz)$" --undupe-blacklist UNDUPE_BLACKLIST Regex specifying which files should always be deleted - even if they are not even a dublicate! Be careful! Example: "^.*\.(nsp|xci)$" --undupe-old-versions Removes every old version as long there is a newer one of the same titleID. -c CREATE, --create CREATE Inverse of --extract. Repacks files/folders to an NSP. Example: --create out.nsp .\in
Example usage
To compress all files in a folder:
nsz -C /path/to/folder/with/roms/
To compress all files in a folder and verifying integrity of compressed files:
nsz --verify -C /path/to/folder/with/roms/
To compress all files in a folder with 8 threads and outputting resulting files to a new directory:
nsz --threads 8 --output /path/to/out/dir/ -C /path/to/folder/with/roms/
To compress all files in a folder with level 22 compression level:
nsz --level 22 -C /path/to/folder/with/roms/
To decompress all files in a folder:
nsz -D /path/to/folder/with/roms/
To view all the possible flags and a description on what each flag, check the Usage section.
Useful links
- Results of how well NSZ performs.
- NSC Builder - An easy to use tool for NSZ compression.
- Ryjin: A NSZ Converter Mod - Another tool for NSZ compression.
File format details
NSZ:
NSZ files are functionally identical to NSP files. Their sole purpose to alert the user that it contains compressed NCZ files. NCZ files can be mixed with NCA files in the same container.
As an alternative to this tool NSC Builder also supports compressing NSP to NSZ, and decompressing NSZ to NSP.
XCZ:
XCZ files are functionally identical to XCI files. Their sole purpose to alert the user that it contains compressed NCZ files. NCZ files can be mixed with NCA files in the same container.
NCZ:
These are compressed NCA files. The NCA's are decrypted, and then compressed using zStandard. The first 0x4000 bytes of an NCZ file is exactly the same as the original NCA (and still encrypted). This applies even if the first section doesn't start at 0x4000.
At 0x4000, there is the variable sized NCZ Header. It contains a list of sections which tell the decompressor how to re-encrypt the NCA data after decompression. It can also contain an optional block compression header allowing random read access.
All of the information in the header can be derived from the original NCA + Ticket, however it is provided pre-parsed to make decompression as easy as possible for third parties.
Directly after the NCZ header, the zStandard stream begins and ends at EOF. The stream is decompressed to offset 0x4000. If block compression is used the stream is splitted into independent blocks and can be decompressed as shown here. CompressedBlockSizeList[blockID] must not exceed decompressedBlockSize. If smaller the block must be decompressed. If equal the block is stored in plain text.
class Section: def __init__(self, f): self.magic = f.read(8) # b'NCZSECTN' self.offset = f.readInt64() self.size = f.readInt64() self.cryptoType = f.readInt64() f.readInt64() # padding self.cryptoKey = f.read(16) self.cryptoCounter = f.read(16) class Block: def __init__(self, f): self.magic = f.read(8) # b'NCZBLOCK' self.version = f.readInt8() self.type = f.readInt8() self.unused = f.readInt8() self.blockSizeExponent = f.readInt8() self.numberOfBlocks = f.readInt32() self.decompressedSize = f.readInt64() self.compressedBlockSizeList = [] for i in range(self.numberOfBlocks): self.compressedBlockSizeList.append(f.readInt32()) nspf.seek(0x4000) sectionCount = nspf.readInt64() for i in range(sectionCount): sections.append(Section(nspf)) if blockCompression: BlockHeader = Block(nspf)
Screenshots
|
Changelog
v4.6.1 2023/12/19
- Fixes MPLUS1p-Medium.ttf not found issue of NSZ 4.6 GUI.
v4.6.0 2023/12/18
- Major new features
- --verify now uses file-level sha256 NSP hash validation. --verify now guarantees that the sha256 hash of the original file and the decompressed file will match and so the file is bit-identical recreated.
- Major XCI/XCZ rewrite to support multiple XCI partitions: Specify --keep to keep all partitions including their content. By default, the content of all but the secure partition is removed but the empty partition itself are kept for Yuzu Emulator compatibility.
- Bugfixes
- Added missing null terminator at the end of the string table. This fixes #151
- Fixed issue with PFS0 header overlapping the first file when the --remove-padding option is used
- Fixed some logic responsible to handle too short PFS0 header sizes
- Fixed PFS0 header padding so it follows the PFS0 specification. This fixes #150
- We don't want to touch the default Kivy font but instead use LabelBase to set DEFAULT_FONT for this specific application. This fixes #154
- Added the ability to read full xci dumps thanks to @alucryd
- Added master_key_10 key hash thanks to @seiya-git
- Better support deleting source files. This fixes #141
- Command line argument changes
- Renamed --remove-padding to --fix-padding. This was done as this option now makes NSP padding match the No-Intro standard.
- Renamed --keep-delta to --keep as it's no longer just used to keep NDV0 files but also to keep all XCI partitions
- --verify and --remove-padding are incompatible with each other's. For compatibility reasons --quick-verify will be automatically used if this situation occurs to match the command line argument behavior prior to NSZ v4.6.0.
v4.5.0 2023/10/01
- Implemented bit-identical FileEntryTable padding recreation
- The size of FileEntryTable including any non-standard zero padding at the end is kept
- The FileEntryTable size is kept even if NDV0 files got removed as they are by default if --keep-delta isn't specified.
- The padding at the end of FileEntryTable will increase in this case
- Use --remove-padding if you don't want to keep the FileEntryTable size and instead make it the lowest possible size according to the no-
- intro standard. Keep in mind that if your input is already no-intro compliant this option is not needed.
- This fixes rare cases of non-standard NSP files not being able to be bit identically recreated due to the removal of non-standard FileEntryTable zero padding as reported by @nitro322
v4.4.0 2023/08/05
- Android CLI support
- To make NSZ Android compatible all uses of shared memory and locks had to be replaced with Android compatible alternatives
- How to install NSZ on Android
- Install "Pydroid 3" and the "Pydroid repository plugin" from the Play Store
- Open "Pydroid 3" and navigate to "Pip"
- Enter "nsz" and unselect "use prebuild" then press install
- Navigate to "Terminal" to use the "nsz" command
- The first time it will tell you where to copy your prod.keys which you should do using the "cp" command
- Use any command line arguments you want like "nsz -D file.nsz" to decompress your game
- Fixed #134 (OverflowError: can't convert negative int to unsigned)
- This was fixed inside #135 (Removal of stringTable padding)
- NSP/NSZ files with a missing stringTable padding are now handled properly
- This fixes some rare edge cases where bit-identical PFS0 recreation silently failed
- Updated all libraries inside the portable Windows version
v4.3.0 2023/07/07
- Bit-identical PFS0 recreation to comply with the no-intro standard
- The hash of the original NSP before compression and the hash of the NSP resulting from decompressing the NSZ will match as they are bit-identical.
- Use the newly introduced --remove-padding option to make existing NSP/NSZ files nxdumptool/no-intro compliant.
- Use the newly introduced --keep-delta option to keep all useless delta fragments (NDV0) during compression so the NSP (PFS0) can be recreated bit-identical during decompression.
- PFS0 hash verification is now part of the --verify process. --verify now requires --keep-delta when used during compression or it will detect removed NDV0 fragments as errors. Use --quick-verify to skip the PFS0 hash verification. --quick-verify only verifies NCA hashes and so does not require --keep-delta when used during compression. For compatibility reasons --quick-verify will be automatically used instead when specifying --verify without --keep-delta to match the command line argument and verification behavior of previous NSZ versions.
- This fixes #98, fixes #101 and fixes #116
- Added console output warning about missing and invalid keys
- Added support for pyinstaller as discussed in #119 (pyinstaller builds for macOS)
- Added crc32 hash of master_key_0f. Thanks to @drizzt for adding it!
v4.2.1 2022/12/14
- Implemented support for zStandard long distance mode inside the GUI.
- This fixes an issue that causes NSZ to crash when trying to compress using the GUI.
- Close GUI window once the NSZ action got started. Not doing so was really bad design.
v4.2.0 2022/12/09
- Resolved title-intaller compatibility with python-zstandard versions later than 0.15.2. NSZ files compressed with modern zstd versions are confirmed to work on the latest Tinfoil, DBI, AtmoXL and TinWoo. Some abandoned title-installer who havn't updated will still be broken. See #120 (zstandard versions newer than 0.15.2 break compatibility with all title installers except DBI) for more information.
- Fixed a mistake in the FileExistingChecks code caused by specifying the wrong target file extension when calling CreateTargetDict. This fixes #123 (XCZ Source File Deleted before decompressing on the same output folder as original).
- Fixed NSP and resulting NSZ will be deleted using --rm-source when solid compression fails. This fixes #124 (NSP and resulting NSZ will be deleted using --rm-source when solid compression fails).
- Added crc32 hash of master_key_0e. Thanks to @16BitWonder for adding it.
- Implemented option to enable zStandard long distance mode for even better compression as requested in #117 (Add support for the zStandard long distance mode).
- Followed the python-zstandard author's feedback regarding flushing to fix all flushing related imperfections.
- Fixed "'Counter' object has no attribute 'write'" error when using --create. This fixes #118 (--create is broken).
- Implemented directory expansion support for --create and improved documentation. This closes #119 (The syntax for --create is stupid and the documentation bad).
- Clarified the plaintext rule of the NSZ file format block compression extension as discussed in #120.
v4.1.0 2022/09/05
- Hotfix 09.09.2022: If you downloaded nsz_v4.1.0_win64_portable.zip before this date please redownload.
- Fixed #84 [GUI] broken input file Selection (Drag & Drop still works). This also fixes #102.
- Fixed undupe not deleting any files, added undupe support for NSX (NSP files without titlekeys) and fixed not whitelisting the first occurrence but undupe-whitelist any other occurrence not deleting the first occurrence.
- Implemented --undupe-rename.
- Adopted this tool to the standard by also skipping the content meta xml and improved the skipping message to follow the print standard.
- Fixed Verify not throwing an exemption when verification fails.
- Implemented --alwaysParseCnmt.
- Handle NSZ verification errors by deleting the corrupted NSZ.
- When verifying, show "Verifying" instead of "Compressing" in progressbars.
- Implemented --undupe-hardlink. Booth --undupe-hardlink and --undupe-rename now uses the output folder as output location..
- Fixed #87: nsz doesn’t respect -o flag when decompressing.
- Add CRC-hash for new master keys 0b, 0c and 0d.
- open file only for reading with 'info' command.
- set executable bit.
- Kivy finally fixed its Linux issues so we can make installing nsz as easy as never before.
- Read masterKeyRevision from either of two positions to resolve some edge cases with strange games.
- Updated dependencies in setup.py to match requirements.txt/requirements-gui.txt.
- There apparently are v0 DLCs so let's remove the safety check that prevents v0 titles to be deleted if there is an update of that exact titleId. The original purpose of this safety check was to protect base titles in case there is a titleID naming issue but is very unlikely to ever be needed. This fixes #105.
- Updated packages inside setup.py to match the new setup.py standard.
v4.0.1 2020/09/22
- I spent a lot of time deleting every piece of the python standard library not required to run nsz in order to save storage and speed up extraction time by having less files to extract.
- This was done because PyInstaller triggered false positive detections on Windows Defender as reported in #82.
- Tested on Windows 7, Windows 10 and Windows Server 2019.
v4.0 2020/09/06
- Implemented Drag & Drop support as requested in #51.
- Implemented CRC32 key validation and added support for future masterkeys.
- Fixed the issue of master_key_0a not being recognized.
- Added CRC32 for master_key_0a.
- Windows 7 support for Windows builds.
- Improved GUI font size scaling.
- Set the NSZ GUI window to be TopMost (always on top) on Windows so Drag & Drop gets much more convenient.
- Added a setting to specify if the Kivy window should be always on top or not.
- XCI/XCZ finally extracts to folders containing the NCA/NCZ files instead of HFS0 partition dumps.
- Fixed a major XCI compatibility bug by implementing compression/decompression support for NCA files with the first section having a smaller or larger offset then 0x4000. This fixes #49.
- Added NSPZ (nsZip legacy file format) extraction support.
- Make GUI an optional install.
- Fixed #59 ncz decompression is not working.
- Cleaned up imports for nsz package.
- Stop bar manager to avoid broken shells.
- Added pywin32 as GUI dependency for Windows. This fixes #56.
- Fixed BlockDecompressorReader.seek with whence = 2 (seek relative to the file's end). This fixes #64.
- Starting nsz.py from within a different working directory finally works.
- Fixes the current Azure Pipeline issue.
- Added solid decompression, block decompression, solid compression and block compression tests to azure-pipelines.yml.
- Made NSZ new returning with error code 1 if there are any exceptions.
- Fixed deadlock in BlockCompressor.py.
- Highly improved block decompression speed by caching the current block.
- Added Visual Studio 2019 Python Project.
- Added titleId and version to the file list and highly improved its design.
- The SelectableLabel items inside the game list now properly scales its height according to the available width and text length of the file path. This fixes #50.
- Added a multi-language supporting open source font for #61.
- Implemented input folder as output folder by default for #61.
- Waiting for Enter before exit when started over GUI so errors and console output can be seen before it closes for #74.
- Fixed install failing on Kubuntu 20.04 and a lot of other modern Linux distributions by improving install_linux.sh for #75.
- Removed code that manipulated the XCI header size for absolutely no reason which fixes #77.
- Improved decompression speed by 400% by heavily reducing the amount of performance intensive status bar refresh calls.
- The decision if the last block should be decompressed or copied now matches the file format specifications by comparing the decompressed block size of that specific block with its compressed size. This issue was caused by missing the (unlikely) edge case that the last block can be larger when compressed without exceeding the general block size. This fixes #79.
- Improved BlockCompression performance and overall CPU usage by reducing the amount of performance intensive status bar refresh calls.
- Ensure that the line right to the curser is clean when the application terminates.
- Implemented undupe, undupe-dryrun, undupe-prioritylist, undupe-whitelist and undupe-old-versions to remove duplicate games.
- Highly improved the missing prod.keys/keys.txt error message by not showing the stack trace and waiting for a user input before exiting.
- Highly improved README.md.
- General system stability improvements to enhance the user's experience.
v3.1.1 2019/12/30
- Fixed XCZ decompression/verification.
v3.1 2019/12/29
- Fixed broken decompression in v3.0.0 due to an optional argument not marked as such and an outdated file extension comparison.
- Allow the selection of individual files inside the OpenFileDialog.
- Made it visible which drive is currently selected inside the OpenFileDialog and SaveFileDialog.
- Replaced the background_down image with the background_normal one as it looks ugly inside the FileDialogs.
- Removed SaveFileDialog as its unused and a pain to maintain.
- Highly improved the OpenFileDialog by allowing switching between the icon and list layout.
- Added filter to the OpenFileDialog so it only displays nsp, nsz, xci, xcz and ncz files.
- No longer showing the empty placeholder for the device selection on non-Windows platforms an made per OpenFileDialog argument specifiable file filters possible.
- Finally enabled the selection of folders and selecting multiple items at the same time.
- Automatically creating the empty gui folder required for settings to be saved in nsz portable which is exactly the fix manually applied to nsz_v3.0.0_hotfix1_win64_portable.zip.
- Fixed the warnings that appeared in the console on every GUI launch.
- Made the file browser view mode buttons in the same design as the Windows device selection buttons.
- Implemented deletion of selected GameList items using the delete or backspace key.
- Highly improved the GameList item selection and deletion.
- Vixed verification only mode not executing when called from GUI.
- Set the default amount of threads for solid compression to 3 while keeping the number of logical cores the default for block compression because the majority is now using task parallelization for solid compression.
- Fixed: AttributeError: 'RootWidget' object has no attribute 'verify'.
- Dirty fixed Kivy throwing "Error in sys.excepthook" during shutdown if the amount of RecycleView data was ever decreasing.
- I tried multiple hours fixing this the proper way without success. I also tried removing all children what makes it to throw the same exception without even decreasing the amount of data and tried filling it with dummy data on shutdown in self.rootWidget.gameList.recycleView.shutdown() which didn’t worked because it’s already too late to spawn everything needed to avoid this exception.
- Improved the wording of the error summary.
- Updated testing and deployment scripts.
- Fixed orphan processes remaining after the main process terminated after block compressing a very short task resulting in the orphan processes spamming "AttributeError: 'ForkAwareLocal' object has no attribute 'connection'" exceptions.
- Fixed NSZ GUI icon not showing.
- General system stability improvements to enhance the user's experience.
v3.0 2019/12/24
- GUI:
- Contains all functions available using command line arguments.
- XCZ support:
- Block compressed (default).
- Solid compressed.
- XCZ to XCI decompression.
- Implemented task parallel solid compression. See --multi.
- Implemented titlekey extraction with titlekeys.txt and titledb support. See --titlekeys.
- Added regex support to the extract option to allow the user to specify exactly which files should be extracted from the container. See --extractregex.
- NCZ decompression directly in nsz #38.
- Updated IndependentNczDecompressor to the latest version inside nsz.
- Fully replace tqdm with enlighten
- Implemented multithreaded multiple process bars communication system for solid compression and verification while avoiding stdout race conditions.
- Endless decompression/verification on a few block compressed games #25.
- Fixed a major bug causing the space between block compressed NCZ files to be filled up with 0x00 to fit their uncompressed size.
- Major Scripts Cleanup.
- Fixed NSZ Decompressor TQDM Progress Bar.
- Fixed --rm-old-version.
- fixed path bug with decompression.
- Removed pycryptodome v3.9.0 restriction as its latest v3.9.3 works fine.
- Improved installation guide.
- Fixed an exception that could occur when trying to receive keys under special circumstances during debugging.
- Fixed a major bug inside the enhanced file existing check leading to files with the same name as a file already existing inside the output directory being overwritten without specifying this behavior using the –overwrite command line argument. This bug was cause by comparing the output file path instead of the output filename with existing filenames.
- Made nsz pip package building Kivy compatible.
- Switched from Nuitka to PyInstaller due to Kivy compatibility.
- Changed how --extract and --verify arguments are handled internally.
- Set up CI with Azure Pipelines using self-hosted server.
- Fully switched to pathlib. This fixes #41 and a lot of other file path related issues.
- Fixed Enhanced File Existing Check. Adapting to pathlib and finally fixing --overwrite and --rm-old-version.
- Improved exception handling related to outdated keys.txt which fixes issue #29 and #40.
- General system stability improvements to enhance the user's experience.
v2.1.1 2019/11/15
- Fixed block compression for pip and Nuitka (nsz_win64_portable) by using
sys.argv[0]
instead of__main__.__file__
so threads no longer need to be prevented from initializing their own nut environment. - Made installation instructions easier to see and understand.
- Scripts to automate testing and publishing.
v2.1 2019/11/14
- 98 commits worth of changes since v2.0.
- Added pip support.
- Added Nuitka standalone windows build support.
- Enhanced File Existing Check #20.
- Skip already compressed/decompressed files by default.
- --overwrite
- --rm-old-version
- Extracting TitleIDs and Versions from filename if possible (#17 and #19).
- The titleID checking is now immensely faster than when extracting from Cnmt
- --parseCnmt to get TitleID/version from Cnmt if not extractable from filename.
- Otherwise it falls back to simple filename checking which is much faster.
- Batch error handling with tracebacks (#16).
- Some debugging codes to find out erroneous files.
- Prevent batch process raising errors.
- Batch error handling with tracebacks.
- The --thread option now works even for solid compression however the progress bar still has some visual glitches.
- NSP/NSZ file hash verification now uses the hashes inside Cnmt instead of the nca filename (#22).
- Fixed decompression memory leak (#21).
- The memory leak only occurs for dctx.stream_reader and was fixed by switching to the simple decompressing API which makes more sense for block decompression anyways.
- Improved pageReadSize calculation speed by using math instead of a while loop.
- Added option --remove-source that deletes the source file after compression or decompression (#24).
- For this we finally properly closed file containers too.
- Fix huge compression memory leak (#13).
- Fixed wrong working directory when starting nut.py from a different directory (#18).
- Improved exception handling during TitleID/Version extraction.
- Added support for prod.keys.
- Reorganized file structure.
- Fixed keys.txt path to always be the folder containing nsz.py.
- Fixed default thread amount to be cpu_count().
- Implemented python version checking to prevent python from showing the users confusing compatibility related exceptions.
- Compatible and tested with Python 3.6 and later.
- General system stability improvements to enhance the user's experience.
v2.0 2019/10/20
- Fully implemented block compression which can be enabled using the --block option.
- Supports for random read access on compressed files.
- Highly multithreaded compression when block compressing.
- Technically supports playing compressed games in the future.
- Current title installers do not support this yet.
- Comes with a low compression ratio cost.
- Implemented NSP/NSZ file hash verification.
- Overwrite/Duplicate protection.
- Reorganized the project's folder structure and enhanced code readability.
- Improved user feedback in form of better understandable messages and errors.
- Fixed a lot of bugs and non-working features.
- Added MIT License so all code inside this project can be used for whatever you like.
- General system stability improvements to enhance the user's experience.
v1.0 2019/10/13
- Scripts to compress and decompress NSZ files.
Credits
Legal:
- This project does not incorporate any copyrighted material such as cryptographic keys. All keys must be provided by the user.
- This project does not circumvent any technological protection measures. The NSZ file format purposely keeps all technological protection measures in place by not decrypting the most crucial parts.
- This project shall only be used for legally purchased games.
Credits:
- SciresM for his hardware crypto functions; the blazing install speeds (50 MB/sec +) achieved here would not be possible without this.
- Thanks to our contributors: nicoboss, blawar, plato79, eXhumer, Taorni, teknoraver, anthonyu, gabest11, KWottrich, maki-chan, thatch, 2weak2live, pR0Ps, 16BitWonder