thash is a tool designed for integrity and security applications, optimized to deliver speed in hash generation and verification for large data volumes.

How it Works

Normal Method

Consider file as a single block, read all and calculate a single hash:

thash Method

Consider File as Blocks of BlockSize, process hash of each block separated and then calculate a chain hash:

Obs. Similar to Blockchain: Hash of hashes

Hash Algorithm: Can be any one, as sha256, sha512, ripemd, etc. For now, only sha256 is implemented and it is the default.

BlockSize:This will be the size of each block with the unit of measurement (KB, MB, GB, TB, etc.). The last Block will be the remainder part (unless FileSize matches a multiple of BlockSize).

thash: will put 't' in front of algorithm, a separator '-' and the BlockSize used.

Examples:

tsha256-50MB = hash algorithm: sha256, BlockSize: 50MB. 

tsha512-1GB = hash algorithm: sha512, BlockSize: 1GB. 

tsha512-15GB = hash algorithm: sha512, BlockSize: 15GB.

Attention! To Verification suceeds its necessary to use the same BlockSize used in generation, otherwise, it will fail! So, the BlockSize used in generation NEEDs to be stored.

Usage

Example of Forensic Generation

thash fgen C:\thash\data\small\ -o -b 10MB

This will generate two files in that folder (-o will overwrite files if exists, -b 10MB use blocksize of 10 MegaBytes):

• thashes.txt: contains the hash of all original files in C:\Folder1\Data\
• thashes_hash.txt: contains the hash of thashes.txt

The files generate has the format showed in setction Output Formats

Example of Forensic Verification

thash fcheck C:\Folder1\Data\

This will:

• 1 - Check integrity of thashes_hash.txt against thashes.txt
• 2 - Check integrity of thashes.txt against original files in C:\Folder1\Data\
• 3 - Check if all files in C:\Folder1\Data\ also exists in thashes.txt and vice versa.

Others parameters

Some others parameters that can be used in the command line, as n_workers, buffersize, overwrite results, change name of files, quiet mode, stop on error, etc. that are ommited here for sake of simplicity.

Check all parameters available, using:

Global Parameters:

thash --help

Parameters of subcommands:

thash fgen --help

thash fcheck --help

Important: Global parameters must be before the subcommand (fgen, fcheck). Parameters specifics to each subcommand must be after subcommand.

Audit Log Integrity

To securely record the audit log and ensure future integrity, you should either print the contents of thashes_hash.txt or digitally sign the file.

If you don't have a digital certificate, you can use a free timestamping authority, such as freetsa.org.

Output Formats

Simple Format: The result is text file where each line contains the hash value, thash with algorithm hash and BlockSize used and the relative file path from the original directory:

281d5d93464f1165ea7c403ca99d63ff4bf9a360864f8df4bd0e8e6c03774e98 ?tsha256-50MB*./file_hashed.bin

Obs. Example of simple format, using thash with algorithm sha256 and BlockSize 50MB.

JSON Format: can be implemented in future, to be used for several others applications.

Download

Latest version of thash:

version 0.1.0 - Windows x64

Benchmarks

Using hyperfine, tests between thash, fsum and hashdeep64 were performed.

Machine configs: Windows 11 Home 64bits, AMD Ryzen 7 7800X3D 8-Core Processor 4.20 GHz, 64 GB of RAM, M2 SSD disk.

Comparisons are presented below.

---

---

---