auditor

auditor is a forensic tool for fast integrity auditing that uses cryptographic hash functions .

It is similar to other popular tools (fsum, hashdeep, sha256sum, etc.), but with features to make digital data auditing simpler and faster.

auditor has support to several hash algorithms. The default is sha256, that is recommended by NIST since 2015 ( see NIST Policy on Hash Functions). By default, thash method is enabled, but this mode can be disabled.

Usage

With auditor installed in your system, you can use it as follows:

hash
check
info
Manual
Important !

hash is the first step of forensic auditing. It generates audit files that allow verifying file integrity.

Examples

auditor hash input_path

This will:

  1. Hash ALL files in input_path with default algorithm (sha256)
  2. Generate audit files:
    • Audit_FullList: contains the hash list of file(s) in input_path. Default name: hashes.txt
    • Audit_Stamp: contains only the hash of Audit_FullList. Default name: hashes_stamp.txt.This is the file that needs to be printed or digitally signed to ensure check of all integrity chain.
  3. After this command, you can perform check command to verify integrity.
  4. To properly ensure integrity check in the future, follow advices in Important!
auditor hash : Benchmarks auditor hash -d : (disable thash mode,to compare with proof.sh in section Proof) Hash no Thash
Others examples:

Example 1: Overwrite audit files (-o), use blockSize 10MB and 'whirlpool' hash function 

auditor hash input_path -o -b 10MB -a whirlpool

Example 2: Overwrite audit files (-o), disable 'thash method' (-d), and using blake3 hash function (-a) 

auditor hash input_path -o -d -a blake3

Example 3: Use 100 workers (-w 100), overwrite audit files (-o), use blockSize 10MB (-b 10MB) and 'whirlpool' hash function (-a whirlpool) 

auditor -w 100 hash input_path -o -b 10MB -a whirlpool

Example 4: Overwrite audit files (-o), include only txt files (-i "**/*.txt" ) only in root folder (-u 1), 

auditor hash input_path -o -i "**/*.txt" -u 1

Example 5: Just generates hashes, but don´t create any files

auditor  hash input_path -l

Others examples with global parameters. Please notice that global parameters must be before the subcommand . See parameters section to details!


check is the second step of forensic auditing. It checks the the integrity of the audit files after hash was performed.

Examples of Check

auditor check input_path

This will:

  1. Check integrity of audit file Audit_Stamp against audit file Audit_FullList
  2. Check integrity of Audit_FullList against all original files in input_path
  3. Check if all files in input_path also exists in Audit_FullList and vice versa.
auditor check : Check
Others examples:

Example 1: Check in quiet mode (-q) and stop on first error (-s) 

auditor -q -s check input_path


info does not perform hash integrity check. It only tests the audit files and the content of input_path and shows useful information.

Example of Forensic Info

auditor info input_path

This will:

  1. Verify if audit files Audit_Stamp and Audit_FullList exists.
  2. Verify if files listed in Audit_Stamp and Audit_FullList exists and listed size is the same that in input_path.
  3. Verify if all files in input_path are listed in Audit_FullList.
  4. Give a report of size of the files and of all input_path.
auditor info: info

Usage

Usage: auditor.exe [OPTIONS] <COMMAND>

Subcommands:

  • hash: Hash <input_path> and generates audit files (<Audit_FullList>/<Audit_Stamp>)
  • check: Checks the integrity of audit files (<Audit_FullList>/<Audit_Stamp>) and <input_path>
  • info: Only tests if audit files and <input_path> are synchronized. This doesn't check the integrity!

Global Options:

  • -x, --stop: Stops and fails immediately on any error
  • -p, --ignore-permissions-errors: Proceed when encountering access permission errors with folders or files. If the '--stop' flag is used without this flag, such errors will cause the operation to fail.
  • -q, --quiet: Runs the program in quiet mode
  • -t, --progress: Track progress of running tasks
  • -w, --n-workers <N_WORKERS>: Number of worker threads. Default is number of cores on your computer.
  • -c, --n-max-concur <N_MAX_CONCUR>: Maximum number of concurrent access to a same file.
  • -l, --log-level <LOG_LEVEL>: Log level: debug, info, warn, error [default: Off]
  • -v, --verbosity...: Increases the verbosity level. -v: minimum , -vv: some, -vvv: many
  • -h, --help: Print help (see more with '--help')
  • -V, --version: Print version

Subcommand hash

Hash <input_path> and generates audit files (<Audit_FullList>/<Audit_Stamp>)

Usage: auditor.exe hash [OPTIONS] <INPUT_PATH>

Arguments:

  • <INPUT_PATH>: Path to the data that require integrity assurance. (will be hashed)

Options:

  • -n, --audit-basename <AUDIT_BASENAME>: If specified, it changes <audit_basename> of the audit files. See with option --help for details. [default: hashes.txt]
  • -f, --audit-full <AUDIT_FULLLIST_FILE>: If specified, uses this whole path as <Audit_FullList>, that can be in anywhere. See with option --help for details.
  • -s, --audit-stamp <AUDIT_STAMP_FILE>: If specified, uses this whole path as <Audit_Stamp>, that can be in anywhere. See with option --help for details.
  • -o, --overwrite-audit-files: Enables Overwrite mode, which will delete existing audit files and create new ones.
  • -b, --block <BLOCKSIZE>: BlockSize for thash mode. Use KB, MB, GB or TB. Ex: 10MB. [default: 50MB]
  • -a, --alg-hash <ALGORITHM_TO_HASH>: Algorithms to hash: sha256, sha512, whirlpool, blake3, k12 (kangarootwelve), sha3_256, sha3_512, keccak256, keccak512. (sha256 and sha512 are recommended by NIST at release date of this version, --help for more info) [default: sha256]
  • -d, --disable-thash: Disable 'thash method' mode. This will force hash files in the normal method, and can be significant slower to big files. See http://thash.org to learn more.
  • -k, --no-stamp: Don’t create the <Audit_Stamp> (but will create the <Audit_FullList>).
  • -l, --no-audit: Just show hashes. Don’t create the <Audit_FullList> neither <Audit_Stamp>.
  • -u, --max-depth <MAX_RECURSIVE_DIR_DEPTH>: Maximum recursive directory depth level. 1: Only one level (current dir), 2: Two levels, etc... Default: '0', infinite. no limit!
  • -i, --include-glob-pattern <INCLUDE_GLOB_PATTERN>...: Include only files that match the Glob patterns. If not used, will include all files. Can be used multiple times. Use the char " to enclose it. Examples: "**/*.txt" "**/*.{txt,doc}" "**/*file1*" .
  • -e, --exclude-glob-pattern <EXCLUDE_GLOB_PATTERN>...: Exclude files that match the Glob pattern. It works over included files. Can be used multiple times. Use the char " to enclose it. Examples: "**/*.txt" "**/*.{txt,doc}" "**/*file1*" .
  • -h, --help: Print help (see more with '--help')

Notice: Global parameters must be before the subcommand (gen, check). Parameters specifics to each subcommand must be after subcommand.

Subcommand check

Checks the integrity of audit files (<Audit_FullList>/<Audit_Stamp>) and <input_path>

Usage: auditor.exe check [OPTIONS] <INPUT_PATH>

Arguments:

  • <INPUT_PATH>: Path to the directory that contains the audit files and the original data to check the entire integrity chain.

Options:

  • -n, --audit-basename <AUDIT_BASENAME>: If specified, it changes <audit_basename> of the audit files. See with option --help for details. [default: hashes.txt]
  • -f, --audit-full <AUDIT_FULLLIST_FILE>: If specified, uses this whole path as <Audit_FullList>, that can be in anywhere. See with option --help for details.
  • -s, --audit-stamp <AUDIT_STAMP_FILE>: If specified, uses this whole path as <Audit_Stamp>, that can be in anywhere. See with option --help for details.
  • -k, --no-stamp: It will not check <Audit_stamp>. Just check hash files inside <Audit_FullList>.
  • -u, --max-depth <MAX_RECURSIVE_DIR_DEPTH>: Maximum recursive directory depth level. 1: Only one level (current dir), 2: Two levels, etc... Default: '0', infinite. no limit!
  • -i, --include-glob-pattern <INCLUDE_GLOB_PATTERN>...: Include only files that match the Glob patterns. If not used, will include all files. Can be used multiple times. Use the char " to enclose it. Examples: "**/*.txt" "**/*.{txt,doc}" "**/*file1*" .
  • -e, --exclude-glob-pattern <EXCLUDE_GLOB_PATTERN>...: Exclude files that match the Glob pattern. It works over included files. Can be used multiple times. Use the char " to enclose it. Examples: "**/*.txt" "**/*.{txt,doc}" "**/*file1*" .
  • -h, --help: Print help (see more with '--help')

Subcommand info

Only tests if audit files and <input_path> are synchronized. This doesn't check the integrity!

Usage: auditor.exe info [OPTIONS] <INPUT_PATH>

Arguments:

  • <INPUT_PATH>: Path to the directory that contains the audit files and the original data.

Options:

  • -n, --audit-basename <AUDIT_BASENAME>: If specified, it changes <audit_basename> of the audit files. See with option --help for details. [default: hashes.txt]
  • -f, --audit-full <AUDIT_FULLLIST_FILE>: If specified, uses this whole path as <Audit_FullList>, that can be in anywhere. See with option --help for details.
  • -s, --audit-stamp <AUDIT_STAMP_FILE>: If specified, uses this whole path as <Audit_Stamp>, that can be in anywhere. See with option --help for details.
  • -k, --no-stamp: It will not check <Audit_stamp>. Just check hash files inside <Audit_FullList>.
  • -h, --help: Print help (see more with '--help')

About the integrity of data

To securely ensure future check of all chain of integrity, you should:

  1. Save all data, including audit files, and either print the contents of Audit_Stamp or digitally sign this file. If you don´t do this, someone can simply change the data an generate new audit files.
  2. In future, when someone performs a check, the content of audit file Audit_Stamp MUST BE the same of the printed or digitally signed version done in step 1. If does not match, the integrity check is not valid.

If you don't have a digital certificate, you can use a free timestamping authority, such as freetsa.org.

Output Formats

The format of audit files are simple. Each line contains:

hash_value ?algorithm|file_size[:hex]*relative_filepath

Example 1:

281d5d93464f1165ea7c403ca99d63ff4bf9a360864f8df4bd0e8e6c03774e98 ?tsha256-50MB|500000*file_hashed.bin

Obs 1. Example of simple format, one using thash with algorithm sha256 and BlockSize 50MB, and other using normal method, just with algorithm sha256.

Obs 2. The size of file is important, because with it we can minimize check verification time. Why hash a big file when already is known that its size does not match with original?

Example 2:

7357b67824d086dc53f5e1ded565f500456bea1812783f1fbcddc08fddc3944c ?sha256|2233:hex*1aCb344356e4e2b2b6

Obs 3. To filepaths with characters like '\n', '\r' or '\0', permitted in some OS, auditor convert them to hex to avoid problems.


Others formats can be implemented in future.

Download

Latest version of auditor:

v0013 - Windows 64 bits v0013 - Linux 64 bits

Benchmarks

Using hyperfine, tests between auditor, fsum and hashdeep64 were performed. The tests were performed on a

Comparisons are presented below.

Machine configs: 
S.O.: Windows 11 Home 64bits
Processor: AMD Ryzen 7 (7800X3D 4.20 GHz)
RAM: 64 GB of RAM (Corsair Vengeance DDR5 64GB - 5200MHz)
Disk: SSD M.2 2TB (Corsair MP600 Pro NVMe)

Data Source:
Benchmarks
Benchmarks: Benchmarks
Have suggestions or found a bug? Contact us at: [email protected]