match_hash.sh

match_hash.sh is the script that can take either a hash .txt file or an entire directory, produces a list of file's hash if necessary and get the list of all unknown files based on the NSRL database.

Requirements

The script is responsible for getting the NSRL tools if they are not installed, install them and run them to start the process. It will use git and wget to download the tools if needed. Be sure to have them installed. If they are not, you can install them like so:

sudo apt-get install git wget

Note that the script requires an admin account to run some commands. Therefore, be sure to have the sudo rights or to have credentials of an account with sudo rights.

How to use

The scripts have to distinguish behavior, depending on what kind of input you want to analyse:

Analyse a directory

If you want to analyse a directory, you have to call the script like so:

./match_hash.sh -d <path_to_directory>

It will then install the NSRL Query tools if they are not installed, start the server, populate it with the latest database release, computes a list of the file's hash inside the given directory, and start to compare the hashes. It will then produces a result file in the hash_outputs/ folder in the form <name_of_directory>-day-hour.txt.

Analyse a hash list file

If you want to analyse a hash list file, you have to call the script like so:

./match_hash.sh -f <path_to_file>

It will then install the NSRL Query tools if they are not installed, start the server, populate it with the latest database release, and start to compare the hashes. It will then produces a result file in the hash_outputs/ folder in the form <name_of_file>-day-hour.txt.

Note that your hash file must follow the md5 convention like so:

hash                              file_path
dezf6589a8f5adadgj5t441c2fdd9989  /home/user/Documents/test/file/example.txt
dd25e97dahjv4887g4t1ju33u48cvuzu  /home/user/Documents/test/file/another_test.png

You can easily get this list by using md5deep:

md5deep -r /path/to/collection > all_hashes.txt
nsrllookup < all_hashes.txt > rds_misses.txt

following the instruction given in the nsrllookup README file example