Today’s post is about one leg of the CIA Triad, Integrity, which means protecting data from unauthorized modification. There are a number of things needed to ensure data integrity, but let’s focus on detection. More specifically, let’s focus on detecting changes to files.

Detecting changes

The easiest way to detect if a file has been changed is to hash it and compare that to a previous hash of the file. Hashing algorithms will detect small changes in files to produce very different outputs. For example, this is the SHA256 hash of Allen Ginsberg’s Howl:


and this is the SHA256 hash of Allen Ginsberg’s Howl with an extra space at the end:


So, having a hash of a file will allow you to quickly determine if it’s been modified. For this reason, hashing is frequently used in File Integrity Monitoring solutions.

Of course, your choice in hashing algorithm matters. A number of security issues have been discovered with the hashing algorithm MD5; a quick search can give you more details. TL;DR don’t use MD5 for integrity checking, use SHA256.

Use Case

You can use it to ensure the integrity of import documents and files. For instance, when submitting an Incident Response Report, you should always add an additional document providing hashes of every file included with the report. This way, it will be easy to detect if the report or supporting documents have been modified.

Here’s a little script I wrote to quickly hash all files in a directory and create a csv



# directory from command line to recursively hash
if [ -z "$1" ]; then
  echo "supply directory with files to hash"
# input must end in / so the awk command later will function properly.
elif [[ $1 = *"/" ]]; then 

# get basename of directory to name output file
SAVETO=$(basename $DIR)

# write CSV header
echo "File, SHA256" > hashed_$SAVETO.csv

# find all files in directory and:
#   hash it if the file is NOT DS_Store
#   print filename and its hash to file
find $DIR -type f -print0 | xargs -0 openssl sha256 | awk -F \/\/ '$2 !~ /DS_Store/ {print $2}' | awk -F \= '{match($1,"\\)")}{OFS=","}{print substr($1, 0, RSTART-1), $2}' >> hashed_$SAVETO.csv

echo "hashes saved to hashed_$SAVETO.csv"

Just include the directory of files as a command line argument and you’re good to go!

Keeping Track of How Tos

Working at a start-up, I get to wear many hats and gain experience in a wide-range of topics.

But that also means picking up a project, learning skills needed to complete it, then moving on to the next project which may not require any of the knowledge I just acquired. For example, as part of an Incident Response project last year, I needed to determine how easily a password hash could be cracked. So, I learned how to use John the Ripper and about hash cracking in general. After concluding that project, I moved onto the next thing. It wasn’t until the next incident a year later that I needed to use John the Ripper again.

Then there are those random questions you never want to waste time figuring out again. Like, “How do I remove newline characters from within quoted text in this csv so that awk sees the quoted text as one field?”

This quick succession and breadth of topics meant that I’d often have to recall something from a few months ago.

So, how do I prevent having to relearn something I knew last week/month/year?

How Tos! lots and lots of How Tos. So many How Tos that managing them became an issue. But that’s a good problem to have, and one I solved with a simple little bash script.


With this script from my repo, you can list, search by topic, open, edit, and create new HowTos.

Using the help option will explain what the Topic Tags mean. Then you can make a new HowTo using the new option. Once you’ve written a couple HowTos you’ll be able to find them easily by searching with the Topic Tag, though sometimes I’ll just list them all out and pipe through grep.

There’s also a batch of Howtos in my repo to help get you started.

It’s made my work much easier having a quick way to parse through my notes and HowTos. I hope it helps you.