Michael G

Using ClamAV with OPNsense provides an additional layer of security by enabling antivirus scanning at the network level. This setup is particularly useful for intercepting malicious traffic before it reaches client devices, which is beneficial in environments where not all devices have up-to-date antivirus protection.

Installation and Configuration

1. Install ClamAV Plugin: Begin by installing the ClamAV plugin (os-clamav) from the OPNsense plugins view. This will add a new menu entry under services for ClamAV.

2. Configure C-ICAP and ClamAV:

   • Navigate to Services > C-ICAP > Configuration and enable the c-icap service.
   • Under the Antivirus tab, enable ClamAV.
   • Go to Services > ClamAV > Configuration to download signatures and enable the clamd and freshclam services.

3. Enable Proxy and ICAP: Set up a basic caching proxy server in OPNsense and enable transparent HTTP and SSL mode. Configure ICAP to work with ClamAV for scanning traffic.

Considerations

• Resource Requirements: ClamAV requires at least 1.5 GB of RAM to function properly, with 2 GB recommended to prevent crashes.
• TLS Traffic: Scanning encrypted TLS traffic requires decrypting it, which can complicate the setup and is often not recommended for small environments due to potential issues with certificate handling.
• Signature Updates: Regularly update ClamAV signatures using the freshclam service to maintain effective virus detection.

By incorporating ClamAV into OPNsense, administrators can enhance network security through proactive virus scanning, although careful consideration of resource requirements and encrypted traffic handling is necessary.

Beginners guide to Bash, Linux commands and Transcoding. Learn Linux and Bash while transcoding your entire video collection to HEVC (x265) recursively.

What you’ll need:

1. A directory full of Videos
2. A Linux Machine – I am using Manjaro
3. Ffmpeg with your hardware transcode compiled. I am using VA-Api with an AMD graphics card which can also be used with intel graphics. This isn’t a guide on how to compile ffmpeg with hardware transcoding enabled but you can either set up a software transcode, have the same set up as mine, or stay tuned for a future guide!
4. Screen to run the script in the background

Getting what you need:

You can install what you need in Manjaro with:

pamac install ffmpeg-amd-full screen

Or Ubunutu
apt-get install ffmpeg screen

Preparing Script File

Start a new empty file with the touch command
touch ~/convert-video
change permissions of this file so that it can be executed by the owner (you) and your primary group. I’ll go a little more in-depth about permissions. You do this with the change mode command
chmod 774 ~/convert-video
The first 7 means read/write/execute permissions for the owner.
The second 7 means read/write/execute permission for the file’s group. This will be your primary group when you touched the file. You can see your primary group by using
id $USER
Mine shows gid=984(users) which means my primary group is users.
The 4 means everyone else only has read capability on this file.
To verify permissions are set correctly:
ls -l ~/convert-video
Now open with a console text editor:
nano ~/convert-video

Start Scripting!

• First line we hash bang this to be a bash script
  #!/bin/bash
• Next we want to check to make sure the user designates the folder that all of our videos are located. We do this with an if-statement.
  if [ -z $1 ];then echo Give target directory; exit 0;fi
  This statement checks if the user put in anything as an argument when executing the script. If the user enters
  ~/convert-video /videofolder
  everything is good and we continue. If they just enter ~/convert-video The script echos a warning message then exits with a code 0. Code 0 is a non-error exit.
• Next line we use the find command. We will search for all video extensions that do not have a HEVC added on by this script. Later in the script, we will add this to the file when transcoded so we can stop the script and resume it without transcoding the same videos. There are better ways of doing this (like using ffprobe to check the video format). I’m doing it this way to make the script simple to understand and fast.
  find "$1" -depth \( -iname "*.mkv" -o -iname "*.mp4" -o -iname "*.mpg" -o -iname "*.mov" -o -iname "*.avi" -o -iname "*.iso" -o -iname "*.mpeg" \) -a -not -iname "*HEVC.mkv" | while read file ; do
  The find command searches the folder the user inputted, checks all sub-folders with -depth and looks for all entered video extensions. -iname is case insensitive, -o means ‘or’, -a -not means ‘and not’ and we pipe ‘|’ that the rest of the code executes while going through our results.
• We now create three variables based on a result from the find

  directory=$(dirname "$file")
  oldfilename=$(basename "$file")
  newfilename=$(basename "${file%.*}")

  directory is the path of the file. Because we are converting all sub-folders, this could be different from file to file.
  oldfilename is the full name of the movie with its old extension, i.e. movie.avi
  newfilename is the file name with the extension removed. This would actually change movie.avi to movie, as we don’t want .avi to be part of the new filename

• Next line we do the conversion with ffmpeg
  nice -n 15 ffmpeg -threads 4 -i "$directory/$oldfilename" -vaapi_device /dev/dri/renderD128 -vcodec hevc_vaapi -vf format='nv12-f matroska "$directory/$newfilename - HEVC.mkv" </dev/null I’ll go through what all this means as it will be specific to your use case. With that information, hopefully you can tailor it as needed.
  nice -n 15 This set the priority or niceness from the default of 0 to 15. That will give most running programs priority over this command so it does not slow down your system.
  -threads 4 sets the amount of threads your processor uses during the decoding of the video. This means that as written, we are doing a software decode and a hardware encode. Why? Because we don’t know the format of the original file. If we did we could specify a hardware decode. Hint, hint. Decoding is much less intensive then encoding but will take up processor power.
  -i "$directory/$oldfilename"
  this uses our variables for the video input.
  -vaapi_device /dev/dri/renderD128 -vcodec hevc_vaapi -vf format='nv12|vaapi,hwupload'
  This is the information specific to my AMD, open source driver and to HEVC (hevc_vaapi). During a software encode to hevc, this all can be replaced with -c:v libx265
-qp:v 20
  sets the quality level. Lower, the higher the quality but larger file. 19, 20 and 21 are ok choices to look at.
  -crf 26
  is the software encode command you want with
  -crf 28
  being default quality.
  -c:a libvorbis -qscale 5 -ac 6
  is the Sound. Vorbis is the lossy sound file we are converting to. We have a quality of 5 with 10 being the highest. Something between 5-7 is good but I’ve found 5 to be very good. -ac 6 converts this to 5.1 surround sound. You can either omit this setting or set it to your sound system. Note that converting a file from something like stereo (2.1) to 5.1 will increase size.
  matroska "$directory/$newfilename - HEVC.mkv
  And finally this is our output file. It is a matroska container with – HEVC at the end. This is so this script and us know which videos have been converted.
  # rm "$directory/$oldfilename"
  This is a commented out remove command. It is commented out so it is not executed normally. In order to uncomment it (remove the hash at the front), you must understand what will happen. Any Failed Transcode will delete your original file. I haven’t had a good video file fail yet but let it be known. And Any Attempt in Stopping of the script with ctl+c will only stop that ffmpeg instance right there, delete the original file and leave with what you got so far then move on to the next file.
• You must stop the script with ctl-z and delete the file it was creating ending in HEVC.mkv before starting it again
  All that is necessary if you uncomment this rm statement. This can easily be considered a bug but much more elaborate scripting is required to make it work differently.
  done now we end our loop

Complete Script

#!/bin/bash
# convert video types to hevc recursively
# By Michael Gray
#####################################
if [ -z $1 ];then echo Give target directory; exit 0;fi
find "$1" -depth \( -iname "*.mkv" -o -iname "*.mp4" -o -iname "*.mpg" -o -iname "*.mov" -o -iname "*.avi"  -o -iname "*.iso" -o -iname "*.mpeg" \) -a -not -iname "*HEVC.mkv" |  while read file ; do
directory=$(dirname "$file")
oldfilename=$(basename "$file")
newfilename=$(basename "${file%.*}")
nice -n 15 ffmpeg -threads 4 -i "$directory/$oldfilename" -vaapi_device /dev/dri/renderD128 -vcodec hevc_vaapi -vf format='nv12|vaapi,hwupload' -qp:v 20 -c:a libvorbis -qscale  5 -ac 6 -f matroska "$directory/$newfilename - HEVC.mkv" </dev/null
# rm "$directory/$oldfilename"
done

Running The Script

This script will take a very long time to transcode many videos. To make it run in the background
screen -S convert-video
then run it with
~/convert-video /media/yourvideos
Exit the screen with ctl-a then d
To check on the progress, screen -r convert-video and ctl-a then d when you are finished.
Also remember if you stop your script with ctl-z, also delete whatever partial output it was creating.

Oh-My-Zsh’s plugin system will turbocharge your linux experience. If you are a Manjaro Linux user, check out this article on how to integrate Oh-My-Zsh with the default terminal.
I have a list of great plugins to add to your ~/.zshrc
You can also check out the full list Oh-My-Zsh Github Plugins

List of My Favorite Plugins

plugins=(
    copybuffer
    copydir
    copyfile
    dirhistory
    emoji
    encode64
    git
    history
    jsontools
    kubectl
    rsync
    sudo
    web-search
)

1. copybuffer

Copy buffer allows you to copy what is already typed on the command line with ctl+o

2. copydir

Copydir will add your current directory the clip board. Just need to type copydir in the terminal

3. copyfile

Just enter copyfile <your_filename> in the terminal and the contents of the file will be added to your clipboard.

4. dirhistory

dirhistory helps one navigate their previous directories. Alt+LeftArrow takes you back to your previous directory while Alt+RightArrow takes you forward. to where your were. Alt+Up takes you to the parent directory (the equivlent of cd ../ )

5. emoji

emoji, as the name suggests, adds emoji support to your terminal

6. encode64

This will encode or decode your input to and from base64
encode64 <your string> and decode64 <your string>

7. git

This creates a bunch of aliases for common git commands. There are too many too list here but you can find them here.

8. history

This aliases the history command that is used to show your past commands. h prints the list of commands hs <search> searches while hsi <search searches case insensitive.

9. jsontools

This will help you work with json files:

• pp_json: pretty prints json.
• is_json: returns true if valid json; false otherwise.
• urlencode_json: returns a url encoded string for the given json.
• urldecode_json: returns decoded json for the given url encoded string.

10. kubectl

Maybe a little narrow but aliases a bunch of useful kuburnetes commands if you manage a cluster

11. rsync

rsync is the correct way to copy files and this will alias key commands for you :

• rsync-copy rsync -avz –progress -h
• rsync-move rsync -avz –progress -h –remove-source-files
• rsync-update rsync -avzu –progress -h
• rsync-synchronize rsync -avzu –delete –progress -h
  

  12. sudo

  saving one of the best for last I don’t know how I lived without this command. Hitting ‘esc’ twice takes your previous command and adds sudo to the front. Very useful if you needed elevated privileges to run that command

  13. web-search

  This one will start your web search from your terminal and open your browser.
  web_search google pizza
google pizza
web_search duckduckgo pizza
duckduckgo pizza