After purchasing a dashboard camera for my car, I started building a private server environment for dashcam upload and storing of video files.

The top-of-the-range BlackVue DR900X dashboard camera Has built-in WiFi features that makes it very convenient to transfer video clips to an external server instead of fiddling with SD cards or USB cables. And BlackVue offers an out-of-the-box solution for this exact purpose.

However, I find BlackVue’s cloud service for storing clips too pricey. At the time of writing, you will be charged $25 a month for a mere 15 Gb. Keep in mind that the front camera delivers 4K resolution HEVC videos.

One minute of recording leaves you with a video file at just below 200 Mb. In addition, the rear camera’s file is about 80 Mb. If you intend to keep a reasonable amount of history, 15 Gb won’t take you far.

Emby media server

Hence, I found a way of transferring the video files from the camera to my local NAS using the scripts showed below. Fortunately, the camera has a convenient API for retrieving video files over HTTP.

I landed on using Emby media server as an internal frontend/streaming solution. It plays very well with my Synology NAS. Emby also provides lightning fast hardware accelerated transcoding on supported devices — if you’re willing to pay 5$ a month for the Premiere edition.

Allthough my installation does not expose Emby directly to the outside world. I still can access it over VPN, either by using the web interface or by installing one of the dedicated apps for Android or iPhone.

Uploading the files from the camera

The dashcam is configured to automatically connect to the Wifi access point each time the car enters it’s parking spot. For this, I have a dedicated network for IoT devices, only allowing internet traffic with no direct access to any of my other private networks.

A low-end Intel NUC server runs a shell script constantly checking whether the camera is present on the network. Once the camera attempts to connect to the BlackVue Cloud service, the NUC starts the dashcam upload process.

I have created a limited user on the NAS, only giving access to its Samba service and dashcam video share. A very narrow firewall rule is put in place to allow SMB traffic from the NUC (on the IoT network), to the NAS network.

The shell script on the NUC is executed at frequent intervals. The source listing is shown below. Initially, the script checks if the camera is present on the network by running netcat against port 80 on the dashcam’s IP address (line 15).

#!/bin/bash

log() {
  touch /var/log/dashcam.log
  printf "%s:   %s\n" "$(date)" "$1" >> /var/log/dashcam.log
  tail -n 1000 /var/log/dashcam.log > /tmp/dashcam.log.tmp
  truncate -s0 /var/log/dashcam.log
  cat /tmp/dashcam.log.tmp >> /var/log/dashcam.log
}

CREDENTIALS="username%password"
NAS_SHARE="\\\\x.x.x.x\\dashcam"
DASHCAM_IP="y.y.y.y"

if nc -z $DASHCAM_IP 80 2>/dev/null; then
  cd "$(dirname "$0")"
  rm -rf upload
  mkdir -p upload
  cd upload
  declare -a sortedfilearray
  log "Collecting files from camera"
  for file in `curl http://${DASHCAM_IP}/blackvue_vod.cgi | sed 's/^n://' | sed 's/,s:1000000//' | sed $'s/\r//'`; do
    filetitle=$(basename "$file")
    if [[ "$filetitle" == *"F.mp4"* ]] || \
       [[ "$filetitle" == *"R.mp4"* ]]; then
      filearray[$i]="$file"
      i=$i+1
    fi
  done
  log "Preparing file array"
  IFS=$'\n' sortedfilearray=($(sort <<<"${filearray[*]}"))
  unset IFS
  existing_files=$( sudo smbclient -U $CREDENTIALS $NAS_SHARE -c "dir Record/*.mp4" )
  for file in "${sortedfilearray[@]}"; do
    filetitle=$(basename "$file")
    if [[ "$existing_files" != *"$filetitle"* ]]; then
      log "Transferring file ${filetitle}"
      curl -o "$filetitle" "http://${DASHCAM_IP}${file}"
      log "Upload to NAS..."
      sudo smbclient -U $CREDENTIALS $NAS_SHARE --directory Record -c "put ${filetitle}" || log "ERROR: Could not upload to NAS"
      rm -f "$filetitle"
    fi
  done
  log "All files scanned"
  cd "$(dirname "$0")"
  rm -rf upload
else
  log "Dashcam offline"
fi

If the camera is found, the script iterates through a list of all video files stored on the camera’s SD card. The video files list is retrieved over plain HTTP directly from the camera (line 22).

Each valid video file is stored in an array. Then a list of existing files is retrieved from the NAS (line 33). The lists are compared to each other (line 36) and any new files are downloaded from the camera to the NUC (line 38) and then uploaded to the NAS (line 40). The new files will be accessible to Emby (that runs in a Docker container on the NAS) — as soon as the permissions and ownership settings are applied to the files most recently added.

Keeping upload within disk quota

Even a NAS has its limitations with regard to disk space. I have set an absolute maximum size for my dashcam upload folder, to prevent the network storage from being flooded by video files.

However, merely relying on a preset maximum disk quota on the NAS will eventually result in abrupt I/O errors. Therefore, we will need some form of file rotation to keep disk usage down, so as to avoid hitting the maximum limit at all.

A script is scheduled to run every 10 minute on the NAS, checking for available space and, if required, deleting old files to make room for new. The script also takes care of setting file permissions and ownership in order for Emby to access recent uploads.

But what about the files I actually want to keep beyond the imposed time limit? There have to be ways to de-schedule important files from ever being deleted by the pruning process.

Using Emby’s API

Fortunately, Emby provides just the means to these ends. Simply mark the relevant files as favorites and use Emby’s API to exclude the favorites from deletion. The dashcam upload script will hence ignore favorited videos when pruning old video content.

The script is shown below. The API key (line 9) can be obtained from Emby’s user interface. User ID can be snatched from the URLs to the API calls under the network tab in your browser’s developer tools.

<?php

  const DIR_RECORD = './Record';

  const MAX_QUOTA =        3900000000000;
  const REQUIRED_FREE_SPACE =  500000000;
  const MAX_FILES_TO_DELETE =       1000;

  const EMBY_API_KEY = '963d0e933dbc7de94822d4f5f385706e';
  const EMBY_USER_ID = 'c35df0516ca6df4ddd2354b51c085bfd';

  function get_available_space() {
    $available_space = MAX_QUOTA;
    if( preg_match( '/^(\d+).*$/', shell_exec( "du -sb " . DIR_RECORD ), $m ) ) {
      $available_space -= intval( $m[ 1 ] );
    }
    return $available_space;
  }

  function get_oldest_emby_items() {
    $baseUrlFmt = "http://localhost:8096/Users/%s/Items";
    $query = [
      'SortBy'           => 'SortName',
      'SortOrder'        => 'Ascending',
      'IncludeItemTypes' => 'Video',
      'Recursive'        => 'true',
      'StartIndex'       => '0',
      'ParentId'         => 'XXXX',
      'api_key'          =>  EMBY_API_KEY
    ];
    $httpQuery = http_build_query( $query );
    $baseUrl = sprintf( $baseUrlFmt, EMBY_USER_ID );
    $json = file_get_contents( "{$baseUrl}?{$httpQuery}" );
    return json_decode( $json, true );
  }

  function delete_if_not_favorite( $embyItem ) {
    $nameOK = preg_match( '/^[0-9]{8}_[0-9]{6}_[A-Z]{2}$/', $embyItem['Name'] );
    $isFavorite = $nameOK && $embyItem[ 'UserData' ][ 'IsFavorite' ];
    if( !$isFavorite ) {
      $filePath = DIR_RECORD . "/{$embyItem['Name']}.mp4";
      if( file_exists( $filePath ) ) {
        echo( "Delete old file in target directory: {$filePath}\n" );
        shell_exec( "rm {$filePath}" );
        return true;
      }
    }
    return false;
  }

  function delete_oldest_files() {
    $embyItems = get_oldest_emby_items();
    $filesDeleted = 0;
    foreach( $embyItems[ 'Items' ] as $embyItem ) {
      if( delete_if_not_favorite( $embyItem ) == false ) {
        continue;
      }
      $spaceOK = get_available_space() >= REQUIRED_FREE_SPACE;
      $reachedDeletionLimit = ( ++$filesDeleted == MAX_FILES_TO_DELETE );
      if( $spaceOK || $reachedDeletionLimit ) {
        echo( "Done!\n" );
        if( $reachedDeletionLimit ) {
          echo( "Stopped deleting after {$filesDeleted} files\n" );
        }
        break;
      }
    }
    return $filesDeleted;
  }

  function prune_target_directory() {
    if( get_available_space() < REQUIRED_FREE_SPACE ) {
      delete_oldest_files();
    }
  }

  prune_target_directory();

?>

The PHP script sends an API query to Emby, requesting the list of videos (line 33), ordered by file title in ascending order (line 24). The file titles correspond to the file names, which are the date and time for the recording in YYYYMMDD_hhmmss format. Therefore, sorting the entries in ascending order will give you the oldest files first.

As long as there are less than REQUIRED_FREE_SPACE bytes left on the disk share, the script iterates through the files from Emby, starting with the oldest ones (line 73). For each file, it checks the isFavorite flag (line 39) and deletes the file only if it is not flagged as favorite.

Disclaimer

As a safety measure, MAX_FILES_TO_DELETE limits the number of files that may be deleted in one run, to somewhat ease the consequences of setting the wrong parameters. However, always remember to take backup copies!

Using my examples is at own risk. I have not fully tested the code. The scripts published here are only intended as proofs of concept, they are not suited for production use and I assume no responsibility for bugs and errors that may lead to data loss!