Merge pull request #62 from mag37/env-rework

Reworked --env-file logic.

.gitignore

@@ -1,2 +1,4 @@
 # ignore users custom notify.sh
 /notify.sh
+# ignore the auto-installed regctl
+regctl

README.md

@@ -12,55 +12,55 @@
 </p>
 
 <h3 align="center">A script checking updates for docker images <b>without pulling</b><br>Then selectively auto-update containers.</h3>
-<h4 align="center">With features like excluding specific containers, filter by name, auto-prune dangling images and more.</h4</h3>
+<h2 align="center">Now with a simple Apprise integration!</h2>
+<h4 align="center">With features like excluding specific containers, filter by name, auto-prune dangling images and more.</h4>
 
+___
+## :bell: Changelog
 
-### :bell: Changelog
+- **v0.3.8**: Fixed `--env-file` logic to work with multiple env-files.
+- **v0.3.7**: Added support for [labels](#bookmark-labels), added the `-f` option (force restart stack).
+- **v0.3.6**: Added pushbullet template.
+- **v0.3.5**: Added a simple progress bar for the registry checkup.
+- **v0.3.4**: Added ntfy.sh template and error message on registry fail.
 - **v0.3.3**: Added Apprise template and the option `-i` inform - to send notifications.
 - **v0.3.2**: Added a notify function to wrap a notify-script, currently DSM/Ssmtp + template script.
-- **v0.3.1**: Addded option `-m` , monochrome mode - no printf color codes.
+
-- **v0.3.0**: Added option `-d N`, age (days) new images have to be before being pulled and updated.
-- **v0.2.6**: regctl check / download logic changed. Now using the scripts directory as primary location.
-- **v0.2.5**: Added a new option `-s` to include stopped containers in the check for updates.
-- **v0.2.4**: Fixed a bug with the Exclude-logic to only exclude exact matches. Added a counter.
-- **v0.2.3**: Added a self updating function (curl/git) and a ugly changenote-message for updates.
-- **v0.2.2**: Fixed breaking errors with multi-compose, odd breakage and working dir error.
-- **v0.2.1**: Added option to exclude a list of containers.
-- **v0.2.1**: Added multi-compose support (eg. override). 
-- **v0.2.0**: Fixed error with container:tag definition. 
-- **v0.1.9:** Fixed custom env-support. 
 ___
 
-## Dependencies
+## :nut_and_bolt: Dependencies
 - Running docker (duh) and compose, either standalone or plugin.   
-- [`regclient/regctl`](https://github.com/regclient/regclient) (Licensed under [Apache-2.0 License](http://www.apache.org/licenses/LICENSE-2.0))   
+- [regclient/regctl](https://github.com/regclient/regclient) (Licensed under [Apache-2.0 License](http://www.apache.org/licenses/LICENSE-2.0))   
   - User will be prompted to download `regctl` if not in `PATH` or `PWD`.   
-  - regctl requires `amd64/arm64` - see [workaround](#workaround-for-non-amd64--arm64) if other architecture is used.
+  - regctl requires `amd64/arm64` - see [workaround](#roller_coaster-workaround-for-non-amd64--arm64) if other architecture is used.
 ___
 
 
 ![](extras/example.gif)
 
-## `dockcheck.sh`
+## :mag_right: `dockcheck.sh`
 ```
 $ ./dockcheck.sh -h
- Syntax:     dockcheck.sh [OPTION] [part of name to filter]
+Syntax:     dockcheck.sh [OPTION] [part of name to filter]
- Example:    dockcheck.sh -y -d 10 -e nextcloud,heimdall
+Example:    dockcheck.sh -y -d 10 -e nextcloud,heimdall
- 
+
- Options:
+Options:"
- -a|y   Automatic updates, without interaction.
+-a|y   Automatic updates, without interaction.
- -d N   Only update to new images that are N+ days old. Lists too recent with +prefix and age. 2xSlower.
+-d N   Only update to new images that are N+ days old. Lists too recent with +prefix and age. 2xSlower.
- -e X   Exclude containers, separated by comma.
+-e X   Exclude containers, separated by comma.
- -h     Print this Help.
+-f     Force stack restart after update. Caution: restarts once for every updated container within stack.
- -i     Inform - send a preconfigured notification.
+-h     Print this Help.
- -m     Monochrome mode, no printf color codes.
+-i     Inform - send a preconfigured notification.
- -n     No updates, only checking availability.
+-l     Only update if label is set. See readme.
- -p     Auto-Prune dangling images after update.
+-m     Monochrome mode, no printf color codes.
- -r     Allow updating images for docker run, wont update the container.
+-n     No updates, only checking availability.
- -s     Include stopped containers in the check. (Logic: docker ps -a)
+-p     Auto-Prune dangling images after update.
+-r     Allow updating images for docker run, wont update the container.
+-s     Include stopped containers in the check. (Logic: docker ps -a).
 ```
 
-Basic example:
+
+### Basic example:
 ```
 $ ./dockcheck.sh
 . . .
@@ -79,7 +79,7 @@ Enter number(s) separated by comma, [a] for all - [q] to quit:
 Then it proceedes to run `pull` and `up -d` on every container with updates.   
 After the updates are complete, you'll get prompted if you'd like to prune dangling images.
 
-### :loudspeaker: Notifications
+## :loudspeaker: Notifications
 Trigger with the `-i` flag.   
 Run it scheduled with `-ni` to only get notified when there's updates available!  
 
@@ -89,24 +89,26 @@ Current templates:
 - Email with [sSMTP](https://wiki.debian.org/sSMTP)  
 - Apprise (with it's [multitude](https://github.com/caronc/apprise#supported-notifications) of notifications)
   - both native [caronc/apprise](https://github.com/caronc/apprise) and the standalone [linuxserver/docker-apprise-api](https://github.com/linuxserver/docker-apprise-api)
+  - Read the [QuickStart](extras/apprise_quickstart.md)
+- [ntfy.sh](https://ntfy.sh/) - HTTP-based pub-sub notifications.
+- [Pushbullet](https://www.pushbullet.com/) - connecting different devices with cross-platform features.
 
 Further additions are welcome - suggestions or PR!   
-Initiated and first contributed by [yoyoma2](https://github.com/yoyoma2).  
+<sub><sup>Initiated and first contributed by [yoyoma2](https://github.com/yoyoma2).</sup></sub>  
 
+## :bookmark: Labels
+Optionally add labels to compose-files. Currently these are the usable labels:
+```
+    labels:
+      mag37.dockcheck.restart-stack: true
+      mag37.dockcheck.update: true
+```
+- `mag37.dockcheck.restart-stack: true` works instead of the `-f` option, forcing stop+restart on the whole compose-stack (Caution: Will restart on every updated container within stack).
+- `mag37.dockcheck.update: true` will when used with the `-l` option only update containers with this label and skip the rest. Will still list updates as usual.
 
-### :warning: `-r flag` disclaimer and warning
+## :roller_coaster: Workaround for non **amd64** / **arm64**
-**Wont auto-update the containers, only their images. (compose is recommended)**   
-`docker run` dont support using new images just by restarting a container.  
-Containers need to be manually stopped, removed and created again to run on the new image.
-
-### :hammer: Known issues
-- No detailed error feedback (just skip + list what's skipped).
-- Not respecting `--profile` options when re-creating the container.
-- Not working well with containers created by Portainer.
-
-### Workaround for non **amd64** / **arm64**
 `regctl` provides binaries for amd64/arm64, to use on other architecture you could try this workaround.
-Run regctl in a contianer wrapped in a shell script. Copied from [regclient/docs/install.md](https://github.com/regclient/regclient/blob/main/docs/install.md):
+Run regctl in a container wrapped in a shell script. Copied from [regclient/docs/install.md](https://github.com/regclient/regclient/blob/main/docs/install.md):
 
 ```sh
 cat >regctl <<EOF
@@ -124,15 +126,23 @@ chmod 755 regctl
 ```
 Test it with `./regctl --help` and then either add the file to the same path as *dockcheck.sh* or in your path (eg. `~/.local/bin/regctl`).
 
+## :hammer: Known issues
+- No detailed error feedback (just skip + list what's skipped).
+- Not respecting `--profile` options when re-creating the container.
+- Not working well with containers created by Portainer.
 
-## `dc_brief.sh`
+## :warning: `-r flag` disclaimer and warning
-Just a brief, slimmed down version of the script to only print what containers got updates, no updates or errors.
+**Wont auto-update the containers, only their images. (compose is recommended)**   
+`docker run` dont support using new images just by restarting a container.  
+Containers need to be manually stopped, removed and created again to run on the new image.
 
-# License
+## :scroll: License
 dockcheck is created and released under the [GNU GPL v3.0](https://www.gnu.org/licenses/gpl-3.0-standalone.html) license.
+
+### :floppy_disk: The [story](https://mag37.org/posts/project_dockcheck/) behind it. 1 year in retrospect.
+
 ___
 
-### Check out a spinoff project: [Palleri/DCW](https://github.com/Palleri/DCW) for a WebUI-front with exporters and notifications.
 
 ## Special Thanks
 - :bison: [t0rnis](https://github.com/t0rnis)   

dockcheck.sh

@@ -1,6 +1,6 @@
 #!/usr/bin/env bash
-VERSION="v0.3.3"
+VERSION="v0.3.8"
-### ChangeNotes: Added option -i, inform - notify. Added Appraise template.
+### ChangeNotes: Fixed --env-file logic to work with multiple env-files.
 Github="https://github.com/mag37/dockcheck"
 RawUrl="https://raw.githubusercontent.com/mag37/dockcheck/main/dockcheck.sh"
 
@@ -24,8 +24,10 @@ Help() {
   echo "-a|y   Automatic updates, without interaction."
   echo "-d N   Only update to new images that are N+ days old. Lists too recent with +prefix and age. 2xSlower."
   echo "-e X   Exclude containers, separated by comma."
+  echo "-f     Force stack restart after update. Caution: restarts once for every updated container within stack."
   echo "-h     Print this Help."
   echo "-i     Inform - send a preconfigured notification."
+  echo "-l     Only update if label is set. See readme."
   echo "-m     Monochrome mode, no printf color codes."
   echo "-n     No updates, only checking availability."
   echo "-p     Auto-Prune dangling images after update."
@@ -42,12 +44,14 @@ c_teal="\033[0;36m"
 c_reset="\033[0m"
 
 Stopped=""
-while getopts "aynprhisme:d:" options; do
+while getopts "aynpfrhlisme:d:" options; do
   case "${options}" in
     a|y) AutoUp="yes" ;;
     n)   AutoUp="no" ;;
     r)   DRunUp="yes" ;;
     p)   AutoPrune="yes" ;;
+    l)   OnlyLabel=true ;;
+    f)   ForceRestartStacks=true ;;
     i)   [ -s $ScriptWorkDir/notify.sh ] && { source $ScriptWorkDir/notify.sh ; Notify="yes" ; } ;;
     e)   Exclude=${OPTARG} ;;
     m)   declare c_{red,green,yellow,blue,teal,reset}="" ;;
@@ -128,9 +132,20 @@ datecheck() {
   fi
 }
 
+progress_bar() {
+  QueCurrent="$1"
+  QueTotal="$2"
+  ((Percent=100*${QueCurrent}/${QueTotal}))
+  ((Complete=50*${Percent}/100)) # change first number for width (50)
+  ((Left=50-${Complete})) # change first number for width (50)
+  BarComplete=$(printf "%${Complete}s" | tr " " "#")
+  BarLeft=$(printf "%${Left}s" | tr " " "-")
+  [[ $QueTotal == $QueCurrent ]] || printf "\r[%s%s] %s/%s " $BarComplete $BarLeft $QueCurrent $QueTotal
+  [[ $QueTotal == $QueCurrent ]] && printf "\r[%b%s%b] %s/%s \n" $c_teal $BarComplete $c_reset $QueCurrent $QueTotal
+}
 
 ### Version check & initiate self update
-[[ "$VERSION" != "$LatestRelease" ]] && { printf "New version available! Local: %s - Latest: %s \n Change Notes: %s \n" "$VERSION" "$LatestRelease" "$LatestChanges" ; [[ -z "$AutoUp" ]] && self_update_select ; }
+[[ "$VERSION" != "$LatestRelease" ]] && { printf "New version available! %b%s%b ⇒ %b%s%b \n Change Notes: %s \n" "$c_yellow" "$VERSION" "$c_reset" "$c_green" "$LatestRelease" "$c_reset" "$LatestChanges" ; [[ -z "$AutoUp" ]] && self_update_select ; }
 
 ### Set $1 to a variable for name filtering later.
 SearchName="$1"
@@ -190,15 +205,20 @@ if [[ -n ${Excludes[*]} ]] ; then
   printf "\n"
 fi
 
+# Variables for progress_bar function
+DocCount=$(docker ps --filter "name=$SearchName" --format '{{.Names}}' | wc -l)
+RegCheckQue=0
+
 ### Check the image-hash of every running container VS the registry
 for i in $(docker ps $Stopped --filter "name=$SearchName" --format '{{.Names}}') ; do
+  ((RegCheckQue+=1))
+  progress_bar $RegCheckQue $DocCount
   ### Looping every item over the list of excluded names and skipping:
   for e in "${Excludes[@]}" ; do [[ "$i" == "$e" ]] && continue 2 ; done 
-  printf ". "
   RepoUrl=$(docker inspect "$i" --format='{{.Config.Image}}')
   LocalHash=$(docker image inspect "$RepoUrl" --format '{{.RepoDigests}}')
   ### Checking for errors while setting the variable:
-  if RegHash=$($regbin image digest --list "$RepoUrl" 2>/dev/null) ; then
+  if RegHash=$($regbin image digest --list "$RepoUrl" 2>&1) ; then
     if [[ "$LocalHash" = *"$RegHash"* ]] ; then 
       NoUpdates+=("$i") 
     else 
@@ -209,7 +229,8 @@ for i in $(docker ps $Stopped --filter "name=$SearchName" --format '{{.Names}}')
       fi
     fi
   else
-    GotErrors+=("$i")
+    # Here the RegHash is the result of an error code.
+    GotErrors+=("$i - ${RegHash}")
   fi
 done
 
@@ -230,6 +251,7 @@ fi
 if [[ -n ${GotErrors[*]} ]] ; then
   printf "\n%bContainers with errors, wont get updated:%b\n" "$c_red" "$c_reset"
   printf "%s\n" "${GotErrors[@]}"
+  printf "%binfo:%b 'unauthorized' often means not found in a public registry.%b\n" "$c_blue" "$c_reset"
 fi
 if [[ -n ${GotUpdates[*]} ]] ; then 
    printf "\n%bContainers with updates available:%b\n" "$c_yellow" "$c_reset"
@@ -257,6 +279,8 @@ if [ -n "$GotUpdates" ] ; then
       ContName=$(docker inspect "$i" --format '{{ index .Config.Labels "com.docker.compose.service" }}')
       ContEnv=$(docker inspect "$i" --format '{{index .Config.Labels "com.docker.compose.project.environment_file" }}')
       ContImage=$(docker inspect "$i" --format='{{.Config.Image}}')
+      ContUpdateLabel=$(docker inspect "$i" --format '{{ index .Config.Labels "mag37.dockcheck.update" }}')
+      ContRestartStack=$(docker inspect "$i" --format '{{ index .Config.Labels "mag37.dockcheck.restart-stack" }}')
       ### Checking if compose-values are empty - hence started with docker run:
       if [ -z "$ContPath" ] ; then 
         if [ "$DRunUp" == "yes" ] ; then
@@ -276,16 +300,28 @@ if [ -n "$GotUpdates" ] ; then
       ### cd to the compose-file directory to account for people who use relative volumes, eg - ${PWD}/data:data
       cd "$ContPath" || { echo "Path error - skipping $i" ; continue ; }
       printf "\n%bNow updating (%s/%s): %b%s%b\n" "$c_teal" "$CurrentQue" "$NumberofUpdates" "$c_blue" "$i" "$c_reset"
+      ### Checking if Label Only -option is set, and if container got the label
+      [[ "$OnlyLabel" == true ]] && { [[ "$ContUpdateLabel" != true ]] && { echo "No update label, skipping." ; continue ; } }
       docker pull "$ContImage"
       ### Reformat for multi-compose:
       IFS=',' read -r -a Confs <<< "$ComposeFile" ; unset IFS
       for conf in "${Confs[@]}"; do CompleteConfs+="-f $conf " ; done 
-      
       ### Check if the container got an environment file set, use it if so:
       if [ -n "$ContEnv" ]; then 
-        $DockerBin ${CompleteConfs[@]} --env-file "$ContEnv" up -d "$ContName" # unquoted array to allow split - rework?
+        ### prepare env-files arguments
+        ContEnvs=$(for env in ${ContEnv//,/ } ; do printf -- "--env-file %s " $env; done)
+        ### Check if the whole stack should be restarted
+        if [[ "$ContRestartStack" == true ]] || [[ "$ForceRestartStacks" == true ]] ; then 
+          $DockerBin ${CompleteConfs[@]} stop ; $DockerBin ${CompleteConfs[@]} ${ContEnvs} up -d 
+        else
+          $DockerBin ${CompleteConfs[@]} ${ContEnvs} up -d "$ContName" # unquoted array to allow split - rework?
+        fi
       else
-        $DockerBin ${CompleteConfs[@]} up -d "$ContName" # unquoted array to allow split - rework?
+        if [[ "$ContRestartStack" == true ]] || [[ "$ForceRestartStacks" == true ]] ; then 
+          $DockerBin ${CompleteConfs[@]} stop ; $DockerBin ${CompleteConfs[@]} up -d
+        else
+          $DockerBin ${CompleteConfs[@]} up -d "$ContName"
+        fi
       fi
     done
     printf "\n%bAll done!%b\n" "$c_green" "$c_reset"
@@ -299,3 +335,4 @@ else
 fi
 
 exit 0
+

extras/apprise_quickstart.md

@@ -0,0 +1,130 @@
+# A small guide on getting started with Apprise notifications.
+
+
+## Standalone docker container: [linuxserver/apprise-api](https://hub.docker.com/r/linuxserver/apprise-api)
+
+Set up the docker compose as preferred:
+```yaml
+---
+version: "2.1"
+services:
+  apprise-api:
+    image: lscr.io/linuxserver/apprise-api:latest
+    container_name: apprise-api
+    environment:
+      - PUID=1000
+      - PGID=1000
+      - TZ=Etc/UTC
+    volumes:
+      - /path/to/apprise-api/config:/config
+    ports:
+      - 8000:8000
+    restart: unless-stopped
+```
+
+Then browse to the webui.
+![](apprise-ex1.png)
+Here you'll click **Configuration Manager**, read the overview and then click on **Configuration**.
+Under **Configuration** you'll craft/paste your notification config. 
+
+![](apprise-ex2.png)
+The simplest way is just paste the url's as is (like in the example above).    
+There are many ways to customize with tags, groups, json and more. Read [caronc/apprise-api](https://github.com/caronc/apprise-api) for more info!   
+
+Look at the [apprise wiki: Notification Services](https://github.com/caronc/apprise/wiki) for more info about how the url syntax for different services works.   
+
+
+You can also use the [caronc/apprise-api](https://github.com/caronc/apprise-api) to host the api as a frontend to an already existing **Apprise**-setup on the host.
+
+
+### Customize the **notify.sh** file.
+After you're done with the setup of the container and tried your notifications, you can copy the `notify_apprise.sh` file to `notify.sh` and start editing it.
+
+Comment out/remove the bare metal apprise-command (starting with `apprise -vv -t...`). 
+Uncomment and edit the `AppriseURL` variable and *curl* line
+It should look something like this when curling the API:
+```bash
+send_notification() {
+Updates=("$@")
+UpdToString=$( printf "%s\n" "${Updates[@]}" )
+FromHost=$(hostname)
+
+printf "\nSending Apprise notification\n"
+
+MessageTitle="$FromHost - updates available."
+# Setting the MessageBody variable here.
+read -d '\n' MessageBody << __EOF
+Containers on $FromHost with updates available:
+
+$UpdToString
+
+__EOF
+
+AppriseURL="http://IP.or.mydomain.tld:8000/notify/apprise"
+curl -X POST -F "title=$MessageTitle" -F "body=$MessageBody" -F "tags=all" $AppriseURL
+
+}
+```
+
+That's all!
+___
+___
+
+
+## On host installed **Apprise**
+Follow the official guide on [caronc/apprise](https://github.com/caronc/apprise)! 
+
+### A brief, basic "get started"
+
+- Install **apprise** 
+    - python package `pip install apprise`
+    - packaged in EPEL/Fedora `dnf install apprise`
+    - packaged in AUR `[yay/pikaur/paru/other] apprise`
+
+- Create a config file with your notification credentials (source of notifications):
+```ini
+mailto://user:password@yahoo.com
+slack://token_a/token_b/token_c
+kodi://example.com
+```
+Then either source the notifications with `-c=/path/to/config/apprise` or store them in *PATH* to skip referencing (`~/.apprise` or `~/.config/apprise`).
+
+- Test apprise with a single notification:
+    - `apprise -vv -t 'test title' -b 'test notification body' 'mailto://myemail:mypass@gmail.com'`
+- Set up your notification URL's and test them.
+    - Look at the [apprise wiki: Notification Services](https://github.com/caronc/apprise/wiki) for more info about how the url syntax for different services works.   
+
+### When done, customize the **notify.sh** file.
+After you're done with the setup of the container and tried your notifications, you can copy the `notify_apprise.sh` file to `notify.sh` and start editing it.
+
+Replace the url's corresponding to the services you've configured.
+```bash
+send_notification() {
+Updates=("$@")
+UpdToString=$( printf "%s\n" "${Updates[@]}" )
+FromHost=$(hostname)
+
+printf "\nSending Apprise notification\n"
+
+MessageTitle="$FromHost - updates available."
+# Setting the MessageBody variable here.
+read -d '\n' MessageBody << __EOF
+Containers on $FromHost with updates available:
+
+$UpdToString
+
+__EOF
+
+# Modify to fit your setup:
+apprise -vv -t "$MessageTitle" -b "$MessageBody" \
+   mailto://myemail:mypass@gmail.com \
+   mastodons://{token}@{host} \
+   pbul://o.gn5kj6nfhv736I7jC3cj3QLRiyhgl98b \
+   tgram://{bot_token}/{chat_id}/
+
+}
+```
+
+That's all!
+___
+___

notify_ntfy-sh.sh

@@ -0,0 +1,31 @@
+### DISCLAIMER: This is a third party addition to dockcheck - best effort testing.
+#
+# Copy/rename this file to notify.sh to enable the notification snippet.
+# Setup app and subscription at https://ntfy.sh
+# Use your unique Topic Name in the URL below.
+
+send_notification() {
+Updates=("$@")
+UpdToString=$( printf "%s\n" "${Updates[@]}" )
+FromHost=$(hostname)
+
+printf "\nSending ntfy.sh notification\n"
+
+MessageTitle="$FromHost - updates available."
+# Setting the MessageBody variable here.
+read -d '\n' MessageBody << __EOF
+Containers on $FromHost with updates available:
+
+$UpdToString
+
+__EOF
+
+# Modify to fit your setup:
+NtfyUrl="ntfy.sh/YourUniqueTopicName"
+
+curl -sS -o /dev/null --show-error --fail \
+  -H "Title: $MessageTitle" \
+  -d "$MessageBody" \
+  $NtfyUrl
+
+}

notify_pushbullet.sh

@@ -0,0 +1,27 @@
+### DISCLAIMER: This is a third party addition to dockcheck - best effort testing.
+#
+# Copy/rename this file to notify.sh to enable the notification snippet.
+# Required receiving services must already be set up.
+# Requires jq installed and in PATH.
+# Modify to fit your setup - set Url and Token.
+
+send_notification() {
+Updates=("$@")
+UpdToString=$( printf "%s\n" "${Updates[@]}" )
+FromHost=$(hostname)
+
+# platform specific notification code would go here
+printf "\nSending pushbullet notification\n"
+
+MessageTitle="$FromHost - updates available."
+# Setting the MessageBody variable here.
+MessageBody="Containers on $FromHost with updates available: $UpdToString"
+
+# Modify to fit your setup:
+PushUrl="https://api.pushbullet.com/v2/pushes"
+PushToken="Your Pushbullet token here"
+
+# Requires jq to process json data
+jq -n --arg title "$MessageTitle" --arg body "$MessageBody" '{body: $body, title: $title, type: "note"}' | curl -sS -o /dev/null --show-error --fail -X POST -H "Access-Token: $PushToken" -H "Content-type: application/json" $PushUrl -d @-
+
+}