Script for mass cloning docker images into self-hosted registry

This script helps us clone a large number of Docker images from Docker Hub into a private registry. It automatically transfers all tags for each image we need. We can also add it to a cronjob to keep our images updated on a schedule.

Create a working directory

mkdir -p ~/docker-sync && cd ~/docker-sync && touch ./images.txt && touch ./download.sh && chmod +x ./download.sh

Define the list of images we need (images.txt)

nginx
node
rabbitmq
traefik
drakkan/sftpgo
gogs/gogs

Write a script that reads the file and fetches available tags for each image (download.sh)

input="./images.txt"
while IFS= read -r image
do
  set -f
  imageParts=($(echo "$image" | tr '/' '\n'))
  imagePartsLength=${#imageParts[@]}
  if [[ imagePartsLength -lt 2 ]]; then
    namespace="library"
    imgName="${image}"
  else
    namespace="${imageParts[0]}"
    imgName="${imageParts[1]}"
  fi
  tags="$(wget -q -O - "https://hub.docker.com/v2/namespaces/$namespace/repositories/$imgName/tags?page_size=25" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$')"
  # Here we will add logic for images download and upload
done < "$input"

Add logic for downloading images from Docker Hub and uploading them to our registry (finalized script)

input="./images.txt"
while IFS= read -r image
do
  set -f
  imageParts=($(echo "$image" | tr '/' '\n'))
  imagePartsLength=${#imageParts[@]}
  if [[ imagePartsLength -lt 2 ]]; then
    namespace="library"
    imgName="${image}"
  else
    namespace="${imageParts[0]}"
    imgName="${imageParts[1]}"
  fi
  tags="$(wget -q -O - "https://hub.docker.com/v2/namespaces/$namespace/repositories/$imgName/tags?page_size=25" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$')"
  while IFS= read -r line; do
    if [[ "$line" =~ ^[0-9]{1,2}\.[0-9]{1,2} ]]; then
      tagname="$image:$line"
      desttagname="$1/$imgName:$line"
      docker pull $tagname &> /dev/null
      docker tag $tagname $desttagname
      docker push $desttagname &> /dev/null
      echo "pushed $desttagname"
    fi
  done <<< "$tags"
done < "$input"

Run the script

cd ~/docker-sync
./download.sh reg.intra:5000