[{"_1":2,"_64":-5,"_65":-5},"loaderData",{"_3":4,"_12":13},"routes/_layout",{"_5":6},"pages",[7],{"_8":9,"_10":11},"slug","about","title","What's going on here?","routes/blog/index",{"_14":15,"_48":49,"_50":49,"_51":-7,"_52":53,"_62":63},"posts",[16],{"_8":17,"_18":19,"_46":47},"script-for-mass-cloning-docker-images-into-self-hosted-registry","meta",{"_10":20,"_21":22,"_23":24,"_25":26,"_40":41,"_42":43},"Script for mass cloning docker images into self-hosted registry","description","This article describes how to clone a large number of docker images into a private registry. All tags for each image in your list will be cloned. The ability to filter tags by a regular expression will be added.","date","2025-01-05","keywords",[27,28,29,30,31,32,33,34,35,36,37,38,39],"docker","registry","images","cloning","self-hosted","private registry","docker hub","containers","automation","script","bulk","sync","infrastructure","author","Denys Bashkatov","tags",[44,45],"Docker","Homelab","html","

Our implementation will help you if you want to have a private docker registry with a large number of cloned images from the original docker hub. In addition, our solution will allow you to automatically transfer all the tags of the images you need to your personal registry. In addition, you can add the script that we will describe in this article to the cronjob, and thus you can update all the images you need according to the schedule.

Create a working directory

mkdir -p ~/docker-sync && cd ~/docker-sync && touch ./images.txt && touch ./download.sh && chmod +x ./download.sh

Describe the format of the file with the list of images that you need (images.txt)

nginx\nnode\nrabbitmq\ntraefik\ndrakkan/sftpgo\ngogs/gogs\n

Now let's write a script that will read the file and download a list of available tags for each image (download.sh)

input="./images.txt"\nwhile IFS= read -r image\ndo\n  set -f\n  imageParts=($(echo "$image" | tr '/' '\\n'))\n  imagePartsLength=${#imageParts[@]}\n  if [[ imagePartsLength -lt 2 ]]; then\n    namespace="library"\n    imgName="${image}"\n  else\n    namespace="${imageParts[0]}"\n    imgName="${imageParts[1]}"\n  fi\n  tags="$(wget -q -O - "https://hub.docker.com/v2/namespaces/$namespace/repositories/$imgName/tags?page_size=25" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$')"\n  # Here we will add logic for images download and upload\ndone < "$input"

Now let's add logic for downloading images from Docker Hub and uploading them to our registry (finalized script)

input="./images.txt"\nwhile IFS= read -r image\ndo\n  set -f\n  imageParts=($(echo "$image" | tr '/' '\\n'))\n  imagePartsLength=${#imageParts[@]}\n  if [[ imagePartsLength -lt 2 ]]; then\n    namespace="library"\n    imgName="${image}"\n  else\n    namespace="${imageParts[0]}"\n    imgName="${imageParts[1]}"\n  fi\n  tags="$(wget -q -O - "https://hub.docker.com/v2/namespaces/$namespace/repositories/$imgName/tags?page_size=25" | grep -o '"name": *"[^"]*' | grep -o '[^"]*$')"\n  while IFS= read -r line; do\n    if [[ "$line" =~ ^[0-9]{1,2}\\.[0-9]{1,2} ]]; then\n      tagname="$image:$line"\n      desttagname="$1/$imgName:$line"\n      docker pull $tagname &> /dev/null\n      docker tag $tagname $desttagname\n      docker push $desttagname &> /dev/null\n      echo "pushed $desttagname"\n    fi\n  done <<< "$tags"\ndone < "$input"

That's it. Now you can run the resulting script in this way:

cd ~/docker-sync\n./download.sh reg.intra:5000

","page",2,"totalPages","tag","allTags",[54,55,44,45,56,57,58,59,60,61],"Architecture","CSS","JavaScript","Linux","Node.js","React","TypeScript","Ubuntu","total",7,"actionData","errors"]

Tech Notes

Tech tips and new discoveries