Everyone is welcome, everyone can contribute, everyone is unique and these are your strengths too!

7. Cafe: Docker Hub Rate Limit: Mitigation, Caching and Monitoring

Highlights

We went from a quick introduction of the Docker Hub Rate Limits into a thoughtful discussion round of whom is affected and how to mitigate further. Every CI/CD pipeline and job being run in a container is affected, and likewise deployments run in containerized environments such as Kubernetes clusters.

Next to caching proxies, we will see a need for building and maintaining your own Docker images. Following the OCI specification, this needs a central repository hosting the Dockerfile definitions, CI jobs for building, tagging and pushing a local container registry, e.g. in GitLab.

There is no clear prediction on whether Docker CLI or podman or $tool will win. As a developer, we probably do not care as they all implement an API specification and “something” does “something” to format, build, test, deploy our code.

We’ve also discussed the changed image names (short image names vs. absolute registry URLs) and the changes required in every CI/CD config file in every project. Niclas shared their best practice on updating CI configuration files with renovate, similar to a bot updating software dependencies and submitting merge requests.

Building and maintaining container images moves more responsibilities to ops, with security deeply tied into this process. Container scanning will become the standard process: A newly tagged image renders older versions potentially out-of-date and needs automated scanning for vulnerabilities and needed updates.

Conclusion was to pick the solution which fits best into the current environment. Invest into a local container registry and own images, with integrated automation and security. If you cannot afford the resources now, invest into the Docker Hub pro tier and buy some time to schedule an infrastructure shift next year.

Recording

Enjoy the session! 🦊

Cafe Notes

Docker Image Creation

Ideas and discussions

GitLab Infra Insights:

Build own Images

As a Developer, I do not care about Docker, Podman, or others. It is just an API. Similar to CI/CD executors.

How to build container images - is it easy/hard?

Rebuilding images, it can be painful to migrate from the short Docker Hub URLs to the longer URLs. Create a CI job, build your image and push it to the registries. Find a good way to update the existing .gitlab-ci.yml configuration file with the image:.

Monitoring

Public registries

Insights

Date published: November 4, 2020