Skip to content

fix(docker): handle TLS, improve ddev utility dockercheck, fixes #8111#8115

Merged
rfay merged 2 commits into
mainfrom
20260206_stasadev_docker
Feb 10, 2026
Merged

fix(docker): handle TLS, improve ddev utility dockercheck, fixes #8111#8115
rfay merged 2 commits into
mainfrom
20260206_stasadev_docker

Conversation

@stasadev

@stasadev stasadev commented Feb 6, 2026

Copy link
Copy Markdown
Member

The Issue

Docker client initialization in docker_manager.go wasn't reading TLS environment variables (DOCKER_TLS_VERIFY, DOCKER_CERT_PATH, DOCKER_TLS), causing "Client sent an HTTP request to an HTTPS server" errors when connecting to Docker daemons that require TLS.

How This PR Solves The Issue

docker_manager.go:
Uses the same technique as in https://github.com/docker/cli/blob/727bc3ea5b2cccf0d1ef77d7049848851e973477/cmd/docker-trust/trust/commands.go#L34-L44 to handle TLS

utility-dockercheck improvements:

  • Downloads docker-compose before version check to ensure it's available
  • Displays TLS configuration status (enabled/disabled, cert paths)

Manual Testing Instructions

  1. Verify TLS environment variables are now respected (on standard local Docker setup):

    # Works in DDEV v1.25.0 - which means it wasn't respected
    DOCKER_TLS=1 ddev utility dockercheck
    DOCKER_TLS_VERIFY=1 ddev utility dockercheck
    
    # but fails with the error in this PR:
    Docker error: unable to resolve docker endpoint: open /home/stas/.docker/ca.pem: no such file or directory
    For help go to: https://docs.ddev.com/en/stable/users/install/docker-installation/#troubleshooting-docker
    
  2. Test dockercheck command displays TLS config:

    ddev utility dockercheck
  3. Using https://gitlab.com/consensus.enterprises/misc/ddev-in-ci

    git clone --recursive https://gitlab.com/consensus.enterprises/misc/ddev-in-ci.git
    cd ddev-in-ci
    source d  # Bootstrap some environment variables
    make

    This will install gitlab-ci-local, and run all of the jobs in the .gitlab-ci-yml.

    Copy ddev binary from this PR (see a comment with artifacts below) to the current directory and create a Dockerfile:

    FROM registry.gitlab.com/consensus.enterprises/drumkit/ddev:24.04-1.25.0
    COPY ./ddev /usr/local/bin/ddev

    Run:

    docker build -t ddev-test . --no-cache

    Then edit https://gitlab.com/consensus.enterprises/misc/ddev-in-ci/-/blob/main/.gitlab-ci.yml, add:

    ddev-1.25.0-pr:
      <<: *test-defaults
      image:
        <<: *image-defaults
        name: ddev-test
      script:
        # Check whether DDEV is working
        - ddev utility dockercheck

    And run:

    $ ./gitlab-ci-local ddev-1.25.0-pr
    parsing and downloads finished in 117 ms.
    json schema validated in 379 ms
    ddev-1.25.0-pr starting ddev-test:latest (test)
    ddev-1.25.0-pr copied to docker volumes in 944 ms
    ddev-1.25.0-pr started service image: docker:29.2.1-dind with aliases: docker in 250 ms
    ddev-1.25.0-pr service image: docker:29.2.1-dind healthcheck passed in 456 ms
    ddev-1.25.0-pr $ docker version
    ddev-1.25.0-pr > Client: Docker Engine - Community
    ddev-1.25.0-pr >  Version:           28.2.2
    ddev-1.25.0-pr >  API version:       1.50
    ddev-1.25.0-pr >  Go version:        go1.24.3
    ddev-1.25.0-pr >  Git commit:        e6534b4
    ddev-1.25.0-pr >  Built:             Fri May 30 12:07:27 2025
    ddev-1.25.0-pr >  OS/Arch:           linux/amd64
    ddev-1.25.0-pr >  Context:           default
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > Server: Docker Engine - Community
    ddev-1.25.0-pr >  Engine:
    ddev-1.25.0-pr >   Version:          29.2.1
    ddev-1.25.0-pr >   API version:      1.53 (minimum version 1.44)
    ddev-1.25.0-pr >   Go version:       go1.25.6
    ddev-1.25.0-pr >   Git commit:       6bc6209b
    ddev-1.25.0-pr >   Built:            Mon Feb  2 17:19:09 2026
    ddev-1.25.0-pr >   OS/Arch:          linux/amd64
    ddev-1.25.0-pr >   Experimental:     false
    ddev-1.25.0-pr >  containerd:
    ddev-1.25.0-pr >   Version:          v2.2.1
    ddev-1.25.0-pr >   GitCommit:        dea7da592f5d1d2b7755e3a161be07f43fad8f75
    ddev-1.25.0-pr >  runc:
    ddev-1.25.0-pr >   Version:          1.3.4
    ddev-1.25.0-pr >   GitCommit:        v1.3.4-0-gd6d73eb
    ddev-1.25.0-pr >  docker-init:
    ddev-1.25.0-pr >   Version:          0.19.0
    ddev-1.25.0-pr >   GitCommit:        de40ad0
    ddev-1.25.0-pr $ docker info
    ddev-1.25.0-pr > Client: Docker Engine - Community
    ddev-1.25.0-pr >  Version:    28.2.2
    ddev-1.25.0-pr >  Context:    default
    ddev-1.25.0-pr >  Debug Mode: false
    ddev-1.25.0-pr >  Plugins:
    ddev-1.25.0-pr >   buildx: Docker Buildx (Docker Inc.)
    ddev-1.25.0-pr >     Version:  v0.31.1
    ddev-1.25.0-pr >     Path:     /usr/libexec/docker/cli-plugins/docker-buildx
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > Server:
    ddev-1.25.0-pr >  Containers: 0
    ddev-1.25.0-pr >   Running: 0
    ddev-1.25.0-pr >   Paused: 0
    ddev-1.25.0-pr >   Stopped: 0
    ddev-1.25.0-pr >  Images: 0
    ddev-1.25.0-pr >  Server Version: 29.2.1
    ddev-1.25.0-pr >  Storage Driver: overlay2
    ddev-1.25.0-pr >   Backing Filesystem: extfs
    ddev-1.25.0-pr >   Supports d_type: true
    ddev-1.25.0-pr >   Using metacopy: false
    ddev-1.25.0-pr >   Native Overlay Diff: true
    ddev-1.25.0-pr >   userxattr: false
    ddev-1.25.0-pr >  Logging Driver: json-file
    ddev-1.25.0-pr >  Cgroup Driver: cgroupfs
    ddev-1.25.0-pr >  Cgroup Version: 2
    ddev-1.25.0-pr >  Plugins:
    ddev-1.25.0-pr >   Volume: local
    ddev-1.25.0-pr >   Network: bridge host ipvlan macvlan null overlay
    ddev-1.25.0-pr >   Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
    ddev-1.25.0-pr >  CDI spec directories:
    ddev-1.25.0-pr >   /etc/cdi
    ddev-1.25.0-pr >   /var/run/cdi
    ddev-1.25.0-pr >  Swarm: inactive
    ddev-1.25.0-pr >  Runtimes: io.containerd.runc.v2 runc
    ddev-1.25.0-pr >  Default Runtime: runc
    ddev-1.25.0-pr >  Init Binary: docker-init
    ddev-1.25.0-pr >  containerd version: dea7da592f5d1d2b7755e3a161be07f43fad8f75
    ddev-1.25.0-pr >  runc version: v1.3.4-0-gd6d73eb
    ddev-1.25.0-pr >  init version: de40ad0
    ddev-1.25.0-pr >  Security Options:
    ddev-1.25.0-pr >   seccomp
    ddev-1.25.0-pr >    Profile: builtin
    ddev-1.25.0-pr >   cgroupns
    ddev-1.25.0-pr >  Kernel Version: 6.12.68-1-MANJARO
    ddev-1.25.0-pr >  Operating System: Alpine Linux v3.23 (containerized)
    ddev-1.25.0-pr >  OSType: linux
    ddev-1.25.0-pr >  Architecture: x86_64
    ddev-1.25.0-pr >  CPUs: 12
    ddev-1.25.0-pr >  Total Memory: 31.19GiB
    ddev-1.25.0-pr >  Name: 1c0e8b6f424f
    ddev-1.25.0-pr >  ID: 0f05efd7-90af-46f0-817a-fb0908fc30a6
    ddev-1.25.0-pr >  Docker Root Dir: /var/lib/docker
    ddev-1.25.0-pr >  Debug Mode: false
    ddev-1.25.0-pr >  Experimental: false
    ddev-1.25.0-pr >  Insecure Registries:
    ddev-1.25.0-pr >   ::1/128
    ddev-1.25.0-pr >   127.0.0.0/8
    ddev-1.25.0-pr >  Live Restore Enabled: false
    ddev-1.25.0-pr >  Product License: Community Engine
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr $ docker run hello-world
    ddev-1.25.0-pr > Unable to find image 'hello-world:latest' locally
    ddev-1.25.0-pr > latest: Pulling from library/hello-world
    ddev-1.25.0-pr > 17eec7bbc9d7: Pulling fs layer
    ddev-1.25.0-pr > 17eec7bbc9d7: Download complete
    ddev-1.25.0-pr > 17eec7bbc9d7: Pull complete
    ddev-1.25.0-pr > Digest: sha256:05813aedc15fb7b4d732e1be879d3252c1c9c25d885824f6295cab4538cb85cd
    ddev-1.25.0-pr > Status: Downloaded newer image for hello-world:latest
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > Hello from Docker!
    ddev-1.25.0-pr > This message shows that your installation appears to be working correctly.
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > To generate this message, Docker took the following steps:
    ddev-1.25.0-pr >  1. The Docker client contacted the Docker daemon.
    ddev-1.25.0-pr >  2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    ddev-1.25.0-pr >     (amd64)
    ddev-1.25.0-pr >  3. The Docker daemon created a new container from that image which runs the
    ddev-1.25.0-pr >     executable that produces the output you are currently reading.
    ddev-1.25.0-pr >  4. The Docker daemon streamed that output to the Docker client, which sent it
    ddev-1.25.0-pr >     to your terminal.
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > To try something more ambitious, you can run an Ubuntu container with:
    ddev-1.25.0-pr >  $ docker run -it ubuntu bash
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > Share images, automate workflows, and more with a free Docker ID:
    ddev-1.25.0-pr >  https://hub.docker.com/
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr > For more examples and ideas, visit:
    ddev-1.25.0-pr >  https://docs.docker.com/get-started/
    ddev-1.25.0-pr > 
    ddev-1.25.0-pr $ docker run --rm -t -p 80:80 -p 443:443 -v "//$PWD:/tmp/projdir" ddev/ddev-utilities sh -c "echo ---- Project Directory && ls /tmp/projdir"
    ddev-1.25.0-pr > Unable to find image 'ddev/ddev-utilities:latest' locally
    ddev-1.25.0-pr > latest: Pulling from ddev/ddev-utilities
    ddev-1.25.0-pr > 43c4264eed91: Pulling fs layer
    ddev-1.25.0-pr > 134d3819e0e6: Pulling fs layer
    ddev-1.25.0-pr > 43c4264eed91: Verifying Checksum
    ddev-1.25.0-pr > 43c4264eed91: Download complete
    ddev-1.25.0-pr > 134d3819e0e6: Verifying Checksum
    ddev-1.25.0-pr > 134d3819e0e6: Download complete
    ddev-1.25.0-pr > 43c4264eed91: Pull complete
    ddev-1.25.0-pr > 134d3819e0e6: Pull complete
    ddev-1.25.0-pr > Digest: sha256:97822ff9a06416ea8ba09853427565b351556fb08a3b1ef944101fcf829b2710
    ddev-1.25.0-pr > Status: Downloaded newer image for ddev/ddev-utilities:latest
    ddev-1.25.0-pr > ---- Project Directory
    ddev-1.25.0-pr > Makefile   README.md  d          drumkit
    ddev-1.25.0-pr $ ddev --version
    ddev-1.25.0-pr > ddev version v1.25.0-5-g4d851e6e1-dirty
    ddev-1.25.0-pr $ ddev utility dockercheck
    ddev-1.25.0-pr > Download complete.
    ddev-1.25.0-pr > Docker platform: linux-docker
    ddev-1.25.0-pr > docker buildx version v0.31.1
    ddev-1.25.0-pr > Using Docker context: default
    ddev-1.25.0-pr > Using Docker host: tcp://docker:2376
    ddev-1.25.0-pr > From DOCKER_HOST=tcp://docker:2376
    ddev-1.25.0-pr > DOCKER_TLS_VERIFY=1 (TLS enabled with verification)
    ddev-1.25.0-pr > DOCKER_CERT_PATH=/certs/client/client
    ddev-1.25.0-pr > docker-compose: v5.0.2
    ddev-1.25.0-pr > Docker version: 29.2.1
    ddev-1.25.0-pr > Docker API version: 1.53
    ddev-1.25.0-pr >  Image ddev/ddev-webserver:20260204_stasadev_avif Pulling 
    ddev-1.25.0-pr >  06465b286bae Pulling fs layer 
    ddev-1.25.0-pr > still running...
    ddev-1.25.0-pr >  Image ddev/ddev-webserver:20260204_stasadev_avif Pulled 
    ddev-1.25.0-pr > Able to run simple container that mounts a volume.
    ddev-1.25.0-pr > Able to use internet inside container.
    ddev-1.25.0-pr > docker buildx is working correctly (trivial build succeeded)
    ddev-1.25.0-pr > Docker authentication is configured correctly
    ddev-1.25.0-pr finished in 39 s
    
    PASS  ddev-1.25.0-pr

    From the output, it works:

    DOCKER_TLS_VERIFY=1 (TLS enabled with verification)
    

Automated Testing Overview

  • Updated TestUtilityDockercheckCmd to verify TLS configuration output

Release/Deployment Notes

This fix enables DDEV to work with Docker daemons that require TLS connections. No breaking changes.

@stasadev stasadev requested a review from a team as a code owner February 6, 2026 18:15
@github-actions github-actions Bot added the bugfix label Feb 6, 2026
@stasadev stasadev changed the title fix(docker): handle TLS connections, improve ddev utility dockercheck, fixes #8111 fix(docker): handle TLS, improve ddev utility dockercheck, fixes #8111 Feb 6, 2026
@github-actions

github-actions Bot commented Feb 6, 2026

Copy link
Copy Markdown

@rfay rfay left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good and my manual testing using a docker-ce on Linux configured for TLS worked fine.

The only question I had, despite being completely successful:

The client certs live in ~/.docker/tls right? But that's the docker client's territory, not the docker API, which we're using. Do we need DOCKER_CERT_PATH=$HOME/.docker/tls ? (It doesn't seem so, but ...)

@stasadev

stasadev commented Feb 9, 2026

Copy link
Copy Markdown
Member Author

But that's the docker client's territory, not the docker API

We use docker/cli in addition to Docker API, so everything goes from there.

stasadev and others added 2 commits February 9, 2026 22:08
## The Issue

Docker client initialization in docker_manager.go wasn't reading TLS
environment variables (DOCKER_TLS_VERIFY, DOCKER_CERT_PATH, DOCKER_TLS),
causing "Client sent an HTTP request to an HTTPS server" errors when
connecting to Docker daemons that require TLS.

## How This PR Solves The Issue

**docker_manager.go:**
- Added InstallFlags() call to populate ClientOptions from environment variables
- Added SetDefaultOptions() to validate TLS certificate file paths
- Uses pflag.NewFlagSet() to enable environment variable parsing
- Continues using client.New(client.FromEnv) for API client creation

**utility-dockercheck improvements:**
- Downloads docker-compose before version check to ensure it's available
- Displays TLS configuration status (enabled/disabled, cert paths)
- Skips buildx checks on Podman (buildx is Docker-specific)
- Skips buildx build test on Podman (uses native podman build)
- Updated test to handle both Docker and Podman environments

## Manual Testing Instructions

1. Verify TLS environment variables are now respected (on standard Docker setup):
   ```bash
   # Should work (TLS without verification)
   DOCKER_TLS=1 ddev version

   # Should fail with certificate error, confirming env vars are respected
   DOCKER_TLS_VERIFY=1 ddev version
   # Expected error: "unable to resolve docker endpoint: open /home/user/.docker/ca.pem: no such file or directory"
   ```

2. Test with TLS-enabled Docker daemon (if available):
   ```bash
   export DOCKER_TLS_VERIFY=1
   export DOCKER_CERT_PATH=/path/to/certs
   ddev version
   ```

3. Test dockercheck command displays TLS config:
   ```bash
   ddev utility dockercheck
   ```
   Verify TLS configuration status is shown

4. Test on Podman (if available):
   ```bash
   ddev utility dockercheck
   ```
   Verify buildx checks are skipped with appropriate messages

## Automated Testing Overview

- Updated TestUtilityDockercheckCmd to verify TLS configuration output
- Added conditional buildx check that only runs on Docker (not Podman)
- Test uses dockerutil.IsPodman() for platform-aware assertions

## Release/Deployment Notes

This fix enables DDEV to work with Docker daemons that require TLS
connections. No breaking changes. Improves Podman compatibility in
dockercheck diagnostic tool.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@stasadev stasadev force-pushed the 20260206_stasadev_docker branch from d0c0bbe to 007cbed Compare February 9, 2026 20:15
@stasadev

stasadev commented Feb 9, 2026

Copy link
Copy Markdown
Member Author

@rfay rfay merged commit f78ab7a into main Feb 10, 2026
42 checks passed
@rfay rfay deleted the 20260206_stasadev_docker branch February 10, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DDEV in CI broken in DDEV-1.24.8+

2 participants