feat: improve images cleanup in ddev delete images, ddev delete, ddev clean, fixes #5073, fixes #7201#7151
Conversation
stasadev
force-pushed
the
20250326_stasadev_delete_images
branch
2 times, most recently
from
629f628 to
17bc652
Compare
|
Download the artifacts for this pull request:
See Testing a PR.
|
|
ideally from my perspective |
|
I imagine you'll want to solve in this one. |
stasadev
force-pushed
the
20250326_stasadev_delete_images
branch
from
17bc652 to
4930d86
Compare
stasadev
changed the title
ddev delete imagesddev delete images, ddev delete, ddev clean, fixes #7201
|
Ready for review, the OP is updated. I didn't add any new automated tests because these commands can't be tested with just one |
@rpkoller, |
|
I used |
There was a problem hiding this comment.
I've tested the artifact. kept all my backlog of images from plugins as well as ddev core for a while: https://gist.github.com/rpkoller/762009ebf6e004c15121f2525471fea5
i love that none images get cleared now as well. i also had some confusion understanding the exact naming pattern. stas already cleared up that confusion and i can attest the cleanup worked very well in my test. the only nitpick to note is that the output you have to confirm when running ddev delete images is sort of overwhelming :
$> ddev delete images
Warning: the following 7 Docker image(s) will be deleted:
Image to be deleted from ddev-core: sha256:6ef8b6687bb46f4eb5961e73f5ae0dde3325368f0c431cad63a1c2e27c78494f
Image to be deleted from ddev-core: ddev/ddev-webserver:20250213_rfay_mariadb_11.8-core-built
Image to be deleted from ddev-lu: ddev-lu-xhgui:latest
Image to be deleted from ddev-uxmeeting: ddev/ddev-webserver:20250213_rfay_mariadb_11.8-uxmeeting-built
Image to be deleted from ddev-vali: ddev-vali-xhgui:latest
Image to be deleted: ddev/ddev-traefik-router:v1.24.6
Image to be deleted: ddev/ddev-webserver:20250213_rfay_mariadb_11.8
Deleting images is a non-destructive operation.
You may need to download images again when you need them.
OK to continue? [Y/n] (yes):
Powering off DDEV to avoid conflicts
The ddev-ssh-agent container has been removed. When you start it again you will have to use 'ddev auth ssh' to provide key authentication again.
All DDEV images discovered were deleted.
i am not sure if i really need the list of images to be deleted. it is a non destructive operation that could cause a data loss. you are only deleting images that could be redownloaded anyway. i guess if something happens and i redownload an image then i might investigate. but that would be just inconvenient for the moment. therefore i would think that list is not really needed imho. and if folks think it would be nice to have then i would at least change the sorting. if you run docker image ls you have a sort order from newst to oldest while the sort order on the confirmation output is alphabetical ascending. i would keep the order consistent. or the longer i think about it, that list is just complicating things, i would simply drop it.
p.s. as i wrote to stas, the only context running that feature might be tricky is with DDEV HEAD. that way you can easily delete more than you want. ;) i guess my rule of thumb will be, i will only run that command when the PR pushing the tags for a new release is committed. that way you are on the save end. but for folks on a stable version of ddev core that PR will be convenient and pure bliss. that is solving one of my biggest issues. thank you!
@rfay, I want to use
@rpkoller, it's grouped by project, then by name. I tried different sortings, and I'm not sure if newest to oldest will be readable at all.
We consider "data loss" only for volumes, not images. |
|
thought about it some more. in regard of readability probably the order might now be the most important aspect. it is rather the redundant block of text: each line for an image is starting with but looking at the inconsistent and rather confusing context with some images having ddev-core some a name related to project (ddev-uxmeeting), some at the end without any context with just the name. i wonder if the following would be the most straight forward ( i would also rename the sha256 to what it is called with or as i said drop the listing of the image names entirely? the dont provide much of an additional benefit? |
@rfay, done,
@rpkoller, sort by name only, without any projects, show short image IDs:
I still think it's nice to see a list of what changes are being made here. |
|
interesting. when you've mentioned display the images in a table i've automatically thought it would mean the table would have borders (which is thought would look and feel too massive), but without any borders it improves the readability significantly. i like that. the sole two nitpicks. would it make sense to add a blank line before and after? that way it makes easier to distinguish the textblocks: and the other detail, i am not sure what benefit the image id has on that deletion step. the ID is only needed if you manually want to trigger the deletion for an image with |
Okay.
The One use case: run
This will only add more complexity to the code, it should already be clear what kind of image it is, as long as it's not |
|
I see
Shouldn't it be |
|
Trying to save away my 182 images before testing. It's probably not worth the effort, because the restore will be too costly.
A better approach would probably be a copy of (Orbstack) in this case images directory. That's also probably too costly. |
No, because |
|
I'm sure you're right, but on my server I get this: I assume it's complaining because I'm using the legacy builder, not because I'm using I do not see the warning on macOS where I'm using buildx: So I'm sure you're right and it's just poor messaging from the docker CLI. |
|
I also have 'docker-buildx' installed and |
|
This is not a request for changes or extra investigation, but I do see one or two suspicious images after |
There was a problem hiding this comment.
Thanks for the great work on this.
ddev clean in a project context seems to delete the ~/.ddev/bin contents, which breaks any other project that might be running. And it even breaks the running project.
I think ddev clean should stop the running project, and shouldn't delete ~/.ddev/bin unless ddev clean -a
(It also doesn't seem to warn about the deletion of ~/.ddev/bin in any case.)
Should ddev clean -a delete the project_list.yaml and/or global_config.yaml ? (I'd probably think it's better to leave them alone.)
rfay@rfay-mba-m4:~/workspace/d11simple$ ddev clean
Warning: snapshots for the following project(s) will be permanently deleted:
No snapshots found for project d11simple
Are you sure you want to continue? [y/N] (no): y
Deleting snapshots and downloads for selected projects...
Deleting unused Docker images that DDEV created...
Finished cleaning DDEV projects.
Optionally, run `docker builder prune` to clean unused builder cache.
rfay@rfay-mba-m4:~/workspace/d11simple$ ls ~/.ddev/bin
ls: /Users/rfay/.ddev/bin: No such file or directory
rfay@rfay-mba-m4:~/workspace/d11simple$ ddev list
┌───────────────────┬────────────────────────────────────────┬───────────────────────────────┬─────────────────────────────┬──────────┐
│ NAME │ STATUS │ LOCATION │ URL │ TYPE │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ d11 │ running (failed to Mutagen sync list d │ ~/workspace/d11 │ https://d11.ddev.site │ drupal11 │
│ │ 11: stderr='', err=fork/exec /Users/rf │ │ │ │
│ │ ay/.ddev/bin/mutagen: no such file or │ │ │ │
│ │ directory nosession for MUTAGEN_DATA_D │ │ │ │
│ │ IRECTORY=/Users/rfay/.ddev_mutagen_dat │ │ │ │
│ │ a_directory) │ │ │ │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ d11simple │ running (failed to Mutagen sync list d │ ~/workspace/d11simple │ https://d11simple.ddev.site │ drupal11 │
│ │ 11simple: stderr='', err=fork/exec /Us │ │ │ │
│ │ ers/rfay/.ddev/bin/mutagen: no such fi │ │ │ │
│ │ le or directory nosession for MUTAGEN_ │ │ │ │
│ │ DATA_DIRECTORY=/Users/rfay/.ddev_mutag │ │ │ │
│ │ en_data_directory) │ │ │ │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ ddev.com │ stopped │ ~/workspace/ddev.com │ │ php │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ hobobiker │ stopped │ ~/workspace/hobobiker.com │ │ drupal6 │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ my-sveltekit-site │ stopped │ ~/workspace/my-sveltekit-site │ │ generic │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ t3v13 │ stopped │ ~/workspace/t3v13 │ │ typo3 │
├───────────────────┼────────────────────────────────────────┼───────────────────────────────┼─────────────────────────────┼──────────┤
│ Router │ OK │ ~/.ddev │ http://127.0.0.1:10999 │ │
└───────────────────┴────────────────────────────────────────┴───────────────────────────────┴─────────────────────────────┴──────────┘
I also believe that it should not touch these files.
Agreed, will change the logic.
Good point. |
|
It does look like we might be leaving busybox out there, because we use it without touching it? Here's after |
rfay
left a comment
rfay
left a comment
There was a problem hiding this comment.
Overall it seems great to me. Minor suggestions in previous reviews and comments, that can be handled as you see fit. But I sure like it, and it doesn't seem to have any risk factors either.
|
Tested this on all my other docker providers, colima, lima, rancher, Docker Desktop. In each case it did marvelous cleanup things, just as I'd hope. |
|
This fixes also, true? |
stasadev
changed the title
ddev delete images, ddev delete, ddev clean, fixes #7201ddev delete images, ddev delete, ddev clean, fixes #5073, fixes #7201
|
All requested changes have been implemented. |
|
I tested it again, and I'll try to fix it. Edit: fixed.
What I've noticed over time: we use Another thing I noticed: if the image doesn't have a specific So if users update This could be solved by always setting the |
not completely. the none images are still being created in the first place. the cleanup got now way more convenient with this PR. |
|
The |
3 participants
The Issue
<none>image is being created when a new and different image of same name is created #5073ddev deleteshould show some name when deleting dangling images #7201ddev delete imageswithout--allflag doesn't clean not properly deleted projects withddev stop --unlistddev cleandeletes all DDEV images, regardless of whether you have passed the--allflag or not.ddev cleanshould work similar to otherddevcommands without project arg inside project directory.How This PR Solves The Issue
ddev deleteshows shasum for dangling images without name (i.e. without tags).ddev delete imagesshows images to be deleted before running cleanup.ddev delete imagesremoves leftovers from not properly deleted projects withddev stop --unlistddev delete images -anow removes every image created bydocker-composein the project context.ddev delete imagescleans leftovers from deleted add-ons.ddev cleancan be run inside a project directory.ddev clean -arunsddev delete images -aunder the hood as before.ddev cleanrunsddev delete imagesunder the hood (notddev delete images -a)ddev cleandoesn't remove global~/.ddev/bindocker builder pruneto the commands above on success.ddev poweroffonly when needed, not all the time.Manual Testing Instructions
Run this with DDEV v1.24.5 (or lower) and
project1:Run this with DDEV v1.24.5 (or lower) and
project2:Run this with DDEV v1.24.5 (or lower) and
project3:Switch to this PR and run:
Expect to see removals for these images:
v1.24.5tags forddev/ddev-webserver,ddev/ddev-dbserver-*,ddev/ddev-traefik-router,ddev/ddev-ssh-agent,ddev/ddev-xhgui.solrimage fromproject1andv1.24.5tags forproject1, but not the current ones.v1.24.5tags forproject2project3images.docker ps -a, it shouldn't contain leftovers.Check out:
The clean command should work in the same way:
To test dangling images in
ddev delete, rebuildsolrseveral times:The dangling image will be listed with shasum.
Automated Testing Overview
I didn't add any new automated tests because these commands can't be tested with just one
ddevbinary.Release/Deployment Notes