gh-121021: zipfile: Support for extended_mtime attribute by Akasurde · Pull Request #121020 · python/cpython

Akasurde · 2024-06-26T03:49:38Z

zipinfo now supports attribute - extended_mtime

📚 Documentation preview 📚: https://cpython-previews--121020.org.readthedocs.build/

Issue: zip: Support for external attributes in zipinfo #121021

danifus · 2024-06-26T13:45:58Z

Info-Zip's proginfo/extrafld.txt description of this extension field has a few details that make getting all three times from the extra fields via python's ZipInfo tricky (their description is attached below)

The python implementation populates the ZipInfo data from the central directory which, according to Zip-Info's description, may only contain the modified time. Python only reads the local header (with all three fields potentially present) when opening the archived entry for reading/ decompression and that read is primarily used for some sanity checks before decompression rather than extracting additional extra fields for further use.

You could rework this PR to focus on extracting the modified time when present if that's still useful for you?

Another potential issue is The time values are in standard Unix signed-long format so will be affected by the year 2038 problem. Might not be a problem as we're just reading values...

Raising a BadZipFile in the case ln is zero (no flags to read) or decoding the time fails would be consistent with the other extra fields.

A test showing reading the field successfully would be useful. You could look to the tests for the utf filename extra fields for inspiration: https://github.com/python/cpython/blob/main/Lib/test/test_zipfile/test_core.py#L1837

         -Extended Timestamp Extra Field:
          ==============================

          The following is the layout of the extended-timestamp extra block.
          (Last Revision 19970118)

          Local-header version:

          Value         Size        Description
          -----         ----        -----------
  (time)  0x5455        Short       tag for this extra block type ("UT")
          TSize         Short       total data size for this block
          Flags         Byte        info bits
          (ModTime)     Long        time of last modification (UTC/GMT)
          (AcTime)      Long        time of last access (UTC/GMT)
          (CrTime)      Long        time of original creation (UTC/GMT)

          Central-header version:

          Value         Size        Description
          -----         ----        -----------
  (time)  0x5455        Short       tag for this extra block type ("UT")
          TSize         Short       total data size for this block
          Flags         Byte        info bits (refers to local header!)
          (ModTime)     Long        time of last modification (UTC/GMT)

          The central-header extra field contains the modification time only,
          or no timestamp at all.  TSize is used to flag its presence or
          absence.  But note:

              If "Flags" indicates that Modtime is present in the local header
              field, it MUST be present in the central header field, too!
              This correspondence is required because the modification time
              value may be used to support trans-timezone freshening and
              updating operations with zip archives.

          The time values are in standard Unix signed-long format, indicating
          the number of seconds since 1 January 1970 00:00:00.  The times
          are relative to Coordinated Universal Time (UTC), also sometimes
          referred to as Greenwich Mean Time (GMT).  To convert to local time,
          the software must know the local timezone offset from UTC/GMT.

          The lower three bits of Flags in both headers indicate which time-
          stamps are present in the LOCAL extra field:

                bit 0           if set, modification time is present
                bit 1           if set, access time is present
                bit 2           if set, creation time is present
                bits 3-7        reserved for additional timestamps; not set

          Those times that are present will appear in the order indicated, but
          any combination of times may be omitted.  (Creation time may be
          present without access time, for example.)  TSize should equal
          (1 + 4*(number of set bits in Flags)), as the block is currently
          defined.  Other timestamps may be added in the future.

Akasurde · 2024-06-26T17:32:11Z

@danifus Thanks a lot for the review. I updated the PR with your suggestions.

danifus

Nice changes. Test cases look good too

danifus · 2024-06-27T11:45:31Z

+                if ln < 5:
+                    raise BadZipFile('Empty extended timestamp extra field (0x5455)') from None
+                if flags & 0x1 and ln >= 5:
+                    self.extended_mtime = int.from_bytes(extra[5:9], "little")


This should be a signed int:

int.from_bytes(b'\x00\x00\x00\x80', byteorder='little', signed=True) -2147483648

are you able to add a test showing the behaviour beyond the 2038 problem date (which would also be the test showing it is a signed field)?

A call to struct's unpack would also be consistent with the other fields (and a little faster) and allow you to catch the case where not enough bytes for a long were supplied:

unpack('<l', extra[5:9])

danifus · 2024-06-27T12:41:17Z


+            elif tp == 0x5455:
+                flags = extra[4]
+                if ln < 5:


ln can be less than 5 and still be valid. The flags in the local header and central header should always be the same and may indicate that any combination of the 3 times are present (even though the central header can only store the mtime). So it is possible that the atime and ctime flags are set but not the mtime and the central header would only have the flags byte and no time data.

The info-zip description says the mtime 'must' be present in the central header if the mtime flag is set but I'd settle for a warning.

You could move this check above the flags = extra[4] line and change it to catch the potential (corrupt) case where ln == 0. If ln == 0 it is either going to raise an IndexError or return the first byte from the next extra field.

danifus

Thanks for adding those last suggestions. These are the last ones I have. Nice work!

danifus

Windows tests were failing on the 2038 test so here's some suggestions about that.

Last changes for sure this time :p

zipinfo now supports attributes like extended_atime, extended_ctime, extended_mtime Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

Akasurde · 2024-07-02T23:14:02Z

@danifus Anything else required over here? Thanks.

danifus · 2024-07-03T00:28:44Z

No more changes from me :) Nice work

LGTM

Akasurde · 2024-08-17T02:34:27Z

@danifus Do I need to do anything to get this merged? Thanks.

danifus · 2024-08-20T00:03:01Z

You'll need to get a review from a core developer and then they can help get it merged (I also have a few waiting :p ). Have a read of the experts index https://devguide.python.org/core-developers/experts/index.html - there maybe someone in there that could review it for you.

Akasurde · 2024-08-21T16:33:19Z

@serhiy-storchaka, @Yhg1s, @gpshead Could you please review this PR? Thanks in advance.

github-actions · 2026-04-17T00:25:14Z

This PR is stale because it has been open for 30 days with no activity.

bedevere-app Bot added the awaiting review label Jun 26, 2024

Akasurde force-pushed the zip_atime branch from 121d4c5 to b3c7a45 Compare June 26, 2024 03:54

Akasurde changed the title ~~zipfile: Support for extended_*time attributes~~ gh-121021: zipfile: Support for extended_*time attributes Jun 26, 2024

bedevere-app Bot mentioned this pull request Jun 26, 2024

zip: Support for external attributes in zipinfo #121021

Open

Akasurde force-pushed the zip_atime branch from b3c7a45 to 4a35758 Compare June 26, 2024 17:31

danifus suggested changes Jun 27, 2024

View reviewed changes

bedevere-app Bot added awaiting core review and removed awaiting review labels Jun 27, 2024

Akasurde force-pushed the zip_atime branch from 4a35758 to c6072a9 Compare June 27, 2024 17:37

Akasurde changed the title ~~gh-121021: zipfile: Support for extended_*time attributes~~ gh-121021: zipfile: Support for extended_mtime attribute Jun 27, 2024

danifus suggested changes Jun 27, 2024

View reviewed changes

Comment thread Lib/zipfile/__init__.py Outdated

Comment thread Lib/zipfile/__init__.py Outdated

Akasurde force-pushed the zip_atime branch from c6072a9 to 257278b Compare June 28, 2024 03:27

danifus suggested changes Jun 29, 2024

View reviewed changes

Comment thread Lib/test/test_zipfile/test_core.py Outdated

Comment thread Lib/test/test_zipfile/test_core.py Outdated

zipfile: Support for extended_*time attributes

03a1553

zipinfo now supports attributes like extended_atime, extended_ctime, extended_mtime Signed-off-by: Abhijeet Kasurde <akasurde@redhat.com>

Akasurde force-pushed the zip_atime branch from 257278b to 03a1553 Compare June 29, 2024 02:46

danifus approved these changes Jul 4, 2024

View reviewed changes

github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 17, 2026

Uh oh!

Conversation

Akasurde commented Jun 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

danifus commented Jun 26, 2024

Uh oh!

Akasurde commented Jun 26, 2024

Uh oh!

danifus left a comment

Choose a reason for hiding this comment

Uh oh!

danifus Jun 27, 2024

Choose a reason for hiding this comment

Uh oh!

danifus Jun 27, 2024

Choose a reason for hiding this comment

Uh oh!

danifus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

danifus left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Akasurde commented Jul 2, 2024

Uh oh!

danifus commented Jul 3, 2024

Uh oh!

Akasurde commented Aug 17, 2024

Uh oh!

danifus commented Aug 20, 2024

Uh oh!

Akasurde commented Aug 21, 2024

Uh oh!

github-actions Bot commented Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Akasurde commented Jun 26, 2024 •

edited

Loading