Skip to content

Accept ISO 8601 (with timezone) for --video_start_time#822

Open
ptpt wants to merge 1 commit into
mainfrom
video-start-time-iso8601
Open

Accept ISO 8601 (with timezone) for --video_start_time#822
ptpt wants to merge 1 commit into
mainfrom
video-start-time-iso8601

Conversation

@ptpt

@ptpt ptpt commented Jun 24, 2026

Copy link
Copy Markdown
Member

Summary

Addresses part of #819 (GoPro MAX timestamping). The --video_start_time option only accepted the proprietary YYYY_MM_DD_HH_MM_SS_sss format, which is always interpreted as UTC. This left no clean way to correct timestamps for cameras whose RTC has no timezone and therefore records local time (e.g. GoPro MAX).

This PR makes --video_start_time also accept ISO 8601, which carries timezone information:

Input Interpreted as Result (UTC)
2020_12_28_13_36_36_508 (legacy) UTC 13:36:36Z
2020-12-28T13:36:36.508Z UTC 13:36:36Z
2020-12-28T13:36:36.508+01:00 offset as given 12:36:36Z
2020-12-28T13:36:36.508 (naive) system local time depends on host tz
  • The legacy format is unchanged (still UTC).
  • An ISO 8601 value with an explicit offset (or Z) is honored.
  • A naive ISO 8601 value is interpreted in the system local timezone, so a wall-clock time copied straight off a local-time camera lands at the right instant.

Implementation

A new _parse_video_start_time() helper in sample_video.py tries the legacy format first, then falls back to datetime.fromisoformat (with a Z+00:00 shim for Python 3.9/3.10 compatibility) and normalizes to UTC via astimezone. The CLI help text is updated accordingly.

Notes

  • Behavior is documented and covered by doctests on the helper.
  • A naive ISO value is host-timezone dependent by design — worth keeping in mind for CI/automated pipelines.
  • This does not address the other items in Video Timestamping Bugs with GoPro MAX #819 (the GPMF-vs-RTC discrepancy, the GPX fallback for empty-GPMF videos). Happy to follow up on the GPX fallback separately.

Test plan

  • Doctests pass (pytest --doctest-modules mapillary_tools/sample_video.py), including under TZ=US/Pacific
  • Full unit suite: 616 passed, 18 skipped, 1 xfailed
  • mypy and ruff clean

The --video_start_time option only accepted the proprietary
YYYY_MM_DD_HH_MM_SS_sss format, which is always interpreted as UTC. This
left no clean way to correct timestamps for cameras whose RTC has no
timezone and records local time (e.g. GoPro MAX), as reported in #819.

Add a _parse_video_start_time() helper that also accepts ISO 8601:
- an explicit UTC offset (or Z) is honored, and
- a naive value is interpreted in the system local timezone.

The legacy format is unchanged and still treated as UTC. Documented via
doctests and updated the CLI help text.
@JakeSmarter

Copy link
Copy Markdown
Contributor

The bug persists. Although --video_start_time does now accept ISO 8601 format timestamps, the sampled output Exif data remains partially incorrect.

mapillary_tools sample_video --video_sample_distance -1 --video_sample_interval 1 --video_start_time '2026-04-06T14:36:07.88+02:00' 'video.mp4'
⋮
exiv2 -pv 'video.mp4/video_v_000001.jpg'
File 1/1: video.mp4/video_v_000001.jpg
0x8769 Image        ExifTag                     Long        1  38
0x9003 Photo        DateTimeOriginal            Ascii      20  2026:04:06 12:36:07
0x9011 Photo        OffsetTimeOriginal          Ascii       7  +00:00
0x9291 Photo        SubSecTimeOriginal          Ascii       7  880000
0x8825 Image        GPSTag                      Long        1  110
0x0007 GPSInfo      GPSTimeStamp                Rational    3  12/1 36/1 197/25
0x001d GPSInfo      GPSDateStamp                Ascii      11  2026:04:06

In other words, GPSDateStamp and GPSTimeStamp are correct but DateTimeOriginal and OffsetTimeOriginal are not.

There are two subsequent issues here though.

  1. Even if this gets fixed and although the user can specify a video start timestamp with a timezone specifier they still have to know beforehand the exact timestamp when video recording was started. So, unless they run ffprobe on the video file beforehand or run the sample_video command, read the creation_time tag from ffmpeg's verbose output, examine the output Exif timestamps, and then run sample_video again with the correct video start timestamp and a timezone specifier, they have no way of knowing the video start timestamp at all. However, most users should usually know their timezone (beforehand). Hence, a --timezone option may be a good idea after all. Making --video_start_time accept ISO 8601 format timestamps is great but it gets us only half way there.

    And yes, one can pass the --offset_time option to the process command later to sort of adjust the timezone but we should not produce incorrect intermediate results either.

  2. When a video file has no GPS clock signal data (which usually is safe to assume as the reliable clock source) we cannot always rely on creation_time timestamps to be in UTC, especially when written by cameras that have no timezone support. I know, it is working around a bug inflicted on you but since GoPro MAX is a Mapillary recommended camera, we kind of have to live and deal with it. In this case, unless the user passes --video_start_time or --timezone, we could have an interactive prompt that asks the user to verify the detected video start timestamp for correctness (including timezone). This way, incorrect timestamps will not slip through and automation would be served too.
    Unfortunately then though, video files with correctly written UTC creation_time tags would kind of be at a disadvantage in that they should not need any user verification but would still require a --timezone or an explicit lengthy --video_start_time option to pass. 😞 There is no easy way out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants