Skip to content

Conversation

erikamov
Copy link
Contributor

@erikamov erikamov commented Sep 12, 2025

Description

This PR changes Unzip and Validate GTFS Schedule Hourly start date to allow run dates from January 1 2025.

[#4294]

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

How has this been tested?

It is just a config setting.

Post-merge follow-ups

  • No action required
  • Actions required (specified below)

Check if unzip_and_validate_gtfs_schedule_hourly DAG is able to run the dates needed in production.

@erikamov erikamov self-assigned this Sep 12, 2025
@erikamov erikamov changed the title Change Parse and Validate RT start date to allow run from July 1st 2025 Change DAGs start date to allow creating jobs for older dates Sep 12, 2025
@erikamov erikamov changed the title Change DAGs start date to allow creating jobs for older dates Change Unzip and Validate GTFS Schedule Hourly start date to allow creating jobs for older dates Sep 12, 2025
@erikamov erikamov force-pushed the mov/4284-change-start-date branch from 4e4c340 to 34f16e4 Compare September 12, 2025 22:30
@cal-itp cal-itp deleted a comment from github-actions bot Sep 12, 2025
@cal-itp cal-itp deleted a comment from github-actions bot Sep 12, 2025
@cal-itp cal-itp deleted a comment from github-actions bot Sep 12, 2025
@erikamov erikamov force-pushed the mov/4284-change-start-date branch from 34f16e4 to 62060b2 Compare September 12, 2025 22:45
Copy link

github-actions bot commented Sep 12, 2025

Terraform plan in iac/cal-itp-data-infra/composer/us

No changes. Your infrastructure matches the configuration.
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

📝 Plan generated in Plan Terraform for Warehouse and DAG changes #720

Copy link

github-actions bot commented Sep 12, 2025

Terraform plan in iac/cal-itp-data-infra/airflow/us

Plan: 0 to add, 1 to change, 0 to destroy.
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~  update in-place

Terraform will perform the following actions:

  # google_storage_bucket_object.calitp-composer["dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"] will be updated in-place
!~  resource "google_storage_bucket_object" "calitp-composer" {
!~      crc32c              = "9/YYeQ==" -> (known after apply)
!~      detect_md5hash      = "2pPOIKu+6ALlfWOnlxTJqg==" -> "different hash"
!~      generation          = 1753225938235511 -> (known after apply)
        id                  = "calitp-composer-dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
!~      md5hash             = "2pPOIKu+6ALlfWOnlxTJqg==" -> (known after apply)
        name                = "dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
#        (17 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

📝 Plan generated in Plan Terraform for Warehouse and DAG changes #720

Copy link

github-actions bot commented Sep 12, 2025

Terraform plan in iac/cal-itp-data-infra-staging/airflow/us

Plan: 0 to add, 1 to change, 0 to destroy.
Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~  update in-place

Terraform will perform the following actions:

  # google_storage_bucket_object.calitp-staging-composer["dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"] will be updated in-place
!~  resource "google_storage_bucket_object" "calitp-staging-composer" {
!~      crc32c              = "9/YYeQ==" -> (known after apply)
!~      detect_md5hash      = "2pPOIKu+6ALlfWOnlxTJqg==" -> "different hash"
!~      generation          = 1753225939203114 -> (known after apply)
        id                  = "calitp-staging-composer-dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
!~      md5hash             = "2pPOIKu+6ALlfWOnlxTJqg==" -> (known after apply)
        name                = "dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
#        (17 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

📝 Plan generated in Plan Terraform for Warehouse and DAG changes #720

@erikamov erikamov force-pushed the mov/4284-change-start-date branch from 62060b2 to 148cd56 Compare September 12, 2025 22:47
@erikamov erikamov marked this pull request as draft September 12, 2025 22:47
@vevetron
Copy link
Contributor

I would like this to go back to 2024-01-01

@ohrite ohrite force-pushed the mov/4284-change-start-date branch from 148cd56 to 71b6b0e Compare September 25, 2025 01:39
@ohrite ohrite force-pushed the mov/4284-change-start-date branch from 71b6b0e to 50858ad Compare September 26, 2025 04:00
Copy link
Contributor

@vevetron vevetron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have this go back to 2024-01-01?

There is unzip failures in 2024 for schedule date I would like to be rebuilt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants