-
Notifications
You must be signed in to change notification settings - Fork 15
Change Unzip and Validate GTFS Schedule Hourly
start date to allow creating jobs for older dates
#4292
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Unzip and Validate GTFS Schedule Hourly
start date to allow creating jobs for older dates
4e4c340
to
34f16e4
Compare
34f16e4
to
62060b2
Compare
Terraform plan in iac/cal-itp-data-infra/composer/us No changes. Your infrastructure matches the configuration.
📝 Plan generated in Plan Terraform for Warehouse and DAG changes #720 |
Terraform plan in iac/cal-itp-data-infra/airflow/us Plan: 0 to add, 1 to change, 0 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~ update in-place
Terraform will perform the following actions:
# google_storage_bucket_object.calitp-composer["dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-composer" {
!~ crc32c = "9/YYeQ==" -> (known after apply)
!~ detect_md5hash = "2pPOIKu+6ALlfWOnlxTJqg==" -> "different hash"
!~ generation = 1753225938235511 -> (known after apply)
id = "calitp-composer-dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
!~ md5hash = "2pPOIKu+6ALlfWOnlxTJqg==" -> (known after apply)
name = "dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
# (17 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy. 📝 Plan generated in Plan Terraform for Warehouse and DAG changes #720 |
Terraform plan in iac/cal-itp-data-infra-staging/airflow/us Plan: 0 to add, 1 to change, 0 to destroy.Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
!~ update in-place
Terraform will perform the following actions:
# google_storage_bucket_object.calitp-staging-composer["dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"] will be updated in-place
!~ resource "google_storage_bucket_object" "calitp-staging-composer" {
!~ crc32c = "9/YYeQ==" -> (known after apply)
!~ detect_md5hash = "2pPOIKu+6ALlfWOnlxTJqg==" -> "different hash"
!~ generation = 1753225939203114 -> (known after apply)
id = "calitp-staging-composer-dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
!~ md5hash = "2pPOIKu+6ALlfWOnlxTJqg==" -> (known after apply)
name = "dags/unzip_and_validate_gtfs_schedule_hourly/METADATA.yml"
# (17 unchanged attributes hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy. 📝 Plan generated in Plan Terraform for Warehouse and DAG changes #720 |
62060b2
to
148cd56
Compare
I would like this to go back to |
148cd56
to
71b6b0e
Compare
…n dates from beginning of 2025 [#4294]
71b6b0e
to
50858ad
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we have this go back to 2024-01-01?
There is unzip failures in 2024 for schedule date I would like to be rebuilt.
Description
This PR changes
Unzip and Validate GTFS Schedule Hourly
start date to allow run dates from January 1 2025.[#4294]
Type of change
How has this been tested?
It is just a config setting.
Post-merge follow-ups
Check if
unzip_and_validate_gtfs_schedule_hourly
DAG is able to run the dates needed in production.