Skip to content

Commit a8515a8

Browse files
committed
Refine logic for Employee Salary History unique key
Why these changes are being introduced: A recently added integrity alert triggered for the 'Employee Salary History' table, indicating we had a row in Quickbase that no longer aligned with data warehouse data. As has happened before, it appears to be shifting data warehouse values, this time for "End Date" and reason and type data, that modifies the unique key. When the unique key changes, we get stale data in Quickbase because we are adding rows instead of updating them. How this addresses that need: After some analysis, and discussion with HR, it was determined we could remove some fields from the unique key logic which keeps it consistent between data warehouse changes. Part of this work was also skipping any salary history with a reason of "Lump Sum" which has only showed up once, and is the only time we might get multiple values for a single person, on a single day, for a single appointment. Similarly, we already skip "Supplemental Pay" for this reason. Side effects of this change: * Improve merge field values, reduce stale data in Salary History table. Relevant ticket(s): * https://mitlibraries.atlassian.net/browse/IN-1450
1 parent 9755a14 commit a8515a8

3 files changed

Lines changed: 2 additions & 4 deletions

File tree

hrqb/tasks/employee_salary_history.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,6 @@ def get_dataframe(self) -> pd.DataFrame:
8282
row.mit_id,
8383
row.position_id,
8484
str(row.appointment_begin_date),
85-
str(row.hr_personnel_action),
86-
str(row.hr_action_reason),
8785
str(row.start_date),
8886
]
8987
),

hrqb/tasks/sql/employee_salary_history.sql

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ CHANGELOG
1111
- 2024-07-24 Added appointment begin/end date to help match appointments
1212
- 2024-09-18 Omit any HR_PERSONNEL_ACTION_TYPE rows where type is "Salary Supplement"
1313
- 2025-02-05 Remove 2019-01-01 date cutoff entirely
14+
- 2025-09-04 Omit any HR_ACTION_REASON rows where reason is "Lump Sum"
1415
*/
1516

1617
select distinct
@@ -47,4 +48,5 @@ left join HR_PERSONNEL_ACTION_TYPE at on at.HR_PERSONNEL_ACTION_TYPE_KEY = a.HR_
4748
left join HR_JOB j on j.HR_JOB_KEY = a.HR_JOB_KEY
4849
left join HR_POSITION p on p.HR_POSITION_KEY = a.HR_POSITION_KEY
4950
where at.HR_PERSONNEL_ACTION not in ('Salary Supplement')
51+
and at.HR_ACTION_REASON not in ('Lump Sum')
5052
order by a.MIT_ID, a.APPT_TX_BEGIN_DATE, a.APPT_TX_END_DATE

tests/tasks/test_employee_salary_history.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,8 +58,6 @@ def test_task_transform_employee_salary_history_key_expected_from_input_data(
5858
emp_salary_row["MIT ID"],
5959
qb_emp_appt_row["Position ID"],
6060
qb_emp_appt_row["Begin Date"],
61-
emp_salary_row["Related Salary Change Type"],
62-
emp_salary_row["Salary Change Reason"],
6361
emp_salary_row["Start Date"],
6462
]
6563
)

0 commit comments

Comments
 (0)