This repository was archived by the owner on Sep 30, 2024. It is now read-only.
Commit 72d8422
[Backport 5.1] internal/event_logs: optimize ping cte (#55714)
The aggregatedCodyUsageEventsQuery is timing out for dotcom. These
changes are cte optimization steps.
## Test plan
Extract query from most expensive ping-supporting query,
[aggregatedCodyUsageEventsQuery](https://sourcegraph.sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/internal/database/event_logs.go?L1594),
analyze original query:
```
GroupAggregate (cost=143042.53..143382.23 rows=270 width=158) (actual time=492.653..608.924 rows=87 loops=1)
Group Key: event_logs.name, (date_trunc('month'::text, '2023-08-01 17:00:00-07'::timestamp with time zone)), ((date_trunc('week'::text, ('2023-08-01 17:00:00-07'::timestamp with time zone + '1 day'::interval)) - '1 day'::interval)), (date_trunc('day'::text, '2023-08-01 17:00:00-07'::timestamp with time zone))
CTE code_generation_keys
-> Function Scan on unnest key (cost=0.00..0.09 rows=9 width=32) (actual time=0.005..0.007 rows=9 loops=1)
CTE explanation_keys
-> Function Scan on unnest key_1 (cost=0.00..0.05 rows=5 width=32) (actual time=0.012..0.013 rows=5 loops=1)
-> Sort (cost=143042.38..143046.21 rows=1530 width=62) (actual time=492.526..496.803 rows=33511 loops=1)
Sort Key: event_logs.name
Sort Method: external merge Disk: 3056kB
-> Gather (cost=1000.16..142961.45 rows=1530 width=62) (actual time=12.381..418.370 rows=33511 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Parallel Seq Scan on event_logs (cost=0.17..141808.45 rows=638 width=62) (actual time=10.540..417.177 rows=11170 loops=3)
Filter: ((name !~~ '%completion:suggested%'::text) AND (name !~~ '%completion:started%'::text) AND (name !~~ '%CTA%'::text) AND (name !~~ '%Cta%'::text) AND (NOT (hashed SubPlan 9)) AND (lower(name) ~~ '%cody%'::text) AND ("timestamp" >= (date_trunc('month'::text, '2023-08-01 17:00:00-07'::timestamp with time zone) - '1 mon'::interval)))
Rows Removed by Filter: 382174
SubPlan 9
-> Function Scan on unnest (cost=0.00..0.13 rows=13 width=32) (actual time=0.007..0.008 rows=13 loops=3)
SubPlan 8
-> CTE Scan on explanation_keys explanation_keys_3 (cost=0.00..0.10 rows=5 width=32) (actual time=0.000..0.001 rows=5 loops=1)
SubPlan 7
-> CTE Scan on explanation_keys explanation_keys_2 (cost=0.00..0.10 rows=5 width=32) (actual time=0.014..0.016 rows=5 loops=1)
SubPlan 6
-> CTE Scan on explanation_keys explanation_keys_1 (cost=0.00..0.10 rows=5 width=32) (actual time=0.000..0.001 rows=5 loops=1)
SubPlan 5
-> CTE Scan on code_generation_keys code_generation_keys_1 (cost=0.00..0.18 rows=9 width=32) (actual time=0.000..0.002 rows=9 loops=1)
SubPlan 4
-> CTE Scan on explanation_keys (cost=0.00..0.10 rows=5 width=32) (actual time=0.000..0.001 rows=5 loops=1)
SubPlan 3
-> CTE Scan on code_generation_keys (cost=0.00..0.18 rows=9 width=32) (actual time=0.007..0.010 rows=9 loops=1)
Planning Time: 1.556 ms
Execution Time: 610.143 ms
```
Add indexes and analyze again in comments. <br> Backport
41e5cf3 from #55538
---------
Co-authored-by: Nathan Downs <[email protected]>1 parent 47f426c commit 72d8422
File tree
10 files changed
+244
-81
lines changed- internal/database
- migrations
- frontend
- 1691043630_event_logs_indexing
10 files changed
+244
-81
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
32 | 32 | | |
33 | 33 | | |
34 | 34 | | |
| 35 | + | |
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| 41 | + | |
| 42 | + | |
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1565 | 1565 | | |
1566 | 1566 | | |
1567 | 1567 | | |
1568 | | - | |
1569 | | - | |
1570 | | - | |
1571 | | - | |
1572 | | - | |
1573 | | - | |
1574 | | - | |
1575 | | - | |
1576 | | - | |
1577 | | - | |
1578 | | - | |
1579 | | - | |
1580 | | - | |
1581 | | - | |
1582 | | - | |
1583 | | - | |
1584 | | - | |
1585 | 1568 | | |
1586 | | - | |
| 1569 | + | |
| 1570 | + | |
1587 | 1571 | | |
1588 | | - | |
| 1572 | + | |
1589 | 1573 | | |
1590 | 1574 | | |
1591 | 1575 | | |
| |||
1595 | 1579 | | |
1596 | 1580 | | |
1597 | 1581 | | |
1598 | | - | |
1599 | | - | |
1600 | | - | |
1601 | | - | |
1602 | | - | |
1603 | | - | |
1604 | | - | |
1605 | | - | |
1606 | | - | |
1607 | | - | |
1608 | | - | |
1609 | | - | |
1610 | | - | |
1611 | | - | |
1612 | | - | |
1613 | | - | |
1614 | | - | |
1615 | | - | |
1616 | | - | |
1617 | | - | |
1618 | | - | |
1619 | | - | |
1620 | | - | |
1621 | | - | |
1622 | | - | |
1623 | | - | |
1624 | | - | |
| 1582 | + | |
| 1583 | + | |
1625 | 1584 | | |
1626 | 1585 | | |
1627 | | - | |
| 1586 | + | |
1628 | 1587 | | |
1629 | 1588 | | |
1630 | 1589 | | |
| |||
1634 | 1593 | | |
1635 | 1594 | | |
1636 | 1595 | | |
1637 | | - | |
1638 | | - | |
| 1596 | + | |
1639 | 1597 | | |
1640 | | - | |
1641 | | - | |
| 1598 | + | |
1642 | 1599 | | |
1643 | | - | |
| 1600 | + | |
1644 | 1601 | | |
1645 | | - | |
| 1602 | + | |
1646 | 1603 | | |
1647 | | - | |
| 1604 | + | |
1648 | 1605 | | |
1649 | | - | |
| 1606 | + | |
1650 | 1607 | | |
1651 | 1608 | | |
1652 | 1609 | | |
1653 | 1610 | | |
1654 | 1611 | | |
1655 | | - | |
| 1612 | + | |
1656 | 1613 | | |
1657 | 1614 | | |
1658 | 1615 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1317 | 1317 | | |
1318 | 1318 | | |
1319 | 1319 | | |
1320 | | - | |
| 1320 | + | |
1321 | 1321 | | |
1322 | 1322 | | |
1323 | 1323 | | |
| |||
1332 | 1332 | | |
1333 | 1333 | | |
1334 | 1334 | | |
1335 | | - | |
| 1335 | + | |
1336 | 1336 | | |
1337 | 1337 | | |
1338 | 1338 | | |
| |||
1361 | 1361 | | |
1362 | 1362 | | |
1363 | 1363 | | |
1364 | | - | |
1365 | | - | |
1366 | | - | |
1367 | | - | |
1368 | | - | |
1369 | | - | |
1370 | | - | |
1371 | | - | |
1372 | | - | |
1373 | | - | |
1374 | | - | |
1375 | | - | |
1376 | | - | |
1377 | | - | |
1378 | | - | |
| 1364 | + | |
| 1365 | + | |
| 1366 | + | |
| 1367 | + | |
| 1368 | + | |
| 1369 | + | |
| 1370 | + | |
| 1371 | + | |
| 1372 | + | |
| 1373 | + | |
| 1374 | + | |
| 1375 | + | |
| 1376 | + | |
| 1377 | + | |
| 1378 | + | |
| 1379 | + | |
| 1380 | + | |
| 1381 | + | |
| 1382 | + | |
1379 | 1383 | | |
1380 | 1384 | | |
1381 | 1385 | | |
1382 | 1386 | | |
1383 | 1387 | | |
1384 | 1388 | | |
1385 | | - | |
1386 | | - | |
1387 | | - | |
1388 | | - | |
1389 | | - | |
1390 | | - | |
1391 | | - | |
1392 | | - | |
| 1389 | + | |
| 1390 | + | |
| 1391 | + | |
| 1392 | + | |
| 1393 | + | |
| 1394 | + | |
| 1395 | + | |
| 1396 | + | |
| 1397 | + | |
| 1398 | + | |
| 1399 | + | |
| 1400 | + | |
| 1401 | + | |
| 1402 | + | |
| 1403 | + | |
1393 | 1404 | | |
1394 | 1405 | | |
1395 | 1406 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
138 | 138 | | |
139 | 139 | | |
140 | 140 | | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
141 | 153 | | |
142 | 154 | | |
143 | 155 | | |
| |||
10400 | 10412 | | |
10401 | 10413 | | |
10402 | 10414 | | |
| 10415 | + | |
| 10416 | + | |
| 10417 | + | |
| 10418 | + | |
| 10419 | + | |
| 10420 | + | |
| 10421 | + | |
| 10422 | + | |
| 10423 | + | |
| 10424 | + | |
| 10425 | + | |
| 10426 | + | |
| 10427 | + | |
| 10428 | + | |
| 10429 | + | |
| 10430 | + | |
| 10431 | + | |
| 10432 | + | |
| 10433 | + | |
| 10434 | + | |
| 10435 | + | |
| 10436 | + | |
| 10437 | + | |
| 10438 | + | |
| 10439 | + | |
| 10440 | + | |
| 10441 | + | |
| 10442 | + | |
| 10443 | + | |
| 10444 | + | |
10403 | 10445 | | |
10404 | 10446 | | |
10405 | 10447 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1355 | 1355 | | |
1356 | 1356 | | |
1357 | 1357 | | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
1358 | 1361 | | |
1359 | 1362 | | |
1360 | 1363 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 11 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
Lines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
Lines changed: 65 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
0 commit comments