Aditya-feat: Add Smart Insights and Predictive Utilization Analysis to the Utilization Chart

[Aditya-gam]

Description

Extends the BM Tool Utilization backend so the Utilization Chart can act as a decision-support surface: per–tool-type classification (under / normal / over) with traffic-light semantics, rules-based forecasting (30-day and schedule-horizon modes), maintenance-style alerts, resource balancing suggestions, and Procurement/Operations export as PDF or CSV. The original chart contract remains supported: GET /api/tools/utilization still returns a JSON array of tool-type rows; additional fields and optional mode are additive where applicable.

Related PRS (if any):

Frontend: OneCommunityGlobal/HighestGoodNetworkApp#5073

• Previous Backend PR: PR #1893
• Previous Frontend PR: PR #3882

Main changes explained:

Created/Updated Files:

• src/constants/toolUtilization.js
  • Centralizes utilization bands (<55% under, 55–85% normal, >85% over), labels, traffic-light tokens, forecast mode enums (historical, forecast30, forecastFull), report formats (pdf, csv), maintenance threshold (85%), regression/EMA tuning (MINIMUM_WEEKS_FOR_REGRESSION, ENSEMBLE_WEIGHTS, CONFIDENCE_THRESHOLDS), degraded condition and non-operational usage strings, and PDF layout constants.
• src/helpers/toolUtilizationHelpers.js
  • Query & aggregation: buildToolFilter (__t: 'tool_item', optional itemType / project when valid ObjectIds or ALL), parseDateRange (default last 30 days), groupToolsByType, calculateCheckedOutHours / calculateGroupUtilization (same pairing rules as the original chart: check-out through check-in, open check-out through range end, pre-range check-out edge case).
  • Caching: buildCacheKey + computeUtilizationData use node-cache (5-minute TTL via src/utilities/nodeCache.js) to cache { utilizationData, rangeStart, rangeEnd, … }. Helpers require require('../utilities/nodeCache')() to use a dedicated cache API instance.
  • Forecasting: bucketUtilizationByWeek → forecastUtilization with regression linear fit + EMA blend (computeEnsemblePredictions); falls back to simple average when fewer than 3 weeks of buckets.
  • Schedule horizon: determineForecastDays reads projectRiskProfile by projectId for forecastFull; falls back to 30 days with an optional warning when project is ALL or no profile/end date.
  • Insights: generateRecommendations, generateMaintenanceAlerts (high utilization, degraded condition, non-operational currentUsage), generateResourceBalancingSuggestions, buildInsightsSummary.
  • Responses: stripInternalDetails removes toolGroupDetails from the public chart payload; buildUtilizationResponse attaches forecast and optional warning per row when mode !== historical; buildReportPayload assembles data for export.
• src/helpers/toolUtilizationReportHelpers.js
  • generatePDFReport: pdfkit stream to response; title “Tool Utilization & Procurement/Operations Report”; sections for summary, utilization rows, maintenance alerts, balancing, recommendations; headers Content-Type: application/pdf, Content-Disposition: attachment.
  • generateCSVReport: json2csv Parser; UTF-8 BOM prefix for Excel; columns include name, utilization, downtime, classification, traffic light, tool count, aggregated maintenance text, recommendation.
• src/controllers/bmdashboard/toolUtilizationController.js
  • getUtilization: Validates mode against VALID_FORECAST_MODES; rejects both startDate and endDate present with startDate > endDate (400); delegates to computeUtilizationData + buildUtilizationResponse; errors via logger.logException.
  • getInsights: Same date-order validation when both dates provided; returns { recommendations, maintenanceAlerts, resourceBalancing, summary }.
  • exportReport: Date-order validation; requires format ∈ { pdf, csv }; streams PDF or sends CSV.
  • Replaces prior inline monolith with thin controller + helpers.
• src/routes/bmdashboard/toolUtilizationRouter.js
  • GET /tools/utilization → getUtilization
  • GET /tools/utilization/insights → getInsights
  • GET /tools/utilization/export → exportReport
  • Mounted under /api in startup/routes.js (unchanged factory signature).
• src/controllers/bmdashboard/__tests__/toolUtilizationController.test.js
  • Mocks helpers + logger; covers getUtilization (default mode, forecast30, invalid mode, errors), getInsights, exportReport (missing/invalid format, pdf vs csv, headers-sent edge case).
• src/helpers/__tests__/toolUtilizationHelpers.test.js
  • Unit coverage for classification, check-out hour math, filters, date range, grouping, forecasting branches, alerts, balancing, summaries, buildUtilizationResponse, buildReportPayload, etc.
• src/helpers/__tests__/toolUtilizationReportHelpers.test.js
  • PDF/CSV headers, piping, BOM, alert/recommendation aggregation behavior.

Key Implementation Details:

• Endpoints (full path with default local API base):
  • GET http://localhost:4500/api/tools/utilization
  • GET http://localhost:4500/api/tools/utilization/insights
  • GET http://localhost:4500/api/tools/utilization/export
• Shared query parameters: tool (ALL or tool type ObjectId), project (ALL or building project ObjectId), startDate, endDate (ISO strings; optional - default window last 30 days).
• Chart-only: mode - historical (default), forecast30, forecastFull. Invalid mode → 400.
• Export-only: format - pdf or csv (required); missing/invalid → 400.
• Forecast math: Weekly buckets from the selected historical window; ensemble of linear regression + EMA when enough weeks; otherwise flat average projection.
• forecastFull caveat: ProjectRiskProfile.projectId is stored as an Issue id in the schema; the BM UI typically sends buildingProject _id as project. If they differ, the lookup may fail, and the API falls back to a 30-day horizon with a warning; reviewers should confirm the product expectation (mapping vs. documentation).

How to test:

1. Check into current branch: Aditya-feat/Add-Smart-Insights-and-Predictive-Utilization-Analysis-to-the-Utilization-Chart
2. Reinstall dependencies and clean cache using rm -rf node_modules package-lock.json && npm cache clean --force
3. Run npm install to install dependencies, then start the backend locally (npm run dev)
4. Use admin/owner Auth token.
5. Test feature / endpoints
   • Historical chart (backward-compatible shape)
     GET /api/tools/utilization?tool=ALL&project=ALL
     Expect 200: JSON array; each element includes name, utilizationRate, downtime, classification (label + trafficLight), toolCount, totalCheckedOutHours, totalPossibleHours, forecast: null.
   • Forecast 30-day
     GET /api/tools/utilization?tool=ALL&project=ALL&mode=forecast30
     Expect 200: each row has forecast object (predictedRate, confidence, forecastEndDate, weeklyPredictions, predictedClassification, method).
   • Forecast full horizon
     GET /api/tools/utilization?tool=ALL&project=<projectId>&mode=forecastFull&startDate=...&endDate=...
     Expect 200: either extended horizon from projectRiskProfiles or fallback warning on each item (warning string) if no profile/end date.
   • Invalid mode
     GET /api/tools/utilization?mode=invalid → 400, error lists allowed modes.
   • Invalid date order
     GET /api/tools/utilization?startDate=2026-02-01&endDate=2026-01-01 → 400, startDate cannot be after endDate.
   • Insights bundle
     GET /api/tools/utilization/insights?tool=ALL&project=ALL
     Expect 200: recommendations, maintenanceAlerts, resourceBalancing, summary.
   • Export CSV
     GET /api/tools/utilization/export?format=csv&tool=ALL&project=ALL
     Expect 200, text/csv; charset=utf-8, body starts with BOM.
   • Export PDF
     GET /api/tools/utilization/export?format=pdf&tool=ALL&project=ALL
     Expect 200, application/pdf, downloadable attachment.
6. Verify
   • Run tests: npm test (or CI-equivalent npm run test:ci).
   • Lint: npm run lint.
   • With real Mongo data, use a date range that overlaps existing logRecord Check In / Check Out events for non-zero utilization.
   • Edge cases: invalid tool/project strings that are not ObjectIds are ignored (filters widen to all tools intentionally).
   • Note: Response cache key is derived from tool, project, startDate, endDate, only switching mode without changing other params may reuse cached utilization data from a prior request; validate in QA or follow up if strict per-mode caching is required.

Screenshots or videos of changes:

• Test Coverage:

• Test Video:

TestVideo.mov

Note:

• Performance Considerations: In-memory aggregation per request; 5-minute node-cache TTL on computeUtilizationData results to reduce repeat DB reads for identical filter/date keys.
• Validation: mode and format validated; startDate/endDate order validated when both present on all three handlers. Invalid dates are not separately validated (invalid Date may yield NaN comparisons - low priority follow-up).
• Backward Compatibility: Same route prefix /api/tools/utilization; new paths are /insights and /export. Existing bmToolsDowntimeRouter still registers other verbs/paths under /tools/utilization - GET remains handled by this router first per startup/routes.js order.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Anusha-Gali]

Hi Aditya,

I have reviewed your API's and the responses are as per requirement.