Update unique fields on standard field - include soft deleted records #14562

etiennejouan · 2025-09-17T13:41:30Z

Done :

migrate standard unique fields (add : -old)
deactivate defaultWhereClause
restore soft deleted if upserted (same for contact creation in mailing sync)

fixes #14443

github-actions · 2025-09-17T13:54:02Z

🚀 Preview Environment Ready!

Your preview environment is available at: http://bore.pub:56949

This environment will automatically shut down when the PR is closed or after 5 hours.

greptile-apps

Greptile Overview

Summary

This PR fixes a critical issue with CSV uploads when mapping multiple unique identifiers by updating the unique field constraint logic to include soft-deleted records.

Key Changes:

Database indexes: Removed "deletedAt" IS NULL constraint from unique indexes so soft-deleted records are included in uniqueness checks
Upsert logic: Enhanced GraphQL resolver to restore soft-deleted records by setting deletedAt: null during upserts
Contact/company creation: Added restoration logic for soft-deleted contacts and companies instead of treating them as conflicts
Database migration: Added v1.7 upgrade command to deduplicate existing unique field violations
Testing: Comprehensive integration tests covering soft-delete restoration scenarios

The changes ensure that when users upload CSV data with existing unique identifiers from soft-deleted records, those records are restored rather than causing constraint violations.

Confidence Score: 4/5

This PR is generally safe to merge with minor syntax issues to address
Score reflects solid implementation of soft-delete restoration logic with comprehensive testing, but contains syntax errors that should be fixed before merge
packages/twenty-server/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts needs syntax fixes

Important Files Changed

File Analysis

Filename	Score	Overview
packages/twenty-server/src/engine/workspace-manager/workspace-migration-builder/factories/utils/workspace-migration-index.factory.utils.ts	5/5	Removed `"deletedAt" IS NULL` constraint from unique indexes to include soft-deleted records in uniqueness checks
packages/twenty-server/src/engine/api/graphql/graphql-query-runner/resolvers/graphql-query-create-many-resolver.service.ts	4/5	Enhanced upsert logic to restore soft-deleted records by setting `deletedAt: null` and updated query to include soft-deleted records
packages/twenty-server/src/modules/contact-creation-manager/services/create-company.service.ts	4/5	Added restoration logic for soft-deleted companies, including filtering and updating deleted companies with `deletedAt: null`
packages/twenty-server/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts	3/5	New command to deduplicate unique fields for existing data, handling workspace members, companies and people with duplicate constraints

Sequence Diagram

sequenceDiagram
    participant U as User
    participant API as GraphQL API  
    participant R as CreateManyResolver
    participant DB as Database
    participant CS as ContactService
    participant COS as CompanyService

    U->>API: CSV Upload with unique identifiers
    API->>R: createMany with upsert: true
    
    R->>DB: Query existing records (including soft-deleted)
    DB->>R: Return existing records
    
    alt Record exists and is soft-deleted
        R->>DB: Update record with deletedAt: null
        DB->>R: Restored record
    else Record exists and not deleted
        R->>DB: Update existing record
        DB->>R: Updated record
    else Record doesn't exist
        R->>DB: Insert new record
        DB->>R: New record
    end

    alt Contact creation workflow
        CS->>DB: Find soft-deleted contacts by email
        DB->>CS: Soft-deleted contacts
        CS->>COS: Create/restore companies for contacts
        COS->>DB: Restore soft-deleted companies
        DB->>COS: Restored companies
        CS->>DB: Restore contacts with company associations
        DB->>CS: Restored contacts
    end

    R->>API: Return processed records
    API->>U: Success response

_{15 files reviewed, 4 comments}

_{Edit Code Review Bot Settings | Greptile}

...r/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts

...src/database/commands/upgrade-version-command/1-10/1-10-deduplicate-unique-fields.command.ts

...r/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts

charlesBochet · 2025-09-25T08:43:35Z

...y-server/src/modules/contact-creation-manager/services/create-company-and-contact.service.ts


-    return this.createContactService.createPeople(
+    const restoredContacts =
+      await this.restorePeopleAndRestoreOrCreateCompanies(


I would have kept one function that's doing both

prastoin · 2025-09-30T12:49:06Z

Hey @etiennejouan is this still up to date ?
Closing this for the moment feel free to re-open

etiennejouan · 2025-09-30T16:01:01Z

Still up to date Mr @prastoin , @charlesBochet could you give a look, please ?

greptile-apps

Greptile Overview

Summary

This review covers only the changes made since the last review, not the entire PR.

The latest changes focus on consolidating and renaming contact creation services to align with a broader refactoring effort that addresses unique field constraints for soft-deleted records. The major update involves renaming CreateContactService to CreatePersonService and CreateCompanyAndContactService to CreateCompanyAndPersonService throughout the codebase, reflecting the actual data model where contacts are stored as Person entities.

Key architectural changes include:

Complete removal of the CreateContactService class, with its functionality migrated to the new CreatePersonService that handles both creation and restoration of Person entities
Enhanced contact creation logic that can now identify whether to create new Person records or restore existing soft-deleted ones, preventing unique constraint violations during CSV imports
Updates to unique index generation to remove the deletedAt IS NULL condition, allowing unique constraints to consider all records including soft-deleted ones
Implementation of a database migration command (DeduplicateUniqueFieldsCommand) that handles existing duplicate unique values by appending -old-{index} suffixes to soft-deleted records
Modification of upsert operations to automatically restore soft-deleted records by setting deletedAt: null

The changes integrate with the existing workspace management, messaging sync, and calendar sync workflows while maintaining backward compatibility through proper service injection and method renaming patterns.

Important Files Changed

Changed Files

Filename	Score	Overview
packages/twenty-server/src/modules/contact-creation-manager/services/create-contact.service.ts	5/5	Complete file deletion - functionality migrated to CreatePersonService
packages/twenty-server/src/database/commands/upgrade-version-command/upgrade.command.ts	5/5	Added version 1.7.0 upgrade with DeduplicateUniqueFieldsCommand
packages/twenty-server/src/modules/contact-creation-manager/services/tests/create-company.service.spec.ts	4/5	Updated tests for createOrRestoreCompanies method with soft-delete handling
packages/twenty-server/test/integration/graphql/suites/upsert/upsert.integration-spec.ts	4/5	Added comprehensive upsert test for restoring soft-deleted records
packages/twenty-server/src/database/commands/upgrade-version-command/1-7/1-7-upgrade-version-command.module.ts	5/5	New module for version 1.7 database upgrade with required dependencies
packages/twenty-server/src/engine/workspace-manager/workspace-migration-builder/factories/utils/workspace-migration-index.factory.utils.ts	4/5	Removed `deletedAt IS NULL` from unique index where clauses
packages/twenty-server/src/modules/contact-creation-manager/utils/filter-out-contacts-that-belong-to-self-or-workspace-members.util.ts	3/5	Renamed function for better clarity with legacy type suppressions
packages/twenty-server/src/modules/calendar/calendar-event-participant-manager/calendar-event-participant-manager.module.ts	3/5	Updated import to use CalendarCreateCompanyAndPersonAfterSyncJob
packages/twenty-server/src/modules/contact-creation-manager/jobs/create-company-and-contact.job.ts	4/5	Updated service imports and method calls to use Person terminology
packages/twenty-server/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts	4/5	New migration command to deduplicate unique fields with comprehensive logic
packages/twenty-server/src/modules/match-participant/utils/add-person-email-filters-to-query-builder.ts	4/5	Added soft-deleted record support with redundant withDeleted() call
packages/twenty-server/src/modules/contact-creation-manager/contact-creation-manager.module.ts	5/5	Module updated with new service names maintaining same structure
packages/twenty-server/src/modules/contact-creation-manager/services/create-company.service.ts	4/5	Enhanced to handle soft-deleted company restoration via createOrRestoreCompanies
packages/twenty-server/src/modules/contact-creation-manager/listeners/auto-companies-and-contacts-creation-calendar-channel.listener.ts	4/5	Updated job types and imports for Person terminology consistency
packages/twenty-server/src/engine/api/graphql/graphql-query-runner/resolvers/graphql-query-create-many-resolver.service.ts	4/5	Added deletedAt: null to restore soft-deleted records during upserts
packages/twenty-server/src/modules/messaging/message-participant-manager/jobs/messaging-create-company-and-contact-after-sync.job.ts	4/5	Service and method name updates with inconsistent logging message
packages/twenty-server/src/engine/metadata-modules/index-metadata/index-metadata.service.ts	5/5	Improved type safety by narrowing objectMetadata parameter type
packages/twenty-server/src/database/commands/upgrade-version-command/upgrade-version-command.module.ts	4/5	Added V1_7_UpgradeVersionCommandModule to support unique field migration
packages/twenty-server/src/modules/contact-creation-manager/services/create-company-and-contact.service.ts	4/5	Major refactor to CreateCompanyAndPersonService with restore functionality
packages/twenty-server/src/modules/contact-creation-manager/services/tests/create-company-and-contact.service.spec.ts	4/5	New test file for CreateCompanyAndPersonService focusing on soft-delete handling
packages/twenty-server/src/engine/workspace-manager/workspace-migration-builder/factories/utils/tests/workspace-migration-index.factory.util.spec.ts	3/5	Updated test expectations to remove deletedAt condition from unique indexes
packages/twenty-server/src/engine/twenty-orm/decorators/workspace-is-unique.decorator.ts	4/5	Modified unique constraint generation by removing prefix and setting whereClause to null
packages/twenty-server/src/modules/contact-creation-manager/services/create-person.service.ts	4/5	New service providing person creation and restoration functionality
packages/twenty-server/src/modules/calendar/calendar-event-participant-manager/jobs/calendar-create-company-and-contact-after-sync.job.ts	4/5	Systematic rename from Contact to Person terminology with method updates

Confidence score: 4/5

This PR involves complex architectural changes with proper testing but requires careful review due to the broad impact on unique constraint handling
Score reflects good implementation of the soft-delete restoration logic but concerns about potential unintended data restoration during upserts
Pay close attention to the database migration command and unique constraint modifications as they affect data integrity across the entire system

Additional Comments (6)

packages/twenty-server/src/modules/contact-creation-manager/utils/filter-out-contacts-that-belong-to-self-or-workspace-members.util.ts, line 25 (link)

syntax: Using object instead of Map for boolean tracking
packages/twenty-server/src/modules/contact-creation-manager/utils/filter-out-contacts-that-belong-to-self-or-workspace-members.util.ts, line 30 (link)

syntax: Inconsistent data structure - declaring Map but using as object
packages/twenty-server/src/modules/contact-creation-manager/utils/filter-out-contacts-that-belong-to-self-or-workspace-members.util.ts, line 40 (link)

syntax: Accessing Map as object - should use .has() method
packages/twenty-server/src/modules/contact-creation-manager/jobs/create-company-and-contact.job.ts, line 8 (link)

style: Type and class names still use 'Contact' terminology while the service uses 'Person'. Consider renaming to maintain consistency throughout the codebase.
packages/twenty-server/src/modules/match-participant/utils/add-person-email-filters-to-query-builder.ts, line 62 (link)

syntax: Duplicate .withDeleted() call - already called on line 38
packages/twenty-server/src/modules/messaging/message-participant-manager/jobs/messaging-create-company-and-contact-after-sync.job.ts, line 111 (link)

syntax: Inconsistent terminology - should use 'people and companies' to match the updated service name and line 39

_{24 files reviewed, 10 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2025-09-30T16:03:56Z

packages/twenty-server/src/modules/contact-creation-manager/services/create-company.service.ts

+    const restoredCompanies = await companyRepository.updateMany(
+      companiesToRestore.map((company) => {
+        return {
+          criteria: company.id,
+          partialEntity: {
+            deletedAt: null,
+          },
+        };
+      }),
+      undefined,
+      ['domainNamePrimaryLinkUrl', 'id'],
+    );


style: Consider checking if companiesToRestore array is empty before calling updateMany to avoid unnecessary database operations

Prompt To Fix With AI

This is a comment left during a code review. Path: packages/twenty-server/src/modules/contact-creation-manager/services/create-company.service.ts Line: 109:120 Comment: style: Consider checking if companiesToRestore array is empty before calling updateMany to avoid unnecessary database operations How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-09-30T16:03:56Z

...creation-manager/listeners/auto-companies-and-contacts-creation-calendar-channel.listener.ts

-  type CalendarCreateCompanyAndContactAfterSyncJobData,
+  CalendarCreateCompanyAndPersonAfterSyncJob,
+  CalendarCreateCompanyAndPersonAfterSyncJobData,
 } from 'src/modules/calendar/calendar-event-participant-manager/jobs/calendar-create-company-and-contact-after-sync.job';


style: Import path references 'calendar-create-company-and-contact-after-sync.job' but the imported types use 'Person' terminology - consider updating the filename to match the new naming convention

Prompt To Fix With AI

This is a comment left during a code review. Path: packages/twenty-server/src/modules/contact-creation-manager/listeners/auto-companies-and-contacts-creation-calendar-channel.listener.ts Line: 14:14 Comment: style: Import path references 'calendar-create-company-and-contact-after-sync.job' but the imported types use 'Person' terminology - consider updating the filename to match the new naming convention How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-09-30T16:03:58Z

...y-server/src/modules/contact-creation-manager/services/create-company-and-contact.service.ts

+        handle.toLowerCase(),
+      )?.existingPerson;
+
+      if (!isDefined(existingPerson) || isNull(existingPerson.deletedAt))


logic: Logic error - condition is inverted. Should check if deletedAt is NOT null for restoration

Suggested change

if (!isDefined(existingPerson) || isNull(existingPerson.deletedAt))

if (!isDefined(existingPerson) || !isNull(existingPerson.deletedAt))

Prompt To Fix With AI

This is a comment left during a code review. Path: packages/twenty-server/src/modules/contact-creation-manager/services/create-company-and-contact.service.ts Line: 367:367 Comment: logic: Logic error - condition is inverted. Should check if deletedAt is NOT null for restoration ```suggestion if (!isDefined(existingPerson) || !isNull(existingPerson.deletedAt)) ``` How can I resolve this? If you propose a fix, please make it concise.

greptile-apps · 2025-09-30T16:03:59Z

...alendar-event-participant-manager/jobs/calendar-create-company-and-contact-after-sync.job.ts

 import { type CalendarChannelWorkspaceEntity } from 'src/modules/calendar/common/standard-objects/calendar-channel.workspace-entity';
 import { type CalendarEventParticipantWorkspaceEntity } from 'src/modules/calendar/common/standard-objects/calendar-event-participant.workspace-entity';
-import { CreateCompanyAndContactService } from 'src/modules/contact-creation-manager/services/create-company-and-contact.service';
+import { CreateCompanyAndPersonService } from 'src/modules/contact-creation-manager/services/create-company-and-contact.service';


style: Import path still references 'create-company-and-contact.service' but imports 'CreateCompanyAndPersonService' - consider updating the file path to match the service name

Prompt To Fix With AI

This is a comment left during a code review. Path: packages/twenty-server/src/modules/calendar/calendar-event-participant-manager/jobs/calendar-create-company-and-contact-after-sync.job.ts Line: 12:12 Comment: style: Import path still references 'create-company-and-contact.service' but imports 'CreateCompanyAndPersonService' - consider updating the file path to match the service name How can I resolve this? If you propose a fix, please make it concise.

charlesBochet

LGTM, it seems that deleted contacts are not restored though. We can check this in a followup

…twentyhq#14562) Done : - migrate standard unique fields (add : -old) - deactivate defaultWhereClause - restore soft deleted if upserted (same for contact creation in mailing sync) fixes twentyhq#14443 --------- Co-authored-by: Charles Bochet <[email protected]>

etiennejouan added the -PR: awaiting author label Sep 17, 2025

etiennejouan added 7 commits September 23, 2025 11:47

include soft deleted records in unique index

4ce7db4

add upgrade command

90cae93

update createMany : restore soft deleted records if updated

a065cdb

fix test

d2f3b92

add restoration logic for contacts and companies - to do test

5627e82

add tests

a281b3b

rebase and move upgrade command to 1-7

47ae5b0

etiennejouan force-pushed the ej/14443 branch from 5aa7acc to 47ae5b0 Compare September 23, 2025 09:51

etiennejouan marked this pull request as ready for review September 23, 2025 09:51

etiennejouan added -PR: awaiting review and removed -PR: awaiting author labels Sep 23, 2025

greptile-apps bot reviewed Sep 23, 2025

View reviewed changes

...r/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts Outdated Show resolved Hide resolved

...r/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts Outdated Show resolved Hide resolved

etiennejouan and others added 3 commits September 23, 2025 12:46

fix tests

09b5f1c

fix test

6c861c8

Merge branch 'main' into ej/14443

12233e7

charlesBochet reviewed Sep 25, 2025

View reviewed changes

...src/database/commands/upgrade-version-command/1-10/1-10-deduplicate-unique-fields.command.ts Show resolved Hide resolved

charlesBochet reviewed Sep 25, 2025

View reviewed changes

...r/src/database/commands/upgrade-version-command/1-7/1-7-deduplicate-unique-fields.command.ts Outdated Show resolved Hide resolved

charlesBochet reviewed Sep 25, 2025

View reviewed changes

charlesBochet and others added 7 commits September 25, 2025 11:15

Fix

31ed431

Fix existing index update

b8bd9ee

Fix

193a5bf

Fixes

cf2fcff

Fix

8b42da6

Fix

db7949b

logic + test

b14f489

prastoin closed this Sep 30, 2025

etiennejouan reopened this Sep 30, 2025

greptile-apps bot reviewed Sep 30, 2025

View reviewed changes

etiennejouan requested a review from charlesBochet September 30, 2025 16:07

charlesBochet added 2 commits October 11, 2025 15:15

Merge branch 'main' into ej/14443

e46a99a

Fix

7d1fbb4

charlesBochet approved these changes Oct 11, 2025

View reviewed changes

charlesBochet merged commit 560e24f into main Oct 11, 2025
42 checks passed

charlesBochet deleted the ej/14443 branch October 11, 2025 13:50

	if (!isDefined(existingPerson) \|\| isNull(existingPerson.deletedAt))
	if (!isDefined(existingPerson) \|\| !isNull(existingPerson.deletedAt))

Update unique fields on standard field - include soft deleted records #14562

Update unique fields on standard field - include soft deleted records #14562

Uh oh!

Conversation

etiennejouan commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Sep 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Greptile Overview

Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

charlesBochet Sep 25, 2025

Choose a reason for hiding this comment

Uh oh!

prastoin commented Sep 30, 2025

Uh oh!

etiennejouan commented Sep 30, 2025

Uh oh!

greptile-apps bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Greptile Overview

Summary

Important Files Changed

Confidence score: 4/5

Additional Comments (6)

Uh oh!

greptile-apps bot Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

charlesBochet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

etiennejouan commented Sep 17, 2025 •

edited

Loading

github-actions bot commented Sep 17, 2025 •

edited

Loading

greptile-apps bot left a comment •

edited

Loading