Export/Import challenges - Data Parity Between Local and Hosted Instance

[LucasZapico]

Summary:

The current options available to create data(entity) parity between instances for example local and Twenty hosted are lacking.

• importing a custom entity into another instance has minor support
• no clear method to maintain relationships between entities
• the current process does not appear to be equipped to support migration without significant work for CRMs like HubSpot, Zoho, Salesforce, etc.

Expectation

The goal is to have a straightforward solution for data parity (export/import) between a local instance and the hosted instance, ensuring:

• Synchronization of records
• Preservation of data model entity relationships
• Support for creating columns dynamically during CSV imports

Implications

• This affects the import/export functionality and its future direction.
• It plays a crucial role in onboarding clients from other CRMs like HubSpot, Zoho, and Salesforce.
• Entity relationship integrity must be maintained during data import/export.

Story

I have been doing nitty, gritty testing on my production CRM and I want to stop to try and maintain a degree of "clean data". I spun up a local instance and am attempting to mirror my hosted instance. While attempting to resolve this, I followed the import/export docs. I found that the current scope of the import/export functionality was too narrow to achieve the expected parity. For example, I attempted to create data parity for an entity called jobs in a local instance by exporting the CSV from the hosted instance as called out in the documentation. However, during the CSV upload process in the local instance, there was no option to “create a column,” or a solution to handle the relationships I have defined in my hosted model.

Thoughts

I believe the most immediate solution would be adding the ability to create columns directly from the imported CSV process. This solution only addresses entity properties and not the relationships between entities, which would still need to be handled manually.

Thank you.

[BOHEUS]

@LucasZapico preservation of data model entities relations is now possible with apps as all standard object ids are unified and if you create an object in app, it'll have the same id across different workspaces, however there can be still a problem with synchronizing records as Twenty doesn't allow for importing morph and junction/many-to-many relations, also, there's no option to create columns dynamically during the import