HiResAs Salesforce consultants, one of the common challenges we see among our clients is related to data storage and retention. As an organization grows, it amasses data, placing more demand on Salesforce storage limits. This puts organizations in a precarious situation and raises several important questions around Salesforce data archiving, including:

1. What data can be removed from the system to keep our storage at an acceptable level?
2. What record retention policies are we legally required to follow based on our industry?
3. Is there a way to shift data storage to a more cost-effective solution and still provide the business access to the information?

That’s where we saw an opportunity to provide a robust Salesforce data archiving solution that utilizes the capabilities of Heroku Connect, Heroku Postgres and Lightning Connect to address data storage demands.

Let’s start with the architecture first. The context diagram below describes the architecture of Cervello’s Salesforce data archiving solution. Here is a summary of each component and the key functionality:

Web Services: provides an interface from Salesforce to request both archived and unarchived actions. The web services interface can be utilized to act on single records or to request a bulk archive of records based on a set of criteria such as last modified date.

Heroku Connect: once configured to the specific objects, both standard and custom, Heroku Connect will provide a bi-directional sync of records between Salesforce and Postgres. This provides the Heroku archiving logic access to both remove (archive) and insert (unarchive) Salesforce records per object.

Lightning Connect: utilizing the Heroku External Objects functionality, we expose the archived records in Salesforce as custom objects. This provides users with seamless access to 100% of their data no matter if the record is considered “live” in Salesforce or archived.

Salesforce-Archiving

To see a full demonstration of the functionality, below is a short video demo of how the solution works.

One of the keys to the Cervello solution is the uniqueness associated with the different scenarios it can handle. For example, it is expected that an object will have other related objects associated with it. This is critical to understand as it impacts the order dependency of what needs to be archived first. In the case of an Account, it will have Contacts associated with it which will need to be archived first. Once archived, it is important to still maintain these relationships when the records are displayed via Lightning Connect. Finally, to unarchive the Account and make it a “live” record in Salesforce you will need to bring over the related Contacts and maintain the relationships. Fortunately, we have thought through these scenarios in our solution design and can address the different object relationships, both standard, and custom.

The next blog in this series will explore a complex scenario with more objects and relationships.

Interested in learning more? Join our mailing list!

Author:

Pete Graham
Jason Reed

Leave a Reply

Your email address will not be published. Required fields are marked *

2 comments on “Salesforce Data Archiving Leveraging Lightning Connect & Heroku: Part I

    • Yes, we support both Notes and Attachments with one caveat. We rely on Heroku Connect, which uses the SOAP API when you are under 50,000 records. Over 50,000 records the Salesforce Bulk API is used, which does not support Attachments. For our solution we will only use Heroku Connect for the objects you want to archive; therefore as long as those objects total less than 50,000 records in a live state then we can archive attachments.
      Let me know if you have any questions.
      Thanks, Jason.