Mocking Apex History Records

There are a few limitations when it comes to the History records created by default when Field Level History is enabled for an object, not the least of which is history records not being created in tests. This can make testing logic out that employs History records difficult to approach, especially from a Test Driven Development standpoint — but it doesn’t need to be so hard. Join me as we walk through how to setup an easily reuseable implementation for mocking History records and retrieving them.

Doing a deep dive on history objects reminded me (extremely wistfully) of One Small Change — in the sense that once we’d started down this route, it quickly became clear that the fast iteration TDD enthusiasts come to expect from their work was about to be rudely interrupted. But let me back up — let’s start from the beginning.

A Short Overview On Prior History Record Mocking

Five years ago, I had newly joined what become my second org — it was there that much of the groundwork you’ve read on this site began to develop. We practiced pair programming and TDD exclusively, and as a result when we had an ask to supplement data on LeadHistory, we thought we knew exactly where to begin:

public class ExampleLeadHistoryBatchable implements Database.Batchable<SObject> {
  public Database.QueryLocator start(Database.BatchableContext bc) {
    return Database.getQueryLocator[
        SELECT CreatedDate, OldValue, NewValue, LeadId, Field
        FROM LeadHistory
        // etc ...
    ];
  }

  public void execute(Database.BatchableContext bc, List<SObject> scope) {
  }

  public void finish(Database.BatchableContext bc) {
  }
}

This was going to be fun! It was going to be easy!

@IsTest
private class ExampleLeadHistoryBatchable {
  @IsTest
  static void queryLocatorRunsOnCorrectDataset() {
    Lead lead = new Lead(LastName = ExampleLeadHistoryBatchable.class.getName());
    insert lead;
    // update some field that's being tracked to generate a history record
    lead.LastName = 'somethingElse';

    Database.QueryLocator queryLocator = new ExampleLeadHistoryBatchable().start(null);

    System.assertEquals(1, Database.query(queryLocator.getQuery()).size());
  }
}

… Except, as I covered in the intro, this test fails. History records don’t get created during unit tests. That’s fine, you might say, I’ve been reading along! I’ll just create the records myself and insert them to be found by the query!

Sure … let’s give that a go?

@IsTest
static void queryLocatorRunsOnCorrectDataset() {
  LeadHistory historyRecord = new LeadHistory(
    Field = 'LastName',
    LeadId = '00Q000000000000',
    OldValue = 'ExampleLeadHistoryBatchable',
    NewValue = 'somethingElse'
  );
  insert historyRecord;

  Database.QueryLocator queryLocator = new ExampleLeadHistoryBatchable().start(null);

  System.assertEquals(1, Database.query(queryLocator.getQuery()).size());
}

… Except that test doesn’t compile. Instead we get the following error on save: Field is not writeable: LeadHistory.OldValue. Ah, but we’re old hats at this. I know what we’ll do! We’ll serialize the record, set the read only field, and then deserialize back to the record! You may recall from elsewhere that I’ve shown off this snippet before:

// In TestingUtils.cls
public static SObject setReadOnlyField(SObject sobj, String fieldName, Object value) {
  return setReadOnlyField(sobj, new Map<String, Object>{ fieldName => value });
}

public static SObject setReadOnlyField(SObject sobj, Map<String, Object> changesToFields) {
  String serializedRecord = JSON.serialize(sobj);
  Map<String, Object> deserializedRecordMap = (Map<String, Object>) JSON.deserializeUntyped(serializedRecord);

  // Loop through the deserialized record map and put the field & value
  // Since it's a map, if the field already exists on the SObject, it's updated (or added if it wasn't there already)
  for(String sobjectField : changesToFields.keySet()) {
    deserializedRecordMap.put(sobjectField, changesToFields.get(sobjectField));
  }

  serializedRecord = JSON.serialize(deserializedRecordMap);
  return (SObject) JSON.deserialize(serializedRecord, SObject.class);
}

OK, sure, let’s give it a go:

@IsTest
static void queryLocatorRunsOnCorrectDataset() {
  LeadHistory historyRecord = new LeadHistory(
    Field = 'LastName',
    LeadId = '00Q000000000000',
  );
  historyRecord = (LeadHistory) TestingUtils.setReadOnlyField(
    historyRecord,
    new Map<String, Object>{
      'OldValue' => 'ExampleLeadHistoryBatchable',
      'NewValue' => 'somethingElse'
    }
  );
  insert historyRecord;

  Database.QueryLocator queryLocator = new ExampleLeadHistoryBatchable().start(null);

  System.assertEquals(1, Database.query(queryLocator.getQuery()).size());
}

Awesome! That compiles. We’ll probably be done with this feature today, at this rate. Except running that test produces:

System.UnexpectedException: Salesforce System Error: 6161127-91129 (-26810738) (-26810738)
(System Code)
Class.TestingUtils.setReadOnlyField: line 15, column 1

That explains the comment I left the unit test in question all those years ago:

// history tables are not populated during Salesforce unit testing 😢

So now you’re up-to-date. If we can’t populate the History records with meaningful information, what recourse do we have access to when looking to test out this functionality? At the time, I was able to get around the read-only nature of some of the history record fields by using the Repository pattern and specifically checking that the where clause was what we needed it to be. That same solution couldn’t work for the current issue though — because our code needs to explicitly act on the OldValue and NewValue fields for each History record. Let’s dive a little deeper.

Getting Into The Thick Of It: How To Mock History Records

As a reminder — the Stub API can’t help us here. If we can’t set the fields we need on the records we expect to get back from the database, any sort of regular mocking strategy has already been derailed. Cue the spotlight, cue the entrance music — meet the DAO (Data Access Object). It’s typically used to abstract away the database from objects within a system, but we’re going to be putting a slight twist on it.

This approach does come with some caveats. For example, using our ExampleLeadHistoryBatchable, we’ll have to exercise the execute() method directly in order to test (since we won’t be able to insert the correctly-setup records at all). The test for the start() method thus has its merits; we can at the very least assert that our where clause (whatever it may be) is formatted correctly — we can also verify that the query that’s produced is valid, in the event that we’re using dynamic SOQL. That part still resembles the “solution” from years back.

Of course, there are also two styles of History records that we’ll have to consider with this approach:

• standard History objects, like LeadHistory
• custom History objects, which end in __History

Let’s dive in to our DAO, starting with custom history objects and coming back to the standard ones later on:

public virtual class FieldLevelHistory {
  public Object OldValue { get; set; }
  public Object NewValue { get; set; }
  public Datetime CreatedDate { get; set; }
  public String Field { get; set; }
  public Id ParentId { get; set; }

  public override String toString() {
    return JSON.serialize(this);
  }
}

The key to this all working is the toString() override. That’s going to allow us to directly set up all of our data in a test without running into the aforementioned Internal Salesforce Error. We can do things like:

// some test code:
static FieldLevelHistory getHistoryRecord(
  String parentId,
  Datetime createdDate,
  String fieldName,
  Object oldValue,
  Object newValue
) {
  FieldLevelHistory history = new FieldLevelHistory();
  history.CreatedDate = createdDate;
  history.Field = fieldName;
  history.ParentId = teamId;
  history.OldValue = oldValue;
  history.NewValue = newValue;
  return history;
}

We’ll have to update our production level code slightly to take in a List<Object>:

public class ExampleLeadHistoryBatchable implements Database.Batchable<SObject> {
  // ...

  public void execute(Database.BatchableContext bc, List<Object> scope) {
    // let's do a really naive implementation!
    // we can't cast directly to List<FieldLevelHistory>
    // because that will only work in a test; in prod,
    // the data will be actual history records that are returned
    List<FieldLevelHistory> historyRecords = (List<FieldLevelHistory>) JSON.deserialize(JSON.serialize(scope), List<FieldLevelHistory>.class);
  }
}

Amazing. We’ll just work with these synthetic DAO records and act on them, same as we would the underlying history records. Except, going back to our test class:

@IsTest
static void someTestConcerningTheExecuteLogic() {
  FieldLevelHistory history = new FieldLevelHistory();
  history.CreatedDate = System.now();
  history.Field = 'LastName';
  history.OldValue = 'ExampleLeadHistoryBatchable';
  history.NewValue = 'somethingElse';

  new ExampleLeadHistoryBatchable().execute(null, new List<FieldLevelHistory>{ history });

  // throws "System.JSONException: Apex Type unsupported in JSON: Object"
}

Oof. That’s the deserializer telling us (coyly) that because FieldLevelHistory has Object-type properties, we can’t directly deserialize to it. Guess I should’ve seen that coming — let’s do the next best thing by spinning up new instances for each record:

public void execute(Database.BatchableContext bc, List<Object> scope) {
  List<FieldLevelHistory> historyRecords = new List<FieldLevelHistory>();
  for (Object obj : scope) {
    // the ternary is a necessary evil since "scope" is either actual History records OR
    // an already initialized mocked version of FieldLevelHistory
    String deserialized = obj instanceof SObject ? JSON.serialize(obj) : obj.toString();
    Map<String, Object> untyped = (Map<String, Object>) JSON.deserializeUntyped(deserialized);
    FieldLevelHistory historyRecord = new FieldLevelHistory();
    historyRecord.ParentId = (String) untyped.get('ParentId');
    historyRecord.CreatedDate = (Datetime) untyped.get('CreatedDate');
    historyRecord.Field = (String) untyped.get('Field');
    historyRecord.OldValue = untyped.get('OldValue');
    historyRecord.NewValue = untyped.get('NewValue');
    historyRecords.add(historyRecord);
  }
}

Of course, it wouldn’t be the Joys Of Apex if, even at what seemed like the crowning moment of getting this working, something weird didn’t happen — running that code gives us:

// the following error is thrown on the below line
// System.TypeException: Invalid conversion from runtime type String to Datetime
historyRecord.CreatedDate = (Datetime) untyped.get('CreatedDate');

… I’m sorry. What? We’ve just initialized what is most definitely a Datetime instance in our test class:

@IsTest
static void someTestConcerningTheExecuteLogic() {
  FieldLevelHistory history = new FieldLevelHistory();
  history.CreatedDate = System.now();
  // ... etc
}

Some comparisons between what was coming out of the map as the CreatedDate property and “stringified” versions of datetimes showed a crucial difference: the T character between the Date portion of the String and the Time portion. Here’s a (really nasty) quick fix:

// ...
String deserialized = obj instanceof SObject ? JSON.serialize(obj) : obj.toString();
Map<String, Object> untyped = (Map<String, Object>) JSON.deserializeUntyped(deserialized);
FieldLevelHistory historyRecord = new FieldLevelHistory();
historyRecord.ParentId = (String) untyped.get('ParentId');
//                                   this "T" is the issue  👇
System.debug(untyped.get('CreatedDate')); // DEBUG|YYYY-MM-ddTHH:mm:ss.000+0000
System.debug(untyped.get('CreatedDate') instanceof String); // outputs: DEBUG|true
historyRecord.CreatedDate = Datetime.valueOfGmt(String.valueOf(untyped.get('CreatedDate')).replace('T', ' '));
historyRecord.Field = (String) untyped.get('Field');
historyRecord.OldValue = untyped.get('OldValue');
historyRecord.NewValue = untyped.get('NewValue');
historyRecords.add(historyRecord);

There’s more that we could do to make this “work” from the perspective of testing, but in reality we’ve reached the end of the line. Consider this example:

LeadHistory firstHistory = [
    SELECT Id, CreatedDate
    FROM LeadHistory
    LIMIT 1
];
String serialized = JSON.serialize(firstHistory);
Map<String, Object> deserialized = (Map<String, Object>) JSON.deserializeUntyped(serialized);
System.debug(deserialized);
System.debug(deserialized.get('CreatedDate') instanceof String); // outputs: DEBUG|true
System.debug(deserialized.get('CreatedDate')); // DEBUG|YYYY-MM-ddTHH:mm:ss.000+0000

So … there’s something wonky about the deserializer in Apex specifically for Datetime records that prevents us from interacting with their CreatedDate field effectively. You may not even need this property to be set; we unfortunately did for what we were working on. Returning to the subject matter at hand, we’re now not that far off from a generic solution that works for other History record types. Returning to LeadHistory for example:

public virtual class FieldLevelHistory {
  private Boolean parentHasBeenAccessed = false;

  public Object OldValue { get; set; }
  public Object NewValue { get; set; }
  public Datetime CreatedDate { get; set; }
  public String Field { get; set; }
  public Id ParentId { get; set; }
  public Id Parent {
    get {
      return this.getParentId();
    }
  }

  public override String toString() {
    return JSON.serialize(this);
  }

  protected virtual Id getParentId() {
    return this.ParentId;
  }
}

public class LeadFieldLevelHistory extends FieldLevelHistory {
  public Id LeadId { get; set; }

  protected override Id getParentId() {
    return this.LeadId;
  }
}

It’s a bit boilerplate-y — but it works. You could go a bit more crazy with validation in FieldLevelHistory — to make sure only the Parent field is used — but in general I don’t recommend taking steps like that. So long as you’re creating classes like LeadFieldLevelHistory as you need to extend the functionality to the standard History objects, simply setting the correct parent field in tests should ensure that the correct parent Id is being used at the production-code level.

Stubs like these aren’t meant to overcomplicate your life — they’re meant to improve upon on it by offering you up feature parity with how the rest of your tests work. For that reason, you can use DAO’s like these in any place where you’re not allowed to instantiate the objects you’d otherwise be acting upon out of the box. Mocking the vanilla Survey object is another place where we’ve done this, recently. This is also a strategy popular amongst ISVs where metadata isn’t necessarily present in all orgs where their package(s) are going to be installed. For instance, Apex Rollup uses a DAO for the vanilla CurrencyInfo object, as it’s only available in orgs where multi-currency is enabled.

There are, of course, notable exceptions to the way that DAOs can be used to mock out of the box objects. You may remember the issues I had with using an AggregateResult DAO in You Need A Strongly Typed Query Builder; in that case, because the fields on AggregateResult can’t be deserialized to, the Repository class needed to set up the DAO records one-by-one. In general, however — where there’s a will, there’s a way.

Wrapping Up

This article brings you through several hours worth of learnings experienced while mobbing together. To say that we started with the best of intentions as far as how we thought TDD and History records would interact with one another is true, but despite the initially slow-going, the lessons we learned here will hopefully continue to bear fruit by proving useful for you in your own Salesforce practices. Knowledge, as they say, is power. Hopefully the examples shown end up demonstrating the usefulness of the DAO pattern; for more reading on patterns you might enjoy Replace Conditional With Polymorphism (my personal favorite pattern)!

As well, I’ve since integrated this pattern into the Repository Pattern Apex DML Mocking repo — be sure to give that a look to see how FieldLevelHistoryRepo can help you out.

Thanks for reading the Joys Of Apex — till next time!