Posted: May 05, 2023

Bulk Lead Conversion

Table of Contents:

The Use Case
Introducing the Lead Converter
Back To The Abandoned Cart Handler
- Overriding Our Default Dependencies
- Rewriting Our Lead Conversion Test
Wrapping Up

There comes a time in every Salesforce implementation when the native lead conversion functionality needs to be extended. Most frequently, this is the result of needing to perform lead conversion in bulk. Lead conversion tends to be a slower process, and an unbulkified process runs the risk of timing out. This makes sense — the worst case scenario involves a contact, account, and opportunity being created, as well as the lead being updated. That’s a lot to take care of, under the hood! Let’s dive in to our use-case, and start writing tests to support a bulkified lead conversion process.

The Use Case

While this is a hypothetical situation, it has absolute roots in reality. Let’s say that we have an e-commerce frontend which allows anonymous users to add items to their cart. At the critical moment — checking out — a guest has the chance to identify themselves via email address. In the background, abandoned cart records are being periodically being created in Salesforce and stamped by device. At set intervals, the website performs a callout, grouping the device ids that have had carts abandoned and sending them to a piece of analytics middleware. The middleware compares those devices with previously logged in users and sends back a message with any users that have been so identified. It also sends back some sentiment analysis for the unidentified users based on the purchasing activity of past customers; if an unidentified guest has comparable activity to a prior converted customer, the analytics middleware identifies them as somebody highly likely to convert. The combined message is then sent to Salesforce.

Because the message contains complex information, and because there is the expectation of high volume when communicating this event to Salesforce, this is a fine use-case for a custom REST endpoint; it prevents the need to insert and then clean up a large number of records.

We can take full advantage of the ease with which a custom REST endpoint can be spun up by using the pattern outlined in Extendable APIs. In order to get a failing test going, we need only define our service class and a class to encapsulate the data we’d like to pass in:

public class ApiHandlerAbandonedCart {

  public class Request {
    public String email { get; set; }

    public String deviceId { get; set; }

    public Boolean isLikelyToConvert {
      get {
        if (isLikelyToConvert == null) {
          isLikelyToConvert = false;
        }
        return isLikelyToConvert;
      }
      set;
    }
  }
}

Which allows us to get started on our test:

@IsTest
private class ApiHandlerAbandonedCartTest {
  @IsTest
  static void convertsLeadsFromRequestIntegration() {
    // we can start with an integration test where a pre-existing lead actually exists
    Lead existingLead = new Lead(
      Company = 'required',
      Email = '' + System.now().getTime() + '@convertsLeadsFromRequestIntegration.com',
      LastName = 'Integration Test'
    );
    insert existingLead;

    ApiHandlerAbandonedCart.Request abandonedCartReq = new ApiHandlerAbandonedCart.Request();
    abandonedCartReq.email = existingLead.Email;
    sendAbandonedCartRequest(new List<ApiHandlerAbandonedCart.Request>{ abandonedCartReq });

    Assert.isTrue(
      [SELECT IsConverted FROM Lead WHERE Id = :existingLead.Id].IsConverted,
      'Lead should have been converted'
    );
  }

  private static void sendAbandonedCartRequest(List<ApiHandlerAbandonedCart.Request> requests) {
    RestRequest req = new RestRequest();
    req.requestURI = '/api/abandonedcart';
    req.requestBody = Blob.valueOf(JSON.serialize(requests));
    RestContext.request = req;
    Api.Response res = ApiService.post();
    Assert.isTrue(res.Success, res.ResponseBody);
  }
}

Note that even with an integration test, the use of the dynamic API pattern means an absolute minimum of ceremony needs to be performed. A lead is inserted, a minimal request is constructed, and an assert is made. Let’s run the test to get our failure!

System.AssertException: Assertion Failed: HTTP method not yet implemented
Class.ApiHandlerAbandonedCartTest.sendAbandonedCartRequest: line 29, column 1
Class.ApiHandlerAbandonedCartTest.convertsLeadsFromRequestIntegration: line 15, column 1

Perfect. Modifying the service class to register it properly as a service:

public class ApiHandlerAbandonedCart extends Api.Handler {

  public override Api.Response doPost(ApiRequestResolver resolver) {
    Api.Response res = new Api.Response('OK');
    res.Success = true;
    return res;
  }

  public class Request {
    // ...
  }
}

Which leads us to the failure we’re looking for, and gives us the justification for writing more production-level code:

System.AssertException: Assertion Failed: Lead should have been converted
Class.ApiHandlerAbandonedCartTest.convertsLeadsFromRequestIntegration: line 17, column 1

Introducing the Lead Converter

By creating a failing test, we’ve also created an interesting boundary, or separation of concerns. We now have two disparate responsibilities to fulfill:

the ApiHandlerAbandonedCart’s duty to match incoming requests with data already in the system
the need to convert Leads

Because the latter is a much more reusable concept, it makes sense for that functionality to be taken care of by another class and injected into the ApiHandlerAbandonedCart service. On the other hand, the behavior associated with lead conversions coming from the abandoned cart service might very well differ from the behavior we otherwise expect elsewhere in the system; for example, here it may make sense to create an opportunity and tie the opportunity to line items coming from the abandoned cart, but if the email address found is already associated with an account that already has an opportunity, it may make more sense to simply tie the abandoned cart record to the opportunity so that a sales rep can make suggestions based off of the line items that differ (if any).

This is starting to sound like behavior that’s conditionally dependent on where a request is coming from which returns a known entity — the Builder Pattern, in other words. Because lead conversion is complicated, and because there are limits involved with lead conversion that need to be addressed, having a default builder which can be optionally overridden by implementations looking to make sure of the converter will be ideal.

We’ll start by definining our LeadConverter object, and registering it in our Factory instance:

public virtual class LeadConverter {
  // we'll come back to this in a bit
  @TestVisible
  private static Integer MAX_CONVERT_BATCH_SIZE = 100;

  private final IDML dml;
  private Builder builder;

  public LeadConverter(Factory factory) {
    this.dml = factory.repoFactory.getDml();
  }

  public LeadConverter setBuilder(Builder builder) {
    this.builder = builder;
    return this;
  }

  public virtual class Builder {
    public virtual String getConvertedStatus(Lead lead) {
      return 'Closed - Converted';
    }

    public virtual Boolean getShouldNotCreateOpportunity(Lead lead) {
      return true;
    }

    public virtual Id getOwnerId(Lead lead) {
      return lead.OwnerId;
    }

    public virtual Boolean getShouldOverwriteLeadSource(Lead lead) {
      return true;
    }

    public virtual void setAccountAndContactInfo(Database.LeadConvert leadConvert, Account acc, Contact con) {
      leadConvert.setAccountId(acc.Id);
      leadConvert.setContactId(con.Id);
    }

    public virtual List<Database.LeadConvert> getLeadConverts(List<Lead> leads, IDML dml) {
      // minimized for now
    }

    protected virtual List<Database.LeadConvert> getLeadConverts(
      List<Lead> leads,
      List<Account> accounts,
      List<Contact> contacts
    ) {
      List<Database.LeadConvert> leadConverts = new List<Database.LeadConvert>();
      for (Integer i = 0; i < accounts.size(); i++) {
        Lead lead = leads[i];
        Database.LeadConvert leadConvert = new Database.LeadConvert();
        leadConvert.setLeadId(lead.Id);
        leadConvert.setConvertedStatus(this.getConvertedStatus(lead));
        leadConvert.setDoNotCreateOpportunity(this.getShouldNotCreateOpportunity(lead));
        leadConvert.setOwnerId(this.getOwnerId(lead));
        this.setAccountAndContactInfo(leadConvert, accounts[i], contacts[i]);
        leadConvert.setOverwriteLeadSource(this.getShouldOverwriteLeadSource(lead));
        leadConverts.add(leadConvert);
      }
      return leadConverts;
    }
  }

  public virtual List<Database.LeadConvertResult> convertLeads(List<Lead> leads) {
    if (this.builder == null) {
      this.builder = new Builder();
    }
    List<Database.LeadConvertResult> leadConvertResults = new List<Database.LeadConvertResult>();

    List<Lead> overflowLeads = new List<Lead>();

    // you can only convert 100 leads at a time
    while (leads.size() > MAX_CONVERT_BATCH_SIZE) {
      overflowLeads.add(leads.remove(0));
    }
    List<Database.LeadConvert> leadConverts = this.builder.getLeadConverts(leads, this.dml);
    leadConvertResults.addAll(Database.convertLead(leadConverts));

    if (overflowLeads.isEmpty() == false) {
      leadConvertResults.addAll(this.convertLeads(overflowLeads));
    }
    return leadConvertResults;
  }
}

And then in the factory:

public virtual LeadConverter getLeadConverter() {
  return new LeadConverter(this);
}

This is pretty nice. Consumers of LeadConverter can pass in a LeadConverter.Builder of their own, but there’s also a default builder ready to assist for simple lead conversion use-cases. In this case, the builder makes a few assumptions (which can also be overridden), like the getLeadConverts(List<Lead> leads, List<Account> accounts, List<Contact> contacts) method assuming that all of the collections are of the same size. Because this is a protected virtual method, consumers can also override that behavior with little in the way of ceremony.

I say it’s “pretty nice” because the LeadConverter is fulfilling one thing above all else — it’s hiding almost all of the details about lead conversion from the rest of the system. Yes, you might need to know a little bit about the Database.LeadConvert and Database.LeadConvertResult classes, but a consumer gets tons of flexibility without necessarily needing to know the rest of the details, although I’ve chosen with getShouldNotCreateOpportunity to preserve the completely awkward naming paradigm of the underlying structure — your mileage may (and possibly even should!) vary as far as naming goes.

We could proceed, from here, in a variety of ways; for now, let’s head back to ApiHandlerAbandonedCart.

Back To The Abandoned Cart Handler

We still have our failing test, and thus our mandate to proceed further with the production-level code:

public class ApiHandlerAbandonedCart extends Api.Handler {
  private final LeadConverter converter;
  private final IRepository leadRepo;

  public ApiHandlerAbandonedCart() {
    this(Factory.getFactory());
  }

  public ApiHandlerAbandonedCart(Factory factory) {
    this.converter = factory.getLeadConverter();
    this.leadRepo = factory.repoFactory.getLeadRepo();
  }

  public override Api.Response doPost(ApiRequestResolver resolver) {
    String responseMessage = 'Successfully received abandoned cart request';
    Boolean isSuccess = true;

    try {
      this.convertLeads((List<Request>) JSON.deserialize(resolver.RequestBody, List<Request>.class));
    } catch (Exception ex) {
      isSuccess = false;
      responseMessage = ex.getMessage() + '\n' + ex.getStackTraceString();
    }
    Api.Response res = new Api.Response(responseMessage);
    res.Success = isSuccess;
    return res;
  }

  private void convertLeads(List<Request> requests) {
    Set<String> emails = new Set<String>();
    for (Request req : requests) {
      emails.add(req.email);
    }
    List<Lead> matchingLeads = this.leadRepo.get(Query.equals(Lead.Email, emails));
    this.converter.convertLeads(matchingLeads);
  }

  public class Request {
    public String email { get; set; }
    // etc ...
  }
}

Which leads to:

=== Test Results
TEST NAME                                                        OUTCOME  MESSAGE  RUNTIME (MS)
───────────────────────────────────────────────────────────────  ───────  ───────  ────────────
ApiHandlerAbandonedCartTest.convertsLeadsFromRequestIntegration  Pass              4230 👀

A couple things to note here:

we have to use two constructors since the dynamic API pattern requires a zero-arg constructor and this is how we inject the factory
you probably wouldn’t actually want to send the stack trace to an external service. Remember to keep the principle of least privilege in mind when designing services; don’t lead data/your schema. The try/catch should be suggestive of doing something actually useful — like logging! — rather than modifying the responseMessage
there’s very little code that’s been added. Yes, we’ve talked about more advanced use-cases for this rest resource, but to simply convert leads using a default lead conversion strategy? That was easy!

But we’ve got a bit more work to do here. For one, we’re integration testing lead conversion and that seems a shame. It’s an unfortunately CPU intensive process, and as such that sort of behavior should really live in its own test class. That would allow our ApiHandlerAbandonedCartTest to get quite a bit faster; after all, it doesn’t really care about what happens during lead conversion — it really just wants to pass that responsibility along to its dependency.

Overriding Our Default Dependencies

If you’ve read The Factory post, what comes next will probably be no surprise. Let’s start with the minimal changes to the test:

@IsTest
private class ApiHandlerAbandonedCartTest {
  @IsTest
  static void convertsLeadsFromRequestIntegration() {
    Factory.factory = new MockFactory.LeadConversion();
    // now it's a unit test, no DML required
    Lead existingLead = new Lead(
      Company = 'E-comm',
      Email = '' + System.now().getTime() + '@convertsLeadsFromRequestIntegration.com',
      LastName = 'Integration Test'
    );
    RepoFactoryMock.QueryResults.add(existingLead);

    ApiHandlerAbandonedCart.Request abandonedCartReq = new ApiHandlerAbandonedCart.Request();
    abandonedCartReq.email = existingLead.Email;
    sendAbandonedCartRequest(new List<ApiHandlerAbandonedCart.Request>{ abandonedCartReq });

    Assert.areEqual(Query.equals(Lead.Email, new Set<String>{ existingLead.Email }), RepoFactoryMock.QueriesMade[0]);
    Assert.isTrue(((Lead) DMLMock.Updated.Leads.firstOrDefault).IsConverted, 'Lead should have been converted');
  }
  // ... etc
}

I’m going a bit out of order here — you haven’t seen MockFactory yet — but there are a couple of things I’d like to review before we get there:

we’re overriding the default Factory. This can only be done within tests
by using RepoFactoryMock.QueryResults, we’re indicating to the overall framework that if there’s an IRepository instance for Leads, it can now be overridden in RepoFactoryMock in order to start mocking query results
while this is optional, the repository pattern framework allows for that first assert - namely, validating that the queries themselves are being formed correctly
we no longer need to query for the updated lead — again, we can reach into DMLMock and its helpers to validate the rest of our code

OK, review time over. Let’s register our lead repository as mockable:

// in RepoFactoryMock
public override IHistoryRepository getLeadRepo() {
  return this.getRepoFromSObjectType(Lead.SObjectType, super.getLeadRepo());
}

And then, finally, the MockFactory implementation. Note that this class as an outer class assumes code reuse; if it was only ever going to be used in ApiHandlerAbandonedCartTest, it could be included as a private inner class within that test class:

@IsTest
public without sharing class MockFactory {
  private MockFactory() {
    //@IsTest classes cannot be abstract :(
  }

  public class LeadConversion extends Factory {
    public LeadConversion() {
      super();
    }

    public override LeadConverter getLeadConverter() {
      return new LeadConverterMock();
    }
  }

  public class LeadConverterMock extends LeadConverter {
    public LeadConverterMock() {
      super(Factory.getFactory().withMocks);
    }

    public override List<Database.LeadConvertResult> convertLeads(List<Lead> leads) {
      List<Database.LeadConvertResult> leadConvertResults = new List<Database.LeadConvertResult>();
      for (Lead lead : leads) {
        leadConvertResults.add(this.convert(lead));
      }
      return leadConvertResults;
    }

    private Database.LeadConvertResult convert(Lead lead) {
      Id oppId = TestingUtils.generateId(Opportunity.SObjectType);
      Id accountId = TestingUtils.generateId(Account.SObjectType);
      Id contactId = TestingUtils.generateId(Contact.SObjectType);
      this.updateLead(lead, accountId, contactId);
      // in a second, you'll also see a reusable version of this method
      // called TestingUtils.setReadOnlyField
      Database.LeadConvertResult res = (Database.LeadConvertResult) JSON.deserialize(
        JSON.serialize(
          new Map<String, Object>{
            'accountid' => accountId,
            'contactid' => contactId,
            'leadid' => lead.Id,
            'opportunityid' => oppId
          }
        ),
        Database.LeadConvertResult.class
      );
      return res;
    }

    private void updateLead(Lead lead, Id accountId, Id contactId) {
      Contact con = new Contact(
        Id = contactId,
        FirstName = lead.FirstName,
        LastName = lead.LastName,
        Email = lead.Email,
        Phone = lead.Phone,
        AccountId = accountId,
        LeadSource = lead.LeadSource
      );

      Map<String, Object> readOnlyFields = new Map<String, Object>{
        'ConvertedContactId' => con.Id,
        'ConvertedAccountId' => con.AccountId,
        'IsConverted' => true
      };
      // here it is!
      lead = (Lead) TestingUtils.setReadOnlyField(lead, readOnlyFields);
      DMLMock.UpdatedRecords.add(lead);
      DMLMock.UpsertedRecords.add(con);
    }
  }
}

Yikes. Well, suffice it to say, there’s more there than would otherwise be necessary, but Database.LeadConvertResult is one of those annoying standard objects that doesn’t let you set fields on it after it’s been constructed, and the constructor isn’t even visible:

Type cannot be constructed: Database.LeadConvertResult

Re-running our test, though, we get the classic 90+% test time improvement — from 4230ms to 302ms!

Rewriting Our Lead Conversion Test

At this point, we’ve successfully isolated the lead conversion code into LeadConverter — but since ApiHandlerAbandonedCart is now unit testing its own lead conversion dependency, it’s time to restore the lead conversion integration test.

We’ll make LeadConverterTest to handle all of our lead conversion testing responsibilities. Note, again, how there’s nothing too crazy going on here — the filling out of required fields, the requisite DML, and then we’re free to test our basic lead conversion scenario.

@IsTest
private class LeadConverterTest {
  @TestSetup
  static void setup() {
    Lead first = new Lead(
      Company = 'required',
      Email = 'one@' + LeadConverterTest.class.getName() + '.com',
      LastName = 'Integration Test One'
    );
    Lead second = new Lead(
      Company = 'required',
      Email = 'two@' + LeadConverterTest.class.getName() + '.com',
      LastName = 'Integration Test One'
    );
    insert new List<Lead>{ first, second };
  }

  @IsTest
  static void properlyConvertsLeadsByChunk() {
    LeadConverter.MAX_CONVERT_BATCH_SIZE = 1;

    Factory.getFactory().getLeadConverter().convertLeads(getLeadsForConversion());

    Assert.areEqual(0, [SELECT COUNT() FROM Lead WHERE IsConverted = FALSE]);
    Assert.areEqual(0, [SELECT COUNT() FROM Opportunity]);
  }
}

We can also go on to test the responsibility of things like custom builders for setting up Accounts, Contacts, and Opportunities:

// in LeadConverterTest
@IsTest
static void itHandlesCustomConversionBuilders() {
  Account exampleExistingAccount = new Account(Name = 'Example Existing');
  insert exampleExistingAccount;

  insert new Contact(LastName = 'Belongs to above account', AccountId = exampleExistingAccount.Id);

  Factory.getFactory().getLeadConverter().setBuilder(new ExampleBuilder()).convertLeads(getLeadsForConversion());

  Assert.areEqual(1, [SELECT COUNT() FROM Opportunity]);
  Assert.areEqual(exampleExistingAccount.Id, [SELECT ConvertedAccountId FROM Lead LIMIT 1].ConvertedAccountId);
}

private static List<Lead> getLeadsForConversion() {
  return Factory.getFactory().repoFactory.getLeadRepo().getAll();
}

private class ExampleBuilder extends LeadConverter.Builder {
  public override List<Database.LeadConvert> getLeadConverts(List<Lead> leads, IDML dml) {
    List<Contact> existingContacts = [SELECT Account.Id, Id FROM Contact];
    return this.getLeadConverts(leads, new List<Account>{ existingContacts.get(0).Account }, existingContacts);
  }

  public override Boolean getShouldNotCreateOpportunity(Lead lead) {
    return false;
  }
}

As the responsibility of the LeadConverter — or its various builders — grow, more integration-style tests can be added here. I think this example test builder go a long way in explaining the power of this pattern — let me know if there’s more you’d like to see!

Our abandoned cart API service can grow in complexity, as well — we’ve already talked about how it might match existing Leads / Contacts based on attributed device Ids, but that complexity doesn’t bleed over into lead conversion. As far as ApiHandlerAbandonedCart is concerned, these are all implementation details that its dependencies can handle for it:

updating records
retrieving records
converting leads

Wrapping Up

If you’d like to take a look through the code mentioned in this post, you can find it here.

Lead conversion is a complicated and CPU-intensive process in Salesforce. Confining the responsibility to a single class that takes care of that responsibility for you serves to allow the other classes in your codebase to focus on their responsibilities in a much cleaner fashion; because lead conversion in particular is slow, you’ll realize significant performance gains by mocking your lead converter across the rest of your codebase. As well, this allows you to bulkify what might otherwise be a one-off process in a consistent fashion.

I like the example of how lead conversion can fit into a much larger set of processes specifically because I believe it’s a good example of an end-to-end example of how dependency injection can best be utilized, either in a greenfield project or in an existing codebase where you’re looking to make improvements. Hopefully you enjoyed seeing how to bulkify lead conversion while also isolating it as a process — thanks for reading the Joys Of Apex, and thanks as always to my Patreon supporters, with a special thanks to Henry Vu for his consistent support!

In the past three years, hundreds of thousands of you have come to read & enjoy the Joys Of Apex. Over that time period, I've remained staunchly opposed to advertising on the site, but I've made a Patreon account in the event that you'd like to show your support there. Know that the content here will always remain free. Thanks again for reading — see you next time!