What's New With Nebula Logger

Join me in again welcoming Jonathan Gillespie to the Joys Of Apex — today, we get to hear about the many exciting updates to Nebula Logger. Thank you, Jonathan, for contributing another great post!

It’s been about 8 months since I wrote my first guest blog post for Joys of Apex covering Nebula Logger. Since that time, so much has changed:

• I now work at Salesforce with James (the author & creator of Joys of Apex) as a software engineer! This job is a blast: we work on an amazing team, and it’s so much fun to be working with James again!
• With 130+ stars (and counting!), Nebula Logger is now the most starred repository on GitHub for Salesforce logging! There has been so much feedback over the last several months from GitHub, SFXD, and more — I really appreciate all of the feedback that the community has provided, it has been (and continues to be) amazing to see how much this project has grown, and how it’s helping other Salesforce professionals monitor & support their orgs.
• A ton of new features have been added to Nebula Logger. We’ll cover some of the big changes from the last 8 months in this article — make sure to check out Nebula Logger's Releases page to see all of the latest changes.

Let’s dive more into the highlights of what’s new with Nebula Logger!

1. Improved Documentation

For years, most of Nebula Logger’s documentation has been maintained in the README file. As Nebula Logger has become more popular, there have also been more questions around how it works overall, how specific features have been implemented, and general questions about the project. Maintaining everything in README has become a bit unmanageable — to help improve the project’s documentation, several changes have been put into place:

• New wiki — this is still a work in progress, but going forward, all documentation will be added to the wiki site, instead of README. Once I’m sure all the relevant content has been added to the wiki, I’ll cleanup & simplify the contents of README.
• New architectural diagram — this (massive) diagram gives an overview of how all of Nebula Logger’s metadata is designed and works together to provide a robust, comprehensive logging tool for Salesforce.
• Enabled GitHub discussions — discussions are now being used for answering questions, annoucing new releases, and discussing possible new enhancements/bugfixes. When applicable, new issues are created from these discussions to track the progress of any new enhancements or bugfixes.
• More ApexDocs — @dancinllama did some great work to add more JavaDoc/ApexDoc—style comments to all of the Apex classes. All of the ApexDoc comments are automatically published to the GitHub Pages site.

2. Development Focus Has Shifted To The Unlocked Package

Originally, my goal for Nebula Logger was to release it as a managed package — the original versions of the managed package was released in v4.0.0, and is still being maintained. However, in v4.4.1, an unlocked package was also released. Since then, the focus has shifted from the managed package to the unlocked package due to several limitations with how managed packages behave compared to unlocked packages. There’s a page within the wiki that provides more context, but essentially, unlocked packages are a better fit for Nebula Logger for several reasons:

• Maintenance is easier: with unlocked packages, it’s much easier to add/remove/rename metadata. Deprecating metadata & Apex classes/methods has several limitations when working with managed packages.
• Unobfuscated code is better for a logging tool: when using the managed package version of Nebula Logger, you cannot see or modify the underlying code. This means it’s much more difficult for you to customize the behavior, and it’s much more difficult to audit what code is running in your org. By using the unlocked package, you can see and modify the underlying code.
• Some features only work in the unlocked package: certain features within Nebula Logger, such as automatically logging a stack trace, do not work within the managed package — the metadata is the same as the unlocked package, but due to limitations in how managed packages work, the unlocked package provides a much better experience.

The unlocked package will continue to receive updates “right away” — as soon as a pull request is merged in GitHub, a corresponding release is published for the unlocked package. For the managed package, it now follows a release cycle that mimics Salesforce’s own release cycle — there are 3 releases planned for (calendar year) 2022. Each release is being tracked in GitHub, using Milestones:

1. Version 4.7.0 -- Spring '22 Release
2. Version 4.8.0 -- Summer '22 Release
3. Version 4.9.0 -- Winter '23 Release

I’m planning to follow this release cycle at least until the end of 2022 — I’ve been considering deprecating the managed package, and only supporting the unlocked package going forward (starting in 2023). If you have thoughts/concerns around this idea, let me know!

3. Improved Tagging System

One of the oldest (and one of my own favorite) features of Nebula Logger is being able to tag LogEntry__c records. Originally released in v0.9.0-beta, the tagging system has historically used the standard objects Topic and TopicAssignment for dynamically labeling each LogEntry__c. This has worked well overall — but using Salesforce Topics does have some limitations:

• Custom fields cannot be added to Topic or TopicAssignment objects
• Support for reporting on Topic and TopicAssignment records is very limited
• The underlying code in Nebula Logger does not work (for whatever reason) when using a managed package, making the feature effectively unusable

To overcome some of these limitations, and to add some new tagging functionality, v4.6.0 introduced new custom objects to provide a better tagging system out of the box:

• LoggerTag__c — this represents your actual tag, such as “my important tag”
• LogEntryTag__c — this is a junction object between LoggerTag__c and LogEntry__c, allowing you to associate multiple tags to a single LogEntry__c record, as well as the ability to associate multiple LogEntry__c records to the same LoggerTag__c record

These new custom objects are now used as the default tagging system — for orgs using the unlocked package, you can still revert to using Salesforce Topics if you prefer, using the custom metadata type LoggerParameter__mdt.UseTopicsForTags.

In addition, this release also included a new custom metadata type called LogEntryTagRule__mdt — this allows you to define rules that automatically append additional tags to any LogEntry__c records that meet criteria that you specify.

4. Reintroduced Logger for Lightning components (LWC & Aura)

Originally released in v0.12.0-beta, Nebula Logger had the ability to log via aura components. Since Salesforce was shifting focus from aura components to lightning web components (lwc), I had removed the original approach altogether, with plans to eventually re-add the functionality using lightning web components (lwc). As of v4.6.4, a new lwc (cleverly called logger) has been added to reintroduce the ability to log from both lwc & aura components.

To add Nebula Logger to your own lightning web component, first embed logger into your component’s markup:

<template>
  <c--logger></c--logger>

  <div>My component</div>
</template>

From your component’s controller, you can then call logger using syntax similar to Logger in Apex:

import { LightningElement } from "lwc";

export default class LoggerDemo extends LightningElement {
  logSomeStuff() {
    const logger = this.template.querySelector("c--logger");

    logger.error("Hello, world!").addTag("some important tag");
    logger.warn("Hello, world!");
    logger.info("Hello, world!");
    logger.debug("Hello, world!");
    logger.fine("Hello, world!");
    logger.finer("Hello, world!");
    logger.finest("Hello, world!");

    logger.saveLog();
  }
}

Similarly, you can use logger for logging within your aura component by first adding it to the markup:

<aura:component implements="force:appHostable">
  <c:logger aura:id="logger" />

  <div>My component</div>
</aura:component>

And then have your aura component’s controller or helper JS files add log entries & save the log:

({
  logSomeStuff: function (component, event, helper) {
    const logger = component.find("logger");

    logger.error("Hello, world!").addTag("some important tag");
    logger.warn("Hello, world!");
    logger.info("Hello, world!");
    logger.debug("Hello, world!");
    logger.fine("Hello, world!");
    logger.finer("Hello, world!");
    logger.finest("Hello, world!");

    logger.saveLog();
  },
});

5. New ‘Log Entry Event Stream’ Tab for Real—Time Monitoring & Filtering of LogEntryEvent__e Platform Events

Since it was first released on GitHub, Nebula Logger has leveraged platform events to ensure that logs are saved, even if an exception occurs (see my original Joys of Apex blog post for more details on why platform events are critical to making this work). But despite their importance to the overall architecture, they were previously only used as a mechanism for creating Log__c and LogEntry__c records — the LogEntryEvent__e platform events were never surfaced directly within Nebula Logger. If you previously wanted to monitor the platform events, you would had to have used an external tool like Workbench (yuck) to view the events.

In v4.6.8, a new component/tab called the “Log Entry Event Stream” was added to the Logger Console app. The new lightning web component logEntryEventStream gives you the ability to monitor & filter LogEntryEvent__e platform events in real-time. This provides a much better way to monitor your logs when trying to deal with real-time issues & support. Currently, it supports filtering on several LogEntryEvent__e fields:

• Logging Level — dropdown, includes all supported LoggingLevel values
• Origin Type — dropdown, includes ‘Apex’, ‘Component’, and ‘Flow’
• Origin Location — textbox, supports partial matches, such as MyClass.someMethod, MyClass, someMethod, etc.
• Logged By — textbox, supports partial matches on username
• Message — textarea, supports partial matches on the log entry’s message

Several enhancements are being planned for this feature via issue #212 — add a comment if you have any feedback for other possible enhancements.

(Side note from James — this component has been one of my favorite of the new Nebula Logger additions. It’s really amazing in an org with a lot of logging setup to watch the events as they’re created!)

6. Log Scenario Field & Rules

One of the newest features of Nebula Logger, this idea came from @fentes and @jverelst and was partially introduced in v4.6.10, with additional functionality added in v4.6.12. With the context that there is 1 Log__c record per transaction, the new “log scenario” feature provides 3 pieces of functionality:

1. Populate the new field Log__c.Scenario__c. This text field gives you the ability to name the transaction, providing a way to report on different scenarios within your org. This can be done when logging via Apex, Flow, and Lightning Components.
2. Use rule-based config to override the user’s logging level for a scenario. Using the new custom metadata type LogScenarioRule__mdt, you can specify a logging level — any time the specified scenario is used, Logger will use the logging level override for the transaction (instead of using the value configured via the custom hierarchy settings LoggerSettings__c). For example, if a user has the logging level WARN configured via LoggerSettings__c, but there is a matching LogScenarioRule__mdt rule configured with logging level DEBUG for the scenario, then the transaction will log/save any log entries with logging level == DEBUG for the scenario.
3. Use rule-based config to override the log retention date for a scenario. When configuring LogScenarioRule__mdt, you can also specify the number of days to retain any logs with the specified scenario. This field also overrides the configuration in LoggerSettings__c — if a user has a default log retention of 14 days in LoggerSettings__c, but a particular scenario has been configured with a different retention (for example, 90 days), then any logs with the specified scenario will use the retention date specified in LogScenarioRule__mdt.

7. Data Masking Rules

Generally speaking, logging tools should try to log as much data as possible so that the data can be used for troubleshooting & monitoring. However, there is some data — such as social security numbers, credit card numbers, and so on — that is highly sensitive data, and you may not want to store it within Nebula Logger. To help manage this type of sensitive data, v4.6.7 introduced the new custom metadata type LogEntryDataMaskRule__mdt. This custom metadata type uses regex—based rules to automatically mask sensitive data in the Message__c and RecordJson__c fields.

Pre-built rules are included for social security numbers, Visa credit card numbers, and Mastercard credit card numbers — additional rules can be configured by creating your own LogEntryDataMaskRule__mdt records.

Logger.error('Here is my fake Visa credit card 4000--1111--2222--0004, please don\'t steal it')
    .addTag('data masking rule')
    .addTag('credit card masking');
Logger.warn('Here is my fake Mastercard credit card 5000--1111--2222--0005, please don\'t steal it')
    .addTag('data masking rule')
    .addTag('credit card masking');
Logger.info('In case you want to steal my identity, my fake social is 400--11--9999, thanks', currentUser)
    .addTag('data masking rule')
    .addTag('social security number masking');
    .addTag('some other tag');
Logger.saveLog();

Running the script above results in 1 Log__c record with 3 LogEntry__c records — each log entry’s message has data masking rules applied to obfuscate sensitive data. This prevents accidentally exposing sensitive data within your logs — data masking rules are applied to the LogEntryEvent__e platform events (which are then transformed/stored in Log__c and LogEntry__c), so you sensitive data is never exposed within Nebula Logger’s objects.

8. Plugin Framework (beta)

If you want to add your own automation to Nebula Logger, you can leverage Apex or Flow to define your own “plugins” — the logger system will then automatically run the plugins after each trigger event (BEFORE_INSERT, BEFORE_UPDATE, AFTER_INSERT, AFTER_UPDATE, and so on) for every custom object (Log__c, LogEntry__c, LogEntryTag__c, and LoggerTag__c objects). This framework makes it easy to build your own plugins, or deploy/install others’ prebuilt packages, without having to modify the logging system directly.

This feature is probably the one I’m most excited about — it is a really powerful way to …

• Provide a way to release advanced features as optional add—on packages — not every feature is useful in all orgs, so being able to create/distribute/install optional plugins really opens up a lot of possibilities for new features that may not make sense to be part of the “core” application. For example, there is now a Slack plugin — not every org uses Slack, so I didn’t want to bake it into the core application. By making it a plugin, only orgs that actually use Slack will have any of the Slack-related configurations.
• Provide a way to easily add your own functionality — I want Nebula Logger to be as robust & feature—rich as possible so it can be used by Salesforce professionals worldwide. However, there may always be additional functionality that certain orgs want to implement. By creating your own plugins for your org, you can customize Nebula Logger & still be able to upgrade to future versions of the package.

Note: the plugin framework is currently in beta — there are a few upcoming changes planned to the plugin framework, which will involve some changes to how existing plugins work. The plan is for plugin framework to be stabilized by either December 2021 or January 2022. There are several official plugins being developed — I’m planning to release these plugins over the next few months.

• Available Beta Plugins
  • Slack Integration — Automatically post critical logs to your Slack channel — additional enhancements documented in issues #177 will add more configurations for the plugin to provide more control over the integration.
  • Logger Admin Dashboard — Includes a new Logger Admin dynamic dashboard, as well as several new reports, to help monitor logs in your org.
• In-Progress Plugins
  • Log Retention Rules — Adds the ability to create & deploy advanced, configurable rules for setting the retention date of Log__c records, using custom metadata types. This will be useful for orgs that have advanced requirements for log retention.
  • Filtered Log Entry Events — Used to create & subscribe to a filtered subset of important LogEntryEvent__e records, using configurable custom metadata types. This will be useful for orgs with external systems that subscribe to LogEntryEvent__e records
• Planned Plugins
  • Big Object Archiving — Will add the ability to archive Log__c and LogEntry__c data in the big object LogEntryArchive__b. This will be useful for orgs that want to retain logging data within Salesforce & have unused BigObject storage space.
  • Loggly Integration — Will add the ability to send log data to Loggly. This will be useful for orgs that already use Loggly as their centralized logging system.

Wrap-Up

Hopefully you’ve found some Nebula Logger’s new features to be helpful! Thanks again for everyone who has contributed to Nebula Logger over the last 8 months — everything from submitting code changes, helping with documentation, testing new beta features, and sharing thoughts on new features to add. Thanks again to James Simone for the opportunity to publish on the Joys of Apex!

Notes from James: make sure you don’t miss out on A Year In Open Source, which has some fun tidbits from Jonathan and myself on the past year in open source for Salesforce (and, in particular, how much Nebula Logger has grown in usage over the past 12 months)!