A TinyMCE plugin for integrating Highlight.js

Introduction

In this post I describe how I integrated the Highlight.js syntax highlighting tool into TinyMCE v5.10.2 for the sake of highlighting code while writing blog posts. So one gets the button saying “HL” as shown here:

I’ve taken a somewhat different approach than usual, mainly in three aspects:

• TinyMCE has a capability to insert code samples, and that includes syntax highlighting. Not good enough though, mainly because formatting can’t be added on top. Besides, the variety of languages is very narrow compared with highlight.js.
• The syntax highlighting is a manual formatting operation, exactly like using “bold” or “italics”: The user selects the code to be highlighted, and presses a button at the toolbar.
• The language is chosen manually.

So first, why not keep the highlighting script running periodically? Well, to begin with, it’s not an advantage when writing a blog post. I never write any code in my posts directly, but rather copy-paste it into the editor window from some other source. After all, it’s supposed to be code that runs, so I’ve obviously tested it somehow. Hence it’s already written somewhere.

On the other hand, continuous highlighting is an obstacle for writing blog posts: I often mark important parts in bold and red, and less important parts in grey. So the formatting made for syntax clarification is often just a starter. Letting the highlighter run uncontrolled would most likely mess up the manually added formatting.

As for selecting the language manually: Obviously, I know which language the code is written in, so why let an algorithm guess it? It may very well miss, in particular when the snippets are short — and in a blog post they often are.

Also, it’s not uncommon that a chunk of code contains more than one language: JavaScript code may contain snippets in HTML, and Perl code may contain snippets in Verilog. So it makes perfect sense to copy-paste the piece of code into the editor’s window, and then select each code segment and highlight it, each with its own language. And then I’m free to go in with colors to mark the important parts.

I’m not saying everyone is wrong. Automatic and periodic highlighting is great for websites with “try it out” windows for experimenting with HTML, JavaScript, CSS etc. This is not my case however.

And one more thing: I’ve deliberately chosen not to turn off spell checking, mainly because it catches spelling mistakes in comments.

This post is organized as follows: First a few words about getting started with Highlight.js. Then an example of integrating Highlight.js for use with automatic language detection (as this covers the main principles). And finally, setting it up to select the language manually.

The code examples in this post were, of course, highlighted with the plugin itself, so it has already one very satisfied user.

Getting started with Highlight.js

As mentioned, the chosen syntax highlighter in JavaScript was Highlight.js v11.3.1. The default set of languages is however web development oriented, and I need a much wider set. No problem, just go to the download page and tick whatever language is needed, and then download a custom zip file.

The zip file is however slightly misleading: Even though it contains quite a few files, the only ones that are needed are highlight.min.js at the zip’s root directory, and one of the CSS files in styles/. The highlight.js file should not be used, as it’s probably the core script without language support. So it does nothing good.

This way or another, there’s nothing to build from this bundle.

Anyhow, I’d mention that adding and removing languages can be done by editing highlight.min.js directly. It actually looks like the download site’s mechanism for supporting languages merely consists appending language related snippets from the language/ subdirectory to some core functions file.

Just look for comments saying something like

/*! `makefile` grammar compiled for Highlight.js 11.3.1 */

The part until a similar comment is the code that installs support for the said language. So cut and paste as necessary.

Note that the language/ directory contains all languages, not just those requested. So save the zip file in a safe place for the possibility to add a language later on without worrying about version compatibility.

There was only one little problem: As I selected which languages to support, I pretty much clicked on anything that I thought might come handy, but it turns out that the module for Python caused Firefox to fail loading highlight.min.js with

SyntaxError: invalid identity escape in regular expression

in the console. Chrome had no such problem. I’ll leave it open if it was a coincidence that Python was somehow related to a failure with a cryptic error message. As the image above implies, I opted out Python.

To give it a test run, I uncompressed the zip file into a directory named highlight/ relative to the file loaded into the browser, and then this correctly highlighted Verilog code:

<link rel="stylesheet" href="highlight/styles/stackoverflow-dark.min.css">
<script src="highlight/highlight.min.js"></script>
<script>hljs.highlightAll();</script>

<pre><code class="language-verilog">
... Verilog stuff ...
</code></pre>

This is the simplest use case: Let Highlight.js look up <pre><code> segments and highlight them automatically. But then, I chose the language manually by virtue of using the language-verilog class (this is not necessary, though).

Simple highlight TinyMCE plugin

So this is the code for the highlighting plugin with automatic language selection. Note that this is just code that is loaded along with the HTML, the good old way, even though it could be used in the common plugin framework (I guess, haven’t tried it).

(function () {
    'use strict';
    var highlighter = function() {
      // First clean up possible leftovers
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
	editor.dom.remove(el, true);
      });
      editor.formatter.apply('highlight_wrapper');
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
        let text = el.innerText;
	let hl = hljs.highlightAuto(text);
	editor.dom.setHTML(el, hl.value);
	editor.dom.remove(el, true);
      });
      editor.undoManager.add(); // Make highlighting operation undoable
    }

    tinymce.PluginManager.add('highlight', function (editor) {
      editor.ui.registry.addIcon('highlight', '<svg width="24" height="24" viewBox="0 0 256 256"><path d="M 57.00,62.00 C 57.00,62.00 41.58,147.00 41.58,147.00 41.58,147.00 34.00,189.00 34.00,189.00 34.00,189.00 50.00,189.00 50.00,189.00 57.50,188.87 56.70,186.77 58.75,175.00 58.75,175.00 63.58,148.00 63.58,148.00 64.08,145.21 64.56,139.55 66.58,137.60 68.58,135.67 72.40,136.01 75.00,136.00 75.00,136.00 111.00,136.00 111.00,136.00 111.00,136.00 102.00,189.00 102.00,189.00 102.00,189.00 124.00,189.00 124.00,189.00 124.00,189.00 133.58,135.00 133.58,135.00 133.58,135.00 141.58,91.00 141.58,91.00 141.58,91.00 147.00,62.00 147.00,62.00 147.00,62.00 131.00,62.00 131.00,62.00 123.09,62.15 124.37,63.75 122.00,76.00 122.00,76.00 117.42,101.00 117.42,101.00 116.92,103.79 116.44,109.45 114.42,111.40 112.42,113.33 108.60,112.99 106.00,113.00 106.00,113.00 70.00,113.00 70.00,113.00 70.00,113.00 79.00,62.00 79.00,62.00 79.00,62.00 57.00,62.00 57.00,62.00 Z M 174.00,62.00 C 174.00,62.00 158.58,147.00 158.58,147.00 158.58,147.00 151.00,189.00 151.00,189.00 151.00,189.00 204.00,189.00 204.00,189.00 204.00,189.00 210.41,187.98 210.41,187.98 210.41,187.98 212.61,182.00 212.61,182.00 212.61,182.00 215.00,167.00 215.00,167.00 215.00,167.00 177.00,167.00 177.00,167.00 177.00,167.00 189.42,99.00 189.42,99.00 189.42,99.00 196.00,62.00 196.00,62.00 196.00,62.00 174.00,62.00 174.00,62.00 Z" /></svg>');

   editor.ui.registry.addButton('hljs', {
     icon: 'highlight',
     tooltip: 'Syntax highlight selection',
     onAction: highlighter,
   });
 });
}());

It relies on the registration of a special class, named highlight_tmp, with this code snippet:

function mce_post_init(editor) {
  editor.formatter.register('highlight_wrapper',
    { inline: 'span', classes: 'highlight_tmp', exact: true } );
}

A few words about the “exact” property. This is required to ensure that a separate <span> tag is created, and that it isn’t merged with another <span> tag that happens to cover the same segment. This can be the case in particular if there’s a single <span> that covers a block element end to end. The “exact” property ensures that if this happens, applying the format creates a <span> within each block, and hence there are two nested <span>’s that overlap exactly.

Merging must be prevented, because the highlight_wrapper format is used merely to isolate the selection, and the <span> element carrying it is removed after the highlighting operation completes. If it has been merged with another <span> element, the formatting the other <span> carried is removed as well.

The call to editor.formatter.register() has to be run after the editor has been fully initialized. Some do this by virtue of a timeout callback, but the clean way for doing this is within a function that is called explicitly when the editor is up and running, mce_post_init() in this case.

This function is declared in the configuration of the editor, so let’s discuss it now (only the parts relevant to highlighting are shown):

tinymce.init({
  init_instance_callback : "mce_post_init",
  content_css: [ 'css/highlight.css', '/js/skins/content/default/content.css'],
  plugins: 'highlight [ ... ]'
  toolbar: '[ ... ] hljs [ ... ]',
  [ ... ]
});

For the plugin to work, it must be listed in the plugins property as shown above. Note that it’s not loaded from a separate file, since it has already been registered by the time the editor is looking for it.

Also, a button needs to be placed in the toolbar, by virtue of the “toolbar” property. And then, the content_css needs to be set up so it includes the classes for highlighting. This can be any of those supplied by Highlight.js’ zip file, for example. The CSS file under /js/skins/ is given explicitly here, as this was the default in my case. When content_css is defined explicitly, the default is dropped.

And of course, the highlighting script must be loaded with something like this in the <head> section of the HTML:

<script src="js/highlight.min.js"></script>

There’s no point loading the relevant CSS file here if the editor is placed in an iframe, which it usually is (unless inline mode is selected).

Note that the icon is given explicitly in text as an SVG. This is quite common, actually. The icon is just the letters “HL” (not very inspirational, but fits nicely).

Simple highlight plugin explained

The actual work is done in this part:

      editor.formatter.apply('highlight_wrapper');
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
        let text = el.innerText;
	let hl = hljs.highlightAuto(text);
	editor.dom.setHTML(el, hl.value);
	editor.dom.remove(el, true);
      });

First, the formatter.apply() method is called to wrap the selected region with a <span> pair of tags, so that the relevant area is isolated and with a DOM element that can be referred to. Note that if there happened to be leftovers from a previous operations (an exception occurred?) they have already been removed by the code that comes before the snippet above.

At this point, a loop runs on all DOM elements having the highlight_tmp class, fetching their pure-text representation with innerText, calling highlightAuto() on this text, and then setting the HTML of the DOM element to the result of this call.

This is followed by a call to dom.remove(), which removes the DOM element, moving all children to the parent (this is what the “true” argument means). In HTML terms, this removes the <span> tags, but leaves anything between them.

At the end of the loop, there’s

editor.undoManager.add();

which adds an undo stage after the highlighting is in place. Without this, a CTRL-Z undoes the step before applying syntax highlighting as well.

Answers to questions nobody asked

First, a couple of notes:

• If no region is selected, but the cursor is in the middle of a word, that word is highlighted — just like “bold” would do. Only at the end of a line, nothing happens (or more precisely, an empty string is highlighted).
• No menu item is added on behalf of this plugin, because adding a menu item to an existing header requires redefining it from scratch, but then what happens if newer items are added by core TinyMCE in future versions? So the correct way to do it is to add another menu header, and that’s a bit too high much.

Now the question: Why wrapping the selection with a bogus class, and then remove it? Why not just grab the text, highlight, and inject it back? Like this?

let text = editor.selection.getContent({format : 'text'});
let hl = hljs.highlightAuto(text);
editor.execCommand('mceInsertRawHTML', false, hl.value);

This works quite nicely, however if an entire <pre> region is selected, the result is that the <pre> tag is dropped, and the highlighted code appears in a plain paragraph format, newlines turned into spaces, so it’s a mess. This happens because when the entire chunk is selected, the surrounding format is also part of the selection. Hence when it’s substituted, the <pre> is dropped as well.

The next question would be why there’s a loop on DOM elements with the class. Why would there be more than one? Why not defining the wrapper by an ID (assigned randomly), with something like

{ inline: 'span', attributes: { 'id': '%id' } }

and then wrapping and lookup of the DOM element would go

let dom_id = "highlight_" + Math.floor(Math.random()*1000000000);
editor.formatter.apply('highlight_wrapper', { id:  dom_id } );
let el = editor.dom.get(dom_id);

instead?

So in what case would there be more than one DOM element to process? The answer is that normally that won’t happen, but if a few paragraphs are chosen (like when pasting multi-line code into a non-<pre> context), formatter.apply() creates a separate <span> for each paragraph (more precisely, for each block element, since inline tags like <span> don’t cross block elements). Which is actually good, because the result is what one would expect.

So even though it’s bit odd to use highlighting not inside a <pre> section, the non-loop version above won’t process nor clean up more than one wrapper. That alone is a good reason to do it in a loop: Even if the user does something unexpected, no junk should be left in the document.

Ehm, I need to correct myself slightly on this: Apparently, sometimes when pasting plain text into the editor window (inside the <pre> region), <br/> tags are inserted instead of newlines. Not clear why, and it’s ugly, but with the loop the highlighting works regardless.

And lastly, is it OK to use innerText, despite its definition being pretty vague saying “it approximates the text the user would get if they highlighted the contents of the element with the cursor and then copied it to the clipboard”. For text inside a preformatted element, it looks safe enough. Maybe using TinyMCE’s API would be better still.

Manual language selection plugin

The code for the manual language is interesting in particular because it demonstrates a split button, i.e. a button that can open a selection drop-drop menu (as shown in the image above). It’s worth mentioning that only the plugin code itself has changed — the configuration in the call to tinymce.init() remains the same.

So here it is:

(function () {
    'use strict';
    // Fetch all languages and create an item list

    var langname = {};
    var items = [];
    var langlist = hljs.listLanguages();

    langlist.forEach(function(codename) {
      let l =  hljs.getLanguage(codename);
      langname[codename] = l.name;
    });

    langlist.sort(function(a, b){
      let x = langname[a].toLowerCase();
      let y = langname[b].toLowerCase();
      if (x < y) return -1;
      if (x > y) return 1;
      return 0;
     });

    var lang = langlist[0]; // Default to first listed language

    langlist.forEach(function(codename) {
      items.push({
        type: 'choiceitem',
        text: langname[codename],
        value: codename,
      });
    });

    // Prepare the GUI stuff
    var highlighter = function() {
      // First clean up possible leftovers
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
	editor.dom.remove(el, true);
      });
      editor.formatter.apply('highlight_wrapper');
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
        let text = el.innerText;
	let hl = hljs.highlight(text, {
          language: lang,
	  ignoreIllegals: true,
        });
	editor.dom.setHTML(el, hl.value);
	editor.dom.remove(el, true);
      });
      editor.undoManager.add(); // Make highlighting operation undoable
    }

    tinymce.PluginManager.add('highlight', function (editor) {
      editor.ui.registry.addIcon('highlight', '<svg width="24" height="24" viewBox="0 0 256 256"><path d="M 57.00,62.00 C 57.00,62.00 41.58,147.00 41.58,147.00 41.58,147.00 34.00,189.00 34.00,189.00 34.00,189.00 50.00,189.00 50.00,189.00 57.50,188.87 56.70,186.77 58.75,175.00 58.75,175.00 63.58,148.00 63.58,148.00 64.08,145.21 64.56,139.55 66.58,137.60 68.58,135.67 72.40,136.01 75.00,136.00 75.00,136.00 111.00,136.00 111.00,136.00 111.00,136.00 102.00,189.00 102.00,189.00 102.00,189.00 124.00,189.00 124.00,189.00 124.00,189.00 133.58,135.00 133.58,135.00 133.58,135.00 141.58,91.00 141.58,91.00 141.58,91.00 147.00,62.00 147.00,62.00 147.00,62.00 131.00,62.00 131.00,62.00 123.09,62.15 124.37,63.75 122.00,76.00 122.00,76.00 117.42,101.00 117.42,101.00 116.92,103.79 116.44,109.45 114.42,111.40 112.42,113.33 108.60,112.99 106.00,113.00 106.00,113.00 70.00,113.00 70.00,113.00 70.00,113.00 79.00,62.00 79.00,62.00 79.00,62.00 57.00,62.00 57.00,62.00 Z M 174.00,62.00 C 174.00,62.00 158.58,147.00 158.58,147.00 158.58,147.00 151.00,189.00 151.00,189.00 151.00,189.00 204.00,189.00 204.00,189.00 204.00,189.00 210.41,187.98 210.41,187.98 210.41,187.98 212.61,182.00 212.61,182.00 212.61,182.00 215.00,167.00 215.00,167.00 215.00,167.00 177.00,167.00 177.00,167.00 177.00,167.00 189.42,99.00 189.42,99.00 189.42,99.00 196.00,62.00 196.00,62.00 196.00,62.00 174.00,62.00 174.00,62.00 Z" /></svg>');

    editor.ui.registry.addSplitButton('hljs', {
      icon: 'highlight',
      tooltip: 'Syntax highlight selection',
      onAction: highlighter,
      onItemAction: function (api, value) {
        lang = value;
	highlighter();
      },
      fetch: function(callback) { callback(items); },
      select: function(x) { return x === lang; },
    });
 });
}());

The larger part of the change is at the beginning of the plugin’s code, which sets up the list of items for the drop-down meny by calling hljs.listLanguages() and then obtaining the human-friendly name of the languages.

Next, hljs.highlightAuto(text) is replaced with hljs.highlight(text, { … } ), so that the language is given explicitly as the @lang local variable.

And finally, addSplitButton() is called instead of addButton(), with the same properties plus:

• onItemAction supplies the function that is called when a menu item is selected. Not surprisingly, is sets @lang to the value of the selection (the “value” property in the item entry) and then calls highlighter() to highlight the relevant chunk.
• fetch supplies a function that is called for supplying the items in the drop-down list. It’s called every time the drop-down list is opened, so the content of this list can change from one time to the other. For this plugin, the same array is supplied each time, as the language list won’t change.
• select supplies a function that evaluates true for the list item next to which a check mark should appear to indicate that it’s selected.

Summary

I have to admit that I’m quite impressed by how TinyMCE’s API was right on spot with supplying the right functions to get the job done easily. Note how short and concise the code for the automatic highlighter is. And the manual language plugin got most of its weight from the code that plays around with languages, and not setting up the GUI.