LG G4 internals: My messy jots

Introduction

These are my really messy jots as I tried out things with my LG G4 H815 phone (Android 5.1, kernel 3.10.49, software version V10e-ISR-COM from Aug 17, 2015, not locked to any specific network). I usually try to get the post tidy so it becomes some kind of how-to post, but this time I just left it as is.

The only important takeaway from all this is that there’s no need to root the phone in order to get the key file that is necessary to decrypt all WhatsApp messages in the phone’s history: The file can be fetched in Download Mode without rooting.

My original idea was to upgrade Android from Lollipop to Nougat, mainly because there are a few Apps out there that won’t run on my outdated OS. So the point was to make the phone work better, or else I just go and buy a new one. As little hassle as possible.

The fact that I have a G4 for six years implies that I’m not fond of these things, and I’m anything but an Android expert. This was really the first time I sat down and tried to see what’s under the hood. It’s more hassle than just buying a new phone (or is it?) but I thought maybe I’ll learn something interesting.

As it turns out, each Android phone has its own story and toolset for the low-level stuff, so it’s really a lot of ugly company-specific things to deal with. In short, not very interesting.

Understanding that upgrading will probably come with a headache, and that rooting the phone will also probably mess up things considerably, I decided to leave all this. The only benefit was the full image backup, which at the very least gives me the WhatsApp secret key.

The original plan

  • Make screenshots of applications, both main and secondary.
  • Try LG Bridge and see what it can do (in particular regarding backup of WhatsApp)
  • Save all WhatsApp chats to cloud (Google Drive).
  • Root the phone for backing up all files. In particular, get the WhatsApp key, without which the backups are worthless (in hindsight, no need to root the phone for this purpose).
  • Upgrade to Nougat, possibly with LG Bridge, and if not with LG UP.
  • Restore with LG’s own backup tool and fetch WhatsApp chats from cloud.

Backups

Before upgrading, make a backup? Here’s my maximalistic checklist.

  • LG’s own backup tool, backing up everything except media (got 760 MB, would have reached 6 GB with media)
  • Back up all in DCIM/Camera (all taken photos and videos).
  • Ran a backup to cloud within WhatsApp. Also copy the files from /storage/sdcard0/WhatsApp/Databases/
  • Then a full filesystem tar backup, as shown below (before rooting, so it’s partial).
  • Full image copies with SALT. Well, no, it didn’t work, see below.
  • A full backup of /data with the phone in Download Mode (including files accessible by root only, but without rooting the phone).

Reading WhatsApp messages

The maybe most important goal of all this was to harvest my WhatsApp history in a sustainable way. That is, so the messages can be read like 20 years from now. Because the grim fact is that as soon as a little key file that is buried inside the phone is lost, all message history goes down the drain forever.

The WhatsApp database files are described on this page with focus on the message files on this page. To make a long story short, the messages are stored in an Sqlite file (or is it? Probably, and it’s a .db file).

These are accessible but encrypted, and so are the backup files (e.g. msgstore.db.crypt14). The encryption key is accessible with root only, and is created specifically for a user when creating a new WhatsApp account as /data/data/com.whatsapp/files/key.

So the files to grab are

  • The encrypted database file: /data/media/0/WhatsApp/Databases/msgstore.db.crypt14
  • The file containing contact information (optional): /data/data/com.whatsapp/databases/wa.db
  • The key: /data/data/com.whatsapp/files/key

With these at hand, WhatsApp viewer (see github repo) can be used to decrypt the db file (it supports crypt14) and then view the messages. It’s a Windows program, and I guess some openssl command for the crypt14 command will be available sooner or later.

SSH session with simpleSSHD

This is a no-cost app, which is essentially the Dropbear server.

Note: It’s also possible to connect with “adb shell” through a plain USB cable, and there’s “adb push” and “adb pull” to transfer files. And there’s shell as root in Download Mode.

Having SimpleSSHD running and started on the phone, I connected with the address provided on the screen

$ ssh -p 2222 user@10.11.12.13

the password appears on the phone’s screen. To use automatic login, go

cat > ~/authorized_keys

and copy-paste the content of ~/.ssh/id_rsa.pub there. Note that the file on the phone is not under a .ssh/ directory, which is probably why the ssh-copy-id utility doesn’t cut. However that once this file is found, a password login is not attempted if the host’s public key doesn’t match, so this can get a bit messy.

In principle, the available executables are in /system/bin. The path contains more directories, but this is the only effective one.

Who am I?

$ id
uid=10141(u0_a141) gid=10141(u0_a141) groups=1015(sdcard_rw),1028(sdcard_r),3003(inet),9997(everybody),50141(all_a141) context=u:r:untrusted_app:s0

Backing up directly from phone (command run on receiving host):

$ ssh -p 2222 user@10.11.12.13 tar -cvz /storage/emulated/0/Pictures/Screenshots/ > ~/Desktop/pics.tar.gz

Data speeds

Because there’s going to be a lot of data copying, I checked the data speeds with scp. So one gets about

  • 2.7 MB/s when the phone is a Wifi client
  • 6.7 MB/s when the phone acts as a hotspot.
  • 20 MB/s when sharing Internet through the USB cable

The winner is clear.

Mounts

The files, as exposed when connecting to the phone with a USB cable, are at /storage/emulated/0/, or /storage/emulated/legacy/. The latter is symlinked to by /storage/sdcard0. Both are mounts of /data/media (as well as /mnt/shell/emulated/). So /media/data/0/ is essentially the root of the filesystem that is visible over USB.

$ mount
rootfs / rootfs ro,relatime 0 0
tmpfs /dev tmpfs rw,seclabel,nosuid,relatime,size=1437392k,nr_inodes=359348,mode=755 0 0
devpts /dev/pts devpts rw,seclabel,relatime,mode=600 0 0
none /dev/cpuctl cgroup rw,relatime,cpu 0 0
adb /dev/usb-ffs/adb functionfs rw,relatime 0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,seclabel,relatime 0 0
selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,relatime 0 0
none /sys/fs/cgroup tmpfs rw,seclabel,relatime,size=1437392k,nr_inodes=359348,mode=750,gid=1000 0 0
pstore /sys/fs/pstore pstore rw,relatime 0 0
none /acct cgroup rw,relatime,cpuacct 0 0
tmpfs /mnt/asec tmpfs rw,seclabel,relatime,size=1437392k,nr_inodes=359348,mode=755,gid=1000 0 0
tmpfs /mnt/obb tmpfs rw,seclabel,relatime,size=1437392k,nr_inodes=359348,mode=755,gid=1000 0 0
/mnt/pstore /mnt/pstore pstore rw,relatime 0 0
/dev/block/bootdevice/by-name/system /system ext4 ro,seclabel,noatime,data=ordered 0 0
/dev/block/bootdevice/by-name/cache /cache ext4 rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,errors=continue,data=ordered 0 0
/dev/block/bootdevice/by-name/userdata /data ext4 rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,resuid=1000,errors=continue,data=ordered 0 0
/dev/block/bootdevice/by-name/persist /persist ext4 rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,errors=continue,data=ordered 0 0
/dev/block/bootdevice/by-name/modem /firmware vfat ro,context=u:object_r:firmware_file:s0,relatime,uid=1000,gid=1000,fmask=0337,dmask=0227,codepage=437,iocharset=iso8859-1,shortname=lower,errors=remount-ro 0 0
/dev/block/bootdevice/by-name/sns /sns ext4 rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,errors=continue,data=ordered 0 0
/dev/block/bootdevice/by-name/drm /persist-lg ext4 rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,errors=continue,data=ordered 0 0
/dev/block/bootdevice/by-name/mpt /mpt ext4 rw,seclabel,nosuid,nodev,noatime,noauto_da_alloc,errors=continue,data=ordered 0 0
/dev/block/platform/f9824900.sdhci/by-name/cust /cust ext4 ro,seclabel,nosuid,nodev,relatime,noauto_da_alloc 0 0
/data/media /mnt/shell/emulated sdcardfs rw,nosuid,relatime,uid=1023,gid=1023,derive=legacy 0 0
/data/media /storage/emulated/legacy sdcardfs rw,nosuid,relatime,uid=1023,gid=1023,derive=legacy 0 0
tmpfs /storage/emulated tmpfs rw,seclabel,nosuid,nodev,relatime,size=1443536k,nr_inodes=360884,mode=751,gid=1028 0 0
/data/media /storage/emulated/0 sdcardfs rw,nosuid,relatime,uid=1023,gid=1023,derive=legacy 0 0
/data/media /storage/emulated/legacy sdcardfs rw,nosuid,relatime,uid=1023,gid=1023,derive=legacy 0 0
$ df
Filesystem                 Size         Used         Free    Blksize
/dev                      1.37G      100.00K        1.37G       4096
/dev/usb-ffs/adb: Permission denied
/sys/fs/cgroup            1.37G       12.00K        1.37G       4096
/mnt/asec                 1.37G        0.00K        1.37G       4096
/mnt/obb                  1.37G        0.00K        1.37G       4096
/system                   3.92G        3.15G      785.26M       4096
/cache                    1.17G        2.11M        1.16G       4096
/data                    22.70G       14.51G        8.19G       4096
/persist                 27.48M       64.00K       27.42M       4096
/firmware                85.95M       74.77M       11.19M      16384
/sns                      3.86M       60.00K        3.80M       4096
/persist-lg               3.86M      604.00K        3.27M       4096
/mpt                     27.48M       11.14M       16.35M       4096
/cust                     5.81M      753.00K        5.07M       1024
/mnt/shell/emulated: Permission denied
/storage/emulated/legacy     22.70G       14.51G        8.19G       4096
/storage/emulated         1.38G        0.00K        1.38G       4096
/storage/emulated/0      22.70G       14.51G        8.19G       4096
/storage/emulated/legacy     22.70G       14.51G        8.19G       4096

Note the mount duplicity of /data/media as sdcardfs. This appears to be some kind of bind mount. /data is the actual userdata mount, and /data/media is what is shown to us users.

Also note that many of the mounts have the nosuid flag set, for good reasons.

Accordingly, backing up everything is done with

ssh -p 2222 user@10.10.0.226 tar -czv --one-file-system / /system /data /persist /firmware /sns /persist-lg /mpt /cust /storage/emulated/0 > ~/Desktop/phone-all.tar.gz

Almost needless to say, if this isn’t done as root, a whole lot of files will not be backed up as permission is denied.

For a non-root (hence partial) backup (5.6 GB), it takes 24 minutes with a Internet-over-USB connection with the phone.

Trying out LG Bridge Setup

Downloaded from the official site and installed version 1.2.68 on a Win10 machine, selecting Israel as my country and with plain installation on C:\. Went smooth.

Running the application, it went “No mobile device connected”. I fiddled around a bit with the phone to enable MTP on the phone, and it was detected properly by the software.

As for backup, it appears to be pretty much like the backup utility on the phone itself. As for software upgrade, that requires network access, so I didn’t go into this further (because enabling network access causes Windows to start upgrading itself).

Some general insights on rooting

In Android, unlike a common Linux system, uids are allocated to apps, and not real users. This is how the system makes sure each app gets access to its own stuff, and to resources they should access, by virtue of groups.

But exactly like any Linux, user 0, root, has access to everything. Android is designed to prevent this state from anything but the operating system itself, among others because many devices are sold for use with a specific cellular network, and rooting the device sets it free.

The idea behind rooting is to replace a component in the boot loading process, so that the root user can be given to plain applications. I didn’t reach this point, so I’m not sure exactly how this works out.

But in principle the procedure is to make some plain tap-on-screen setups to make the phone more permissive towards changes, then load a software image (e.g. TWRP) that is used when the phone wakes up in Recovery Mode (more on this next), which then allows injecting software that allows root access.

Because the phone is designed to prevent users from doing this, it will have to be “bootloader unlocked” somewhere through the process. It’s quite interesting that my phone needs that too, even though it was bought standalone, so who cares what I do with it.

TWRP is basically a utility that allows fiddling with the phone. One way to run it is with fastboot, in which case it runs from memory. To root the phone, its copied into the /recovery partition, so when the phone is powered up in Recovery Mode, TWRP runs.

The bootloader unlock thing

The procedure for rooting a phone always involves “unlocking the bootloader”. This appears to mean to allow writing raw image files (sometimes mistakenly called “ROMs”) into the phone’s flash partitions. This is often referred to as “flashing” and is crucial, because one of the steps for rooting the device is to replace the partition that is loaded in Recovery Mode.

Since flashing is typically made when the phone talks in the fastboot protocol (i.e. it’s in Fastboot Mode), and this protocol is commonly implemented in the bootloader, I guess that’s why they call it to unlock the bootloader.

See my failed attempt to write an image file in Fastboot Mode below.

Until the end of 2021, LG had a web interface for submitting the phones identification and obtain an unlock.bin file, which could be written to the device, and that would unlock the phone. Maybe the idea was to void warranty and allow this to phones that were sold with no strings attached. But I was too late for this.

But here’s the twist. LG (and probably other vendors) added a “Download Mode” that allows writing to partitions even if the phone is locked. My speculation is that they needed a way to update the software on any phone, without any hassle. So they created their own secret protocol (LG LAF) that works in Download Mode and allows full access, for use only by their own software. But then this protocol was reverse engineered.

By doing this, they created a huge backdoor which bypasses the limitations on fastboot mode completely. Security by obscurity. It always fails.

Boot modes

  • Regular Android mode (MTP) — when the phone works normally. adb talks with the phone in this mode. USB ID 0x1004/0x633e.
  • Recovery mode — the phone loads the recovery image, whatever happens to be there. Not sure if there’s any USB communication, I guess it depends on the image.
  • Fastboot mode — apparently this is the pre-Linux bootloader. The device presents itself with a different USB Vendor / Product ID, intended for tweaking and flashing the device. The fastboot utility talks with the phone in this mode. USB ID 0x18d1/0xd00d.
  • Download mode — the most permissive mode it seems. Allows some file related commands directly to the phone as root, as well as shell commands. For this mode, use SALT, or the LG LAF utilities directly. USB ID 0x1004/0x633e (same as MTP).

“adb devices” and “fastboot devices” will never detect the phone at the same time. The former expects it to be functioning normally, and the latter in Fastboot Mode, as in after “adb reboot bootloader”. The phone appears on the USB bus with a completely different ID.

There’s also “adb shell” which opens a plain shell on the phone, with uid 2000. The “su” command doesn’t exist, and neither would it work on a non-rooted phone. It’s also possible to copy files to and from the phone from adb. See this page.

On the boot process

The boot process is a bit in the mist, but apparently there’s a vendor-proprietary bootloader that chooses to boot one of the partitions on the flash, depending on button presses and whatnot. It’s probably aboot.img, being an ELF 32-bit executable, so it’s fair to guess that it’s the initial bootloader. Apparently this executable implements the fastboot protocol. In other words, fastboot mode is probably the most initial boot level.

The important part is that each of the other boot images contain the kernel plus a ramdisk image. So each is standalone.

To get a closer look on a boot image, follow the suggestion on this page,

# apt install abootimg

So this allows looking into what’s inside an Android boot image:

$ abootimg -i laf.bin 

Android Boot Image Info:

* file name = laf.bin 

* image size = 50331648 bytes (48.00 MB)
  page size  = 4096 bytes

* Boot Name = ""

* kernel size       = 26700384 bytes (25.46 MB)
  ramdisk size      = 10118136 bytes (9.65 MB)

* load addresses:
  kernel:       0x00008000
  ramdisk:      0x02200000
  tags:         0x02000000

* cmdline = console=ttyHSL0,115200,n8 androidboot.console=ttyHSL0 user_debug=31 ehci-hcd.park=3 lpm_levels.sleep_disabled=1 androidboot.hardware=p1

* id = 0x3fc917e5 0xb1c78118 0x83d0740e 0x1a124148 0x836687d1 0x00000000 0x00000000 0x00000000

and extract the kernel and ramdisk:

$ abootimg -x laf.bin
writing boot image config in bootimg.cfg
extracting kernel in zImage

And extract the RAM image in a new directory:

$ zcat ../initrd.img | cpio -vid

Preparations for rooting

Annoying abbreviations: LP = Lollipop, MM = Marshmallow, N = Nougat (all codenames for Android versions).

I went for the instructions on this page as well as those specific to H815, even though it’s probably the same with all G4′s. So:

  • Went to Settings > About Phone > Software Information (מידע תוכנה) and tap 7 times on build number (מספר יסוד in Hebrew, an awful translation). And it told me that I’m a software developer now.
  • Now under General (כללי) I had a new entry, Developer Options (אפשרויות פיתוח, another horrible translation), confirmed the warning, and enabled USB Debugging (ניפוי באגים). And confirm the warning.
  • Also checked “Allow disable OEM lock” (or something like that, this is a translation from Hebrew). Otherwise “adb reboot bootloader” fails (boots normally).

Getting the tools up

I went for Windows as the platform for the tools, mainly because most flashing tools are for Windows.

  • Downloaded ADB and fastboot (platform tools) from here (following suggestion on this site). Note that there’s platform tools for Linux (unlike other utils), and it’s probably a better idea.
  • Extracted the zip file into c:\platfrom-tools and ran
    adb devices

    from DOS window. The phone requested permission to allow access from the computer, and once granted, the phone was listed as “unauthorized”. But that was just because I granted a one-time access when the phone asked me about it. Giving it permanent access made the phone listed as “device”, as it should:

    C:\>adb devices
    List of devices attached
    LGH81530828bea  device
  • Ah, if “adb devices” lists it as “offline”, just move to another USB port. Don’t know why it matters, but it works.
  • I went “adb reboot bootloader” to invoke Fastboot Mode, and indeed I got some very tiny text on the phone, above the LG logo. The phone identified as USB 0x18d1/0xd00d, and Windows classified it as “Other Device” named “Android”, and claimed having no driver for it. So I reinstalled adb with adb-setup-1.4.3 downloaded from here. This made adb installed system-wide, albeit the device was still undetected. Going further, I downloaded the driver package from here, even though I could see that the device ID wasn’t listed in the .inf file, and indeed that didn’t help. The version of the Mobile Drivers was the latest at the time (4.8.0). I found the solution ultimately here, downloading the file called CT_HsPhone_General_Drivers.rar, and the driver for the device, in fastboot mode was there, and I installed it from the Device Manager. This would probably be much easier on Linux.
  • I tried “adb reboot recovery” but the phone woke up with a dark screen and it said “Žádný příkaz” which apparently means “No command” in Polish. So clearly, no proper recovery image on my phone.

Money time (not)

  • Download TWRP 3.6.1_(-0 for H815 from this page. There’s a different (?) TWRP for virtually every model, but apparently there’s one covering all G4′s.
  • I intended to install TWRP in the recovery partition as suggested on this page, but never got to that.
  • I also downloaded SuperSU, but likewise didn’t get to use it.

And why did I stop here? Well, in Fastboot Mode, this session says it all:

C:\>fastboot devices
30828bea        fastboot

C:\>fastboot flash recovery S:\android-upgrade-stuff\twrp-3.6.1_9-0-g4.img
target reported max download size of 536870912 bytes
sending 'recovery' (21976 KB)...
OKAY [  0.510s]
writing 'recovery'...
FAILED (remote: device is locked. Cannot flash images)
finished. total time: 0.541s

C:\>fastboot boot S:\android-upgrade-stuff\twrp-3.6.1_9-0-g4.img
Sending 'boot.img' (21976 KB)                      OKAY [  0.488s]
Booting                                            FAILED (remote: 'unlock device to use this command')
fastboot: error: Command failed

so the device is locked.

I wonder what would happen if I wrote the image to the partition in Download Mode, with a shell command, as described below. I guess the TWRP would run, and I guess SuperSU would get installed, but would that root the device? Given that the trick in Download Mode got me what I wanted (a full backup of /data), it was pointless to risk messing up a working phone for this purpose.

SALT and FWUL

SALT is the superior tool for accessing the device (explanations here), and is probably the only way to fully backup the phone, including things that are inaccessible without rooting the phone. It’s a GUI tool written in Bash shell script (!) with yad, and is to a great extent based upon the lglaf utility set.

Its main advantage is that the phone is in Download Mode, running a simple protocol (LG LAF) that allows simple commands to be made as root. So it’s possible to read and write into any file, including the block devices that represent flash partitions in /dev.

Another interesting feature of SALT is the ability to open a shell as root (under the advanced menu). However even as root, certain operations are not permitted, apparently because of SELinux. And it’s a fake shell window — each command is sent individually, so e.g. changing directory has no effect, and it’s not possible to invoke background processes.

Since all partitions can be read from, any information on the phone is available. It’s also possible to replace to bootloader completely, along with other images, which is how the UsU unlock is made possible.

To use SALT, I went for FWUL (Forget Windows, Use Linux), downloaded from this page. Then ran the ISO image inside a VMPlayer virtual machine.

FWUL is Manjaro Linux, based upon Arch Linux (hello pacman) without ability to install an NFS mount by default (or else how am I supposed to store a 30 GB image?). And once I did install mount utilities with pacman, and overcame some Glibc conflicts, it turned out that I had to add the “insecure” option in the /etc/exports file of the NFS host, because it connects from an port < 1024.

In short, it’s not FWUL, but AWFUL. If it’s intended for Linux newbies, it can’t be based upon an unfriendly distribution. It has no netcat, there’s only leafpad for a GUI text editor (first time I heard about it) or go for vim (even the command “vi” didn’t work).

The password for both “android” user and root is “linux”.

“adb devices” failed to detect the phone, on the virtual machine as well as on bare metal, even though the phone’s internal storage was available through the USB interface, so it was clearly properly detected by the system.

However SALT did detect the phone in download mode, so it was good enough.

To invoke download mode: Turn the phone off, disconnect whatever was in its USB / power input, press the volume up button, and insert a USB cable connected to a computer (not just a power source).

I attempted a full partition backup into an NFS mount, and it got stuck, so wrote the ISO image on a USB stick (plain cat xxx.iso > /dev/sda) and ran it on a computer directly. But that wan’t the problem. The whole partition backup into images doesn’t really work for large images, even not on bare metal: Somewhere after 400-600 MB, the data transfer got stuck (the image file written to stopped growing). Sometimes even earlier. Which can explain why the “Basic” backup offered by the SALT grabs images of up to 260 MB.

Unlocking the UsU way

The official LG way was not supported when I got to this, so I went for the scary option.

The description for how to unlock the phone is on this page. It can’t be too emphasized that getting this wrong will hard brick the phone. The basic idea is to overwrite three crucial partitions with alternative images, and good luck.

SALT identified the ARB (AntiRollBack) version as 0, as expected from this page, so this was a green light to proceed.

However at this point I realized that UsU pretty much messes up the phone’s internals (there’s a warning about reducing video frame rate…!), so I just left it. Given that the reason for doing this is to make a full backup as root, the chance for bricking the phone made all this pretty much pointless.

Backing up all data as root

Since Download mode allows sending shell commands as root, and there’s the tar utility available, why not make one big tarball, and bypass the access restriction of a regular user? It turns out it’s not that simple. This is what happens in Download Mode (and not otherwise):

# gzip
-: gzip: Permission denied
# tar
-: tar: Permission denied

So probably by virtue of SELinux or something, these two functions can be run as a regular user, but not in Download Mode. How clever is that? Not at all. Security by obscurity at its worst.

The basic rule is that if you have shell access, you can do everything. I might as well have compiled my own utilities and put them on the filesystem. But why bother? Both utilities are there, so just copy their content:

# cat /system/bin/tar > /mytar
# cat /system/bin/gzip > /mygzip
# chmod 0777 mygzip mytar

And all that is left to do is

# ./mytar -c --exclude=data/a.tgz data | ./mygzip -c > data/a.tgz

A few notes:

  • All commands are given from the root directory. So I use with and without trailing slash interchangeably.
  • The -v flag caused the shell window to crash right away. Too much output, I guess.
  • The -z flag caused tar to use the regular gzip executable, so it failed. Hence the pipe to my own gzip.

So this ran for a while, and then I got:

/home/android/programs/lglafng/lglaf.py:564: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  _logger.warn(e)
LGLAF.py: WARNING: [Errno 110] Operation timed out

Like, really? No kidding. This takes time. And after this message, any attempt to do something with the phone failed until it was repowered with the battery. So maybe the command kept running, and therefore? I don’t know.

To make a long story short, adding an ampersand at the end of the command doesn’t make the command go into background. So that doesn’t solve anything. So I tried with a little shell script that makes the call go into background, maybe?

Using echo commands, I set up this little file as together.sh:

#!/system/bin/sh
./mytar -c --exclude=data/a.tgz data | ./mygzip -c > data/a.tgz &

and then I went

# chmod 0777 together.sh

and then I ran together.sh. That didn’t run in the background either, and soon enough I got the timeout message again. Sigh.

But this time I decided to just let the phone be and not touch anything for a few hours. Maybe it still runs.

After a while I got back to the shell window, tried some random command, and it emitted the following:

./mytar: data/misc/camera/cam_socket1: socket ignored
./mytar: data/misc/camera/cam_socket2: socket ignored
./mytar: data/misc/location/mq/alarm_svc: socket ignored
./mytar: data/misc/location/quipc/gsiff_socket: socket ignored
./mytar: data/misc/perfd/mpctl: socket ignored
./mytar: data/data/com.zhiliaoapp.musically/files/socket_pipe: socket ignored
./mytar: data/property/.temp.oflBVX: Cannot open: Permission denied
./mytar: data/property/persist.sys.multi-cust: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.default: Cannot open: Permission denied
./mytar: data/property/persist.sys.ntcode-changed: Cannot open: Permission denied
./mytar: data/property/persist.sys.ntcode: Cannot open: Permission denied
./mytar: data/property/persist.sys.mccmnc-list: Cannot open: Permission denied
./mytar: data/property/persist.sys.subset-list: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.integration: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.subca-prev: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.subca-prop: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.next-root: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.prev-rootdir: Cannot open: Permission denied
./mytar: data/property/persist.sys.cupss.changed: Cannot open: Permission denied
./mytar: data/property/persist.service.bdroid.a2dp_con: Cannot open: Permission denied
./mytar: data/property/persist.service.bdroid.scms_t: Cannot open: Permission denied
./mytar: data/property/persist.sys.emmc_size: Cannot open: Permission denied
./mytar: data/property/persist.profiled.build.version: Cannot open: Permission denied
./mytar: data/property/persist.lge.appman.installstart: Cannot open: Permission denied
./mytar: data/property/persist.sys.thermald_miti_off: Cannot open: Permission denied
./mytar: data/property/persist.lge.appman.errc_done: Cannot open: Permission denied
./mytar: data/property/persist.radio.adb_log_on: Cannot open: Permission denied
./mytar: data/property/persist.radio.ril_payload_on: Cannot open: Permission denied
./mytar: data/property/persist.sys.first-mcc: Cannot open: Permission denied
./mytar: data/property/persist.sys.first-boot: Cannot open: Permission denied
./mytar: data/property/persist.sys.mcc-list: Cannot open: Permission denied
./mytar: data/property/persist.sys.sim-changed: Cannot open: Permission denied
./mytar: data/property/persist.sys.clientid-changed: Cannot open: Permission denied
./mytar: data/property/persist.sys.first-mccmnc: Cannot open: Permission denied
./mytar: data/property/persist.lge.appbox.ntcode: Cannot open: Permission denied
./mytar: data/property/persist.sys.iccid: Cannot open: Permission denied
./mytar: data/property/persist.sys.iccid-mcc: Cannot open: Permission denied
./mytar: data/property/persist.radio.add_power_save: Cannot open: Permission denied
./mytar: data/property/persist.radio.eons.enabled: Cannot open: Permission denied
./mytar: data/property/persist.sys.dalvik.vm.lib.2: Cannot open: Permission denied
./mytar: data/property/persist.sys.language: Cannot open: Permission denied
./mytar: data/property/persist.sys.country: Cannot open: Permission denied
./mytar: data/property/persist.sys.localevar: Cannot open: Permission denied
./mytar: data/property/persist.radio.first-set: Cannot open: Permission denied
./mytar: data/property/persist.lg.data.dsqn: Cannot open: Permission denied
./mytar: data/property/persist.sys.profiler_ms: Cannot open: Permission denied
./mytar: data/property/persist.sys.theme0: Cannot open: Permission denied
./mytar: data/property/persist.sys.theme: Cannot open: Permission denied
./mytar: data/property/persist.sys.cnd.iwlan: Cannot open: Permission denied
./mytar: data/property/persist.service.bdroid.bdaddr: Cannot open: Permission denied
./mytar: data/property/persist.radio.sms_ims: Cannot open: Permission denied
./mytar: data/property/persist.security.sdmediacrypto: Cannot open: Permission denied
./mytar: data/property/persist.radio.flex.orgVals: Cannot open: Permission denied
./mytar: data/property/persist.radio.flex.orgSBPs: Cannot open: Permission denied
./mytar: data/property/persist.data.sbp.update: Cannot open: Permission denied
./mytar: data/property/persist.radio.first-mccmnc: Cannot open: Permission denied
./mytar: data/property/persist.sys.factory.status: Cannot open: Permission denied
./mytar: data/property/persist.audio.nsenabled: Cannot open: Permission denied
./mytar: data/property/persist.audio.voice.clarity: Cannot open: Permission denied
./mytar: data/property/persist.camera.ais.hal: Cannot open: Permission denied
./mytar: data/property/persist.data.rear.minfps: Cannot open: Permission denied
./mytar: data/property/persist.data.front.minfps: Cannot open: Permission denied
./mytar: data/property/persist.mms.pre-install: Cannot open: Permission denied
./mytar: data/property/persist.gsm.sms.forcegsm7: Cannot open: Permission denied
./mytar: data/property/persist.gsm.mms.enabled: Cannot open: Permission denied
./mytar: data/property/persist.gsm.mms.roaming.enabled: Cannot open: Permission denied
./mytar: data/property/persist.sys.wificountrymcc: Cannot open: Permission denied
./mytar: data/property/persist.sys.cust.latamspanish: Cannot open: Permission denied
./mytar: data/property/persist.sys.timezone: Cannot open: Permission denied
./mytar: data/property/persist.sys.usb.config: Cannot open: Permission denied
./mytar: data/property/persist.sys.security: Cannot open: Permission denied
./mytar: data/property/persist.radio.iccid-changed: Cannot open: Permission denied
./mytar: data/property/persist.radio.sim_mccmnc: Cannot open: Permission denied
./mytar: data/property/persist.radio.sim-spn: Cannot open: Permission denied
./mytar: data/property/persist.data_netmgrd_mtu: Cannot open: Permission denied
./mytar: data/property/persist.radio.clir: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_lons_0_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_lons_1_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_lons_2_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_lons_3_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_sons_0_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_sons_1_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_sons_2_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_sons_3_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.nitz_plmn_0: Cannot open: Permission denied
./mytar: data/property/persist.radio.keyBlockByCall: Cannot open: Permission denied
./mytar: data/property/persist.radio.airplane_mode_on: Cannot open: Permission denied
./mytar: data/property/persist.sys.webview.vmsize: Cannot open: Permission denied
./mytar: data/property/persist.sys.theme10: Cannot open: Permission denied
./mytar: data/property/.temp.gaiLAw: Cannot open: Permission denied
./mytar: data/property/.temp.E6U7qM: Cannot open: Permission denied
./mytar: data/system/ndebugsocket: socket ignored
./mytar: Error exit delayed from previous errors

and then I restarted the shell window because apparently the responses to commands I got didn’t make sense.

I don’t know if this is all output that the tar operation generated, and neither do I know if it made any difference that I ran the command from this special script file as opposed to the original attempt — both ended with a timeout, after all.

But the fact that I got tar’s exit message, and it wasn’t that it ran out of disk space: The command finished successfully (with some 200 MB disk space left).

So that left me with moving the file where it will be visible as a regular user:

# mv data/a.tgz data/media/0/
# chmod 666 data/media/0/a.tgz

And that’s it. I got a full backup, minus the files listed above (I think, maybe there were more). The backup includes WhatsApp’s private files, and the key, which is the really important part.

I fetched the file with scp using Dropbear, with that app’s regular credentials.

To remove the file, I had to get into Download Mode again.

The phone was never rooted.

Jots while trying to find resources on the web

Incomplete, sometimes duplicates of things said above.

  • There’s Terminal Emulator (running on the machine itself).
  • There’s an Dropbear, and 4SSH server (go for former).
  • How to root G4, or more specifically a H815: Here and here (probably the same). In essence, the process consists of unlocking the bootloader and install TWRP recovery. This allows running SuperSU root tool (or something else). But this is the most accurate page to relate to.
  • LG Bridge is the tool for upgrading and backing up an LG phone. Available only for Windows and Mac. There are indications that is supports G4.
  • Upgrading to Nougat: Unofficial guide
  • There are two unoffical, yet popular tools for flashing an LG: LGUP and LG Flash Tool, and it seems like LGUP is generally preferred, Not clear why use them and not LG Bridge. There’s also Uppercut, which is a utility allowing LGUP to detect virtually any LG device.
  • There’s a tool for extracting LBF (backup files), however written in Python. But it worked with no issue on ruhe. That said, the backup content was pretty opaque and minimal, so it’s not all that useful.

What updating to Nougat would have meant

Even though I didn’t eventually update my phone to Nougat, I did fetch the official LG-H815_Official_v29A.zip file. Opening this zip file, it’s a whole lot of .bin files, each to be written to a different flash partition.

Some are ext4 filesystems, some are Android boot files, and some are well, who knows. The easy way to tell is to use the “file” utility, e.g.

$ file system.bin
system.bin: Linux rev 1.0 ext4 filesystem data, UUID=da594c53-9beb-f85c-85c5-cedf76546f7a, volume name "system" (extents) (large files)

So it’s ext4? Great, let’s mount it (create /mnt/tmp first, right?):

# mount -o loop system.bin /mnt/tmp

What’s really unpleasant is that aboot.bin and boot.bin are also updated. That’s bootloaders. And laf.bin, which I guess is the image for Download Mode. In short, if this fails, your phone is really bricked.

The procedure is outlined in META-INF/com/google/android/updater-script inside the said ZIP file, which is apparently executed by the “update-binary” program in the same directory. Anyhow, the updater-script file is as follows:

ui_print("");
ui_print("*****************************************");
ui_print("*                                       *");
ui_print("*                Nougat                 *");
ui_print("*                                       *");
ui_print("*            for LG G4 H815             *");
ui_print("*                                       *");
ui_print("*****************************************");
ui_print("");
ui_print("             ..please wait..             ");
ui_print("");
ui_print("Un-Mounting partitions...");
ifelse(is_mounted("/system") == "/system", unmount("/system"));
ifelse(is_mounted("/data") == "/data", unmount("/data"));
ifelse(is_mounted("/modem") == "/modem", unmount("/modem"));
ifelse(is_mounted("/recovery") == "/recovery", unmount("/recovery"));
ui_print("Flashing Nougat...");
ui_print(" ");
package_extract_file("aboot.bin", "/dev/block/bootdevice/by-name/aboot");
package_extract_file("factory.bin", "/dev/block/bootdevice/by-name/factory");
package_extract_file("hyp.bin", "/dev/block/bootdevice/by-name/hyp");
package_extract_file("laf.bin", "/dev/block/bootdevice/by-name/laf");
package_extract_file("pmic.bin", "/dev/block/bootdevice/by-name/pmic");
package_extract_file("raw_resources.bin", "/dev/block/bootdevice/by-name/raw_resources");
package_extract_file("rpm.bin", "/dev/block/bootdevice/by-name/rpm");
package_extract_file("sbl1.bin", "/dev/block/bootdevice/by-name/sbl1");
package_extract_file("sdi.bin", "/dev/block/bootdevice/by-name/sdi");
package_extract_file("sec.bin", "/dev/block/bootdevice/by-name/sec");
package_extract_file("tz.bin", "/dev/block/bootdevice/by-name/tz");
package_extract_file("boot.bin", "/dev/block/bootdevice/by-name/boot");
package_extract_file("modem.bin", "/dev/block/bootdevice/by-name/modem");
package_extract_file("system.bin", "/dev/block/bootdevice/by-name/system");
package_extract_file("recovery.bin", "/dev/block/bootdevice/by-name/recovery");
ui_print("Finished!");

Unrelated: Prevent Windows from messing around

This is a little private note on how to prevent the computer that runs Windows from accessing the real Internet. Because if it does, it starts an update frenzy.

I run this on the computer that is the gateway to the internet on the LAN, of course.

# iptables -I FORWARD -s 10.1.1.11 -j droplog

A TinyMCE plugin for cleaning up HTML copy-paste disasters

Introduction

This post is about a plugin I wrote for TinyMCE v5.10.2, which cleans up some mess that often occurs in the HTML code when copy-pasting text from the document to itself. It doesn’t cover copy-pasting from other sources, however the plugin is easily modified to wipe away all kinds of superfluous HTML. See this as boilerplate code.

Canonicalize plugin iconThe plugin adds a button to the toolbar with an icon of a cleaning mop, as shown to the left. When clicked, the selected text is cleaned up, or as I preferred calling it, canonicalized.

It’s a bit of work in progress. For now, it addresses two issues:

  • Unnecessary “style” formatting attributes
  • Break tags inside <pre> blocks

I’ll explain both of these issues, and then I’ll show the plugin, with source and explanations.

Note that the selection granularity is blocks (as in <p> and <div> segments). This means that if a word within a block is selected, all text in the block is canonicalized. I’ll explain why further down, but I wanted to have this on the table right away, as it may appear to be a bug. Anyhow, the way I will probably use this plugin is by selecting all text, so I doubt it will matter much. At least to me.

Also note that this plugin considers a “style” formatting attribute superfluous if the formatting of the enclosed text remains exactly the same even when it’s removed. Hence if there are multiple views of the same page (e.g. for printing, or a narrower viewport) it may remove formatting that would have made a difference on another view. It may be correct to remove that formatting nevertheless. I discuss this later too.

My anecdotal tests with Chrome and Firefox resulted in bytewise exactly the same HTML output code. But that can change in the future.

The superfluous “style” attribute problem

Rich text editing inside web browsers has the well-known problem that the same keyboard input can produce different HTML, depending on which browser (and which version) is used. But that’s small potatoes compared with what happens when text is pasted into the editing window.

The problem is that unless “Paste Text Only” is used, formatting attributes go along with the pasted text. Browsers can be pretty creative in how they turn these attributes into the DOM representation of the text, and consequently the HTML it produces once saved (for example, this thread and this too, both regardless of TinyMCE). This is kind-of understandable when the formatted text comes from an external source (a word processor or a web page), but it gets really obnoxious when junk formatting is inserted as a result of copy-pasting within the same editing window.

So yes, a plain copy-paste sequence from one part of the editing window to another can end up with things like:

<p style="font-family: verdana, Arial, Helvetica, sans-serif; font-size: 16px; font-weight: 400;">Text</p>

when the copied text was just

<p>Text</p>

Break tags inside <pre>

Another annoying thing is that when pasting into a preformatted block (that is, within a <pre> tag), newlines are sometimes translated into <br> tags. That’s completely unnecessary, ending up with a very long single line instead of HTML that is easily read as plain code.

Why do browsers add the “style” attribute?

What apparently happens is that the browser (Google Chrome in this case) sees importance in preserving the formatting of the text as copied, and adds an inline “style” attribute with the explicit definition of the font formatting. What’s really annoying is when the text would have been rendered exactly the same without this “style” attribute. That is, when the calculated formatting of the text part is exactly the same with and without it.

One may wonder why Chrome (and other browsers) doesn’t optimize away this unnecessary formatting assignment. The truth is that I don’t know, but I speculate that the rationale is that it’s not 100% clear whether the formatting is necessary or not. The thing is that the calculated formatting (the visible result) of the environment can be altered by virtue of CSS manipulations. For example, a different viewport, or even resizing the window can change the CSS definition of the document. Hence the fact that the pasted text happens to be formatted exactly like its environment doesn’t necessarily mean that it will stay so: The environment can change dynamically.

And that leaves us with the question: If the environment’s formatting changes, should the pasted text change along with it, or should it retain the format it had in its origin? If the pasted text is formatted the same as its environment, does it necessarily means it’s the same in essence?

That answer is very easy to answer in my case, since I don’t make any CSS manipulations. Hence if it looks the same, it is the same. Therefore if the inline style formatting doesn’t change anything, it should go away.

Selection with block granularity

How do you manipulate all DOM elements within a selection? Loop on all DOM elements inside it, of course. But if all text in a paragraph is selected, does it include the <p> tag that encloses it? And the last character is excluded from the selection, possibly by mistake, should the <p> tag be excluded or included?

This isn’t a philosophical question: Often the “style” attributes that need to be removed are placed in the <p> tag itself.

So I could have developed a clever algorithm for this (and fixed its bugs eternally), but I went for a simpler approach: Apply a block-level formatting for the entire selected region. The trick is to call formatter.apply with a format that is registered as block-level formatting, so TinyMCE wraps the entire selection with a <div> element defining a dedicated class. Now all that is left is to request all DOM elements that are descendants of this specific class. That’s a simple CSS filter.

However because this is block-level formatting, it applies to all text in the block, not just exactly the segment that has been selected. It’s like requesting a block quote: It won’t be done on part of the block.

One could ask why not use inline formatting, like I did with the syntax highlighting plugin. That would, in theory, generate a <span> element on the exact part of the selection, nothing else. But that wouldn’t work at all: If the selection includes more than one block, a <span> tag is inserted inside each of these blocks. Hence this method is guaranteed to miss <p> and other block tags that have “style” attributes that need removal.

So even if I wrote a clever algorithm, there would have been quirky behavior no matter what choices I would make. Applying the plugin to the entire block is relatively intuitive.

The source code

Without further ado, this is the JavaScript code for the said plugin.

(function () {
    'use strict';

    let canonicalize = function(api) {
      // First clean up possible leftovers
      editor.dom.select('div.canonicalizer_tmp').forEach(function(el) {
	editor.dom.remove(el, true);
       });

      editor.formatter.apply('canonicalizer_wrapper');

      // Replace <br> tags under <pre> with newlines
      editor.dom.select('div.canonicalizer_tmp pre br').forEach(function(el) {
        var newline = el.ownerDocument.createTextNode("\n");
	editor.dom.insertAfter(newline, el);
	editor.dom.remove(el, false);
      });

      editor.dom.select('div.canonicalizer_tmp *').forEach(function(el) {
        var style = editor.dom.getAttrib(el, 'style');

	if (!style)
	  return;

	var thewindow = el.ownerDocument.defaultView;
	var from_inline = editor.dom.parseStyle(style);
	var prestyle = thewindow.getComputedStyle(el, null);
	var before = { };
	var p;

	for (p in from_inline) {
	  before[p] = prestyle[p];
	}

	editor.dom.setAttrib(el, 'style', null); // Remove the attribute
	var after = thewindow.getComputedStyle(el, null);
	var survivors = { };

	for (p in from_inline) {
	  // If some style names should be removed anyhow, this is the place
	  // to look it up and issue a "continue" on a match.

	  if (before[p] === after[p])
	    continue;
	  survivors[p] = from_inline[p];
	}

	editor.dom.setAttrib(el, 'style', editor.dom.serializeStyle(survivors));
      });

      // Cleanup the <div> wrapper
      editor.dom.select('div.canonicalizer_tmp').forEach(function(el) {
        editor.dom.remove(el, true);
      });

      editor.undoManager.add(); // Make operation undoable
    }

    tinymce.PluginManager.add('canonicalize', function (editor) {
      editor.ui.registry.addIcon('canonicalize-icon', '<svg width="24" height="24" fill-rule="evenodd"><path d="M 5.86,22.51 C 5.51,22.44 3.79,21.81 3.41,20.46 3.03,19.11 3.38,16.66 5.80,13.83 8.23,10.99 9.54,10.29 10.70,10.15 10.72,9.79 11.26,9.23 12.12,9.48 12.67,8.38 19.15,0.70 19.69,0.00 20.10,-0.16 20.29,-0.27 20.76,0.00 19.25,2.28 13.81,9.44 13.14,9.95 13.46,10.27 13.69,10.74 13.47,11.10 14.08,11.86 13.74,13.06 13.00,15.04 12.39,16.57 10.84,19.72 15.22,19.71 15.04,20.17 14.85,20.60 13.57,20.74 13.97,21.05 14.21,21.32 14.64,21.52 14.04,21.50 13.08,21.68 12.27,21.21 12.54,21.77 12.79,22.06 13.03,22.30 12.07,22.21 11.08,21.96 10.53,21.55 10.54,22.06 10.66,22.43 10.77,22.69 9.90,22.51 8.41,22.07 7.69,21.46 7.74,22.44 7.81,22.27 7.93,22.63 6.30,21.91 6.29,21.75 5.59,21.07 5.68,21.84 5.68,22.05 5.86,22.51 Z M 5.92,14.23 C 5.92,14.23 6.92,13.04 7.05,12.91 7.74,14.46 9.31,14.87 12.82,14.54 12.50,15.40 12.33,15.66 12.28,15.81 9.93,16.07 7.12,16.52 5.92,14.23 Z" /></svg>');

      editor.ui.registry.addButton('canonicalize', {
	icon: 'canonicalize-icon',
        tooltip: 'Canonicalize (Cleanup unnecessary styles and <br>)',
        onAction: canonicalize
      });
   });
}());

For this to work, the string “canonicalize” should be added to the plugin property, as well as to somewhere in the toolbar property in the editor’s initialization (the tinymce.init call).

Also, the “init_instance_callback” property should be set to “mce_post_init”, so that the canonicalizer_wrapper format is registered during initialization with

function mce_post_init(ed) {
  editor.formatter.register('canonicalizer_wrapper',
    { block: 'div', classes: 'canonicalizer_tmp', wrapper: true } );
}

So we have something like

tinymce.init({
  selector: '#mytextarea',
  init_instance_callback : "mce_post_init",
  [ ... ]
  plugins: 'canonicalize [ ... ]',
  toolbar: '[ ... ] canonicalize [ ... ]',
  [ ... ]
});

Walking through the code

The canonicalize function starts with cleaning up possible <div> wrappers that may have be left from a previous messup. This should never be necessary, but better safe.

The call to editor.formatter.apply() applies the format defined as canonicalizer_wrapper to the current selection. Because this format is defined as “block” and has the “wrapper” property true, TinyMCE wraps the selected region with a <div> having the class “canonicalizer_tmp”. This is a block-level wrap, including the entire blocks that contain the selection.

The next thing is to replace <br> tags inside <pre>. The tags to manipulate are found with editor.dom.select(‘div.canonicalizer_tmp pre br’), i.e. by virtue of a plain CSS selection expression that says “find me all <br> tags that are descendants of <pre> tags, that are descendants of <div> tag with class “canonicalizer_tmp”. Or for short, inside the selection.

For each such <br> element found, a text DOM element containing a newline is created by the browser API’s createTextNode() (does TinyMCE have something parallel? I didn’t find any). This element is put after the <br> element, after which the latter is removed from the DOM with editor.dom.remove(). The second argument in this call, “false”, means that any sub-elements should be removed as well — but <br> shouldn’t have any, so this doesn’t matter much.

That was the easy part. Now to getting rid of useless “style” assignments.

Similar to before, editor.dom.select(‘div.canonicalizer_tmp *’) finds all DOM elements that are descendants of <div> tag with class “canonicalizer_tmp”. An anonymous function is called for each.

If the “style” attribute isn’t present at all, nothing is done for the element (“return” inside a looped function is like “continue” inside a loop).

Then a few preparations: “thewindow” is assigned with what is usually referred to as “window” in JavaScript, and odds are that the latter could be used anyhow. I use this form because it appears somewhere in TinyMCE’s internal code, so it may be an overkill, and maybe not.

Then “from_inline” is assigned with an object that is a key-value representation of the text in the “style” attribute. And then “prestyle” is assigned with the object that represents the browser’s computed style of the element. In other words, the key-value attributes of the formatting, as displayed, with keys like “font-size”, “font-family” and many many more. This is done by calling getComputedStyle() on the Window Object.

Next, there’s a loop on all attributes that were assigned in the DOM element’s “style” attribute — they are the keys of “from_inline”. In this loop, the “before” object is populated with keys taken from the “style” attribute, and values from the currently calculated formatting for that DOM element.

Note that stringwise comparing the value in “from_inline” with that of “prestyle” wouldn’t have helped much, because there’s more than one way to express formatting. For example, the value in “from_inline” could be “red” and in “prestyle” it would probably say “rgb(255,0,0)”. Different strings meaning the same thing.

Because there are gazillion key-value pairs in the browser’s computed style, a copy of the relevant ones is made, as a preparation for the next step:

The “style” attribute is removed from the DOM element, as to say: Now let’s see if that made a difference.

And here comes the most misleading part of this code: “after” is assigned with the value of getComputedStyle(). This is most likely unnecessary: The values in the “prestyle” object (previously returned by getComputedStyle() ) magically change as the DOM element changes, at least on Chrome. Hence I could have used the same object to check the formatting even after removing the “style” attribute.

However as far as I could tell, the documentation doesn’t say anything on this peculiar feature. So the bottom line is that “prestyle” can’t be relied upon to be updated after the change, but neither can be be relied upon to retain the values before the change in the DOM object. The easy way to solve it: Call getComputedStyle() again, after the change. That surely works either way.

That’s why the relevant key-value pairs are copied into “before” object prior the DOM change. It’s a simple object, and nothing fishy will happen with it. I nevertheless call getComputedStyle() again to get a fresh “after” object, just to be sure.

So after the removal of the “style” attribute, there’s a loop again on the keys of “from_inline”, this time comparing the computed value before the change, as copied into “before”, and the currently computed value. If they are different, the “survivors” object is assigned with the key-value pair as it appeared in the original “style” attribute.

Note the comment in this loop: If you want to wipe out certain style attributes whether they make a difference or not, just make sure that a “continue” is called when “p” equals their name, and you’re done.

After this loop, the DOM element’s “style” attribute is assigned with the translation to text (using serializeStyle() ) of the key-value pairs that were stored in the “survivors” object. If the object is empty, no “style” attribute is generated.

This concludes the loop for each DOM element in the selection. Prior to exiting, all “canonicalizer_tmp” <div> elements are removed. Note that the second argument of the remove() call method is true, which means that the children of this element should be retained. Or else all content in the selection would be wiped away.

Plus undoManager.add() is called to make the entire operation undoable.

Some extra notes

  • The plugin code doesn’t remove <span> elements that have no “style” nor “class” attributes. Such elements are pointless, and they may very well be the result of a <span> element that carried a “style” attribute that was removed throughout the process. As it turns out, TinyMCE removes these in the output given to getContent(), so they don’t appear in the saved document nor when viewing the source code inside the editor. In fact, they can be removed just by viewing the source code and then saving it. So I found it pointless to add an additional manipulation to fix a problem that practically doesn’t exist.
  • TinyMCE has a getStyle() method for obtaining the computed format. It’s based upon getComputedStyle() and does a whole lot of API friendliness stuff. I opted out mainly because I just needed a before-after check.
  • It may be possible to make the cleanup as the text is pasted, so there won’t be a need to do it explicitly. For example, this page suggests hooking on “paste_preprocess” to get a chance to clean up the content. By the same coin, maybe the “paste_postprocess” callback can be used to call canonicalize() every time new text is pasted, so it gets cleaned up immediately. The problem with this approach is the on-the-fly intervention, which can be confusing if something goes wrong. I prefer that witchcraft is done when I request it, so I have an eye on if something fishy happened. And by the way, this requires the Paste premium tinyMCE plugin.

Vivado: Failed to install all user apps

Every now and then Vivado whines with

[Common 17-356] Failed to install all user apps.

And every time I’m looking up how to to solve this, and then I find that the command that fixes this is

tclapp::reset_tclstore

in the Tcl command window. And then quit Vivado, and start it again. Why? I’ll never know.

Just wanted it written down where I’ll be looking for it.

Notes while deploying TinyMCE v5

Introduction

These are my notes as I adopted TinyMCE v5 for use as the editor for writing things to be published on the Internet. Like this post, for example.

My overall verdict is that it’s excellent, and I’m writing this after having added quite some unusual features to it. In particular, syntax highlighting where the both the segment and language are chosen manually. The API offered for writing custom plugins plays along with you. The examples in the docs are usually more or less copy-paste for the actual need.

I can’t say I felt very comfortable by the commercial attempt to make the impression that using tinyMCE requires a paid-for license to get a decent editor. The contrary is true. The free package covers functionality that is perfectly fine for comfortable editing. So if there’s anyone out there considering to adopt this editor for a business web venture, I warmly suggest getting down to the details, and in particular to understand the LGPL. Maybe the paid-for plans suit some scenarios, but keep in mind that Tiny’s website is quite misleading in this respect. Unfortunately, they’re really not alone.

Getting started

Download the self-hosted Dev Package from here (the Community package doesn’t have the non-minified JS file).

Create the following HTML file where the tinymce.min.js file is, inspired by this page:

<!DOCTYPE html>
<html>
<head>
  <script src="tinymce.min.js" referrerpolicy="origin"></script>
  <script type="text/javascript">
  tinymce.init({
    selector: '#mytextarea'
  });
  </script>
</head>

<body>
<h1>TinyMCE Quick Start Guide</h1>
  <form method="post">
    <textarea id="mytextarea">Hello, World!</textarea>
  </form>
</body>
</html>

and load that file.

Plugins

Official plugins are loaded from js/plugins/{name}/plugin.js (or plugin.min.js, depending on whether tinemce.js or tinymce.min.js was used, even though this can be changed). The {name} part is the name that appears in the call to tinymce.init(), in the “plugins” property.

The plugin files themselves contain a function enclosure that is executed when the plugin is loaded. That’s it. For example, this plugin.js just makes an alert window while the editor is being loaded. Useless, but makes the point:

(function () {
    'use strict';
    alert("Hello!");
}());

Apparently, TinyMCE just executes the plugin.js’ content, which in this case is a function enclosure that executes itself.

For this to work, the snippet above needs to be named plugin.js, and be in a directory whose name is listed in the “plugins” configuration parameter.

To get a really minimal example of a plugin, I suggest looking at hr/plugin.js. With the understanding that it just gives a function that is executed, it’s quite simple to figure out what it does.

It’s quite odd that TinyMCE’s way to load plugins is one file for each plugin — that is, one HTTP request for each plugin used. As there can easily be a dozen of these, it slows down the page’s load, and also increases the chance for a failure. I haven’t found an official solution for loading all at once, even though it’s quite easy to achieve. For example, concatenate all plugin files into a single file:

$ cat $(find plugins/ -name plugin.js) > all.js

And then make sure that the concatenated file is loaded along with the web page with a line like this in the <head> section:

<script src="/js/all.js"></script>

(or actually, virtually anywhere in the page). A smaller file can be obtained by using the .min.js files. And of course, this JavaScript file may also contain other things.

Note that the plugin’s name still needs to be listed in the ”plugins” configuration parameter. Using the concatenated script file just eliminates the attempt to load a dedicated plugin.js file from a URL that is based upon the plugin’s name. What counts is that the plugin name has been registered with tinymce.PluginManager.add(). If it has, the tinyMCE doesn’t bother trying to load it separately.

If loading the concatenated file fails, or it lacks plugins that are listed in the “plugins” configuration parameters, errors of this sort appear in the web console:

GET https://thesite.com/js/plugins/link/plugin.js net::ERR_ABORTED 404 (Not Found)
Failed to load plugin: link from url plugins/link/plugin.js

It’s worth mentioning that tinyMCE doesn’t initialize immediately on the call to tinymce.init(), so the script that registers the plugin can be executed after this call. Which is quite unexpected, but in the positive direction.

Creating icons (with GIMP)

Basically, draw anything in a square image. Then create a path (possibly by using Select > To Path) and then in the Path tab, right-click the path and choose Export Path to create an SVG file.

The SVG then needs editing: Remove everything in the <svg> tag except for width and viewBox. The latter doesn’t exist in MCE’s default icons, but it’s necessary to scale the coordinates to the target size. Change the width and height to 24.

In the <path> tag, remove everything except for the “d” assignment. If there are internal areas in the image (there usually are), a “fill-rule” attribute needs to be set to “evenodd”. Otherwise, the default “nonzero” is good enough.

The evenodd is added inside the <svg> attribute, so it terminates with something like

[ ... ] fill-rule="evenodd"/></svg>'

Then turn it all into a single line with this Perl one-liner:

$ perl -e 'local $/; $a=<>; $a =~ s/[ \n\r\t]+/ /g; $a =~ s/> </></g; print $a;' < myicon.svg

Then, register the icon with something like:

editor.ui.registry.addIcon('highlight', '<svg width="24" height="24" viewBox="0 0 256 256"><path d=" [ ... ] " /></svg>');

The coordinates of viewBox may vary, of course, depending on the original image size.

Adding items to the main menu

Apparently, there’s no dynamic way to add items to the main menu. It’s a bit understandable, since doing that would require to define where, within each drop-down of the menu, the item should go.

So the way to do it, is to define it on the init call’s configuration, typically by adding another header to the menu. So the configuration reads like this:

tinymce.init({
  [ ... ]
  plugins: 'myplugin [ ... ]',
  menubar: 'file edit view insert format tools table mine help',
  menu: { mine: { title: "Eli's extras", items: 'myitem' } },
  [ ... ]

And then the registration for both toolbar and menu goes (the addButton() call is here for reference, and isn’t relevant):

   editor.ui.registry.addButton('myitem', {
     icon: 'myicon',
     tooltip: 'Do this and that',
     onAction: doit,
   });

   editor.ui.registry.addMenuItem('myitem', {
     icon: 'myicon',
     text: 'Do this and that',
     onAction: doit,
   });

It’s also possible to define one of the built-in menu headers, but that’s not forward-compatible, because it requires listing all items that appear under that header. So now imagine upgrading the editor, expecting to find new features, and not finding them because that specific menu header is overridden by some JavaScript snippet you wrote years ago, buried somewhere.

General notes

Some random jots.

  • In the example code, the selector property is set to ‘#mytextarea’, which results in tinyMCE creating an iframe instead of the DOM element identified by the said ID. The main advantage of this is that the CSS of the edited area is separated from the surrounding page. To just edit any DOM element, add an “inline: true” property.
  • Either way, the edited area doesn’t have to be a <textarea>, nor does it have to be enclosed in a <form> if the data submission to the server is done by other means than a plain form submit (e.g. with AJAX).
  • The “body_class” property of the init object allows adding classes to the document enclosure. Using the same main class in the editor and the published document gives a true WYSIWYG effect.
  • Setting the editor’s width property is somewhat tricky: The specified width includes the border and the scrollbar if such exists. Hence it’s wiser to set it wider by 17 pixels (or 20 pixels to be on the safe side) and add a max-width CSS property to the editor window’s body tag. In the absence of a scroll bar, there will be little unused space to the right. This might become apparent when something is wide and centered, so it reaches both side’s edges. And only in the absence of a scroll bar, of course, which doesn’t happen much when editing for real.
  • The API for plugins is excellent. For operations directly on the DOM, use the DOMUtils API. It has rather advanced features like split().
  • Formatting: There’s also a useful API for generating custom formats.
  • There’s also an execCommand set of API calls, with something like
    editor.execCommand('mceInsertContent', false, '<hr />');

    The “false” in the command above, like in many other command usage examples shown on that page, goes to a function parameter named “ui”, apparently meaning that the call isn’t made by the user directly.

  • Don’t enable the “quickbars” plugin, unless you want annoying popups whenever you select anything, or just press ENTER for that matter.
  • The code that produces the menu and the toolbar, and its submenus, belongs to the theme, e.g. js/themes/silver/theme.js.
  • The editor isn’t really initialized when tinymce.init() returns. Among others, this means that attempting to obtain a handle to it with e.g. tinymce.get() will fail immediately after that. My workaround was to register a function for init_instance_callback.
  • getContent() returns the HTML with line breaks between block elements, which is a blessing (unlike innerHTML which is everything in one line).
  • Toggle buttons are quirky, maybe even buggy: The “active” property was ignored flat (or did I mess up?), and this is worked around with an “onSetup” callback using setActive(). Even more important, if the toggle button doesn’t change anything in the editing area, the focus will remain on the button itself, which makes it appear to be in the active state even when it’s not. This is solved with a simple editor.onfocus(false) in the onAction handler, so the focus is moved away from the button itself.
  • TinyMCE’s “change” event isn’t suitable for indicating to user that the document needs saving, since certain edits don’t fire it off. Some use the isDirty flag in function that is called recurrently, but I went for adding a mutation observer:
      function mutation_callback(mutationsList, observer) {
        document.getElementById("savestatus").innerHTML = '';
      }
    
      const observer_config = {
        attributes: true,
        characterData: true,
        childList: true,
        subtree: true,
      };
    
      let observer = new MutationObserver(mutation_callback);
      observer.observe(editor.getBody(), observer_config);

    Surprisingly enough, mutation_callback gets called back once after each edit operation, and isn’t bombarded as one might fear. This is because there’s a list of mutations that is passed over on each call (which is ignored in this case), so it’s one call for a lot of possible changes.

  • The DOM tree may get obscured due to formatting. For example, selecting a region within a text area and toggling bold on and off results in three consecutive text areas (in Google Chrome), that aren’t optimized into one. This is pretty harmless as it doesn’t have any visual effect. Opening the Source Code window and clicking “Save” fixes this, as the DOM is rebuilt from HTML. And of course, when the document is reloaded from HTML. So this is a temporary and invisible quirk.

And a couple of CSS related jots:

  • When there’s a <span> inside a <span>, class formatting of the inner span overrides that of the external one, regardless of anything. In particular, it may also override a CSS assignment made with !important. All formatting that isn’t declared explicitly by the span element itself is considered inherited format, and there’s no specificity calculation nor any other conflict resolution. All the relevant resolution rules apply only when several formats are imposed by the same DOM element.
  • When the exact same selector is used in a CSS file (same level of specificity), the one appearing later overrides the previous. Citing this site: “When multiple declarations have equal specificity, the last declaration found in the CSS is applied to the element”.

Random JavaScript jots

This is just a bunch of jots I’ve made about JavaScript, in no particular order. Posted here, because the alternative was to delete them.

  • JavaScript is single-threaded by its nature. Nothing runs in parallel, hence no need for any synchronization or atomics. There is actually an Atomic class, but for use in conjunction with shared memory (which is far away from browser JavaScript).
  • In Firefox, use the Web Console for spotting errors: Hamburger > Web Developer > Web Console, or just go Ctrl-Shift-K. Or F12. Make sure that the Errors button is enabled. Requests is also nice.
  • Chrome as well as Firefox have a JavaScript debugger which allows inserting breakpoints anywhere in the code as well as when when a DOM element changes. Including stack trace and the ability to watch the calling code. Just use the inspector and right-click on the DOM element, and select “Break on…”. Code breakpoints can be conditional, i.e. the user can write an expression so that the breakpoint is skipped if it isn’t true. Firefox’ debugger is somewhat unstable when resuming execution, so go for Chrome.
  • To figure out what code does what, set a breakpoint on a DOM element, and make it change.
  • Use console.log() to generate printf-style debug output. alert() is also available, but so 90′s.
  • There’s also a JavaScript debugger allowing to print out expressions: Same interface as the Web Developer.
  • In particular, enable “Pause on exceptions”, and possibly even “Pause on caught exceptions” (the latter on Chrome only, maybe?). It helps a lot.
  • JavaScript’s built-in keys() method returns an Array of keys in the order they were inserted.
  • Use “let” rather than “var” to declare variables, so that they are locally scoped (?).
  • When assigning callbacks, it’s better to insert execution code rather than to refer to a function, as this allows telling the reason for the call in the debugger’s stack trace.
  • [].slice.call(this.domNode.querySelectorAll(‘*’)).forEach(function(node): Runs the function on every node under this current position. See explanation.
  • (args) => { code } expressions: Shorthand for function (args) { code }
  • Node.js is a standalone JavaScript interpreter, typically appearing as /usr/bin/nodejs. It runs JavaScript files just like Bash, Perl, Python or other script languages.
  • npm is the Node.js package manager, allowing installation of JavaScript modules.
  • .ts and .tsx are extensions for TypeScript, an extension of JavaScript maintained by Microsoft.
  • React is a JavaScript library for building user interfaces.

Perl: Matching apparently plain space in HTML with regular expression

I’ve been using a plain space character in Perl regular expressions since ages, and it has always worked. Something like this for finding double spaces:

my @doubles = ($t =~ / {2,}/g);

or for emphasis on the space character, equivalently:

my @doubles = ($t =~ /[ ]{2,}/g);

but then I began processing HTML representation from the Mojo::DOM module (or TinyMCE’s output directly) and this just didn’t work. That is, \s detected the spaces (with Perl 5.26) but the plain space character didn’t.

As it turns out, TinyMCE put &nbsp; instead of the first space (when there was a pair of them), which Mojo::DOM correctly translated to the 0xa0 Unicode character (0xc2, 0xa0 in UTF-8). Hence no chance that a plain space, i.e. a 0x20, will match it. Perl was clever enough to match it as a whitespace (with \s).

Solution: Simple. Just go

my @doubles = ($t =~ /[ \xa0]{2,}/g);

In other words, match either the good old space or the non-breakable space.

A TinyMCE plugin for integrating Highlight.js

Introduction

In this post I describe how I integrated the Highlight.js syntax highlighting tool into TinyMCE v5.10.2 for the sake of highlighting code while writing blog posts. So one gets the button saying “HL” as shown here:

Example of Highlight.js plugin in TinyMCE

I’ve taken a somewhat different approach than usual, mainly in three aspects:

  • TinyMCE has a capability to insert code samples, and that includes syntax highlighting. Not good enough though, mainly because formatting can’t be added on top. Besides, the variety of languages is very narrow compared with highlight.js.
  • The syntax highlighting is a manual formatting operation, exactly like using “bold” or “italics”: The user selects the code to be highlighted, and presses a button at the toolbar.
  • The language is chosen manually.

So first, why not keep the highlighting script running periodically? Well, to begin with, it’s not an advantage when writing a blog post. I never write any code in my posts directly, but rather copy-paste it into the editor window from some other source. After all, it’s supposed to be code that runs, so I’ve obviously tested it somehow. Hence it’s already written somewhere.

On the other hand, continuous highlighting is an obstacle for writing blog posts: I often mark important parts in bold and red, and less important parts in grey. So the formatting made for syntax clarification is often just a starter. Letting the highlighter run uncontrolled would most likely mess up the manually added formatting.

As for selecting the language manually: Obviously, I know which language the code is written in, so why let an algorithm guess it? It may very well miss, in particular when the snippets are short — and in a blog post they often are.

Also, it’s not uncommon that a chunk of code contains more than one language: JavaScript code may contain snippets in HTML, and Perl code may contain snippets in Verilog. So it makes perfect sense to copy-paste the piece of code into the editor’s window, and then select each code segment and highlight it, each with its own language. And then I’m free to go in with colors to mark the important parts.

I’m not saying everyone is wrong. Automatic and periodic highlighting is great for websites with “try it out” windows for experimenting with HTML, JavaScript, CSS etc. This is not my case however.

And one more thing: I’ve deliberately chosen not to turn off spell checking, mainly because it catches spelling mistakes in comments.

This post is organized as follows: First a few words about getting started with Highlight.js. Then an example of integrating Highlight.js for use with automatic language detection (as this covers the main principles). And finally, setting it up to select the language manually.

The code examples in this post were, of course, highlighted with the plugin itself, so it has already one very satisfied user.

Getting started with Highlight.js

As mentioned, the chosen syntax highlighter in JavaScript was Highlight.js v11.3.1. The default set of languages is however web development oriented, and I need a much wider set. No problem, just go to the download page and tick whatever language is needed, and then download a custom zip file.

The zip file is however slightly misleading: Even though it contains quite a few files, the only ones that are needed are highlight.min.js at the zip’s root directory, and one of the CSS files in styles/. The highlight.js file should not be used, as it’s probably the core script without language support. So it does nothing good.

This way or another, there’s nothing to build from this bundle.

Anyhow, I’d mention that adding and removing languages can be done by editing highlight.min.js directly. It actually looks like the download site’s mechanism for supporting languages merely consists appending language related snippets from the language/ subdirectory to some core functions file.

Just look for comments saying something like

/*! `makefile` grammar compiled for Highlight.js 11.3.1 */

The part until a similar comment is the code that installs support for the said language. So cut and paste as necessary.

Note that the language/ directory contains all languages, not just those requested. So save the zip file in a safe place for the possibility to add a language later on without worrying about version compatibility.

There was only one little problem: As I selected which languages to support, I pretty much clicked on anything that I thought might come handy, but it turns out that the module for Python caused Firefox to fail loading highlight.min.js with

SyntaxError: invalid identity escape in regular expression

in the console. Chrome had no such problem. I’ll leave it open if it was a coincidence that Python was somehow related to a failure with a cryptic error message. As the image above implies, I opted out Python.

To give it a test run, I uncompressed the zip file into a directory named highlight/ relative to the file loaded into the browser, and then this correctly highlighted Verilog code:

<link rel="stylesheet" href="highlight/styles/stackoverflow-dark.min.css">
<script src="highlight/highlight.min.js"></script>
<script>hljs.highlightAll();</script>

<pre><code class="language-verilog">
... Verilog stuff ...
</code></pre>

This is the simplest use case: Let Highlight.js look up <pre><code> segments and highlight them automatically. But then, I chose the language manually by virtue of using the language-verilog class (this is not necessary, though).

Simple highlight TinyMCE plugin

So this is the code for the highlighting plugin with automatic language selection. Note that this is just code that is loaded along with the HTML, the good old way, even though it could be used in the common plugin framework (I guess, haven’t tried it).

(function () {
    'use strict';
    var highlighter = function() {
      // First clean up possible leftovers
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
	editor.dom.remove(el, true);
      });
      editor.formatter.apply('highlight_wrapper');
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
        let text = el.innerText;
	let hl = hljs.highlightAuto(text);
	editor.dom.setHTML(el, hl.value);
	editor.dom.remove(el, true);
      });
      editor.undoManager.add(); // Make highlighting operation undoable
    }

    tinymce.PluginManager.add('highlight', function (editor) {
      editor.ui.registry.addIcon('highlight', '<svg width="24" height="24" viewBox="0 0 256 256"><path d="M 57.00,62.00 C 57.00,62.00 41.58,147.00 41.58,147.00 41.58,147.00 34.00,189.00 34.00,189.00 34.00,189.00 50.00,189.00 50.00,189.00 57.50,188.87 56.70,186.77 58.75,175.00 58.75,175.00 63.58,148.00 63.58,148.00 64.08,145.21 64.56,139.55 66.58,137.60 68.58,135.67 72.40,136.01 75.00,136.00 75.00,136.00 111.00,136.00 111.00,136.00 111.00,136.00 102.00,189.00 102.00,189.00 102.00,189.00 124.00,189.00 124.00,189.00 124.00,189.00 133.58,135.00 133.58,135.00 133.58,135.00 141.58,91.00 141.58,91.00 141.58,91.00 147.00,62.00 147.00,62.00 147.00,62.00 131.00,62.00 131.00,62.00 123.09,62.15 124.37,63.75 122.00,76.00 122.00,76.00 117.42,101.00 117.42,101.00 116.92,103.79 116.44,109.45 114.42,111.40 112.42,113.33 108.60,112.99 106.00,113.00 106.00,113.00 70.00,113.00 70.00,113.00 70.00,113.00 79.00,62.00 79.00,62.00 79.00,62.00 57.00,62.00 57.00,62.00 Z M 174.00,62.00 C 174.00,62.00 158.58,147.00 158.58,147.00 158.58,147.00 151.00,189.00 151.00,189.00 151.00,189.00 204.00,189.00 204.00,189.00 204.00,189.00 210.41,187.98 210.41,187.98 210.41,187.98 212.61,182.00 212.61,182.00 212.61,182.00 215.00,167.00 215.00,167.00 215.00,167.00 177.00,167.00 177.00,167.00 177.00,167.00 189.42,99.00 189.42,99.00 189.42,99.00 196.00,62.00 196.00,62.00 196.00,62.00 174.00,62.00 174.00,62.00 Z" /></svg>');

   editor.ui.registry.addButton('hljs', {
     icon: 'highlight',
     tooltip: 'Syntax highlight selection',
     onAction: highlighter,
   });
 });
}());

It relies on the registration of a special class, named highlight_tmp, with this code snippet:

function mce_post_init(editor) {
  editor.formatter.register('highlight_wrapper',
    { inline: 'span', classes: 'highlight_tmp', exact: true } );
}

A few words about the “exact” property. This is required to ensure that a separate <span> tag is created, and that it isn’t merged with another <span> tag that happens to cover the same segment. This can be the case in particular if there’s a single <span> that covers a block element end to end. The “exact” property ensures that if this happens, applying the format creates a <span> within each block, and hence there are two nested <span>’s that overlap exactly.

Merging must be prevented, because the highlight_wrapper format is used merely to isolate the selection, and the <span> element carrying it is removed after the highlighting operation completes. If it has been merged with another <span> element, the formatting the other <span> carried is removed as well.

The call to editor.formatter.register() has to be run after the editor has been fully initialized. Some do this by virtue of a timeout callback, but the clean way for doing this is within a function that is called explicitly when the editor is up and running, mce_post_init() in this case.

This function is declared in the configuration of the editor, so let’s discuss it now (only the parts relevant to highlighting are shown):

tinymce.init({
  init_instance_callback : "mce_post_init",
  content_css: [ 'css/highlight.css', '/js/skins/content/default/content.css'],
  plugins: 'highlight [ ... ]'
  toolbar: '[ ... ] hljs [ ... ]',
  [ ... ]
});

For the plugin to work, it must be listed in the plugins property as shown above. Note that it’s not loaded from a separate file, since it has already been registered by the time the editor is looking for it.

Also, a button needs to be placed in the toolbar, by virtue of the “toolbar” property. And then, the content_css needs to be set up so it includes the classes for highlighting. This can be any of those supplied by Highlight.js’ zip file, for example. The CSS file under /js/skins/ is given explicitly here, as this was the default in my case. When content_css is defined explicitly, the default is dropped.

And of course, the highlighting script must be loaded with something like this in the <head> section of the HTML:

<script src="js/highlight.min.js"></script>

There’s no point loading the relevant CSS file here if the editor is placed in an iframe, which it usually is (unless inline mode is selected).

Note that the icon is given explicitly in text as an SVG. This is quite common, actually. The icon is just the letters “HL” (not very inspirational, but fits nicely).

Simple highlight plugin explained

The actual work is done in this part:

      editor.formatter.apply('highlight_wrapper');
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
        let text = el.innerText;
	let hl = hljs.highlightAuto(text);
	editor.dom.setHTML(el, hl.value);
	editor.dom.remove(el, true);
      });

First, the formatter.apply() method is called to wrap the selected region with a <span> pair of tags, so that the relevant area is isolated and with a DOM element that can be referred to. Note that if there happened to be leftovers from a previous operations (an exception occurred?) they have already been removed by the code that comes before the snippet above.

At this point, a loop runs on all DOM elements having the highlight_tmp class, fetching their pure-text representation with innerText, calling highlightAuto() on this text, and then setting the HTML of the DOM element to the result of this call.

This is followed by a call to dom.remove(), which removes the DOM element, moving all children to the parent (this is what the “true” argument means). In HTML terms, this removes the <span> tags, but leaves anything between them.

At the end of the loop, there’s

editor.undoManager.add();

which adds an undo stage after the highlighting is in place. Without this, a CTRL-Z undoes the step before applying syntax highlighting as well.

Answers to questions nobody asked

First, a couple of notes:

  • If no region is selected, but the cursor is in the middle of a word, that word is highlighted — just like “bold” would do. Only at the end of a line, nothing happens (or more precisely, an empty string is highlighted).
  • No menu item is added on behalf of this plugin, because adding a menu item to an existing header requires redefining it from scratch, but then what happens if newer items are added by core TinyMCE in future versions? So the correct way to do it is to add another menu header, and that’s a bit too high much.

Now the question: Why wrapping the selection with a bogus class, and then remove it? Why not just grab the text, highlight, and inject it back? Like this?

let text = editor.selection.getContent({format : 'text'});
let hl = hljs.highlightAuto(text);
editor.execCommand('mceInsertRawHTML', false, hl.value);

This works quite nicely, however if an entire <pre> region is selected, the result is that the <pre> tag is dropped, and the highlighted code appears in a plain paragraph format, newlines turned into spaces, so it’s a mess. This happens because when the entire chunk is selected, the surrounding format is also part of the selection. Hence when it’s substituted, the <pre> is dropped as well.

The next question would be why there’s a loop on DOM elements with the class. Why would there be more than one? Why not defining the wrapper by an ID (assigned randomly), with something like

{ inline: 'span', attributes: { 'id': '%id' } }

and then wrapping and lookup of the DOM element would go

let dom_id = "highlight_" + Math.floor(Math.random()*1000000000);
editor.formatter.apply('highlight_wrapper', { id:  dom_id } );
let el = editor.dom.get(dom_id);

instead?

So in what case would there be more than one DOM element to process? The answer is that normally that won’t happen, but if a few paragraphs are chosen (like when pasting multi-line code into a non-<pre> context), formatter.apply() creates a separate <span> for each paragraph (more precisely, for each block element, since inline tags like <span> don’t cross block elements). Which is actually good, because the result is what one would expect.

So even though it’s bit odd to use highlighting not inside a <pre> section, the non-loop version above won’t process nor clean up more than one wrapper. That alone is a good reason to do it in a loop: Even if the user does something unexpected, no junk should be left in the document.

Ehm, I need to correct myself slightly on this: Apparently, sometimes when pasting plain text into the editor window (inside the <pre> region), <br/> tags are inserted instead of newlines. Not clear why, and it’s ugly, but with the loop the highlighting works regardless.

And lastly, is it OK to use innerText, despite its definition being pretty vague saying “it approximates the text the user would get if they highlighted the contents of the element with the cursor and then copied it to the clipboard”. For text inside a preformatted element, it looks safe enough. Maybe using TinyMCE’s API would be better still.

Manual language selection plugin

The code for the manual language is interesting in particular because it demonstrates a split button, i.e. a button that can open a selection drop-drop menu (as shown in the image above). It’s worth mentioning that only the plugin code itself has changed — the configuration in the call to tinymce.init() remains the same.

So here it is:

(function () {
    'use strict';
    // Fetch all languages and create an item list

    var langname = {};
    var items = [];
    var langlist = hljs.listLanguages();

    langlist.forEach(function(codename) {
      let l =  hljs.getLanguage(codename);
      langname[codename] = l.name;
    });

    langlist.sort(function(a, b){
      let x = langname[a].toLowerCase();
      let y = langname[b].toLowerCase();
      if (x < y) return -1;
      if (x > y) return 1;
      return 0;
     });

    var lang = langlist[0]; // Default to first listed language

    langlist.forEach(function(codename) {
      items.push({
        type: 'choiceitem',
        text: langname[codename],
        value: codename,
      });
    });

    // Prepare the GUI stuff
    var highlighter = function() {
      // First clean up possible leftovers
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
	editor.dom.remove(el, true);
      });
      editor.formatter.apply('highlight_wrapper');
      editor.dom.select('span.highlight_tmp').forEach(function(el) {
        let text = el.innerText;
	let hl = hljs.highlight(text, {
          language: lang,
	  ignoreIllegals: true,
        });
	editor.dom.setHTML(el, hl.value);
	editor.dom.remove(el, true);
      });
      editor.undoManager.add(); // Make highlighting operation undoable
    }

    tinymce.PluginManager.add('highlight', function (editor) {
      editor.ui.registry.addIcon('highlight', '<svg width="24" height="24" viewBox="0 0 256 256"><path d="M 57.00,62.00 C 57.00,62.00 41.58,147.00 41.58,147.00 41.58,147.00 34.00,189.00 34.00,189.00 34.00,189.00 50.00,189.00 50.00,189.00 57.50,188.87 56.70,186.77 58.75,175.00 58.75,175.00 63.58,148.00 63.58,148.00 64.08,145.21 64.56,139.55 66.58,137.60 68.58,135.67 72.40,136.01 75.00,136.00 75.00,136.00 111.00,136.00 111.00,136.00 111.00,136.00 102.00,189.00 102.00,189.00 102.00,189.00 124.00,189.00 124.00,189.00 124.00,189.00 133.58,135.00 133.58,135.00 133.58,135.00 141.58,91.00 141.58,91.00 141.58,91.00 147.00,62.00 147.00,62.00 147.00,62.00 131.00,62.00 131.00,62.00 123.09,62.15 124.37,63.75 122.00,76.00 122.00,76.00 117.42,101.00 117.42,101.00 116.92,103.79 116.44,109.45 114.42,111.40 112.42,113.33 108.60,112.99 106.00,113.00 106.00,113.00 70.00,113.00 70.00,113.00 70.00,113.00 79.00,62.00 79.00,62.00 79.00,62.00 57.00,62.00 57.00,62.00 Z M 174.00,62.00 C 174.00,62.00 158.58,147.00 158.58,147.00 158.58,147.00 151.00,189.00 151.00,189.00 151.00,189.00 204.00,189.00 204.00,189.00 204.00,189.00 210.41,187.98 210.41,187.98 210.41,187.98 212.61,182.00 212.61,182.00 212.61,182.00 215.00,167.00 215.00,167.00 215.00,167.00 177.00,167.00 177.00,167.00 177.00,167.00 189.42,99.00 189.42,99.00 189.42,99.00 196.00,62.00 196.00,62.00 196.00,62.00 174.00,62.00 174.00,62.00 Z" /></svg>');

    editor.ui.registry.addSplitButton('hljs', {
      icon: 'highlight',
      tooltip: 'Syntax highlight selection',
      onAction: highlighter,
      onItemAction: function (api, value) {
        lang = value;
	highlighter();
      },
      fetch: function(callback) { callback(items); },
      select: function(x) { return x === lang; },
    });
 });
}());

The larger part of the change is at the beginning of the plugin’s code, which sets up the list of items for the drop-down meny by calling hljs.listLanguages() and then obtaining the human-friendly name of the languages.

Next, hljs.highlightAuto(text) is replaced with hljs.highlight(text, { … } ), so that the language is given explicitly as the @lang local variable.

And finally, addSplitButton() is called instead of addButton(), with the same properties plus:

  • onItemAction supplies the function that is called when a menu item is selected. Not surprisingly, is sets @lang to the value of the selection (the “value” property in the item entry) and then calls highlighter() to highlight the relevant chunk.
  • fetch supplies a function that is called for supplying the items in the drop-down list. It’s called every time the drop-down list is opened, so the content of this list can change from one time to the other. For this plugin, the same array is supplied each time, as the language list won’t change.
  • select supplies a function that evaluates true for the list item next to which a check mark should appear to indicate that it’s selected.

Summary

I have to admit that I’m quite impressed by how TinyMCE’s API was right on spot with supplying the right functions to get the job done easily. Note how short and concise the code for the automatic highlighter is. And the manual language plugin got most of its weight from the code that plays around with languages, and not setting up the GUI.

Quill: The DOM, Parchment, and Delta views of the document

Everything in this post relates to Quill v1.3.7. I don’t do web tech for a living, and my knowledge on Quill is merely based upon reading its sources. Besides, I eventually picked another editor for my own use.

The trio

The most remarkable thing with the Quill editor, is that it constantly retains three representations of the edited document, all of which are in sync with each other:

  • The DOM, which is the browser’s view on the visible edit window
  • The Parchment, which is Quill’s parallel representation to the DOM (kept as the quill.scroll object)
  • The Delta, which is a sequence of editing operations for creating the document from scratch (kept as quill.editor.delta). This can be though of as what to type, from top to bottom, to get the document that is currently in the editor window.

This post discusses these three representations and how the relation among them is maintained.

It’s worth mentioning that apparently Quill was conceived with the idea that the Delta representation should be used exclusively, instead of the HTML code. In other words, it was envisioned that the web page for publishing would load the Delta from the server in JSON format (or something of that sort), and then a Quill parser, running on the browser as JavaScript code, would extract it into the DOM representation for view.

This is probably the reason Quill’s formal API doesn’t provide a means to obtain the HTML of the document.

Positions: index and length

Even though not directly related to either of these three, it’s worth mentioning that there’s also an index / length representation, in which the document is viewed as a linear sequence of characters.

Formatting (e.g. bold, italics, links) doesn’t occupy any width, however a new paragraph has a length of one, on behalf of the newline that it results from (even though it isn’t included in the DOM, and hence not in innerText). Embedded objects, such as images, are also considered to have a length of one.

The index and length are used extensively in Quill’s API as well as internal machinery to define positions inside the document.

Quill and DOM

As the document is edited, it’s of course shown in the browser. Hence it necessarily has a DOM representation in the browser, as a subtree of the DOM element that encloses the editing area (kept as the quill.root property). There is nothing special about this area, in fact, except that it’s editable and manipulated by Quill.

It’s however worth to make a quick recap on how the DOM tree is structured. So without going into the gory details, one can say that the DOM’s tree is a reflection of the HTML that would represent it properly: Whenever there’s a tag that needs a terminator (e.g. <p>, <div>, <b>, <a>), the node belonging to the tag becomes the parent for everything between the tag and its terminator. The siblings in this subtree are of course placed in the same order they appear in this theoretic HTML document.

Note that the tree structure doesn’t imply the graphical packing of the elements: A <div> tag creates a subtree containing everything that is within its vertical limits, but <b> does nothing of that sort. On the other hand, <br> and <hr> tags don’t generate any subtree, but do influence vertical packing.

Another thing to be aware of is that superfluous tags create DOM nodes as they appear in the originating HTML. So if the HTML says <b><i><b>Text</b></i></b>, the superfluous, internal <b> tag creates a subtree. There’s no such optimization when building the DOM.

The Parchment

The Parchment is a tree that mimics the DOM’s tree structure quite accurately, but is based upon a completely different set of classes. Accordingly, the objects in the Parchments, the blots, have different properties and hence mostly contain different kinds of information.

There are differences between the structure of these two trees only where the Cursor blot is inserted (see a separate post on this) or in some other cases that go beyond plain document editing (e.g. when something that Quill can’t digest has been pasted into the editing area, in which case that chunk becomes uneditable, and hence the relevant subtree isn’t covered in the Parchment in detail). Other than that, a difference in the tree structure indicates a bug somewhere, most likely in some add-on module.

As mentioned before, the Parchment for a document is given as quill.scroll. The nodes in the Parchment is called Blots, which is a collective name for the nodes in the Parchments (i.e. the JavaScript objects) as well as the classes of these objects. Each node has a .next and .prev property pointing at its next and previous sibling (possibly null when there’s no such). When a node has children, .children.head points at the first child (and likewise .children.tail at the last). Otherwise these are null.

Even more interesting, each blot also has a .domNode property, which points at the DOM object it represents. Conversely, each DOM object inside the editor’s area has __blot property, so that __blot.blot points back to representing blot (except for those DOM nodes that are not covered).

The important difference between a blot and DOM node it represents, is that blot classes represent the intention of the graphical elements they generate. For example, an editor may be customized to support two kinds of links: To sites of type A and sites of type B. They should have different formatting and possibly attributes. By representing each link type as different blot classes, the user creates them with different buttons, possibly with different UI for feeding their details, and the formatting (i.e. selection of the correct class) is done automatically. Nevertheless, both blots end up as a <a> DOM element.

The vast majority of functions and methods in Quill operate on the Parchment tree and the DOM tree simultaneously. The low-level methods that relate to the Parchment tree separately are implemented as the Registry class.

The Delta format

Delta is a serialized, and completely different way to look at the document. Note that the Delta format is also used to represent differences and changes, which is discussed further down. For now, I’ll focus on Delta as a representation of the entire document.

The Delta object, which is kept as quill.editor.delta, but should be obtained by application with the the quill.getContents() method, consists of a single element, an array called ops. This array consists of the sequence of operations required to reproduce the document, starting from an empty one.

So the document isn’t described in terms of its structure, but how it would have been typed into the editing window from beginning to end. It’s worth to try out the live editor example at the bottom of this page which shows the Delta object side by side (and also explains the format in more detail).

The said ops array contains only insertion operations: Each operation is represented with an object, which has at least one property, called “insert”. If it contains a string, that’s the text to add at the specific point. Newlines in this string are newlines as typed on keyboard. These end up as some block blot (paragraph, header etc. depending on the context).

If the “insert” property is an object, that requests the generation of blot. The name of the property of this object is the class of the blot, and the value of the property is that value to be assigned as the “value” of the blot object.

An insert operation may also have another property, “attributes”, which is an object. Its properties modify the insert op in a variety of ways: The font, color and also a “link” property turns the inserted element into a link.

There is hence a fundamental difference between how text is represented in Delta format vs. with HTML and the DOM. In the latter case, it goes “bold starts here, text, bold ends here”. With Delta, it goes “this is a segment of uniformly formatted text, and complete description for the formatting is this and that (among others, bold)”. So the textual parts of the document are chopped into chunks with uniform formatting, and they may span several lines.

To produce a pretty-printed JSON string of the document in Delta format:

JSON.stringify(quill.getContents(), null, '  ');

Formatting in Delta ops

It may come counterintuitive that a link is a formatting of the text segment, but when considering that links are almost always segments with uniform formatting, it turns out to be the natural solution: The link is just an attribute of the text.

Another confusing thing might be that a header (as in <h1>Header</h1>) is inserted as two ops: The first is the text, and the second is just a “\n” insert with the attributes object containing a “header” property giving the rank of the header (i.e. <h3> gets @header 3). This is because attributes that relate to blocks are applied only to the newline character(s) in the the text, controlling which block-level blot is the parent of the text before the newline.

It also goes along with the fact that Quill ingests pasted HTML by traveling through the DOM of the pasted text in post-order, meaning that the children of a parent are scanned from left to right, and then the parent. Hence when scanning e.g. “<h1>This is <i>important</i></h1>” it goes “This is”, “important”, italic tag, header tag. Those used to RPN calculators will find this familiar.

Basic API for manipulating the document

The Quill API (mainly) supplies two methods for inserting things into the document: insertText() and insertEmbed().

insertText() is exactly like typing the text with the keyboard. Newlines (“\n”) are treated like pressing Enter. For example, if the text is inserted inside a bulleted list, a new line and bullet are created, exactly as pressing Enter would.

Inserting text with updateContents(), where plain text is given in the .insert property works completely differently, because the information is treated as Delta operations. For example, a newline in a Delta op may appear to have a quirky behavior unless the Delta format is properly understood.

Another aspect of updateContents() is that it doesn’t respect surrounding formatting. So for example, if insertText() adds text where the context is bold, the added text will be bold too. updateContents() will add non-bold text in this case, unless the Delta has been set up to generate bold text. Every “insert” entry in a Delta op lists all formatting that should be applied explicitly, regardless of the surroundings.

All calls to updateContents() relate to the beginning of the document. In order to reach the place to manipulate, “retain” is used to skip to that position (using index metrics) and possibly “delete” to remove parts, as described in the API page.

In summary: insertText() is like typing, and updateContents() injects blots and text directly, with possibly counterintuitive behavior.

Several manipulation methods are listed in the Partchment’s API, in particular insertAt(), formatAt() and deleteAt(). The first too are used directly by insertText() (see core/editor.js) however these shouldn’t be used directly except for when implementing blots and other internal functionality, since they don’t update the Delta view of the document.

For usage as a replacement for insertText() and friends, remember that these are methods of the Parchment and not Quill, so a typical call would go

quill.scroll.insertAt(index, text);

It’s also possible to call these methods on any blot, however note that the index is then related to the blot’s beginning. This is in fact the case with calling the Scroll object too, since its zero index is the beginning of the document.

As mentioned in a separate post of mine, formats are in principle divided into Inline and Block formats. formatText() works with the Inline formats only (or with the Block formats when targeting the newline character), and formatLine() only with Block formats. format() checks if the format description is in the Block or Inline group, and delegates the call to formatLine() or formatText() accordingly (see core/quill.js).

So this snippet turns the selected part into red font, and gets the line in which the selection (or cursor) is included into a block quote:

quill.format("blockquote", "true");
quill.format("color", "red");

These are the functions that are called by the toolbar, so using them in a script is equivalent to that. Note however that formatLine() or formatText() allow changing any place in the document, not just where the selection is.

Assigning innerHTML directly

There are certain situations, where it’s easier to assign a DOM object’s innerHTML directly, as a quick and somewhat dirty way to update the document’s content. One use for direct innerHTML assignment is integration with Highlight.js’ module in syntax.js. It’s also a possibility for ugly hacks instead of modifying the document with Quill’s API. If you choose to do so, kindly do not refer to this post on where you got the idea from.

If and when an assignment is made to innerHTML anywhere in the editor’s area, the related blots are updated to follow suit, and the internal Delta representation is updated immediately as well. This happens as a result of the browser reporting a change (mutation) to Quill, and consequently the synchronization takes place, as explained in the next section.

This doesn’t affect blots that are away from the DOM hierarchy that was affected by the innerHTML update. In effect, this means that if these blots have object properties that are not reflected in the DOM object, they are retained nevertheless: It’s not like the entire document is refreshed from the DOM.

Hence it’s OK to hold hidden information in the blots’ objects, as long as their relevant DOM elements aren’t updated with an innerHTML assignment.

Synchronizing the Parchment and DOM with update()

In principle, the editor window is managed by the browser. In order to keep the Parchment in sync with the editor window’s content, Quill registers itself to listen to several events involving pressing keyboard keys, pasting etc. On top of that, the ScrollBlot class, which is what the top-level scroll blot is made from, registers itself as follows during its construction (from parchment/src/blot/scroll.ts):

    this.observer = new MutationObserver((mutations: MutationRecord[]) => {
      this.update(mutations);
    });
    this.observer.observe(this.domNode, OBSERVER_CONFIG);

The MutationObserver class is defined by the browser. The result of this registration is that when the browser makes any change in the editor’s window, the update() method is called with the mutation array as provided by the browser. Each entry in this array defines which DOM element has changed, and how.

Note that this doesn’t relate just to direct changes of innerHTML, but to the vast majority of user edits on the document.

The update() method that is defined in the same class (and file) goes

  update(mutations?: MutationRecord[], context: { [key: string]: any } = {}): void {
    mutations = mutations || this.observer.takeRecords();
    // TODO use WeakMap
    mutations
      .map(function(mutation: MutationRecord) {
[ ... ]
      })
      .forEach((blot: Blot | null) => {
[ ... ]
      });
[ ... ]

It loops on the array of mutations, finds the corresponding blot object for each DOM object, and calls its update() method. This allows the blot object to update itself, possibly by changing its attributes and content, or update its subtree structure to match the updated DOM tree (see update() method in src/blot/abstract/container.ts).

Note that if update() is called with no arguments, takeRecords() is called to fetch any pending mutation records from the browser. This ensures that when update() returns, any changes in the DOM have been registered in the Parchment, and hence they are in sync.

It’s important to note that this mechanism covers only changes to the DOM that are initiated by the browser, e.g. when typing text or when text is pasted. Changing the selection, pressing the Enter or Delete key initiate events that are handled otherwise — this is handled by the Keyboard module. Quill calls update() when such events involve changes in the Parchment and/or DOM, typically calling quill.update() defined as follows in core/quill.js:

  update(source = Emitter.sources.USER) {
    let change = this.scroll.update(source);   // Will update selection before selection.update() does if text changes
    this.selection.update(source);
    return change;
  }

As this call is made without any mutations, the purpose of this call is to ensure that the Parchment is in sync with the DOM.

Looking at insertText()

As calling insertText() is equivalent to typing text manually, it’s worth looking at its simple implementation to get an idea how Quill processes input. This is contrary to the complicated handling of interactive input.

This function is defined in core/quill.js, and essentially calls the editor object’s insertText method, which is defined as follows in see quill/core/editor.js:

  insertText(index, text, formats = {}) {
    text = text.replace(/\r\n/g, '\n').replace(/\r/g, '\n');
    this.scroll.insertAt(index, text);
    Object.keys(formats).forEach((format) => {
      this.scroll.formatAt(index, text.length, format, formats[format]);
    });
    return this.update(new Delta().retain(index).insert(text, clone(formats)));
  }

What this demonstrates is that insertAt() is called to insert text into the DOM and Parchment. formatAt() then adds formatting as required, once again affecting both DOM and Parchment.

But then a Delta object that represents this change is generated, and the update() method is called with it. Note that “this” refers to the editor object, so this.update() is the method of the Editor class, and not the Scroll class. This is important, because the Editor’s class’ implementation of update() is completely different: Unlike the Scroll class’ implementation, which updates the Delta according to the Parchment, the Editor’s class implementation updates the Delta by manipulating Delta ops only.

This maintains the parallel view of the document in the quill.editor.delta.ops array. If this isn’t done properly, the Delta structure that is then used to save the document won’t match what’s seen on the editor window. It’s actually quite remarkable that this works.

This update() method is for low-level use, as it updates the Delta view of the document only. To apply Delta operations to a document, the API’s updateContents() should be used.

Internals: How attributes in Delta are applied

The Delta format has also other purposes, and is important within Quill’s API, in particular for requesting certain changes in the documents. This is however of less interest for those not writing or modifying Quill modules. Anyhow, this is a deep dive into the machinery that makes this happen.

The function that is used both by setContents() and updateContents() is applyDelta(), the latter defined in core/editor.js.

Aside from inserting text and objects, the attributes are applied. That’s done with this simple loop:

Object.keys(attributes).forEach((name) => {
 this.scroll.formatAt(index, length, name, attributes[name]);
});

Or simply put, the keys method is applied to the attributes to fetch the attribute keys, and then formatAt() is called for each. formatAt() is hence called in the order as returned by the JavaScript’s built-in keys() method, which is the order they were inserted. However Quill is designed to organize the blot structure (and hence DOM and HTML) in a canonical manner, no matter the order of formatting, so the ordering doesn’t matter effectively.

To complete this issue, I’ll just mention what @attributes equals when the loop above is executed: At this point, the @attributes object contains the updates that are required relative to the format that exists anyway in the current position. Or in more detail, this is done by first fetching them from the Delta op:

let attributes = op.attributes || {};

and if it’s a text op (i.e. the “insert” property is a string) , the current position’s format is calculated by querying the blots above and adjacent to the left for the format they contribute. This is done by recursively calling bubbleFormats() (defined in Quill’s blots/block.js), which calls the blots’ formats() function. @attributes is modified with

attributes = DeltaOp.attributes.diff(formats, attributes) || {};

The said diff() method is defined in quill-delta/lib/op.js, and it loops through all keys in both object arguments (concatenation of the keys() array of the first argument and the second, in this order), and returns the properties in the second argument that have different values from the first argument’s object. Properties that are present only in the first argument are returned with the key set to null.

So all in all, @attributes ends up with the changes needed to update the format to the one required in the Delta op: The value if it wasn’t defined at all or was different, and null if it should be removed.

Quill internals and the base blot classes: Block, Inline, Embed and friends

Everything in this post relates to Quill v1.3.7. I don’t do web tech for a living, and my knowledge on Quill is merely based upon reading its sources. Besides, I eventually picked another editor for my own use.

Introduction

Quill offers a variety of blot classes, which are the base for creating custom blot classes by virtue of extension. Understanding the ideas behind these classes is important, in particular as extending the most suitable class for custom blots is required to ensure its intuitive behavior in the editor.

This post focuses on the three main base classes: Block, Inline and Embed, which are intended to be the base for custom blot classes. There are however a few other base classes, which are required to understand how things work under the hood.

As discussed in a separate post of mine, the DOM and Parchment tree (i.e. the Scroll) have the same tree structure, and hence there’s a blot object in the Parchment tree for each node in the DOM and vice versa, with a few exceptions that are irrelevant for the current topic.

Recall that a pair of HTML tags that form an enclosure (e.g. <strong> and </strong>) correspond to a single DOM node, having the elements within the closure as its children in the DOM tree. Hence if some text is marked as bold in the editor, a blot is created to correspond to the DOM object for the enclosure of <strong> tags, and the bold text itself is placed as a child of this blot.

A few base blot classes

In essence, there isn’t a single blot class that isn’t interesting when developing a custom class, since imitating an existing implementation is the safest way to get it right. But these are those most important to know about:

  • TextBlot (exported as blots/text, defined in blots/text.js), effectively same as Parchment.Text (parchment/src/blot/text.ts), corresponding to #text DOM items, and is hence always a leaf (i.e. it has no children). Not surprisingly, this is the blot class used for text.
  • Embed (exported as blots/embed, defined in blots/embed.js) extending Parchment.Embed (parchment/src/blot/embed.ts): Intended for tags such as <img>, this blot class is used for insertion of elements that come instead of text, but isn’t text. In the Delta model, it occupies a horizontally packed element with a length of one character.
  • BlockEmbed (exported as blots/block/embed, defined in blots/block.js) extending Parchment.Embed (parchment/src/blot/embed.ts) is essentially an Embed blot that is allowed where a Block blot would fit in, so it’s occupies vertical space of its own, rather than being horizontally packed.
  • Inline (exported as blots/inline, defined in blots/inline.js) extending Parchment.Inline (parchment/src/blot/inline.ts): This is the blot class intended for formatting tags such as bold, italic and also links (with <a href=”"> tags), and is used for any tag enclosure that don’t cause the browser to jump to a new line. The default tag for this blot is <span>.
  • Block (exported as blots/block, defined in blots/block.js) extending Parchment.Block (parchment/src/blot/block.ts): Its default tag is <p>, which implies its intention: Usage for tag enclosures that create vertical segments. Its direct children are allowed to be of blots of the classes Inline, Parchment.Embed or TextBlot, or classes derived from these. In other words, child blots that create horizontal packing of elements.
  • Container (exported as blots/container, defined in blots/container.js) effectively same as Parchment.Container (parchment/src/blot/abstract/container.ts): This class has no default tag, and is used for vertical segment enclosures that must be nested, for example <ul> and <li>. Its allowed direct children may only be other block-type blot classes, that is Block, BlockEmbed and Container.

Almost all blot classes are somehow extensions of these six.

Note that first three blot classes listed here are fundamentally different from the other three: The first three, TextBlot, Embed and BlockEmbed, represent content, and hence their related Parchment classes extend LeafBlot. Inline and Block, on the other hand, represent formatting, and hence their related Parchment classes extend FormatBlot. Container, unlike the other five, extend ShadowBlot: It can’t be generated directly by virtue of formatting, but only internally to create a tree structure that is needed indirectly by some formatting command.

Block blots are considered to represent a newline (“\n”) character, and their length is accordingly one. In other words, the index in the document after a Block blot is higher than the one before by one.

Quill’s built-in format blots is listed here. The division into Block, Inline and Embed in that list is somewhat inaccurate, but accurate enough for end-user purposes (in particular regarding List being a Container, not Block).

Quill’s Parchment tree model

The relationship between the browser and Quill is bidirectional, so the browser makes certain changes to the document, and Quill controls the structure of the Scroll (i.e. the Parchment tree), and hence also the DOM tree.

The main influence of the tree model is on the document’s top hierarchy node, which is the editor’s root DOM node, or interchangeably, the Parchment tree’s root (the Scroll blot). All children of this top node are the document’s lines, and all document lines are children of this top node. In other words, all <p>, <div>, <h1>, <h2> and similar DOM elements are always direct children of the root DOM node. Accordingly, the corresponding blot classes for these tags always extend the Block blot class (possibly indirectly).

All other blots, which represent horizontally packed DOM elements, form a subtree of a single block blot. In other words, there’s a linear sequence of lines from the document’s beginning to end, each represented by a block blot. Inside each line, there’s only text, inline formatting or inline embedded objects. Vertical packing occurs only at the top level, horizontal formatting can have any depth.

The only exception is the Container blot, however its use doesn’t conflict with the concept of document lines. Rather, it allows grouping block-like blots, as the children of a Container can only be Block, BlockEmbed and Container. This allows a not completely flat tree structure from the top level, but the tree can still be traversed from its beginning to end, and walk from line to line, each represented by either a BlockEmbed blot, or a Block blot with children that constitute horizontally packed elements. Containers merely group block-like blots.

The Container blot is applied when nesting is inevitable. For example, bulleted and enumerated lists are interesting cases, because they require a blot to correspond to the <ul> or <ol> tag enclosure, and then a blot for each <li> enclosure. So clearly, the blot that corresponds to <ul> or <ol> must be a direct child of the Scroll blot. On the other hand, the former blot must have children which are Block blots, corresponding to <li> enclosures.

By making the blots referring to <ul> and <ol> extend the Container class, and make the <li>’s blot extend the Block class, the latter can be children of the former, which is necessary to mimic the DOM tree structure (see formats/list.js). But since <li> is a Block blot (it must be, or else how could its children be text?) it can’t have a Container nor Block blot as a direct child. As a result, nested lists are not generated by Quill. When such are needed, CSS is used to indent <li> items visually, to make an appearance of a nested list.

Not surprisingly, the Scroll blot class extends Parchment.Container, and allows only Block, BlockEmbed and Container as its direct children (see blots/scroll.js).

From a user’s point of view, this means that everything in the document is in the context of a line that is of a single formatting type. It’s either a header, a list, a plain paragraph or something of that sort. One can’t insert a header nor a code block into a bulleted list, for example, even though that wouldn’t violate the DOM structure. In fact, one can’t insert a <p> paragraph enclosure into a list either, nor a code block.

Or as said in this Quill’s doc page:

While Inline blots can be nested, Block blots cannot. Instead of wrapping, Block blots replace one another when applied to the same text range.

Had it not been for this simple line structure, it would have been significantly more difficult to obtain a concise Delta representation.

The concept of a “line”

Another way to understand the tree structure, is looking at the implementation of API’s getLines() (defined in core/quill.js), which is described “Returns the lines contained within the specified location”. It would be more accurate to say “within the specified range”. Anyhow, this function merely calls lines() as defined in blots/scroll.js:

  lines(index = 0, length = Number.MAX_VALUE) {
    let getLines = (blot, index, length) => {
      let lines = [], lengthLeft = length;
      blot.children.forEachAt(index, length, function(child, index, length) {
        if (isLine(child)) {
          lines.push(child);
        } else if (child instanceof Parchment.Container) {
          lines = lines.concat(getLines(child, index, lengthLeft));
        }
        lengthLeft -= length;
      });
      return lines;
    };
    return getLines(this, index, length);
  }

First, I’d mention that forEachAt() (and similar methods) is implemented in parchment/src/collection/linked-list.ts. As its name implies, it calls a function on all blots within a range (index, length), setting the index and length for each call relative to the blot being processed.

Also, isLine() is a local function defined as:

function isLine(blot) {
  return (blot instanceof Block || blot instanceof BlockEmbed);
}

With this information, the mechanism is quite clear: All blots in the requested loop are scanned. Only those that are extended from Block or BlockEmbed classes are added to the list. If a blot that is extended from the Container class is encountered, lines() calls itself recursively on that blot, or in other words, the subtree is scanned in the same manner.

The takeaway from this code dissection is that only Block and BlockEmbed based blots are considered “lines”, and that Container blocks are just a way to create a parent node for a subtree.

Likewise, getLine() is defined as to “Returns the line Blot at the specified index within the document”. This method just wraps line(), which is also defined in the same file:

line(index) {
  if (index === this.length()) {
    return this.line(index - 1);
  }
  return this.descendant(isLine, index);
}

@this in the code above is the scroll object. So once again, the same principle. In this case, the “descendant” method is used to look up a blot that is extended from either Block or BlockEmbed somewhere down the tree. Container blots aren’t related to directly here, because they are just passed through as the tree is traversed.

Quill internals and the jumping cursor + the Selection and Cursor classes

Everything in this post relates to Quill v1.3.7. I don’t do web tech for a living, and my knowledge on Quill is merely based upon reading its sources. Besides, I eventually picked another editor for my own use.

TL;DR

If you’ve reached this page, odds are that you’re trying to figure out why the cursor is jumping under certain conditions, for example as in this still unresolved issue.

While I may not answer that directly, I can suggest one thing to do: With your browser’s JavaScript debugger (Google Chrome recommended), put a breakpoint on the Selection class’ setNativeRange() method, and an additional breakpoint on the return statement marked below in read. It will typically be something like this in a non-minimized quill.js:

    key: 'setNativeRange',
    value: function setNativeRange(startNode, startOffset) {
      var endNode = arguments.length > 2 && arguments[2] !== undefined ? arguments[2] : startNode;
      var endOffset = arguments.length > 3 && arguments[3] !== undefined ? arguments[3] : startOffset;
      var force = arguments.length > 4 && arguments[4] !== undefined ? arguments[4] : false;

      debug.info('setNativeRange', startNode, startOffset, endNode, endOffset);
      if (startNode != null && (this.root.parentNode == null || startNode.parentNode == null || endNode.parentNode == null)) {
        return;
      }

Now, notice the part marked with red. If setNativeRange() returns because of this, odds are that an attempt to put the cursor in its right position failed, and that caused the cursor to jump to where the browser chose to put it.

And if that isn’t the case, maybe the position that is chosen by this call’s parameters is wrong.

If this method isn’t called at all when the cursor jumped, odds are that Quill wasn’t clever enough to know that a cursor repositioning was necessary. All requests to set the selections, by API or by internal functions, end up with this method. So if this method isn’t called, Quill didn’t even try.

The rest of this post describes the three reasons I found for my case of jumping cursor (well, actually two and a half), and how I solved them. All of which were because of the same reason: For the sake of DOM tree optimization, Quill merged two DOM text elements while the cursor was part of one of them. As a result, the browser moved the selection the end of the text element that had been deleted. And then the mechanisms that are in place in Quill to return it to the right place failed to do their work.

And here’s the bad news: To understand the reason and fix, one must first understand a few things about the Cursor and Selection classes. So take a deep breath.

The Cursor blot class

First and foremost, this blot’s name is misleading: The cursor that is displayed is always the one produced by the browser. This blot isn’t involved in generating anything visual. It would have been more accurate to call it FormatHolder or something like that.

This is going to cause confusion, so let me reiterate: There’s the Cursor blot object, which isn’t a cursor at all, and it may or may not be on the same place as the visual cursor. So when I say “cursor” below, I refer to the visual thing, not the Cursor class or object.

The purpose of the Cursor class is to allow an Inline format (e.g. bold and italic) to be enabled when there’s no text related to it. The typical situation is when we click on the “Bold” button with nothing selected. Nothing happens, but if we type text, it’s in bold. Which is the same as when typing text immediately after text that is already bold.

So the Cursor class is used to mark the existence of single 0xfeff Unicode character, which is a non-breakable zero-width white space. In the DOM, the equivalent of <span class=”ql-cursor”>&#65279</span> is inserted. The #text DOM element itself doesn’t have a corresponding blot, which is quite unusual in Quill (except for Iframes’ content). The declared length of the Cursor blot and its content is zero (which is unusual too for a #text DOM element with something in it).

Once the enclosing Inline blot is filled with some real text, the Cursor blot is removed. If it becomes empty again due to deletion of characters, the Cursor blot is inserted again.

Effectively what happens is that this character becomes the “character before”, and hence prevents the formatting to get optimized away. It’s purpose is to influence the characters to be added after it. For example, with Bold formatting, it means that the <strong></strong> pair has something between them, and that the text that is typed at this point will be in bold. Had it not been for the insertion of the cursor, it would have become impossible to enable a format. Pressing the “Bold” button would do nothing, for example, but selecting text and turning it to bold would still work.

The generation of the single Cursor object for the entire Quill object is done by the constructor of the Selection class (core/selection.js), and is kept as the class’ this.cursor. This class also applies the Cursor format when the selection is of zero length. More precisely, it calls its own getNativeRange() method to find the range of nodes in the DOM that correspond to the current selection. If the collapsed property is true (indicating a zero-length selection), plus the format is not for a Block blot, and selection isn’t the cursor itself, it ends up with

let after = blot.split(nativeRange.start.offset);
blot.parent.insertBefore(this.cursor, after);

which cuts the underlying blot where the Cursor blot will go, and then it’s inserted there. insertBefore() also takes care of updating the DOM.

But even more important, if a Cursor blot is inserted (or was already present), the selection is always turned around it:

this.setNativeRange(this.cursor.textNode, this.cursor.textNode.data.length);

In other words, that single cursor character is selected. This means that if the user types or pastes anything, the browser puts it instead of the cursor character.

When the Cursor blot should go away because some text has come instead, CursorBlot’s update() method calls its own restore() method (see blots/cursor.js). Alternatively, this method might be called by the Selection class’ update() method if it deems that the selection range has changed.

And by the way, the fact that the Cursor blot includes a character in the DOM, makes it appear in in the innerHTML property, which is unfortunate for those who want to save the innerHTML of the editor container. It’s not a really big deal, as it’s a transparent zero-width element, but it might break a word in the middle, making it unsearchable. A common solution is to inject the the Delta into a second, bogus, Quill instance, and use its HTML.

The restore method

As just mentioned, restore() is defined blots/cursor.js. This method is called to remove the Cursor blot from where it has been inserted. Or more precisely, to clean up: Since the cursor character itself was marked as selected when it was inserted, it might have been replaced with user text. So just removing the entire package might remove the newly inserted text. As this is highly related to the jumping cursor, here’s a walkthrough of the method’s code:

  restore() {
    if (this.selection.composing || this.parent == null) return;
    let textNode = this.textNode;
    let range = this.selection.getNativeRange();
    let restoreText, start, end;

After checking if the method should be executed at all, some variable initialization: @textNode is the DOM node that is wrapped by the <span> pair that the Cursor blot relates to. @range is assigned with the range, in DOM terms, of the current selection.

A short word about this.selection.composing: Composition is the use of several keystrokes to create an Asian character (e.g. Korean Hangul). This flag is set on compositionstart events and turned off on compositionend, both events generated by the browser to indicate such a keystroke session. Hence nothing should happen while this is going on.

    if (range != null && range.start.node === textNode && range.end.node === textNode) {
      [restoreText, start, end] = [textNode, range.start.offset, range.end.offset];
    }

If the Cursor blot’s text DOM element, and only it, is included in the selection, @restoreText, @start and @end are assigned with @textNode and the positions of the selection inside it. This is the case if a character was typed with the keyboard or text was pasted, but may not be if the browser’s cursor was just moved.

    // Link format will insert text outside of anchor tag
    while (this.domNode.lastChild != null && this.domNode.lastChild !== this.textNode) {
      this.domNode.parentNode.insertBefore(this.domNode.lastChild, this.domNode);
    }

The comment here speaks for itself (and it’s quite irrelevant). And then:

    if (this.textNode.data !== Cursor.CONTENTS) {
      let text = this.textNode.data.split(Cursor.CONTENTS).join('');
      if (this.next instanceof TextBlot) {
        restoreText = this.next.domNode;
        this.next.insertAt(0, text);
        this.textNode.data = Cursor.CONTENTS;
      } else {
        this.textNode.data = text;
        this.parent.insertBefore(Parchment.create(this.textNode), this);
        this.textNode = document.createTextNode(Cursor.CONTENTS);
        this.domNode.appendChild(this.textNode);
      }
    }

The condition of the first if statement is true iff the data inside the cursor has been modified. So this entire piece of code relates to when text has been typed or pasted, and hence wiped out the cursor character

So first, @text is assigned with the (updated) content of textNode, with the cursor character removed from the string, if it’s still present (why would it be?). In short, the browser put the text typed or pasted by the user instead of the cursor, and now it’s given as @text.

Then, the sibling to the right in the Parchment is checked if it’s a TextBlot. It will be if the cursor was placed along with text, but if it’s next to an Embed blot, it might not be.

So if the next sibling is a text blot, the text in the Cursor blot is injected into it before the existing text and Cursor blot gets back to having its special cursor character. Also @restoreText is updated to become the the sibling’s DOM element, indicating that the selection should be restored to that element, and not to the Cursor blot, which is now out of the game.

If the next sibling isn’t a text blot, the DOM node containing the text is assigned with @text (effectively this removes the cursor character, if it was present), and then a new text blot is created and inserted into the Parchment (and hence also into the DOM). So basically, insert text. But oops, now we lost the the this.textNode to play around with. So a new one is created, just like in the constructor method.

And then the Cursor blot is taken off the tree:

    this.remove();

Keep in mind that @this stands for the Cursor blot in the parchment. Its removal merely takes it off the Parchment and the DOM tree. And then…

    if (start != null) {
      [start, end] = [start, end].map(function(offset) {
        return Math.max(0, Math.min(restoreText.data.length, offset - 1));
      });
      return {
        startNode: restoreText,
        startOffset: start,
        endNode: restoreText,
        endOffset: end
      };
    }
  }

… the grand finale: If the Cursor blot, and only it was the selection, return the range to be selected when everything is over, in terms of the DOM text node and starts and stop offsets. These offsets are limited to between zero and the length of the text, which is quite understandable.

Note that if this isn’t the case, the function returns nothing. This is important, because the Cursor class’ update method goes

  update(mutations, context) {
    if (mutations.some((mutation) => {
      return mutation.type === 'characterData' && mutation.target === this.textNode;
    })) {
      let range = this.restore();
      if (range) context.range = range;
    }
  }

so if the restore() returns something valid, it’s stored as the @range property in the @context, which is juggled around as the call progresses.

For now, remember that there’s a context variable floating around. I need to make a not-so-small detour now.

Why the cursor jumps, and Quill’s take on the problem

Well, there might be more than one reason for a cursor jump, but I’ll discuss the one at hand: The DOM element, on which the browser’s cursor stands, is manipulated by Quill. Typically, it’s a text DOM element that is either changed or deleted. This could be the result of a direct edit, or internal manipulations, such as splitting a blot to insert formatting blots, or optimizing away blots (and hence DOM elements).

Quill handled each of these cases separately to prevent a cursor jump, and I’ll focus on the mechanisms that are related to the Cursor class.

The first mechanism takes a simple approach: update() shouldn’t change the selection. Therefore, get the selection before update() is processed, remember it, and restore it before it’s about to finish.

So the constructor of the Selection class (core/selection.js) goes:

    this.emitter.on(Emitter.events.SCROLL_BEFORE_UPDATE, () => {
      if (!this.hasFocus()) return;
      let native = this.getNativeRange();
      if (native == null) return;
      if (native.start.node === this.cursor.textNode) return;  // cursor.restore() will handle
      // TODO unclear if this has negative side effects
      this.emitter.once(Emitter.events.SCROLL_UPDATE, () => {
        try {
          this.setNativeRange(native.start.node, native.start.offset, native.end.node, native.end.offset);
        } catch (ignored) {}
      });
    });

As their names imply, a SCROLL_BEFORE_UPDATE is injected by Quill’s own code before an update() is executed, and SCROLL_BEFORE_UPDATE is injected just before it returns. See towards the end of this post how the said method fires off these events.

The idea is quite simple: The handler of SCROLL_BEFORE_UPDATE gets the current selection and stores it in the local variable @native, and then registers a second callback for execution when SCROLL_UPDATE is fired off, i.e when the update() is done. Since the second callback is defined within the first callback, it’s exposed to @native, and uses it to set the selection to where it was before.

This works almost all the time. The recurring problem, which is discussed further below, is that things that happened during the update() processing caused the DOM nodes that appear in @native to be removed from the DOM tree. As a result, they surely can’t convey information on where the selection was.

I guess that it was found by trial and error that the problem occurred only as a result of optimizations after the restore() method mentioned above was called, with the selection set on the cursor itself. So a hack to work this around was introduced in Quill git repo’s commit 56ce0ee54 (June 2017, included in v1.3.0.), in which there were several changes made to use the @context in order to restore the selection at a late stage. There were other commits as well related to this issue.

It’s a pin-point fix for a specific problem, and it boils down to this: restore() might return the position of the selection. If it does, store it in the @context variable, and use it instead of the SCROLL_UPDATE mechanism mentioned above (note the “if (native.start.node === this.cursor.textNode) return;” part).

For this purpose, the Selection class’ constructor added another listener:

    this.emitter.on(Emitter.events.SCROLL_OPTIMIZE, (mutations, context) => {
      if (context.range) {
        const { startNode, startOffset, endNode, endOffset } = context.range;
        this.setNativeRange(startNode, startOffset, endNode, endOffset);
      }
    });

So what happens is that when an optimize() is completed, the Quill-defined SCROLL_OPTIMIZE is generated (see code snippet for that towards the end of this post), and the @context property is the used to resume the selection to its original position. In other words, what restore() returned.

All in all, if restore() is called, with the selection spanning a single text node (in practice the Cursor blot’s text node), the selection position, in terms of the DOM, is stored in the “context” object that is juggled throughout, and is then (hopefully) restored.

In all other cases, the selection position is restored by virtue of the SCROLL_UPDATE mechanism. Also, hopefully.

Reason #1 for jumping cursor

Actually, this is quite unlikely to be the reason, but anyhow:

As mentioned above, the current selection is passed on if and only if it’s limited to the Cursor blot’s text DOM element. So if the selection is moved towards the end of the document (e.g. with a right arrow), it won’t be part of the Cursor blot, so no restoration of the selection.

Nevertheless, it might very well be that the text region, on which the browser cursor or selection stands on, is eliminated by the Text blot’s optimization (see the code in “Text blot optimization” below), and then it isn’t corrected. To the user, the cursor jumped to the end of the segment with the same formatting.

That said, the cursor won’t jump because of this in the common use case, because the SCROLL_UPDATE will handle this. But what about when interrupting a character composition with moving the cursor? Will the SCROLL_UPDATE do its magic? Haven’t tried.

Reason #2 for jumping cursor

The second problem relates to restore()’s context.range mechanism: The crux is that the selection to be restored is given in “native form”, i.e. in terms of DOM elements. Now, recall that the text node (and possibly more of them) were split into two in order to give place for inserting the Cursor blot. What happens if the removal of the Cursor blot caused the two remaining text nodes to merge into one, by virtue of optimization? In that case, @startnode, which is originally @restoreText as defined in the Cursor class, doesn’t necessarily exist any more, or more precisely, isn’t on the DOM tree anymore: The optimization necessarily removed at least one text node from the DOM tree when merging two text nodes into one.

In this case, the selection range will be defined by virtue of a DOM element that isn’t in the DOM tree anymore. What happens practically? Consider setNativeRange(), also defined in the Selection class, which starts with this:

if (startNode != null && (this.root.parentNode == null || startNode.parentNode == null || endNode.parentNode == null)) {
  return;
}

So if setNativeRange() is called with a DOM node that isn’t on the DOM anymore, at least one of startNode or endNode won’t have a parent node (that’s the essence of not being on the DOM tree), so the call returns without doing anything. What can it do?

And once again, the text node with the selection was eliminated, and the selection wasn’t restored, hence a jump.

This seems to happen when user-defined blot classes are added to Quill, in particular if they provoke optimization scenarios that are legit, however didn’t happen before.

Reason #3 for jumping cursor

The SCROLL_UPDATE mechanism actually suffers from the same problem: If the selection includes a text element that is then removed from the DOM tree during the processing of the update() call, most likely due to an optimization, setNativeRange() will return doing nothing, for the same reason as with reason #2.

This too is related to the removal of the Cursor blot and a merge of text blots, but as a result of moving the browser’s cursor or the selection away from the Cursor blot.

The recurring problem

At this point, it should be clear that the real problem is that there’s no robust mechanism for maintaining the selection after text DOM elements are eliminated by virtue of optimization and other manipulations. More specifically, the recurring problem is the reliance on the DOM elements. Which may appear to be a strange decision: Why not define the selection in terms of the index and length, in document-global terms? That is guaranteed to work correctly.

The answer is that I don’t know, and I’m sure someone had a good reason. I think the hint lies in the comment of this little snippet from core/quill.js:

  getSelection(focus = false) {
    if (focus) this.focus();
    this.update();  // Make sure we access getRange with editor in consistent state
    return this.selection.getRange()[0];
  }

getRange() is the internal method used to get the selection in terms of index and length. And as the comment implies, it might not work well unless the editor is after a call to update(), in order to sync the Parchment with the DOM. But one can’t call update() in code that processes update(), quite naturally.

Bug fix strategy

First and foremost, this is a good time to mention that the Embed blot (blots/embed.js) has a similar mechanism for preventing cursor jumps, most likely with the same problems, which I haven’t tended to. Mostly because I haven’t seen any actual problem with it. Left as an exercise to the reader.

Second, I’m not really going to fix the root cause of this issue, but rather add yet another hack that makes it work again for me. It seems to me that the correct way to fix this once and for all is to change the strategy altogether: That a property of the main Scroll object would hold the selection to resume to, in terms of start and end DOM nodes and the offsets within. And then, every piece of code that might remove a DOM node first checks if it appears in that selection property, and if it does, it makes sure to update it so it points at the corresponding updated position in the DOM tree. I guess the places to do that is in implementations of split() and optimize() as well as a few others.

So now back to reality and my own little hack. There are two fundamental differences between my solution and the way it was before, both relating to the Cursor class’ restore() method:

  • restore() always sets a selection for restoring.
  • restore() specifies the selection in terms of index and length, and is therefore indifferent to changes in the DOM tree structure

The main assumption behind this solution is that the call to restore() for the removal of the Cursor blot can be a result of two reasons only: Insertion of text (by typing, pasting or some other insertText() call) or a change in selection.

The solution relies on the notion that restore() already makes a distinction between whether new text has been inserted instead of the cursor character or not. So the algorithm goes

  • If text has been added, put a zero-length selection after the inserted text. This is what editors always do. So there’s no need to query for the selection in this case. Calling the Cursor blot’s offset() method gives the index.
  • Otherwise, getRange() is called to obtain the index and length. This is very likely to give correct result, as restore() was called following a change in selection, and not in the middle of processing an edit.
  • As an unlikely fallback, if getRange() returns null, set the selection at the Cursor blot’s position. Not clear if this will ever happen, but even if it does, it’s more likely while processing a selection change, in which case the cursor will just jump back, and the user will click again.

The truth is that only the first bullet here seems to matter. In the two other cases, it seems like the update() related mechanism will have the last word on where the selection ends anyhow.

Speaking of which, the way to work around reason #3 is to obtain the index-based selection range alongside with the DOM-based one. And then use the latter if the former is going to fail anyhow. It’s not just better than nothing, but my own anecdotal experience shows that it works.

The code changes

A word of truth: I didn’t really make the changes in Quill’s sources, but rather hacked the mangled quill.js file. Setting up the build environment was a bit too much for me. So what is shown below is a manually made reconstruction of the changes I made. Hopefully without mistakes.

The main change is done in the Cursor class, with restore() changed to:

  restore() {
    if (this.selection.composing || this.parent == null) return;
    let textNode = this.textNode;
    let restore_range = { index:0, length: 0 };

    // Link format will insert text outside of anchor tag
    while (this.domNode.lastChild != null && this.domNode.lastChild !== this.textNode) {
      this.domNode.parentNode.insertBefore(this.domNode.lastChild, this.domNode);
    }
    if (this.textNode.data !== Cursor.CONTENTS) {
      let text = this.textNode.data.split(Cursor.CONTENTS).join('');

      restore_range.index = this.offset(this.scroll) + text.length;

      if (this.next instanceof TextBlot) {
        this.next.insertAt(0, text);
        this.textNode.data = Cursor.CONTENTS;
      } else {
        this.textNode.data = text;
        this.parent.insertBefore(Parchment.create(this.textNode), this);
        this.textNode = document.createTextNode(Cursor.CONTENTS);
        this.domNode.appendChild(this.textNode);
      }
    } else {
      let range = this.selection.getRange()[0];

      if (range) {
	restore_range = range;
      } else {
	restore_range.index = this.offset(this.scroll);
      }
    }

    this.remove();

    return { restore_range: restore_range };
  }

Note that both @range and @restoreText have been eliminated. Instead, there’s restore_range, which is index and length based. And the method always returns a value.

Then there are adaptions in the Selection class. First, the handler of SCROLL_UPDATE, which changes to:

    this.emitter.on(Emitter.events.SCROLL_BEFORE_UPDATE, () => {
      if (!this.hasFocus()) return;
      let [range, native] = this.getRange();
      if (native == null) return;
      if (native.start.node === this.cursor.textNode) return;  // cursor.restore() will handle
      // TODO unclear if this has negative side effects
      this.emitter.once(Emitter.events.SCROLL_UPDATE, () => {
        try {
          if (native.start.node.parentNode == null || native.end.node.parentNode == null) {
	    this.setRange(range, Emitter.sources.SILENT);
	  } else {
	    this.setNativeRange(native.start.node, native.start.offset, native.end.node, native.end.offset);
	  }
        } catch (ignored) {}
      });
    });

The trick is simple: Rather than obtaining just the DOM-based selection range with getNativeRange(), the index-based range is obtained as well, with getRange(); Then, if setNativeRange() is doomed to fail miserably, fall back on the index-based range.

The handler for SCROLL_OPTIMIZE is changed to this:

    this.emitter.on(Emitter.events.SCROLL_OPTIMIZE, (mutations, context) => {
      if (context.range) {
        if (context.range.hasOwnProperty('restore_range')) {
	  this.setRange(context.range.restore_range, Emitter.sources.SILENT);
	} else {
	  const { startNode, startOffset, endNode, endOffset } = context.range;
	  this.setNativeRange(startNode, startOffset, endNode, endOffset);
	}
      }
    });

So it now plays along with restore_range as well as the old native range. Why keep the old? Because the Embed blot still emits a context old school, as mentioned above.

And finally, handleComposition() has one line changed, so it treats the result of cursor.restore() correctly. The part going

setTimeout(() => {
  this.setNativeRange(range.startNode, range.startOffset, range.endNode, range.endOffset);
}, 1);

changes to

setTimeout(() => {
  this.setRange(range.restore_range, Emitter.sources.SILENT); }, 1);

And this is the time I wonder about the concept that the selection position restoration is pushed 1 millisecond later, with the idea that surely that will be “after everything”. This is in fact a common way to defer work in Quill’s code, which doesn’t make me wonder less.

For reference: The code doing Text blot optimization

Just to save the need to look it up in Quill’s sources. I have made no changes. So this is implemented in parchment/src/blot/text.ts as follows:

  optimize(context: { [key: string]: any }): void {
    super.optimize(context);
    this.text = this.statics.value(this.domNode);
    if (this.text.length === 0) {
      this.remove();
    } else if (this.next instanceof TextBlot && this.next.prev === this) {
      this.insertAt(this.length(), (<TextBlot>this.next).value());
      this.next.remove();
    }
  }

Nothing surprising here: If the current text element contains nothing, just remove it. Otherwise, if there’s a sibling to the right that is also a Text blot, append its content to yourself, and remove that sibling. But why checking this.next.prev === this? When could that not be true?

For reference: The emitters of SCROLL events

Just to show how these events are generated explicitly, excerpt from blots/scroll.js:

  optimize(mutations = [], context = {}) {
    if (this.batch === true) return;
    super.optimize(mutations, context);
    if (mutations.length > 0) {
      this.emitter.emit(Emitter.events.SCROLL_OPTIMIZE, mutations, context);
    }
  }

[ ... ]

  update(mutations) {
    if (this.batch === true) return;
    let source = Emitter.sources.USER;
    if (typeof mutations === 'string') {
      source = mutations;
    }
    if (!Array.isArray(mutations)) {
      mutations = this.observer.takeRecords();
    }
    if (mutations.length > 0) {
      this.emitter.emit(Emitter.events.SCROLL_BEFORE_UPDATE, source, mutations);
    }
    super.update(mutations.concat([]));   // pass copy
    if (mutations.length > 0) {
      this.emitter.emit(Emitter.events.SCROLL_UPDATE, source, mutations);
    }
  }