Remote Update from ECPQ flash on Altera Cyclone IV

Introduction

This post relates to Altera (or should I say Intel FPGA?) Cyclone IV FPGAs loaded from an ECPQ flash in Active Serial x 1 (AS x 1) mode. Things written below are probably relevant to other Altera FPGAs as well, but keep in mind that Cyclone IV FPGAs have several peculiarities you won’t find on other Altera device families.

“Remote Update” is the feature in some Altera FPGAs, which allows application logic / software to safely update the bitstream from which the FPGA is loaded. The trick is to always have a Factory (“Golden”) bitstream image on the flash, and update the “Application” image only. When powers goes up, the FPGA ends up with the Application bitstream if it’s OK, or the Factory bitstream if it’s absent or corrupt.

Since the bitstreams carry a CRC, it’s guaranteed that only valid bitstreams are used. It’s therefore safe to overwrite a previous Application bitstream image: If something goes wrong in the middle of writing, it won’t be deemed a valid bitstream, so the FPGA will end up with the Factory bitstream.

The basics

To implement a remote update feature on an FPGA design, there are two functional elements needed:

• The ability to write data into the configuration flash with user-designed logic / software. This is discussed in this post.
• The logic / software that makes sure the FPGA ends up with the right configuration (and, in particular, prevents an endless configuration loop as explained next)

Note that the Remote Update IP Core has nothing to do with flash programming: Its function is merely to allow the FPGA’s logic to issue a reconfiguration, and offer some information on how and why the current bitstream was loaded.

When an FPGA powers up, it always configures from a constant address of the flash, which is zero on ECPQ flashes. In other words, the FPGA always powers up from the Factory bitstream, no matter what. It’s the user application logic / software’s duty to force the configuration of the Application bitstream when adequate. This means that during normal operation, there are always two configurations of the FPGA at powerup, one for the Factory bitstream, and one for the Application. This doubles the configuration time, of course.

How it happens: The FPGA is powered up, and loads the Factory bitstream from a fixed address. Through the Remote Update IP Core, the logic / software in the FPGA sets the address of the Application image at the flash, from which the FPGA should configure itself. It then can triggers a reconfiguration of the FPGA.

The FPGA’s configuration state machine attempts to load a bitstream from the flash at the given address. If the bitstream’s magic words are in place and the CRC is OK, it starts running on the new bitstream. If not, it loads the Factory bitstream again as a fallback.

By virtue of a register of the Remote Update IP Core, the software / logic in the Factory bitstream detects that it was loaded due to a failure, and takes action (or no action) accordingly. It may try another address at the flash, or refrain from another reconfiguration altogether (i.e. stay with the “Golden Image”). The decision depends on the design requirements, but the crucial point here is to prevent an endless loop of configurations.

Some reading

This post is not a user guide or a substitute for these two must-read documents:

• Chapter 8 in the Cyclone IV device handbook, volume 1
• Altera Remote Update IP Core User Guide

Spoiler

This Nios II code implements the loading of the Application bitstream. It written so it can be used on any bitstream, as is does nothing when run from an Application bitstream.

It’s also safe for use with JTAG configuration: It won’t issue a reconfiguration in that case (well, sort of: see note on peculiarity below).

void do_remote_update(void) {
  alt_u32 app_bitstream_addr = 0x100000;

  alt_u32 mode = IORD_32DIRECT(REMOTE_UPDATE_0_BASE, 0) & 3;
  alt_u32 config_reason = IORD_32DIRECT(REMOTE_UPDATE_0_BASE, 0x64);

  if ((mode == 0) && (config_reason == 0)) {
    IOWR_32DIRECT(REMOTE_UPDATE_0_BASE, 0x30, 0); // Turn off watchdog
    IOWR_32DIRECT(REMOTE_UPDATE_0_BASE, 0x40, app_bitstream_addr);

    IOWR_32DIRECT(REMOTE_UPDATE_0_BASE, 0x74, 1); // Trigger reconfiguration

    while (1); // Wait briefly until configuration takes place
  }
}

do_remote_update() should be called first thing in the Nios II code entry. If the function returns, the FPGA is either running on the Application bitstream, or the Factory (“Golden”) bitstream with a good reason not to reconfigure (i.e. a previous failure to load the Application bitstream or after a JTAG bitstream load).

Please refer to the “Programming the flash with NIOS software” section in this post on how to generate the image of the Application bitstream.

The code above works with the following setting:

• The FPGA’s NCONFIG pin is tied high. This will not work if the NCONFIG pin is driven by some power supply watch logic or alike, because config_reason won’t be zero if NCONFIG triggered the configuration.
• REMOTE_UPDATE_0_BASE is the base address in NIOS’ address space of a Remote Update IP core, which has the “writing configuration parameters” option enabled.
• The application bitstream image is loaded at flash address 0x100000 (i.e. can be read with epcs_read_buffer() using this address)
• The Golden image is at address zero, of course.

If loading the application image fails once, no other attempts are made. This is the straightforward thing to do if there’s no additional image to try from. There’s no sensible reason to try the same image again, unless the PCB designer has done a really bad job.

Now the peculiarity note promised above: If the Factory bitstream didn’t have the Remote Update IP instantiated (or STRATIXIII_UPDATE_MODE as mentioned below not set? Not clear), the first JTAG bitstream loaded after it will not be detected as a JTAG, so the bitstream loaded from JTAG might mistake itself for a first-attempt Factory bitstream and attempt to load the Application bitstream immediately. This kind-of makes sense, because the Current State register always reflects Factory mode after a JTAG bitstream load, and the difference is told from the Trigger Condition register, which reflects the reason for triggering the bitstream load. However this is a Read Past Status 1 (read_source = 1) register, reflecting something stored before the current bitstream load. In the absence of the Remote Update feature on the previous bitstream, it seems like this register wasn’t updated at all before the JTAG load, read all zeros after it, and hence the misinterpretation on the current bitstream.

This scenario is however irrelevant in all but rather messed up settings (why wouldn’t the Factory bitstream support Remote Update?). Anyhow, see another note below on another issue with reflecting the status after a JTAG configuration.

How this function works, briefly:

• It verifies that the configuration mode is 0, that is Factory mode. If we’re in Application mode, the function returns.
• It verifies that the trigger for configuration was a powerup by checking config_reason, or it returns. This prevents an endless loop of configurations in the case of a fallback into the Factory bitstream in the event of a failed attempt to load the Application bitstream.
  Note that if the configuration was triggered as a result of an assertion of the FPGA’s NCONFIG pin, or on a JTAG configuration, config_reason will read 0x10 (most of the time, see note below).
• The watchdog is disabled, so the Application bitstream doesn’t have to deal with it
• The Application bitstream’s address is set
• A configuration is forced by writing to the dedicated register
• An endless while (1) loop is invoked for preventing the execution to go on — not that it would go anywhere far.

General notes

• It’s important to observe that the terminology of Factory / Application configuration modes, which is used in the docs, isn’t just for the sake of clarity: The Remote Update IP Core exposes different registers, based upon whether it considers itself to be in either of the modes: In particular, when in Application mode, there is very little the logic can do, except for jumping back to Factory mode or to reset the watchdog.
• When generating the Remote Update block (most likely in Qsys), be sure to check “Add support for writing configuration parameters”. Or you’ll keep wondering why writing to the NIOS registers has no effect at all.
• Also be sure to set configuration mode to remote for the FPGA project. There should be a line as follows in the QSF file:

  set_global_assignment -name STRATIXIII_UPDATE_MODE REMOTE

• When setting the boot address register, use the actual boot address with the two LSBs forced to zero. When it’s read back after a configuration as the previous boot address, it’s shifted two bits to the left. The docs are a bit confusing about this too. Go figure.
• The watchdog is enabled by default, so unless it’s tended to in the application bitstream, it must be explicitly turned off before firing off reconfiguration.
• The watchdog timer runs on the internal configuration clock, which is 10 MHz unless the external CLKUSR is applied..

Accessing registers

Put short, the registers map is a mess. Out of the long list given in Tables 20 and 21 in the Remote Update IP Core User Guide, only a handful have a meaning.

It’s important to realize that some registers are valid when the Remote Update IP core is in Factory mode, and others when it’s in Application mode. These two register sets are mutually exclusive (except for the CURRENT_STATE_MODE register). The test program shown further down this post demonstrates which registers are valid in each mode.

This is a list of things to keep in mind regarding these registers:

• Reading from a Factory mode register in Application mode (and vice versa) returns meaningless (and rather confusing) data.
• The way to make sense of the registers from the docs is to refer to tables 16 and 17 in the Remote Update IP Core User Guide to tell what you want to access in terms of which param and which read_source, and then find the address for them in table 21. Several registers in table 21 constitute combinations of param and read_source that aren’t listed in table 17, which probably renders them meaningless.
• … except for RU_RESET_TIMER and RU_RECONFIG, which are interpreted in logic to generate a reset signal / reconfiguration signal respectively, and and therefore not listed in table 17.
• Too add more confusion, readbacks don’t work as one might expect. For example, the boot address for the next configuration is set at address offset 0x40, but reading back from the same address always yields the factory boot address. To get the boot address for the next configuration (i.e. the one written to 0x40), read it back at 0x4c.
• More confusion: The translation from the param numbers to the Nios access register isn’t some arithmetic operation, but rather some lookup logic of the avl_controller_cycloneiii_iv module in Qsys_remote_update_0_remote_update_controller.sv, which is generated automatically by Qsys.
• The registers listed in the BSP’s drivers/inc/altera_remote_update_regs.h are those of all Altera FPGAs except Cyclone IV. For example, the docs as well as the Qsys Verilog file () place RU_WATCHDOG_TIMEOUT at address 0x08 (actually, addresses 0x08-0x0b) but the BSP’s altera_remote_update_regs.h
• Note that compared with other FPGA families, Cyclone IV’s register interface is considerably more extensive, allowing the controller to query the status of two configuration cycles back in history. Seems like this feature was dropped on later FPGAs (due to lack of interest vs complication…?)

The RU_RECONFIG_TRIGGER_CONDITIONS register

This register is interesting in particular, as it tells us why that caused the FPGA to configure the bitstream that is currently running:

IORD_32DIRECT(REMOTE_UPDATE_0_BASE, 0x64); // Register 0x19 in the guide

And to obtain the reason for the configuration before that:

IORD_32DIRECT(REMOTE_UPDATE_0_BASE, 0x68); // Register 0x1a in the guide

These read the remote config core’s param 3′b111 with read source 2′b01 and 2′b10 respectively. Note that the translation from the param number of 3′b111 to the Nios access register isn’t just a multiplication, but rather some lookup logic as mentioned (but not detailed) above.

Running some tests of my own (with the test program below), I got the following values. There’s nothing surprising about these results; they are exactly as documented.

• On cold configuration: 0
• When not disabling the watchdog (not handling it after configuration): 2 (bit 1 set, User watchdog timer timeout)
• After failed application configuration due to lack of image: 4 (bit 2 set, nSTATUS asserted by an external device as the result of an error)
• After failed application configuration due to damaged image: 8 (bit 3 set, CRC error during application configuration)
• On configuration from JTAG; 0x10 (bit 4 set, External configuration reset (nCONFIG) assertion)

Ah, but there’s a thing with configuration from JTAG: Some other tests I’ve ran showed that if loading an application image with a CRC error (bit 3 set), this remains even after JTAG configurations (note the plural — even after several consecutive JTAG configurations). So instead of reading 0x10, this register reads 0x08, no matter how many times the bitstream was correctly loaded into the FPGA after that.

So the bottom line is that this register doesn’t work well with JTAG configurations (see peculiarity note above).

Without NIOS/Avalon interface

It’s also possible to instantiate a Remote Update IP without a NIOS processor. These are my few observations as I interfaced such IP without Avalon interface:

• The clock frequency should be below 20 MHz (or 10 MHz on some other device families. Refer to the Altera Remote Update IP Core User Guide on this.
• All of the Remote Update IP’s signals toggle on the rising edge of the clock.
• A read_param request assertion make busy rise on the rising edge of the clock for which it is asserted, and held high for about 62 clocks. In other words, (read_param && busy) is constantly zero (because of the one-clock nature of read_param) but (read_param || busy) will be logic ’1′ from the first assertion of read_param and until the read cycle is done. See scope shot below.
• write_param follows the same relation with busy, but the busy pulse is shorter: Around 47 clock cycles.
• The read_source bits are always bits 3:2 of the address as used by NIOS software to access the respective register.
• The param address is often bits 6:4 of the address as used by NIOS software to access the respective register with the exception when these are 3′b101, 3′b110 or 3′b111, in which cases the first to are replaced with 3′b110 and 3′b111 respectively. The last, 3′b111 addresses the FPGA logic directly (asserts a reconfiguration, among others).
• Nevertheless, refer to Table 17 of ug_altremote.pdf for the outline of param and read_source.
• The data_out is updated on the same clock cycle that busy goes low. In other words, for any clock cycle, if busy is low, data_out is valid. See scope shot below.
• data_out remains valid until the following read cycle. It seems like data_out goes zero soon a few clocks after busy goes high, but the actual value is of course valid only when it goes low again.

A test program

On my way to understanding how the whole thing works, I wrote a small test program that ran on the Nios II processor, which dumps all registers that are relevant for each mode. As a bonus, it can be used as a register reference, as it lists all registers available for reading Factory vs. Application mode in the respective structures.

#include <system.h>
#include <alt_types.h>
#include <io.h>
#include "sys/alt_stdio.h"
#include <unistd.h>

int main()
{
  int mode;

  struct regitem {
    int read_source;
    int param;
    const char *desc;
  };

  const struct regitem factoryparams[] = {
    { 0, 0x00, "Current Machine State Mode" },
    { 0, 0x10, "Factory Boot Address" },
    { 1, 0x10, "Previous Boot Address" },
    { 1, 0x18, "Previous reconfiguration trigger source" },
    { 2, 0x10, "One before previous Boot Address" },
    { 2, 0x18, "One before previous reconfiguration trigger source" },
    { 3, 0x04, "Early confdone check bits" },
    { 3, 0x08, "Watchdog timeout value" },
    { 3, 0x0c, "Watchdog enable bit" },
    { 3, 0x10, "Boot address" },
    { 3, 0x14, "Force internal oscillator" },
    {}
  };

  const struct regitem applicationparams[] = {
    { 0, 0x00, "Current Machine State Mode" },
    { 1, 0x08, "Watchdog timeout value" },
    { 1, 0x0c, "Watchdog enable bit" },
    { 2, 0x10, "Boot address" },
    {}
  };

  const struct regitem unknownparams[] = { {} };

  const struct {
    const struct regitem *list;
    char *desc;
  } modetab[4] = {
    { factoryparams, "Factory mode" },
    { applicationparams, "Application mode" },
    { unknownparams, "Unknown mode" },
    { applicationparams, "Application mode with watchdog enabled" },
  };

  const struct regitem *item;

  alt_putstr("\r\n----------------   BASE IMAGE   ---------------------\r\n\r\n");

  mode = IORD_32DIRECT(REMOTE_UPDATE_0_BASE, 0) & 3;

  alt_printf("Remote update register dump\r\nMode: %s\r\n",
	     modetab[mode].desc);

  alt_putstr("\r\nParameters:\r\n");

  for (item = modetab[mode].list; item->desc; item++) {
    int addr = (item->param + item->read_source) * 4;
    alt_printf("%s (0x%x) = 0x%x\r\n", item->desc, addr,
	       IORD_32DIRECT(REMOTE_UPDATE_0_BASE, addr));
  }

  if (mode == 0) { // Factory mode only
    usleep(500000);
    IOWR_32DIRECT(REMOTE_UPDATE_0_BASE, 0x30, 0); // Turn off watchdog
    IOWR_32DIRECT(REMOTE_UPDATE_0_BASE, 0x40, 0x100000);
    IOWR_32DIRECT(REMOTE_UPDATE_0_BASE, 0x74, 1);
  }

  /* Event loop never exits. */
  while (1);

  return 0;
}

Tests results

The test program above was compiled and included in the bitstream that was loaded into flash address 0 (Factory image).

The first alt_putstr was then changed to say “Application Image”, and the compiled version of that was included in the bitstream loaded at address 0x100000 of the flash (Application Image).

Standard output was directed to a physical UART (instead of the JTAG UART) for the purpose of this test (Eclipse’s JTAG UART console didn’t like these games with configurations).

And then I powered on:

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0xc
Previous reconfiguration trigger source (0x64) = 0x0
One before previous Boot Address (0x48) = 0xc
One before previous reconfiguration trigger source (0x68) = 0x0
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

----------------   APPLICATION IMAGE   ---------------------

Remote update register dump
Mode: Application mode

Parameters:
Current Machine State Mode (0x0) = 0x1
Watchdog timeout value (0x24) = 0x1ffe0008
Watchdog enable bit (0x34) = 0x0
Boot address (0x48) = 0x400000

Note that if the register writes in the example are done before showing the registers, these following two lines would replace their respective outputs in the Base Image parameter list:

Watchdog enable bit (0x3c) = 0x0
Boot address (0x4c) = 0x100000

The same, with the application image wiped out (zeros):

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0xc
Previous reconfiguration trigger source (0x64) = 0x0
One before previous Boot Address (0x48) = 0xc
One before previous reconfiguration trigger source (0x68) = 0x0
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0x400000
Previous reconfiguration trigger source (0x64) = 0x4
One before previous Boot Address (0x48) = 0xc
One before previous reconfiguration trigger source (0x68) = 0x0
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0x400000
Previous reconfiguration trigger source (0x64) = 0x4
One before previous Boot Address (0x48) = 0x400000
One before previous reconfiguration trigger source (0x68) = 0x4
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

[ ... etc ... ]

The same, with the Application image loaded in place, but with a small error (changed a single bit):

(this caused a CRC error)

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0xc
Previous reconfiguration trigger source (0x64) = 0x0
One before previous Boot Address (0x48) = 0xc
One before previous reconfiguration trigger source (0x68) = 0x0
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0x400000
Previous reconfiguration trigger source (0x64) = 0x8
One before previous Boot Address (0x48) = 0xc
One before previous reconfiguration trigger source (0x68) = 0x0
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0x400000
Previous reconfiguration trigger source (0x64) = 0x8
One before previous Boot Address (0x48) = 0x400000
One before previous reconfiguration trigger source (0x68) = 0x8
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

[ ... etc ... ]

Loading with JTAG: I set up both flash images properly, powered up so the FPGA stayed on the Application Image. At that point, I loaded the SOF of the Factory bitstream into the FPGA through JTAG (with a USB Blaster). The JTAG operation yielded this:

----------------   BASE IMAGE   ---------------------

Remote update register dump
Mode: Factory mode

Parameters:
Current Machine State Mode (0x0) = 0x0
Factory Boot Address (0x40) = 0x0
Previous Boot Address (0x44) = 0x400000
Previous reconfiguration trigger source (0x64) = 0x10
One before previous Boot Address (0x48) = 0xc
One before previous reconfiguration trigger source (0x68) = 0x0
Early confdone check bits (0x1c) = 0x1
Watchdog timeout value (0x2c) = 0x0
Watchdog enable bit (0x3c) = 0x1
Boot address (0x4c) = 0x0
Force internal oscillator (0x5c) = 0x1

----------------   APPLICATION IMAGE   ---------------------

Remote update register dump
Mode: Application mode

Parameters:
Current Machine State Mode (0x0) = 0x1
Watchdog timeout value (0x24) = 0x1ffe0008
Watchdog enable bit (0x34) = 0x0
Boot address (0x48) = 0x400000

When loading the same bitstream through JTAG once again the same result is obtained, only with “One before previous reconfiguration trigger source” set to 0x10 as well.