my tech blog

i.MX51 EIM bus clarified

These are my notes as I made my way in grasping how the EIM bus works. Unfortunately, the information in the reference manual was far from complete, so except for the list of acronyms, this page consists of things I found out by reverse engineering the bus.

The actual bus cycle outlines and timings are given in section 4.6.7 of the datasheet. There are also timing diagrams in section 63.8 of the reference manual, which include the internal AXI bus as well, which may be a bit confusing.

I worked with an Armadeus APF51 board, which has a 16-bit multiplexed bus connected to the Xilinx Spartan-6 FPGA. My preferences and observations are related to this board.

I wrote some code for the FPGA and processor on the board, for the sake of this reverse engineering, which is available in another post of mine. I’ve also published some oscilloscope shots during the process, which you may look at here.

EMI, EIM and WEIM

Freescale’s nomenclature regarding the external bus is somewhat inconsistent, so here’s a quick clarification.

EMI is the external memory interface, which is the general crossconnect for making AXI masters talk with their slaves, internal or external. EIM is the external interface module, which is embodied as the Wireless External Interface Module (WEIM). Pin names in the datasheet and schematics are used in EIM terms, but the reference manual uses WEIM nomenclature. Section 4.6.7.1 in the datasheet contains a table which connects between the different names used for the same signals. It’s a must to look at.

Parameter acronyms

Section 63 in the reference manual covers the external memory interface. As it uses acronyms pretty extensively, sometimes with forward reference, I ended up making a cheat sheet.

Here’s a list of bus parameter acronyms, along with the values I chose by default for my tests with the bus (which are pretty much my final preferences). The acronyms themselves were taken directly from chapter 63.4.3 in the Reference Manual.

From Chip Select x General Configuration Register 1

• PSZ = 0, Page Size
• WP = 0, Write Protect
• GBC = 1, Gap Between Chip Selects. That is, the gap between asserting one CS pin and then asserting another pin. Not to be confused with CSREC. At least 1 in sync mode.
• AUS = 1, Address Unshifted.
• CSREC = 1, CS Recovery, minimal unused clock cycles on the bus between operations on the same CS pin (back to back access). At least 1 in sync mode.
• SP = 0, Supervisor Protect
• DSZ = 1, Data Port Size (16 bit on Armadeus)
• BCS = 0, Bus Clock Start, for fine shifting of the burst clock
• BCD = 0, Bus Clock Divider (zero means don’t divide). Note that for BCM = 1 (BCLK running continuously), BCD only controls the bus signals’ rate, and not the BCLK signal itself. More about this is the “gotcha” section.
• WC = 0, Write Continuous
• BL= 0, Burst Length. When the internal DMA functional unit accesses the EIM bus, BL must be set to cover the longest burst possibly required (typically 32 bytes), or data is corrupted when the SDMA engine induces a burst longer than BL allows.
• CREP = 1, Configuration Register Enable Polarity
• CRE = 0, Configuration Register Enable (disabled in my case)
• RFL/WFL = 1, Read/Write Fix Latency. Whether WAIT should be ignored (it is for RFL= WFL =1)
• MUM = 1. Multiplexed Mode. If data and address are multiplexed on the same lines. True in this case.
• SRD/SWD = 1, Synchronous Read/Write Data. Whether bus operations are synchronous. They are, of course.
• CSEN = 1, CS Enable. If this isn’t set, attempting to write to the relevant region ends up with a bus error (and an oops in the Linux kernel).

From Chip Select x General Configuration Register 2

• DAP = 0, Data Acknowledge polarity, irrelevant in sync mode
• DAE = 0, Data Acknowledge Enable, irrelevant in sync mode
• DAPS = 0, Data Acknowledge Polling Start, irrelevant in sync mode
• ADH = 0, Address Hold Time

From Chip Select x Read/Write Configuration Register 1 and 2

• RWSC/WWSC = 1, Read/Write Wait State Control. The number of wait states on a bus transaction, given in BCLK cycles (as opposed to WEIM cycles). Must be at least 1.
• RADVA/WADVA = 0, Read/Write ADV Assertion. Tells when ADV is asserted in WEIM cycles. Note that while ADV is asserted, the address is present on the multiplexed address/data lines, no matter what, even at the cost of some or all data not appearing on the bus at all.
• RADVN/WADNV = 0, Read/Write ADV Negation. How many extra WEIM cycles ADV stays asserted. The formula given in the reference manual says that by default, ADV is asserted for a BCLK worth’s of time, starting as required by RADVA/WADVA, and whatever is given by RADVA/WADVA is added to that.
• RAL/WAL = 0, Read/Write ADV Low. When this bit is set, RADVN/WADVN are ignored, and ADV is asserted during the entire bus operation.
• RCSA/WCSA = 0, Read/Write CS Assertion. The number of WEIM clocks the CS’s assertion is delayed.
• RCSN/WCSN = 0, Read/Write CS Negation. Ignored in sync mode.
• RBEA /WBEA= 0, Read/Write BE Assertion. The number of WEIM clocks to delay BE assertion.
• RBEN/WBEN=0, Read/Write BE Negation. Ignored in sync mode.
• RBE = 1, Read BE enable
• WBED = 0, Write BE disable
• OEA = 0, (Read) OE Assertion. How many WEIM clock cycles to delay the OE signal assertion. Note that unlike other delay parameters, OEA is relative to the first data clock cycle, so OE will mean “expecting data on data lines on this clock cycle” for OEA=0. It works this way in multiplexed mode, at least.
• OEN = 0, (Read) OE Negation. Ignored in sync mode.
• APR = 0, (Read) Asynchronous Page Read. Must be held zero in sync mode.
• PAT = 0, (Read) Page Access Time. Ignored when APR=0 (and hence ignored in sync mode)
• RL = 0, Read Latency.
• WEA = 0, WE Assertion. How many WEIM clock cycles to delay WE assertion
• WEN = 0, WEN Negation. Ignored in sync mode
• WBCDD = 0, Write Burst Clock Divisor Decrement

And finally, from the WEIM Configuration Register

• WDOG_LIMIT = 0, Memory Watchdog. Not really necessary in sync mode
• WDOG_EN = 0, Memory Watchdog Enable.
• INTPOL = 1, Interrupt Polarity
• INTEN = 0, Interrupt Enable
• BCM = 1, Burst clock mode. When asserted, the BCLK runs continuously, instead of only during bus transactions.
• GBCD = 0, General Burst Clock Divisor. Used globally for all CS spaces instead of each space’s BCD when BCM=1. See warning below.

Some gotcha notes

• Clock division with BCD and GBCD is a messy issue, and clock division is best avoided when running the clock continuously (BCM=1). The thing is that while GBCD indeed controls the division of the BCLK signal itself, the bus signals are governed by the clock divided by the individual BCD. So if BCD != GBCD for a specific CS region, the bus signals are completely unrelated to BCLK. But even if BCD and GBCD are equal, there is no guaranteed phase relation between them (as has been observed) because they’re generated by two unrelated clock dividers. So the BCLK signal is useless unless BCD = GBCD = 0.
• Most delays are given in WEIM clocks, not BCLK clocks. This makes no difference as long as BCLK runs at WEIM rate (BCD=0), but if BCLK is divided, even for the sake of getting clear transitions on an oscilloscope, this needs to be taken into account.
• For most parameters, delays in assertions make the signal’s assertion duration shorter. It’s not a time shift, as the deassertion doesn’t move.
• All signals, except OE, are asserted at the same first clock cycle of the bus access, unless delayed by the parameters below. This includes WE and BE, which one could mistakenly expect to be asserted when data is available. OE is indeed asserted when data is due to be supplied, and its assertion timing parameter works relatively to that clock cycle.
• Delaying and/or extending the ADV signal with RADVA/WADVA  and RADVN/WADVN on a multiplexed bus causes address to be present during the relevant time periods without changing other timings, with precedence to address. So time slots which would otherwise be used for data transmission are overridden with address on the bus, possibly eliminating data presence on the bus completely. This can be compensated with wait states, but note that wait states count in BCLK cycles, while the ADV adjustments count WEIM cycles.
• The OE signal is somewhat useless as an output-enable when BCLK runs at 95 MHz: If used directly to drive tri-state buffers, the round-trip from its assertion to when data is expected is ridiculously short: The data-to-rising clock setup time is 2 ns, according to the datasheet (section 4.6.7.3, table 53, parameter WE18). OE is asserted on the falling edge of the clock just before the rising edge, for which the data is sampled, with a delay of up to 1.75 ns (same table, WE10). At a clock cycle of 10.5 ns (95 MHz), this half-clock gap between these two events is 5.25 ns, leaving 5.25 – 2 – 1.75 = 1.5 ns for the bus slave to take control of the bus. Not realistic, to say the least. So the bus slave must deduce from WE whether the bus cycle is read or write, and drive the bus according to predefined timing. As for bursts, I’m not on the clear on whether bursts can be stalled in the middle and how OE behaves if that is possible. The timing diagram in section 63.8.7 of the Reference Manual does not imply that OE may get high in the middle of a burst. On the other hand, it shows OE going down together with ADV, which surely isn’t the case as I observed (maybe because I ran on a multiplexed bus?).

Write data cycles

Reminder: This entire post relates to a 16-bit address/data multiplexed EIM bus.

The simplest write data cycle (as defined by parameter settings above) consists of three BCLK cycles. On the first one, the lower 16 bits of the address is present on the bus, and ADV is held low. On the two following clock cycles, ADV is high and the 32-bit word is transferred over the data lines. CS and WE are held low during the entire cycle (three BCLKs). And no, the upper 16 bits of the address are never presented on the bus.

For BCD=0 (BCLK = WEIM clock), the master toggles its signals on the falling edge of BCLK, and samples signals from the slave on its rising edge. This holds true for all bus signals.

Data is sent in little endian order: A 32-bit word is sent with its lower 16-bit part (bits 15:0) in the first clock cycle, and the higher 16 bits (bits 31:16) in the second cycle. The 16-bit words are sent naturally (that is, each DA[15:0] is a consistent 16-bit word).

With AUS=1 (address unshifted) the address’ lower 16 bits appear naturally on the address cycle. For example, writing to offset Ox30 (to the CS2 address range)  sets bits DA[4]=1 and DA[5]=1 (only) in the address clock cycle.

With AUS=0 (address shifted according to port size) the address shown on the bus is shifted by one bit, since there are two bytes in the port size’s width. Hence writing to offset Ox60 sets bits DA[4]=1 and DA[5]=1 (only) in the address clock cycle.

As said above (and verified with scope), WADVA and WADVN don’t just move around the ADV signal’s assertion, but also the times at which the address is given on the bus, possibly overriding time which would otherwise be used to transfer data. It’s the user’s responsibility to make sure (possibly with wait states) that there is enough time for data on the bus.

Wait states, as set by WWSC extend the first data cycle, so the lower 16 bits of data are held on the bus for a longer time. If WWSC=0, only the upper 16 bits are shown (the first data cycle is skipped) but this is an illegal setting anyhow. Again, note that WWSC counts BCLK cycles, as opposed to almost every other timing parameter.

For BCD=0 (only) the data lines’ levels are held with the last written value until the next bus operation. This feature (which AFAIK is not guaranteed by spec) is used by the FPGA bitstream loader: A word is written, and the FPGA’s clock is toggled afterwards to sample the data which is left unchanged (which is maybe why you can’t load the bitstream file from the on-board flash, as indicated in Armadeus’ wiki page). When BCD>0, the data lines go to zero after the cycle is ended.

Read data cycles

Read data cycles are in essence the same as write cycles, only the bus slave is expected to drive the data lines in the same time slots for which the master drove them on a bus write operation. On read cycles, the master samples the data lines on rising BCLK edges, which is symmetric to the slave sampling the same lines on write cycles. The endianess is the same of course.

OE is asserted on the same WEIM cycle for which ADV is deasserted, which is one BCLK cycle after CS’s assertion with the default parameters given above. WE is not asserted in write cycles, of course.

As mentioned in the “gotcha notes” above, the OE line is pretty useless in high bus rates, as it’s asserted on the falling edge of BCLK coming just before the rising edge on which the data is sampled. This gives by far too little time for the slave to respond. So the slave should figure out the correct behavior and timing according to WE and CS.

Using the WAIT signal

In order to use the WAIT signal for adding wait states on the fly, the respective RFL/WFL parameter need to be zeroed. If RFL=0, also set RWSC=2 (instead of the minimal 1), or four useless and unused wait states will be added to each bus cycle, most likely due to an illegal condition in the master’s bus state machine. This is not necessary for writes (i.e. it’s OK to have WFL=0 and WWSC=1).

WAIT is active low. When the master samples WAIT low (on a rising BCLK edge) it considers the following rising BCLK edge as a wait state. It’s or course legal to assert WAIT for consecutive BCLK cycles to achieve long waits. If WAIT is not asserted, the bus runs according to RWSC/WWSC. Each BCLK cycle is considered independently, so when a 32-bit word is transmitted on two BCLK cycles, wait states can be inserted between 16-bit words, resulting in expected behavior. There is no need to consider the fact that these 16-bit words form a 32-bit word when dealing with wait state behavior.

As one would expect, if WAIT is asserted with respect to a bus cycle that wouldn’t occur anyhow (i.e. the last BCLK cycle in a transmission), it’s ignored.

In read cycles, all this boils down to that if the master sampled WAIT low on BCLK rising edge n, no data will be sampled from data lines on rising edge n+1, and the entire bus operation is extended by another BCLK cycle. RWSC must be set to a minimum of 2, since WAIT is ignored on the first bus cycle (on which ADV is asserted) , so the first chance to request a wait state is on the second cycle, which must be a wait state anyhow. If RWSC=1 and RFL=0, the master will insert this wait state anyhow, but misbehave as just mentioned above. Even though counterintuitive, the master may very well sample data from the bus on a BCLK rising edge for which WAIT is asserted. This will make the following bus cycle a wait state, as one can deduce from the mechanism. But it may come intuitively unnatural that an asserted WAIT and valid data are sampled on the same BCLK.

For write cycles, if the master samples WAIT asserted on a rising edge of BCLK, it will behave as usual on the falling BCLK immediately following it, but will not update data lines on the falling BCLK edge afterwards (and hold the internal state machine accordingly). This follows the overall scheme of wait states described above. Unlike read bus operations, this holds true for the ADV cycle as well, so it’s possible to get wait states on the first data transaction by asserting wait on the first BCLK cycle. For WWSC=1, this means in practice to have WAIT asserted while there is no bus activity, because there’s half a clock cycle between the assertion of CS, ADV and other bus signals, and the sampling of WAIT in this case. In order to give the slave time to assert WAIT depending on the bus operation’s nature, WWSC has to be increased to 2 at least.

Bus frequency

The bus EIM bus frequency is derived by default from PLL2 (at least on Armadeus), which is a 665 MHz clock, divided according to the emi_slow_podf field in the CBCDR register (see 7.3.3.6 in the reference manual). On the Armadeus platform, this field is set to 6 by default, so the clock is divided by 7, yielding a bus clock of 95 MHz. To change it, the following code snippet applies:

#define MXC_CCM_CBCDR 0x14
u32 temp_clk;
const emi_slow_podf = 7;

temp_clk = __raw_readl( MX51_IO_ADDRESS(MX51_CCM_BASE_ADDR)+ MXC_CCM_CBCDR );

__raw_writel( (temp_clk & (~0x1c00000)) | (emi_slow_podf << 22),
               MX51_IO_ADDRESS(MX51_CCM_BASE_ADDR)+ MXC_CCM_CBCDR )

The above code reduces the EMI clock to 83.125 MHz (divide by 8).

Note that there’s a little remark in section 4.6.7.3 of the datasheet (Table 53, WEIM Bus Timing Parameters), in footnote 4, saying “The lower 16 bits of the WEIM bus are limited to 90 MHz”. Indeed, running 95 MHz has proven to have rare bus malfunctions (after half a gigabyte of data or so, probably causing some local heating), taking the form of sporadic bus cycle missed, causing injection of bogus data or data being missed.

The FPGA+ARM Armadeus APF51 board: Buildroot notes

Scope

This post was spun off the main post regarding setting up the Armadeus board for Embedded Linux on ARM and Xilinx Spartan-6 FPGA. It covers my own little war story as I set up the Buildroot SDK, so I could have my own cross compiler and Linux kernel to work with.

As kernel.org happened to be down, this was a (forced) opportunity to look a bit deeper into how things work under the hood. These are my notes, reflecting my own experience, and isn’t a substitute for any official documentation.

Setting up the build environment

Downloaded armadeus-4.0.tar.bz2 from the SourceForge page. There are binaries there too (Linux kernel, rootfs and U-boot image). Opened the tarball, changed directory, and followed the instructions and deduced that the apf51_defconfig is valid even though not documented:

$ make apf51_defconfig

that downloaded buildroot-2010.11.tar.bz2 from http://buildroot.uclibc.org/downloads/buildroot-2010.11.tar.bz2 ran some script (a lot of patches applied) and then opened the config menu.

Config menu settings:

• Under Target Options > Armadeus Device Support > Size of a single RAM chip, change it to 512 MB (to match my board). On second thought, the kernel detects all 512MB anyhow (despite this parameter set at 256 MB) so it looks like there’s no need to change this.
• Under Build options, set Number of jobs to run simultaneously to the number of cores in the compiling computer (8 for a quadcore with hyperthreading). So it finishes today.

and then simply went “make”. The computer spits a lot of mumbo-jumbo. Loads, really.

In retrospective, I would consider running “make source” to download whatever needs to be downloaded. In my own build, this turned out to be where problems occured.

Note: Always run make from armadeus-4.0 and not from “buildroot”. Doing the latter will work properly for several components, but will fail every now and then with weird errors. It’s very easy to make this mistake, in particular when kickstarting the build after download failures.

After the build finishes, the interesting stuff is in

• buildroot/output/images — Root, kernel and bootloader images.
• buildroot/output/build/staging_dir — Things to run on the host computer, such as the cross-compiler (e.g. buildroot/output/build/staging_dir/usr/arm-unknown-linux-uclibcgnueabi/bin/gcc)

Issues resulting from kernel.org being down

The build got stuck at downloading the Linux kernel. kernel.org happened to be down for maintenance, and Armadeus’ FTP site didn’t supply it. So I fetched  linux-2.6.38.1.tar.bz2 from a Linux mirror and put the file in /path/to/armadeus-4.0/buildroot/downloads and ran make again.

And then a git clone for linux-firmware.git failed, since it’s from kerner.org as well. So I went for the safe “trust anyone” strategy, and went for any repository I could find. So instead of the failed

git clone git://git.kernel.org/pub/scm/linux/kernel/git/dwmw2/linux-firmware.git /home/eli/armadeus/build/armadeus-4.0/buildroot/output/build/firmware-af5222c5ded5d944267acfbd001571409bea7eeb

I went

$ git clone https://github.com/mdamt/linux-firmware.git /home/eli/armadeus/build/armadeus-4.0/buildroot/output/build/firmware-af5222c5ded5d944267acfbd001571409bea7eeb

which wasn’t really helpful, since it didn’t satisfy some Make rule. Running “make -d” I found out that the Make rules would be happy with a tarball, so I went

$ tar -czf ../../downloads/firmware-af5222c5ded5d944267acfbd001571409bea7eeb.tar.gz firmware-af5222c5ded5d944267acfbd001571409bea7eeb

and ran “make” again after removing the directory from which I had created the tarball.

Other problems

I also needed to download iproute2-2.6.35.tar.bz2 from Fedora’s repository into armadeus-4.0/buildroot/downloads, as it couldn’t be downloaded from linuxfoundation.org. This was starting to become routine.

And then I got

/usr/bin/ld: cannot find -lc

while building host-module-init-tools-3.12. And as buildroot’s docs guessed correctly, I am using Fedora, so it was all down to a

# yum install glibc-static

and go “make” again.

Just a silly mistake

When attempting to build the kernel I got

drivers/usb/Kconfig:169: can't open file "drivers/armadeus/Kconfig"

which was a direct result of the broken link of the “armadeus” subdirectory in “output/build/linux-2.6.38.1/drivers/”.  But that was just a silly mistake: I ran “make” from “buildroot” and not from the armadeus-4.0 directory, so the Makefile setting up the necessary environment variable was never set.

Doing it all over again

The idea is to rerun everything offline, i.e. without any downloads from the internet. One can’t always rely on that servers will be up and ready… The correct way to do this is to define the download directories before starting off, but I didn’t bother.

My motivation for doing this was that after kickstarting the build so many times, it crossed my mind that I may have messed up something without noticing. So I figured it would be best to rerun it all in one go.

So first of all, move the already built directory to finished-armadeus-4.0. It’s some 3.5 GB, but we’ll need only the download directories.

$ tar -xjf armadeus-4.0.tar.bz2
$ cd armadeus-4.0
$ cp -r ../finished-armadeus-4.0/downloads/ .
$ cp -r ../finished-armadeus-4.0/buildroot/downloads/ buildroot/
$ make apf51_defconfig

Which brings us back to the configuration menu, after which a simple “make” does the work.

It’s also worth to mention, that according to the docs/README file, “make source” will download all necessary sources, which is a good starter.

The FPGA+ARM Armadeus APF51 board: Setup notes

Scope

I got myself an Armadeus APF51 board for some work combining a fullblown ARM processor running embedded Linux with a recent Xilinx FPGA. I wrote down some setup notes for possibly future need while setting it up for work, and they are below. There is not really something undocumented here, but it’s more convenient to have the info organized according to my own workflow.

This is not a substitute for Armadeus own documentation, of course. Just my own jots.

First steps

Power supply: Completely inconsistent about the voltage. In the datasheet is says both 5V and 8V, the Wiki says 6V. So I looked up the power regulator on the schematics, and it turns out it can take 3V to 28V.

But there’s a 1.6A fuse on the power input, so maybe they’re afraid that the fuse will blow due to a low input voltage (= high current for the same power needs). And still.

It’s also worth to note that the IO_L41N_GCLK8_M1CASN_1 wire, which goes to the board’s FPGA button, also goes to one of the board’s pin headers. This is an unfortunate miswiring, because the wire is pulled up, so it’s not so good as an output from the FPGA. I wanted a continuous row of pins to use for debugging, but it didn’t work out that well on this pin header. I suppose this wouldn’t happen, had the names of the wires been less quirky.

And if we’re at it, the button appears to have had some mechanical problems, so there was a need to push it firmly quite a few times before it got fairly responsive.

The connection to the PC goes through a Serial-to-USB adapter, MCP2200, to be accurate. Too bad I needed to download the drivers for Windows XP here (on this page). And there’s also a hotfix that needs to be downloaded from Microsoft. This is more or less where I asked myself why I bother to set it up on Windows.

Surprisingly enough, the serial communication works like a charm with my Fedora 12. This is one of those rare cases where Linux has better driver support…

Issuing the “dhcp” command to U-boot makes a DHCP request on the network, and also sets the Linux boot parameters so Linux also has the same network configuration. If the network connection isn’t necessary with U-boot, it’s also OK to go

# udhcpc --now

after Linux has booted.

To get telnet access, activate inetd simply with

# inetd

(I added this as a last row in /etc/init.d/rcS. It’s a simple, working and politically incorrect hack).

There are two users of interest in the system: “root” and “default”. Neither require a password, but only “default” works on a telnet connection (an old useless security precaution, I guess).

Setting up the buildroot software environment

This involves downloading the buildroot bundle, setting it up, and running the build process through. I’ve dedicated a separate post to my own experiences doing this.

Note that I assume that this process has been run through properly below.

Booting over TFTP

# mkdir /var/lib/tftpboot/armadeus
# cp /path/to/armadeus-4.0/buildroot/output/images/apf51-linux.bin /var/lib/tftpboot/armadeus

And then on the U-boot console:

BIOS> dhcp                                                                     
FEC_MXC: Link is up - 100/Full                                                 
BOOTP broadcast 1                                                              
DHCP client bound to address 10.1.1.112                                         
BIOS> tftpboot 0x90800000 10.1.1.111:/armadeus/apf51-linux.bin                   
FEC_MXC: Link is up - 100/Full                                                 
Using FEC_MXC device                                                           
TFTP from server 10.1.1.111; our IP address is 10.1.1.111
Filename '/armadeus/apf51-linux.bin'.                                          
Load address: 0x90800000                                                       
Loading: #################################################################     
 #################################################################     
 ################################################                      
done                                                                           
Bytes transferred = 2611736 (27da18 hex)                                       
BIOS> setenv bootargs console=ttymxc2,115200 mtdparts=mxc_nand:1M(U-boot)ro,1M(U-boot_env),1M(firmware),8M(kernel),-(rootfs) ubi.mtd=rootfs root=ubi0:rootfs rootfstype=ubifs
BIOS> bootm 0x90800000                                                         
## Booting kernel from Legacy Image at 90800000 ...                            
 Image Name:   Linux-2.6.38.1                                                
 Image Type:   ARM Linux Kernel Image (uncompressed)                         
 Data Size:    2611672 Bytes =  2.5 MB                                       
 Load Address: 90008000                                                      
 Entry Point:  90008000                                                      
 Verifying Checksum ... OK                                                   
 Loading Kernel Image ... OK                                                 
OK                                                                              

Starting kernel ...                                                             

Uncompressing Linux... done, booting the kernel.                               
Linux version 2.6.38.1 (eli@ocho.localdomain) (gcc version 4.4.5 (Buildroot 2010.11) ) #1 PREEMPT Mon Oct 3 12:22:26 IST 2011
CPU: ARMv7 Processor [412fc085] revision 5 (ARMv7), cr=10c53c7f                
CPU: VIPT nonaliasing data cache, VIPT aliasing instruction cache              
Machine: Armadeus APF51
(...etc)

Note that the changes done with setenv do not survive to the next boot.  Use the “saveenv” command to save the environment to flash.

The above can be shortened with

BIOS> setenv netkernel 'dhcp; tftpboot 0x90800000 ${serverip}:/armadeus/apf51-linux.bin; setenv bootargs ${console} ${mtdparts}; run addipargs addubifsargs; bootm 0x90800000'
BIOS> run netkernel

The ${serverip} variable was used here, so the underlying assumption is that the DHCP server is the same as the TFTP server (I’m not on the clear on whether ${serverip} points at the DHCP server or the server given as the “next server” by DHCP).

And the nice thing is that “saveenv” will save the netkernel command if issued after setenv.

Root over NFS

Create a directory to be exposed over NFS, and go

# tar -xf /path/to/armadeus-4.0/buildroot/output/images/apf51-rootfs.tar

in the directory just created.

For running the kernel from flash, but root on NFS,:

BIOS> setenv rootnfsboot 'dhcp; setenv bootargs ${console} ${mtdparts} rootfstype=nfs root=/dev/nfs nfsroot=${serverip}:/armadeus_root; run addipargs; setenv autostart yes;nboot.jffs2 90800000 0 ${kernel_offset}'

Again, ${serverip} is used, so this works if the NFS server is the same as DHCP server (or the “next-server” option has been fiddled with, which may or may not be helpful).

For running both kernel and root from server, the command goes:

BIOS> setenv netboot 'dhcp; tftpboot 0x90800000 ${serverip}:/armadeus/apf51-linux.bin; setenv bootargs ${console} ${mtdparts} rootfstype=nfs root=/dev/nfs nfsroot=${serverip}:/armadeus_root; run addipargs; bootm 0x90800000'

And when this is to be permanent, change bootcmd (‘run ubifsboot’ by default):

BIOS> setenv bootcmd 'run rootnfsboot'
BIOS> setenv bootdelay 2
BIOS> saveenv

The bootdelay variable makes the time window for halting automatic boot significantly shorter, but hey, I’m quick.

This was my final preference: Kernel from NAND (it’s read only anyhow, so it won’t suffer from crashes) but root over NFS.

Compiling a userspace applications

A typical makefile for crosscompilation of a simple userspace application can look like

GNUPREFIX=/path/to/armadeus-4.0/buildroot/output/build/staging_dir/usr/bin/arm-unknown-linux-uclibcgnueabi-

CC=$(GNUPREFIX)gcc
AR=$(GNUPREFIX)ar
AS=$(GNUPREFIX)as
CXX=$(GNUPREFIX)g++
LD=$(GNUPREFIX)ld
STRIP=$(GNUPREFIX)strip

CFLAGS=-Wall -I. -O3 -lm

APPLICATION=sine

OBJECTS=#somefile.o somefile2.o etc

all: $(APPLICATION)

$(APPLICATION): $(OBJECTS) $(APPLICATION).o
 $(CC) $(CFLAGS) $(OBJECTS) $(APPLICATION).o -o $(APPLICATION)

clean:
 rm -f *~ $(APPLICATION) *.

Of course, “/path/to” is the path to where the Armadeus Buildroot is placed.

Note the -lm flag, which is there to demonstrate support of libm.

Compiling a kernel module

It’s worth to note, that CONFIG_MODVERSIONS is not set on the default kernel configuration (as seems to be the widespread setting), so one can compile a kernel module against a different kernel than the one the module will run on. This is pretty convenient, but it’s also an opening to nasty crashes if the kernel module expects a different API than the one it finds in the running kernel. So by all means, keep improvisations to a minimum.

This is the Makefile for compiling the frandom module with a simple “make”:

# Makefile for 2.6 kernels

export CROSS_COMPILE=/path/to/armadeus-4.0/buildroot/output/build/staging_dir/usr/bin/arm-unknown-linux-uclibcgnueabi-

ifneq ($(KERNELRELEASE),)
obj-m    := frandom.o

else
KDIR := /path/to/armadeus-4.0/buildroot/output/build/linux-2.6.38.1
PWD := $(shell pwd)

default:
 $(MAKE) CROSS_COMPILE=$(CROSS_COMPILE) -C $(KDIR) SUBDIRS=$(PWD) modules

clean:
 @rm -f *.ko *.o modules.order Module.symvers *.mod.? *~
 @rm -rf .tmp_versions module.target
endif

Again, “/path/to” is the path to where the Armadeus Buildroot is placed. Note that the sub-make is given CROSS_COMPILE explicitly. It’s not clear to me why this is necessary, but without this, the native compiler runs and complains about not recognizing the architecture.

The external memory interface (EIM)

This issue is covered in several posts. You may want to start on this one.

Pad multiplexing on the i.MX51 processor

A lot of pins can be connected to several different internal modules, depending on the setting of the IOMUX module (there are independent settings for more or less each pin), as detailed in Appendix A of the device’s reference manual (MCIMX51RM.pdf). The MUX settings is done with registers named e.g. IOMUXC_OBSERVE_MUX_n, IOMUXC_SW_MUX_CTL_PAD_EIM_DAn and IOMUXC_SW_MUX_CTL_PAD_EIM_An (where “n” is a number). These registers are based at 0x73FA8000 (IOMUXC, or MX51_IOMUXC_BASE_ADDR in mx51.h header file), to which the offsets given in Appendix A are added.

The mx51_map_io() function in arch/arm/mach-mx5/mm.c calls mxc_iomux_v3_init() (defined in arch/arm/plat-mxc/iomux-v3.c) to setup the base pointer then used by mxc_iomux_v3_setup_pad() and mxc_iomux_v3_setup_multiple_pads() in the same C file. The latter is called by apf51_board_init() in arch/arm/mach-mx5/board-apf51.c with the apf51_pads array, defined in the same C file. The initialization of the array is based upon constants in arch/arm/plat-mxc/include/mach/iomux-mx51.h, which is the actual place to look for how the pads are set up. The IOMUX_PAD() macro is defined in arch/arm/plat-mxc/include/mach/iomux-v3.h, according to which the second argument is the offset of the IOMUX register, and the third one is the mux mode to set.

To make things even more complicated, the iomux-mx51.h defines pad definitions such as _MX51_PAD_DI1_D0_CS__GPIO3_3 (note the ‘_’ prefix) pointing at offset 0x2b4 and IOMUX mode 4 (ALT4) but without any pad control. Later down in the file, MX51_PAD_NANDF_WE_B__GPIO3_3 (no prefix) is defined by ORing the prefixed constant with the similar name with MX51_GPIO_PAD_CTRL. So it all makes sense, but is nevertheless pretty tricky to figure out. See page A-184 (page 3370 in the pdf file) for a confirmation of that this is the right thing to do.

The pad mux configuration is by no means complete. For example, the EIM_DAn pads only have the definition for ALT0 (which is using these pads as address-data lines), but the legal ALT1 definition is absent. Not that someone is expected to use it, and still. It also appears like the corresponding registers are never set, but that the default, which is ALT0, is relied upon.

So the bottom line of all this is that the if you want to know how the pads are multiplexed, the apf51_pads array in arch/arm/mach-mx5/board-apf51.c tells the story.

The initialization of apf51_fpga_pre()

In drivers/armadeus/fpga/dev_tools/loader/apf51-fpga-loader.c the apf51_fpga_pre() function has a section going as follows:

 temp_rcr1 =  __raw_readl( MX51_IO_ADDRESS(MX51_WEIM_BASE_ADDR) + MXC_CS1RCR1_ADDR );
 __raw_writel( 0x01000010, MX51_IO_ADDRESS(MX51_WEIM_BASE_ADDR) + MXC_CS1RCR1_ADDR );

 temp_wcr1 = __raw_readl(MX51_IO_ADDRESS(MX51_WEIM_BASE_ADDR) + MXC_CS1WCR1_ADDR);
 __raw_writel( 0x01000008, MX51_IO_ADDRESS(MX51_WEIM_BASE_ADDR) + MXC_CS1WCR1_ADDR );

 /* change emi_clk_sel to ensure blck smaller than 50MHz */
 temp_clk = __raw_readl( MX51_IO_ADDRESS(MX51_CCM_BASE_ADDR)+ MXC_CCM_CBCDR );
 __raw_writel( temp_clk | EMI_CLK_SEL, MX51_IO_ADDRESS(MX51_CCM_BASE_ADDR)+ MXC_CCM_CBCDR );

MXC_CS1RCR1_ADDR is defined as Ox20 and MXC_CS1WCR1_ADDR defined Ox28 in the same file. MX51_WEIM_BASE_ADDR is Ox83fda000, indeed the base address for WEIM related registers. MX51_CCM_BASE_ADDR is Ox73fd4000, and MXC_CCM_CBCDR is Ox14.

CS1RCR1 means Chip Select 1 Read Configuration Register 1. The way it’s sets read wait state control to 1 (minimal value) and sets one clock cycle (not minimum) between the beginning of a read access and CS assertion. CS1WCR1is Chip Select 1 Write Configuration Register 1. It’s set up so write wait states to 1 (minimum) and 1 clock cycle between beginning of write access and CS assertion. So these two don’t supply any drama.

CBCDR is CCM Bus Clock Divider Register. It’s changed to set bit 26, which means to derive clock from AHB clock root.

The WEIM configuration register (having offset Ox90) was found to have the value 00000021, which is the reset value + bit 0 set. Setting bit 0 causes BCLK to run all the time, and not only during bursts, as the reference manual keeps warning.

Some board related linux files:

• Board-specific definitions: arch/arm/mach-mx5/board-apf51.c
• Memory map define file: arch/arm/plat-mxc/include/mach/mx51.h
• Main platform file (?): arch/arm/mach-integrator/include/mach/platform.h

Capture data at 250 MBytes/sec with Linux on Microblaze

The problem

The Xilinx Microblaze soft processor, which is implemented on the FPGA’s logic fabric, is indeed a stable and fully capable processor, but its rather low clock frequency — 70-100 MHz on a Spartan-6 — makes it a problematic candidate for data capture and frame grabbing.

When running Linux on Microblaze, the current kernel allows for a data rate of approximately 1 MByte/sec due to internal overhead. It appears like there’s a lack of optimization of the parts in the kernel copying data.

So while Linux on Microblaze is a great solution for making the FPGA talk with storage and network in a high-level manner, it suffers from a very slow I/O, rendering it useless for data capture to a network shared disk, for example.

How it’s tackled

Technically speaking, the solution is to capture data directly into the processor’s DDR memory using DMA. Since the 32-bit bus’ frequency is the same as the processor’s, even the lower end of 70 MHz allows for a theoretic throughput of 280 MBytes/sec. In practice, the Xillybus IP core has the proven capability of capturing data arriving at a continuous rate of 250 MBytes/sec, on bursts of 8 MBytes each.

Keep it simple

Another bonus with using Xillybus, is that the data is fed into a standard asynchronous FIFO on the FPGA. There is no need to interface with Microblaze’s buses, just connect data and read enable to a FIFO. The IP core supplies additional signals for synchronizing events with the processor, but their use is optional.

On the Linux side, it all boils down to opening a device file, reading data normally, and closing the file. The FPGA can signal EOF (end-of-file), so making a high speed data capture can be done from the shell prompt with the “cat” or “dd” commands. There’s no need to write complicated software nor a driver. Just a single standard UNIX command, and the data is stored in a regular disk file.

One thing to take into account, is that even though an 8 MBytes chunk of data is captured into the processor’s RAM in a split second, the I/O operation of copying it into some other media will typically take around 8 seconds. The memory access is fast, but the processor isn’t all that so.

A few technicalities

A working Linux distribution for Microblaze is available for download at Xillybus’ site. While this distribution has the Xillybus IP core and kernel driver included, that version captures data at the processor’s slow rates. For an evaluation kit supporting fast data capture, please contact Xillybus directly.

Another thing to mention is the reason for the 8 MBytes limit: The DDR memories come in larger sizes, but DMA memory is inherently within Linux kernel space. Allocating large physically continuous segments of RAM is difficult, and doing too well on that can make the entire system unstable.

There is a well-known workaround for this, though: It’s possible to give the kernel a boot parameter limiting the RAM it’s allowed to access. Using this simple trick, it’s possible to use the untouched chunk as a huge buffer. This requires a simple modification on the Xillybus driver. So it’s not so difficult to allow a capture segment of any size, as long as there’s enough RAM for both the buffer and the kernel itself.

Download a Linux distribution for Xilinx’ Microblaze

If you want to get to it, just go to the Xillybus mini-distro for Microblaze’s page.

This distribution is a software kit, which allows you to run a functional Linux system on the SP605 or ML605 hardware evaluation kit (for Spartan-6 and Virtex-6, respectively). All components necessary to build it will be available for download at no cost (partly from Xilinx’ site). The distribution is not a demo kit; it’s a kickstart for a real-life application.

The steps to reaching a functional system include installing pieces of software, and perform specific and rather trivial operations as described in detail. No prior knowledge on either FPGA nor Linux is necessary. Your computer may run either Windows or Linux for running this through.

This distribution will allow you to

• Run a Linux system which is ready for working with, including a DHCP client, telnet, ftp, a small web server, mounting of NFS as well as Windows (SMB) shares. The CompactFlash card will also serve as the local disk which can be written to.
• Easily compile your own user-space applications in with “make” and gcc (a ready-to-go cross-compilation environment will be available for download as well). Compilation is possibly dynamic against libraries which are part of the distro.
• Easily develop your own Linux-reachable peripherals on the FPGA using the Xillybus IP core, and transport data between the FPGA and Linux user space applications with minimal effort.

Required hardware

• A Xilinx SP605 evaluation kit or ML605 ditto.
• A Sandisk CompactFlash card, with at least 512 MB. Let me emphasize this: Make it a Sandisk card, and not anything else. You may get away with using other cards for a one-off loading with data and multiple reads, but the Linux system is going to use this storage as its hard disk. Anything else than Sandisk, including the card which comes with the evaluation kit itself, is most likely to have reliability problems, which will give you a very wrong impression regarding the Linux system’s stability. Believe me, I’ve tried quite a few. Also, try to make the card not bigger than 8 GB, as the SysACE chip’s support of larger cards is questionable.
• A CompactFlash to USB adapter. There is no special requirements here, as long as your operating system recognizes it, and it’s reliable. Which holds true even for those cheap universal card readers available on Ebay.

As mentioned above, this is in the Xillybus mini-distro for Microblaze’s page.

GNUPREFIX=../gnutools/microblazeel-unknown-linux-gnu/bin/microblazeel-unknown-linux-gnu-

CC=$(GNUPREFIX)gcc
AR=$(GNUPREFIX)ar
AS=$(GNUPREFIX)as
CXX=$(GNUPREFIX)g++
LD=$(GNUPREFIX)ld
STRIP=$(GNUPREFIX)strip

CFLAGS=-Wall -I. -O3

APPLICATION=hello

OBJECTS=#somefile.o somefile2.o etc

all: $(APPLICATION)

$(APPLICATION): $(OBJECTS) $(APPLICATION).o
	$(CC) $(CFLAGS) $(OBJECTS) $(APPLICATION).o -o $(APPLICATION)

clean:
	rm -f *~ $(APPLICATION) *.o

# mount -o remount,rw /

# mount -o remount,rw /dev/xsa3

# mount -o remount,ro /

# mount -o remount,ro /dev/xsa3

# halt

#!/bin/sh  

ifconfig eth0 10.12.14.16 netmask 255.0.0.0

/ # chmod a+x /rw/system/userrc

/ # dhcp
udhcpc (v1.17.1) started
Sending discover...
PHY: c0729a0c:07 - Link is Up - 100/Full
Sending discover...
Sending select for 10.12.14.16...
Lease of 10.12.14.16 obtained, lease time 21600
deleting routers
route: SIOCDELRT: No such process
adding dns 10.0.0.1
adding dns 10.0.0.2
/ # eth0: no IPv6 routers present

/ # ifconfig eth0 10.12.14.16 netmask 255.0.0.0

/ # route add default gw 10.11.12.13

# mount -o user=theuser,password=thepass //10.11.12.13/mydisk /mnt/try

# mount 10.11.12.13:/thedir -o nolock,proto=tcp /mnt/try

/ # mount -o remount,rw /

Gnome: Using your own alert sound

As usual with those fancy Gnome themes (as came with Fedora 12 in my case), the basic assumption is that you’re a not-so-clever user, so not too many choices should be left for you. So pick one of those premade alert sounds. Picking your own sound file? Noooo. That’s way too much freedom for a dummy user. So what if that’s possible on Windows? Linux freedom doesn’t mean that end users should get too much control over their computers…

OK, so here’s how to use a custom sound file as your alert. First, generate the desired sound file if necessary, for example, mybeep.ogg, and put is somewhere. Say /usr/share/sounds/ (where there are a lot of other sound clips). Make it a stereo file, possibly WAV (or .ogg) with a sample rate of 44100 Hz.

Then edit /usr/share/gnome-media/sounds/gnome-sounds-default.xml, and add an entry just before the last line (which is </sounds>). It could be something like this:

<sound deleted="false">
 <name>My own beep</name>
 <filename>/usr/share/sounds/mybeep.ogg</filename>
</sound>

All that’s left is to open Preferences > Sound (or gnome-volume-control at the command line) and select the new entry in the list of possible alert sounds.

That’s it. It’s actually simple, once the configuration file is unburied.

Random notes about git

Merely for myself, so I’ll remember how to do it. If I’m doing stupid things, please comment below.

Rule #1

If you’re about to do anything with git being unsure about the consequences, always protect yourself with

$ git branch bettersafe
$ git commit -a
$ git checkout whatever

Assuming that you want to mess on the branch “whatever”

No matter what happens next, you can always return things to where they were by checking out bettersafe. Really. As long as you don’t mess up the bettersafe branch, of course. That is, don’t mention it in any subsequent command, and git won’t touch it. Don’t delete it, don’t use it in merge or rebase commands. Only check it out if things go wrong.

Otherwise, you can rebase, mess around, delete files and squash commits. No commits or other data will be lost, because bettersafe depends on them.

Delete bettersafe at some later time, of course. When you’re sure everything turned out OK.

To keep in mind

• Git manages changes, not versions. Think of a commit as a patch, not a snapshot. Even though a commit’s ID happens to be a representation of the project’s current snapshot.
• A branch is just a pointer to some commit’s ID (possibly base) saying “my next commit will be based on this commit” (the latter is the branch’s HEAD).
• Being on a branch controls where your next commit will go
• Checking out a branch is like checking out the root commit, and running all commits (think patches) up to that branch’s HEAD
• Hence rebasing actually means moving the branch’s first commit’s parent ID.

Backing up and restoring the whole repo

To backup the entire repository:

$ git bundle create mybundle.git --all

This creates the mybundle.git file, which is an efficient storage of the entire repository.

To restore all branches and tags, the following sequence applies:

$ git clone mybundle.git myclone
$ cd myclone/
$ for i in `git bundle list-heads mybundle.git` ; do git checkout ${i##*/} ; done
$ git checkout master

The for-loop produces as lot of output, and eventually leaves the working tree on some random ref. So the final checkout gets you where you want to start, presumably “master” in the example above.

I will be most delighted to know if there exists a single command doing this. As it seems, just “git clone” copies only the commits belonging to the current branch (i.e. HEAD), and any refs that were on its way. This probably makes sense when cloning from a remote repository with gazillions of commits. The concept of a local backup was probably neglected, because of the way git is usually used (that is, in collaboration).

For a partial backup of a certain sequence of commits, say from “master” to “playaround” go

$ git bundle create mybundle.git master..playaround

To use this bundle, “git pull” must be used on the target repository to get the commits. Needless to say, it must already have the commit pointed to by “master” when the bundle was made, or the entire bundle is completely worthless.

Oops. I messed up.

Git doesn’t delete anything except for during periodical (or initiated) garbage collections, so if something was committed in the past, there’s a change it’s still there, only invisible. The trick is to use

$ git reflog

and if the lost commit is found, tag it with

$ git tag rescue_me 7cb12d6

Tagging it makes the commit (and those leading to it) visible in gitk (given that it’s set to show all refs) and also prevents their removal on the next garbage collection.

Git sequence for making a patch for submission

See Documentation/SubmittingPatches in the Linux tree.

First, clone the main git that appears in the MAINTAINERS part for the subsystem the patch is going to. Probably better, add the repo to the existing main Linux repository, initially fetched with

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Remember: The patch must apply (as in “git am”) against the “next” branch of the relevant subsystem’s repository.

Make the changes, verify that the latest version of sparse doesn’t complain with e.g.

$ make M=drivers/staging/mydriver C=2

The C=2 forces as sparse run even on files that need no compilation. C=1 only checks modified files.

Always grab the latest version of sparse and check against the latest possible kernel. Really. I really had new warnings popping up just by checking against a newer kernel.

When all is well, commit. If the patch is a response to a bug report, add a “Reported-by:” header at the end of the commit message manually. Same goes for Tested-by, Suggested-by and Reviewed-by. There is no automatic mechanism for this (like –signoff).

If the patch fixes a bug, add a Fixes: header as well, with a commit ID and title of where the bug first occurs. If the bug was there from the beginning, put the commit ID where the file was inserted. The decision on whether the patch should be applied to stable kernels as well is made based upon this.

Also: Always have something meaningful written in the commit description. Just a title isn’t good enough (Greg rejected a very short patch of mine on the grounds that it had only a subject and no “changelog information”).

$ git commit -a
$ git format-patch master -o patch --signoff

This will create a directory called “patch” and put the patch there. If several commits are made, a different patch file is created for each. That what the commit -a is for.

To get all commits in one patch, use the –to-stdout flag, and of course redirect stdout to a file.

If this is a second submission (version 2 = v2), use

$ git format-patch master -o patch --signoff --subject-prefix="PATCH v2"

It’s also a desired to add a short description on the difference from earlier versions. The place to do it, is under the “—” mark of the signoff. This can be done manually, or with git notes (see below) for creating a note for a commit, and then use the –notes flag with git format-patch. The result in the patch file is something like this:

Signed-off-by: Eli Billauer <eli.billauer@example.com>
---

Notes:
 This is a really horrible patch

Check the patch with (run from kernel tree’s root)

$ scripts/checkpatch.pl --strict patch/0001-name-of-patch

(note that checkpatch.pl has a -f flag, allowing to check a source file rather than a patch)

I marked the –strict flag red, because I forgot to use it in an important occasion. It’s turned on automatically, by the way, when checkpatch detects that the target directory is one of drivers/net, net/ or drivers/staging/. Yes, the directories are hardcoded in the script itself. Perl or not?

Whitespace cleanup:

$ scripts/cleanpatch patch/0001-name-of-patch

Send it through email (the way the kernel guys like it)

git send-email --to 'linux-kernel@vger.kernel.org' patch/0001-name-of-patch

Add a –cc flag to send a copy to someone else than yourself. The mail won’t kick off before confirming it, so don’t worry…

Now, since I’m using my Gmail address as the “From”, the mail must come from a Gmail mail server. In order to relay the mail through their servers (and not my own), there’s a whole story about that. See this post.

git notes

This is a neat mechanism for adding extra information to commits. Its main use is for change log information to patches, so they appear in the email but not in the commit (if and when it’s applied).

To make thing simple, this is the only command needed to maintain notes:

$ git notes edit

It adds and/or edits the notes for the current commit with the selected editor.

As of gitk that goes along with 2.17.1 (2016 edition? It doesn’t say its version), the notes appear at the commit view, below the commit title, after a hard update (Ctrl-Shift-F5). There are also small yellow boxes next to the commit title in the tree view to mark that there are notes related to the commit.

Following the suggestion on this page, add the following lines to .git/config of the relevant repository.

[notes "rewrite"]
    amend = true
    rebase = true
[notes]
    rewriteRef = refs/notes/commits

The problem this solves: Notes refer to commits by their object ID, which changes when the commit is rebased or amended. As a result, the note becomes detached from the commit it related to. This chunk tells git to update the notes’ refs to follow the commits.

It’s also possible to add it to ~/.gitconfig, but if rebasing or commit amending is done on a repository that doesn’t have any notes, one gets a “warning: notes ref refs/notes/commits is invalid”.

Applying a Linux patch into a local project

The path of the files in the Linux project is deeper, so the application is with something like

$ git am -p4 patchdir/0001-this-is-the.patch

or the other way around: Applying a local project’s patch into the Linux kernel:

$ git am --directory drivers/mypath/mydriver/ 0001-this-is-the.patch

Applying a dirty patch

If the patch was made not to be applied directly (e.g. apply changes made in one file to another, by editing the patch file) format-patch can be used to generate the patch, and then go

$ git apply --reject --whitespace=fix patchdir/0001-This_is_the_patch

I think it’s best to remove the header containing the commit ID from which the patch was originally made, but haven’t really tried not doing this.

Preventing diff from working on a binary file

In particular, gitk has a tendency to try diffing *.ngc files and therefore freezing, since they start with text and then go blob. Create (or edit) .gitattributes in the project’s root directory (man gitattributes) to say

*.ngc binary

(as usual with git, the solution is painfully simple) and it probably makes sense to commit this file into the project as well.

Messing around with commits

For a graphical representation of the branches and commits, change directory to where the git commands are run from and simply go

$ gitk &

To see a the complete tree of local branches (recommended) go to View > New View and check “All (local) branches”.

Add files to be watched by git (and hence relevant on the next commit)

$ git add filename filename2 ...

This can also be done as one step of a commit command. Committing all watched files:

$ git commit -a

Oops…? Made some changes which would fit best in the last commit? Want to fix the last commit, and re-edit the commit message? Easy, just go

$ git commit -a --amend

Note that this changes the commit message as well as updating the commit according to the working tree.

And change the date of the commit to “now”:

$ git commit --date "`date -R`" --amend

… but that changes only the authoring date. To set the committing date to the same, go

$ git rebase --committer-date-is-author-date HEAD~1

And of course, the latter command can go deeper into history, depending on what’s instead of “HEAD~1″.

For setting the date to the current one on several commits, use e.g.

$ git rebase --ignore-date origin/master

To commit only part of the changes:

$ git add --all -p
$ git commit

To use an existing commit’s log message (in particular if it’s cherry-picked or even orphaned):

$ git commit -c 77ddd228c8cc26801eb83f421048d30fa1c31564

To move the current branch’s head, so it doesn’t include the latest commit (but leave the changes in the sources), a.k.a. “remove the commit”:

$ git reset HEAD~1

Note that the commit stays in the repository until garbage collected, and its changes remain in the worktree. This actually says “move the branch one step (as in HEAD~1) back. A “git checkout -f” will remove the commit’s effect on the worktree as well.

To really go back one commit (that is, revert its changes in the working tree),

$ git reset --hard HEAD~1

Even though the commit stays in the repository until garbage collected, that may happen sooner than desired. So if the commit may be useful in the future, create a new branch (e.g. “delme”) before backing off the current one, so the commit isn’t lost.  If this is done by mistake, just cherry-pick the reverted commit with gitk (hoping it’s still there).

And check the latest commits with

$ git log

Rebasing your own experiments (branch “foolaround”) on top of the master branch (so your games are on the real thing + your changes)

$ git rebase master foolaround

Note that the third argument, “foolaround” tells git to check out this branch first, and then rebase it to master. It’s otherwise assumed that the current branch is rebased, so this format is somewhat safer. But “git rebase master” is fine as well (and rebasing “master” on itself just says that the branch is up to date, which happens to be always be true).

Move some commit to the top of some other branch, say to master:

$ git rebase 77ddd228c8cc26801eb83f421048d30fa1c31564 master

where that blob in the middle is the commit ID, of course. Unlike “git cherry-pick” which applies the changes only, without changing the tree of commits)

To fool around with the 4 last commits (reorder, remove, squash several commits into one:

$ git rebase -i HEAD~4

Note that when squashing, the commit marked to squash is mixed with the commit one the row above in the list (which was committed later in time). An opportunity to edit the commit message will be given anyhow, so just mark the commits for squashing and go on with it. Of course, several commits can be marked in a row for a multi-squash.

What have I changed since the last commit?

$ git diff

What is the difference between now and some branch?

$ git diff somebranch

What is the commit ID of a given tag, branch or other commit one can refer to (HEAD in this example)?

$ git rev-parse HEAD

What’s the last tag issued in the current branch?

$ git describe --tags --abbrev=0

To check for another branch, add its name as the last argument for the same command above.

Apply changes as if checking out another branch, but stay in place:

$ git checkout -p thatbranch

Each hunk (that is, piece to change) is prompted for. Say “y” to all, and the working tree will be like “thatbranch”. Say “n” to all, and nothing changes. This way or another, you don’t switch branch, only the files changes. This is, in fact, not a checkout.

Find the directory where .git/ sits:

$ git rev-parse --show-toplevel

Untracking files already in index

This recipe works when it’s OK to temporarily remove the files from the worktree.

First, edit the .gitignore so that the relevant files will be ignore in the future. Then commit .gitignore and create a tag on the last commit, say “hold”. Make sure that the files are indeed tracked:

$ git ls-files | less

Now remove the files from the index, possibly with several commands e.g.:

$ git rm --cached useless-file

This changes nothing in the working tree for now, but only marks the files as gone in the index.

And then commit. The files will be removed in the working tree.

$ git commit

To have the files back, patch-checkout the previous commit. HEAD~1 would work instead of “hold”, but the purpose of hold was also to be an anchor for messups:

$ git checkout -p hold

When applying the relevant hunks, git asks as if these will be applied to the index as well, but in fact they don’t go to the index because of the updated .gitignore. So when the process is finished, git remains in sync (unlike it would in a normal checkout -p session).

If all is well, the “hold” tag can be removed.

To get a list of branches so a script can handle the output (as opposed to just “git branch” which isn’t safe):

$ git for-each-ref --format='%(refname:short)' refs/heads/

Cleaning up the directory tree

Use with caution, and think twice before doing this: It deletes all files not tracked in the repo. All those nice private scripts and stuff? Gone.

To clean untracked files (those appearing in “git status”) go first

$ git clean -n

and see what the damage is, and then remove the -n if being sure. To delete all files, regardless of .gitignore (this is “make mrproper” just a little more aggressive):

$ git clean -n -d -fx

Windows: Remove git from Explorer right-click menus

Uninstalling git involves closing explorer.exe (!) and didn’t work even so on my computer, so I took the registry editing route.

Basically removed the HKEY_CLASSES_ROOT\*\shellex\ContextMenuHandlers\Git-Cheetah, HKEY_CLASSES_ROOT\Directory\Background\shellex\ContextMenuHandlers\Git-Cheetah, HKEY_CLASSES_ROOT\Drive\shellex\ContextMenuHandlers\Git-Cheetah registry keys and other places where I found Git-Cheetah under ContextMenuHandlers.

Also removed anywhere the string {ca586c80-7c84-4b88-8537-726724df6929} appeared under something having to do with shell extensions. A bit scary, but harmless.

Setting up my own little git server

A git server is just like any repository, only the data is kept “bare”, that is without the working tree, and with the files usually in the .git subdirectory residing directly on the repo’s root. A simple “git clone” with the repo’s root yields the full git repository, in case of doubt. The gitk utility can also be run from the repo’s root.

First thing first:

# yum install git-daemon

Among others, this adds /etc/xinetd.d/git, which is disabled by default.

Edit the file to read (changed parts in red):

# default: off
# description: The git dæmon allows git repositories to be exported using \
#       the git:// protocol.

service git
{
 disable         = no
 socket_type     = stream
 wait            = no
 only_from       = 10.1.1.0/24 127.0.0.1
 user            = git
 server          = /usr/libexec/git-core/git-daemon
 server_args     = --base-path=/home/git/public_git --export-all --syslog --inetd --verbose --enable=upload-pack --enable=receive-pack
 log_on_failure  += USERID
}

Create a new user “git”, create public_git in its home directory, and then go

# service xinetd restart

And setup a new repo, e.g. (as user “git” in its home directory):

$ mkdir test
$ cd test/
$ git --bare init

And then, as any user, bind the origin. This is not necessary if the repository is cloned from the origin anyhow.

$ git remote rm origin
$ git remote add origin git://localhost/test/

And possibly push and fetch (I don’t like pulling, because the merging can fail). See my note about a problem with “git push” on Windows.

$ git push --all
$ git fetch --all

Note that the the new commit will not necessarily appear after fetching, unless the display mode is set to “all refs”.

To push a certain branch to the remote repo, despite “push –all” failing because some history will be killed, go

$ git push -n origin +master
$ git push origin +master

for the pushing the master branch by force (the -n option is for a dry-run). This is OK in particular after some commit munching on a branch, which no other development entity depends on. Or the other side will have to cherry-pick its latest commits. Or move its local “master” branch to the new one, and rebase from there (losing tags).

To synchronize with the remote repository, either merge or rebase. The remote branch is called origin/branch, e.g.

$ git rebase origin/master

rebases the local changes on the remote ones on the master branch.

To push all tags to the remote server, go

$ git push --tags

but remember not to do this when there are temporary tags (because the only way to get rid of them is from the server).

To turn the remote into an exact copy of the local repo, use

$ git push --mirror

with care: No questions are asked. The remote repo is just overridden. Whatever is gone on the local repo will be gone in the remote one. This is important in particular if commits have been squashed etc. Try “push –all” first. If that fails, it’s likely that the other copies of the repository will have to be re-cloned. Which is fairly OK if they’ve all pushed –all and the current repo has at least fetched –all.

Always make sure there is a “master” branch when pushing. Otherwise attempts to clone will result in “remote HEAD refers to nonexistent ref, unable to checkout” and a useless local copy. This is because HEAD points at “master” by default.

Remote repos

$ git branch --set-upstream localbranch theorigin/master

This makes the local branch track the “master” branch on some remote repository. Unfortunately, for pushing, one still has to be explicit:

$ git push theorigin localbranch:master

This makes git treat “localbranch” as the remote’s “master”, and it’s necessary despite the upstream setting above.

Uploading to github

Definitely, go for ssh authentication. It requires copying the content of ~/.ssh/id_rsa.pub into a text window under Github’s web interface: Click on the icon at the top right, choose Settings and then select “SSH and GPG keys”. Then be sure to define git’s remote as “git@github.com:billauer/theproject.git” and not the https: thing. And from this point on, it’s just like password-less ssh. That is, forget about authentication.

Ah, but it’s not that simple if you have multiple accounts on Github. Because if an ssh key is already used for another user, it’s rejected when trying to apply it on a new one, with the error message “Key is already in use”. The solution is to maintain multiple keys. This is mildly annoying once you get the hang of how to do that. See section named “Having multiple SSH keys” in another post of mine. The trick is to invent a name for a host, say, “sillygit”. Set git’s remote as “sillygit:billauer/theproject.git”. Then set SSH’s config file to recognize “sillygit” as the name of a host, and apply a set of parameters: The real host name, the user name and the SSH key to use.

Don’t confuse this Deploy keys, which are a per-repository SSH keys. These keys allow access to a specific repository only.

Access to multiple users

If the repository should be accessible to multiple users that belong to a group, add this to the config file:

sharedRepository = true

See git’s doc on configuration parameters.

Microblaze + Linux: Sample design of a custom peripheral

Scope

Even though Xilinx supplies a cute wizard for creating peripherals in its EDK (version 13.2 in my case), it’s just enough to work as a demo. For a real-life case there’s no escape from getting down to the system’s guts. As it turns out, things are pretty well organized under EDK’s hood, which makes the attempt to cover it all up with a wizard even more questionable.

This post is a jot-down of the technicalities behind designing a minimal bare-boned peripheral and its Linux driver. With “bare-boned” I mean that it has the absolutely simplest bus interface (AXI4 Lite without even decoding the addresses), and that it’s in pure Verilog. No external IP is used.

This peripheral connects to the SP605′s four LEDs. Any write to its address region updates the LED’s state according to the written value’s four LSBs. Reading back from any of its covered addresses gives the four LSBs back. That’s all.

Sources of information

This post assumes that you’re familiar with running Linux on Microblaze. I have a pretty extensive tutorial on the subject for reference.

These are worth to look at:

• The official cores’ sources (in VHDL) can be found at ISE_DS\EDK\hw\XilinxProcessorIPLib\pcores (path from where ISE is installed). It’s interesting in particular to look at axi_lite_ipif_v1_00_a.
• The AMBA AXI protocol specification, downloaded free from ARM’s site.
• Platform Specification Format Reference Manual (UG642, psf_rm.pdf): Describes the file formats in detail. Necessary when editing the files.
• EDK Concepts, Tools, and Techniques (UG683, edk_ctt.pdf) : The chapter about Creating Your Own Intellectual Property is somewhat helpful to try out the Wizard.

Understanding the process

It looks like the missing link in Xilinx’ documentation is to explain how the whole machinery works with regard to adopting a custom made peripheral. I’ll try to fill in that gap now.

Generally speaking, the minimal core consists of the following files, which should be in a dedicated directory under the “pcores” directory, which is under the EDK project’s top directory:

• data/minimal_v2_1_0.mpd: This file is what EDK looks at when an IP is added to a project. It contains all the information used directly by the EDK. The peripheral’s GUI is set up according to this information, and it’s also used when the EDK generates wrappers and connections for it. Its format is well documented, but it looks like it’s easier to just copy snippets from existing core’s MPD files. It’s also possible to generate this file automatically with PsfUtility from the toplevel source file, but it’s not clear if it’s worth the effort to learn yet another tool.
• data/minimal_v2_1_0.pao: This file supplies EDK with a list of HDL files which need to be synthesized to create the peripheral. It also sets the order of synthesis.
• hdl/verilog/minimal.v: The Verilog file constituting the peripheral. Of course there may be several files, which need to be mentioned in the PAO file.
• Note that “black box” modules (presynthesized netlists) are listed in BBD files, which are not necessary in this example. When used, the MPD file is set to reflect this.

The file names above relate to a peripheral called “minimal”. They change according to the project’s setting and version numbers.

All in all, the flow is pretty simple: Only the MPD file is considered by EDK, and only at platform netlist generation are the HDL files synthesized according to the PAO file. The instantiation and connection depend on the settings within the EDK (actually, the MHS file).

It’s easiest to create just any peripheral with the wizard, see what they do, and then modify the files.

Going from Wizard’s output to minimal peripheral

This is a short outline of the stages. The result is given in the next section.

• Edit the data/*.pao file: Remove all files and insert the single Verilog file, changing the type to verilog.
• In the data/*.mpd file, change OPTION HDL = VHDL to VERILOG too. Also add, under ##ports, PORT minimal_leds = “”, DIR = O, VEC = [3:0] (so that the I/O port is known. Note the =”" part).
• Remove data/*.prj file  so no unnecessary files are included (even though this file seems to be ignored).
• Roughly translate the VHDL file to Verilog. Make it Verilog parameters instead of VHDL generics.
• Rename/remove the devl directory, since its information is not in sync with the new situation, and Custom IP Wizard can’t do any good at this point.
• And finally, in EDK, Project > Rescan User Repositories
• Remove the LED_4bits core from the project, choosing “Delete instance and any connections to internal nets”. This will keep the net names used for connecting to the LEDs, and make them available for connection to the new peripheral. Otherwise, the external net names need to be set, and the system.ucf given at the “project” tab updated to reflect the new nets.
• Add the minimal core to the project, and connect the just released LEDs_4Bits_TRI_O to its minimal_leds port.
• Create bitfile

The synthesis of the peripheral’s HDL takes place during the “create netlist” flow (which is, of course, part of generating bitfile). For example, the synthesis of an instance named minimal_0 will appear as follows in the console

INSTANCE:minimal_0 - C:\tryperipheral\system.mhs line 424 - Running XST
synthesis
PMSPEC -- Overriding Xilinx file
<C:/ise13_2/ISE_DS/EDK/spartan6/data/spartan6.acd> with local file
<C:/ise13_2/ISE_DS/ISE/spartan6/data/spartan6.acd>

And if there are errors in the HDL, they will show up at this point.

Sample files

These are the files used for the minimal peripheral. They are a sloppy adoption of the files generated by the Custom IP Wizard, so they’re very likely to contain unnecessary declarations.

First, the Verilog file:

module minimal #(
 parameter C_S_AXI_DATA_WIDTH = 32,
 parameter C_S_AXI_ADDR_WIDTH = 32,
 parameter C_S_AXI_MIN_SIZE = 'h000001FF,
 parameter C_USE_WSTRB = 0,
 parameter C_DPHASE_TIMEOUT = 8,
 parameter C_BASEADDR = 'hFFFFFFFF,
 parameter C_HIGHADDR = 'h00000000,
 parameter C_FAMILY = "spartan6",
 parameter C_NUM_REG = 1,
 parameter C_NUM_MEM = 1,
 parameter C_SLV_AWIDTH = 32,
 parameter C_SLV_DWIDTH = 32
 )
 (
 input S_AXI_ACLK,
 input S_AXI_ARESETN,
 input [(C_S_AXI_ADDR_WIDTH-1):0] S_AXI_AWADDR,
 input S_AXI_AWVALID,
 input  [(C_S_AXI_DATA_WIDTH-1):0] S_AXI_WDATA,
 input  [((C_S_AXI_DATA_WIDTH/8)-1):0] S_AXI_WSTRB,
 input S_AXI_WVALID,
 input S_AXI_BREADY,
 input  [(C_S_AXI_ADDR_WIDTH-1):0] S_AXI_ARADDR,
 input S_AXI_ARVALID,
 input S_AXI_RREADY,
 output S_AXI_ARREADY,
 output  [(C_S_AXI_DATA_WIDTH-1):0] S_AXI_RDATA,
 output  [1:0] S_AXI_RRESP,
 output S_AXI_RVALID,
 output S_AXI_WREADY,
 output [1:0] S_AXI_BRESP,
 output reg S_AXI_BVALID,
 output S_AXI_AWREADY,

 output reg [3:0] minimal_leds
 );

 assign  S_AXI_RDATA = minimal_leds;
 assign  S_AXI_RRESP = 0; // OKAY on AXI4
 assign  S_AXI_ARREADY = 1; // Always ready for read address
 assign  S_AXI_AWREADY = 1; // Always ready for write address
 assign  S_AXI_RVALID = 1; // Read data always valid (ILLEGAL)
 assign  S_AXI_WREADY = 1; // Always ready to write
 assign  S_AXI_BRESP = 0; // OKAY on AXI4

 // This will not work OK if several "bursts" are sent with no BVALIDs
 // inbetween. Not an expected scenario.

 always @(posedge S_AXI_ACLK)
   if (S_AXI_WVALID)
     begin
        S_AXI_BVALID <= 1;
        minimal_leds <= S_AXI_WDATA;
     end
   else if (S_AXI_BREADY && S_AXI_BVALID) // Active BRESP cycle
     S_AXI_BVALID <= 0;

endmodule

Most of the parameters at the top can be removed, I believe. It appears like they are necessary only when creating the MPD file with PsfUtility.

All ports, except minimal_leds are standard AXI4 lite ports. The implementation of the interface isn’t example for anything except a quick and dirty peripheral which responds to bus requests. The only thing it does actively is to update minimal_leds when necessary, and toggle the AXI_BVALID, so that only one burst response is sent for each write cycle (which is always one clock long in AXI4 lite). It’s OK not to decode the address, since it’s the interconnect’s job to make sure each peripheral gets only what it directed to it.

Holding S_AXI_RVALID high all the time violates the AXI4 spec, since it’s required to be asserted only after ARVALID and ARREADY. But the interconnect tolerated this anyhow.

Now to minimal_v2_1_0.mpd:

BEGIN minimal

## Peripheral Options
OPTION IPTYPE = PERIPHERAL
OPTION IMP_NETLIST = TRUE
OPTION HDL = VERILOG
OPTION IP_GROUP = MICROBLAZE:USER
OPTION DESC = MINIMAL
OPTION LONG_DESC = A minimal peripheral to start off with
OPTION ARCH_SUPPORT_MAP = (others=DEVELOPMENT)

## Bus Interfaces
BUS_INTERFACE BUS = S_AXI, BUS_STD = AXI, BUS_TYPE = SLAVE

## Generics for VHDL or Parameters for Verilog
PARAMETER C_S_AXI_DATA_WIDTH = 32, DT = INTEGER, BUS = S_AXI, ASSIGNMENT = CONSTANT
PARAMETER C_S_AXI_ADDR_WIDTH = 32, DT = INTEGER, BUS = S_AXI, ASSIGNMENT = CONSTANT
PARAMETER C_S_AXI_MIN_SIZE = 0x000001ff, DT = std_logic_vector, BUS = S_AXI
PARAMETER C_USE_WSTRB = 0, DT = INTEGER
PARAMETER C_DPHASE_TIMEOUT = 8, DT = INTEGER
PARAMETER C_BASEADDR = 0xffffffff, DT = std_logic_vector, MIN_SIZE = 0x0, PAIR = C_HIGHADDR, ADDRESS = BASE, BUS = S_AXI
PARAMETER C_HIGHADDR = 0x00000000, DT = std_logic_vector, PAIR = C_BASEADDR, ADDRESS = HIGH, BUS = S_AXI
PARAMETER C_FAMILY = virtex6, DT = STRING
PARAMETER C_NUM_REG = 1, DT = INTEGER
PARAMETER C_NUM_MEM = 1, DT = INTEGER
PARAMETER C_SLV_AWIDTH = 32, DT = INTEGER
PARAMETER C_SLV_DWIDTH = 32, DT = INTEGER
PARAMETER C_S_AXI_PROTOCOL = AXI4LITE, TYPE = NON_HDL, ASSIGNMENT = CONSTANT, DT = STRING, BUS = S_AXI

## Ports
PORT S_AXI_ACLK = "", DIR = I, SIGIS = CLK, BUS = S_AXI
PORT S_AXI_ARESETN = ARESETN, DIR = I, SIGIS = RST, BUS = S_AXI
PORT S_AXI_AWADDR = AWADDR, DIR = I, VEC = [(C_S_AXI_ADDR_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_AWVALID = AWVALID, DIR = I, BUS = S_AXI
PORT S_AXI_WDATA = WDATA, DIR = I, VEC = [(C_S_AXI_DATA_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_WSTRB = WSTRB, DIR = I, VEC = [((C_S_AXI_DATA_WIDTH/8)-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_WVALID = WVALID, DIR = I, BUS = S_AXI
PORT S_AXI_BREADY = BREADY, DIR = I, BUS = S_AXI
PORT S_AXI_ARADDR = ARADDR, DIR = I, VEC = [(C_S_AXI_ADDR_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_ARVALID = ARVALID, DIR = I, BUS = S_AXI
PORT S_AXI_RREADY = RREADY, DIR = I, BUS = S_AXI
PORT S_AXI_ARREADY = ARREADY, DIR = O, BUS = S_AXI
PORT S_AXI_RDATA = RDATA, DIR = O, VEC = [(C_S_AXI_DATA_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_RRESP = RRESP, DIR = O, VEC = [1:0], BUS = S_AXI
PORT S_AXI_RVALID = RVALID, DIR = O, BUS = S_AXI
PORT S_AXI_WREADY = WREADY, DIR = O, BUS = S_AXI
PORT S_AXI_BRESP = BRESP, DIR = O, VEC = [1:0], BUS = S_AXI
PORT S_AXI_BVALID = BVALID, DIR = O, BUS = S_AXI
PORT S_AXI_AWREADY = AWREADY, DIR = O, BUS = S_AXI
PORT minimal_leds = "", DIR = O, VEC = [3:0]

END

This file is exactly as generated by the Wizard, except for the HDL option in the beginning changed to VERILOG, and the added port minimal_leds at the end. Note its assignment to “”. This file is best created by looking at examples of existing cores.

Now to minimal_v2_1_0.pao:

lib minimal_v1_00_a minimal verilog

which was rewritten to reflect that the peripheral consists of one single Verilog file.

The device tree file

The device tree file needs to be generated as described in one of my posts. The relevant section is given here, since it relates to kernel code presented next:

minimal_0: minimal@7ae00000 {
 compatible = "xlnx,minimal-1.00.a";
 reg = < 0x7ae00000 0x10000 >;
 xlnx,dphase-timeout = <0x8>;
 xlnx,family = "spartan6";
 xlnx,num-mem = <0x1>;
 xlnx,num-reg = <0x1>;
 xlnx,s-axi-min-size = <0x1ff>;
 xlnx,slv-awidth = <0x20>;
 xlnx,slv-dwidth = <0x20>;
 xlnx,use-wstrb = <0x0>;
 }

It’s pretty evident that some of these parameters have no use.

The driver

First, it’s convenient to create a makefile for cross compilation. Even though the correct way is to set the environment variables in the shell, and run the module compilation in the same way the kernel itself is compiled, it’s much more convenient to go just “make” or “make clean” with this makefile. It’s not good for distribution, as the paths to both the kernel tree and cross compiler are hardcoded.

So here’s a dirty, but yet convenient makefile:

export CROSS_COMPILE=/path/to/microblazeel-unknown-linux-gnu/bin/microblazeel-unknown-linux-gnu-
export ARCH=microblaze

ifneq ($(KERNELRELEASE),)
obj-m    := minimal.o

else
KDIR := /path/to/linux-2.6.38.6

default:
 @echo $(TARGET) > module.target
 $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

clean:
 @rm -f *.ko *.o modules.order Module.symvers *.mod.? .minimal.* *~
 @rm -rf .tmp_versions module.target

minimal.ko:
 $(MAKE)
endif

And now to the driver itself, minimal.c:

#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <asm/io.h>

/* Match table for of_platform binding */
static struct of_device_id minimal_of_match[] __devinitdata = {
 { .compatible = "xlnx,minimal-1.00.a", },
 {}
};

MODULE_ALIAS("minimal");

static void __iomem *regs;
static struct resource res;

static int __devinit
minimal_of_probe(struct platform_device *op, const struct of_device_id *match)
{

 const int *width;
 int ret;
 int val;

 ret = of_address_to_resource(op->dev.of_node, 0, &res);
 if (ret) {
 printk(KERN_WARNING "minimal: Failed to obtain device tree resource\n");
 return ret;
 }

 printk(KERN_WARNING "minimal: Physical address to resource is %x\n", (unsigned int) res.start);

 if (!request_mem_region(res.start, 32, "mimimal")) {
 printk(KERN_WARNING "minimal: Failed to request I/O memory\n");
 return -EBUSY;
 }

 regs = of_iomap(op->dev.of_node, 0); /* Verify it's non-null! */

 printk(KERN_WARNING "minimal: Access address to registers is %x\n", (unsigned int) regs);

 width = of_get_property(op->dev.of_node, "xlnx,slv-dwidth", NULL);

 printk(KERN_WARNING "minimal: Obtained width=%d\n", be32_to_cpu(*width));

 val = ioread32(regs);
 printk(KERN_WARNING "minimal: Read %d, writing %d\n", val, val+1);

 iowrite32(++val, regs);

 return 0; /* Success */
}

static int __devexit minimal_of_remove(struct platform_device *op)
{
 iounmap(regs);
 release_mem_region(res.start, 32);
 return 0; /* Success */
}

static struct of_platform_driver minimal_of_driver = {
 .probe = minimal_of_probe,
 .remove = __devexit_p(minimal_of_remove),
 .driver = {
 .name = "minimal",
 .owner = THIS_MODULE,
 .of_match_table = minimal_of_match,
 },
};

int __init minimal_init(void)
{
 int ret;
 ret = of_register_platform_driver(&minimal_of_driver);
 return ret;
}

void __exit minimal_exit(void)
{
 of_unregister_platform_driver(&minimal_of_driver);
}

module_init(minimal_init);
module_exit(minimal_exit);

MODULE_AUTHOR("Eli Billauer");
MODULE_DESCRIPTION("Microblaze minimal module");
MODULE_LICENSE("GPL")

It doesn’t do anything special, except for change the state of the LEDs every time it’s loaded. The drivers also reads one of the parameters from the device tree structure. Not fascinating, but keeps the code, well, minimal.

This code should be pretty straightforward to programmers who are familiar with PCI device drivers, with probing and removal working in more or less the same way. I’ve chosen a hardcoded segment of 32 bytes as the requested region. This depends on the peripheral, of course.

A test run

This is the transcript of the session on the UART console, as run on a freshly booted system. LEDs did indeed go on and off as reflected by the numbers.

/ # insmod minimal.ko
minimal: Physical address to resource is 7ae00000
minimal: Access address to registers is c87e0000
minimal: Obtained width=32
minimal: Read 0, writing 1
/ # lsmod
minimal 1978 0 - Live 0xc8056000
ipv6 305961 10 - Live 0xc8763000
/ # cat /proc/iomem
40600000-4060000f : uartlite
40a00000-40a0ffff : xilinx_spi
40e00000-40e0ffff : xilinx_emaclite
7ae00000-7ae0001f : mimimal
/ # rmmod minimal
rmmod: module 'minimal' not found
/ # cat /proc/iomem
40600000-4060000f : uartlite
40a00000-40a0ffff : xilinx_spi
40e00000-40e0ffff : xilinx_emaclite
/ # lsmod
ipv6 305961 10 - Live 0xc8763000
/ # insmod minimal.ko
minimal: Physical address to resource is 7ae00000
minimal: Access address to registers is c8820000
minimal: Obtained width=32
minimal: Read 1, writing 2

Note that rmmod produces an error message, which makes it look as if it failed to remove the module, when in fact all went well.

The physical address was indeed detected correctly (see device tree), and mapped to another kernel virtual address each time.

Linux on Microblaze HOWTO (part IV)

This is part IV of my HOWTO on running Linux on Microblaze. The outline is as follows:

• Part I: Introduction and setting up the Microblaze processor
• Part II: Compiling the kernel
• Part III: Preparing for boot and booting
• Part IV: Compiling user space applications (this page)

Compiling user space applications

We shall now look at how to compile applications for execution under the Microblaze Linux machine. This is pretty straightforward for programs written for the specific environment. The problems may occur when compiling sources which were originally written for fullblown computers, as the build system may not have taken cross compilation into account. And as software projects tend to be hacked to death, with new features added all the time, the code may depend on libraries which are installed on every desktop, but not necessarily on an embedded system. These dependencies are at times a result of a completely offbeat feature, but it’s often simpler to compile the necessary library than to remove the feature. Since there is no single recipe for solving that kind of problems, we’ll stick to the basics of compiling for user space.

Background

Cross compilation of user space applications is actually more difficult than compiling the kernel, mainly because the kernel itself is, after all, a standalone application. There are a few things to take care of:

• Make sure libraries for dynamic linking are in place in the target runtime filesystem, as well as the dynamic linker itself.
• The compilation should be done against the header files corresponding to the libraries present in the target.
• The linked libraries used during compilation should correspond to those in the target.
• The C Runtime stubs (crt1.o, crti.o, crtbegin.o, crtend.o and crtn.o) should fit the Linux user space environment (different files with similar names are used for Microblaze standalones).

This may sound complicated, but most of the job has already been done. So it all boils down to a few simple things to bare in mind, as shown next.

Preparing the target’s file system

In part II it was shown how to download and extract the cross compiler for Microblaze. The same tarball also has the entire package for the target’s root under microblazeel-unknown-linux-gnu/microblazeel-unknown-linux-gnu/sys-root. This directory should be copied into the target’s root as is.

But the target’s root directory is already populated with files as necessary to boot Linux, and run command line utilities with busybox. Some of the files in sys-root, dynamic libraries in particular, already exist in target root, and they’re not identical. But since busybox is statically linked, overwriting the dynamic libraries seems harmless. Overwriting the previous files where applicable is therefore the way to resolve these conflicts, since dynamically linked applications will be compiled against the newer libraries.

All in all, the sys-root directory is ~164 MB, which isn’t too bad when stored on a flash memory. ~129 of these are /lib. The library files include libm, libpthread, libresolv and several other important libraries.

/usr/include takes up ~14 MB, which is probably not necessary on the target system.

Compilation

For cross compilation we use the same compiler used for the kernel. To compile an application:

/path/to/microblazeel-unknown-linux-gnu-gcc --sysroot=/path/to/nfsroot/ -o hello hello.c -lm

where /path/to/nfsroot is the directory which will be the root directory when the executable runs, or a copy of it. The truth is I’m not really sure –sysroot is really necessary, but given the pretty wild search the GNU tools do to find include files and libraries, it looks like a good measure to point directly at where these should be found.

Note that if we omit –sysroot, compiling for Microblaze Linux user space is done simply by using the cross compiler normally, and that works too. This happens because the compiler was configured to look for libraries and includes from its own sys-root. The C runtime stubs are always taken from the compiler’s own.

This is a good time to repeat something said in part II: There is no need to “install” the Microblaze compiler, and neither do the files need to be owned by root. A simple untar anywhere is fine. The compiler uses relative paths to find its resources.

Static linking

To be somewhat safer, static compilation may be preferred, in particular when the whole system’s functionality consists of a single application. The executable file is considerably larger, but doesn’t depend on libraries, so it works even without the sys-root copy mentioned above. Just add the –static flag to gcc. e.g.

/path/to/microblazeel-unknown-linux-gnu-gcc --static --sysroot=/path/to/nfsroot/ -o hello hello.c -lm

The -lm flag is here to demonstrate that libraries should be at the end of the command line. This has no significance when the executable is compiled to be dynamic, but for static linking, failing to put the -lm (and other loadables) at the end will cause misleading errors such as:

/tmp/cckqzZqo.o: In function `main':
: undefined reference to `sin'
collect2: ld returned 1 exit statu

So put the -lm at the end, OK?

Compiling from SDK

Note: I don’t present a working solution in this subject.

Since the BSP, which is generated by the SDK along with the compilation directories, are just include files and libraries, it was appealing to try compiling Linux user space applications from the SDK. The method suggested is to overwrite the BSP created by SDK with include files and libraries for the Linux environment.

The problem I didn’t bother to solve in the end, was that the C runtime libraries used by the linker remained those for a standalone application, so the executable couldn’t run on Linux. Most likely, this can be solved easily, but since I don’t like IDEs myself, I left this issue as is.

Another problem with this method is that the BSP is erased every time the project is rebuilt (but it survives recompilation, of course). So it’s best to keep a copy of the entire BSP directory structure.

In short, this is the procedure, minus the part that makes the linker work with the Linux-related CRTs.

• Create a BSP by creating a C project
• Copy all .a files from sys-root’s  /usr/lib and replace the lib directory in the BSP with a directory containing these (only)
• Replace the BSP’s include directory with sys-root’s /usr/include as is
• In SDK (Eclipse, actually), right-click the C project on the Project Explorer and pick Properties. Under C/C++ Build > Settings > Microblaze gcc linker > Linker script,  remove the linker script given (edit the text, and remove the part saying ../src/lscript.ld)

And again, this almost works. If someone takes this to the finish line, please let me know.

Running a VoIP phone adapter through a Linux machine

The setting

I got an AudioCodes MP202B as a phone line adapter from my Israeli ISP, Netvision. The normal way to connect it is putting it between the computer and the ADSL modem, so it does the “dialing” (sending username and password). This was a no-no for me, because I have a little home network with my own NAT and fake DNSes, so the last thing I wanted was to reconfigure my own network.

My twisted, and not really optimal solution was to let the phone adapter think my own computer is the ISP, so it connects to my computer with pppoe, gets a bogus IP address and DNS details, and then connect to the VoIP network through my computer.

That means, among others, that the phone adapter’s packets undergo NAT just like anything else going through that computer. What about incoming calls, I asked myself. They are initiated by the far end. How will iptables know the that its address needs to be mangled, so it goes to the phone adapter, and not the the host?

The answer, as usual with iptables, is don’t worry, be happy. As it turns out, the phone keeps sending initiation packets on UDP port 5060 periodically, so iptables can easily see the session. The voice packets also find their way. In short, it simply works. As usual.

Note that the bogus IP address the adapter sees is exposed to the VoIP operator in the headers of connection establishment packets. So if the ISP suddenly decides to check if the IP address appearing in these packets is in the valid range, the trick is revealed. Actually, it’s enough to check if the IP address in the headers matches the source address of the UDP packet itself (which has been altered by iptables to the real address given by the ISP). A software update or change in the (security?) configuration in the ISP’s infrastructure can lead to a sudden disconnection of the phone line. But this is not likely to happen: Assuming that the ISP checks the phone number against the ISP’s login name, this leaves no room for malicious tricks. Any extra restrictions are unnecessary, and as any network maintainer knows, add any extra filter, and some manager will shout at you soon.

This solution has an inherent flaw, though: Putting the phone between the computer and ADSL modem allows it to prioritize its own packets over data packets. Without this, voice quality can go down as a result of a massive upload to the web. But running a few tests, I didn’t hear any difference.

Getting the link up

Connect the phone adapter through its WAN jack to the desired Ethernet card (eth3 in my case).

Run a sniffer on the port, and verify that the adapter attempts to start a pppoe session. Something like this:

No. Time        Source                Destination           Protocol Info
 1 0.000000    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 2 0.999256    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 3 2.999042    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 4 6.998632    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)

Set up /etc/ppp/pppoe-server-options, which has the options to pppd, to read something like this:

lcp-echo-interval 10
lcp-echo-failure 2
noauth
ms-dns 10.2.0.1
ms-dns 10.2.0.2

with emphasis on the “noauth” option, since it’s pretty obvious that whoever is connected to the Ethernet jack doesn’t need authentication. Otherwise, the login name and password configured in the phone adapter must be added to pap-secrets or chap-secrets (whichever applies).

The ms-dns option contains DNS addresses for the adapter. These are fake addresses, which are NATed in the hosting machine, so they are real DNSes to the adapter.

Start off the pppoe server (as root, of course) with

# pppoe-server -I eth3 -L 10.192.0.0 -R 10.192.0.1 -N 1

where -N 1 limits the server to only one connections. The -L and -R set the local and remote addresses.

And by the way, to kill the pppoe server along with its connections go:

# killall pppoe-server

The packet capture should now look like:

No. Time        Source                Destination           Protocol Info
 1 0.000000    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 2 0.000369    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Offer (PADO)
 3 0.000650    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPPoED   Active Discovery Request (PADR)
 4 0.001711    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Session-confirmation (PADS)
 5 1.020712    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Request
 6 1.021534    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 7 1.021687    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Reject
 8 1.023117    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Ack
 9 1.024261    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 10 1.024434    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Ack
 11 1.024513    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 12 1.024565    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Request
 13 1.025981    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Request
 14 1.026087    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Reply
 15 1.026740    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 16 1.026854    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Nak
 17 1.028069    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 18 1.028197    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Request
 19 1.028267    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Reject
 20 1.029948    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 21 1.031115    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Ack
 22 1.036625    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 23 1.037321    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Ack
 24 1.038305    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Ack
 25 1.040164    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 26 1.043146    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Ack
 27 1.572454    10.192.0.1            10.2.0.1              DNS      Standard query A ntp.netvision.net.il
 28 3.070230    10.192.0.1            10.2.0.1              DNS      Standard query A centrex.res.netvision.net.il
 29 5.584155    10.192.0.1            10.2.0.1              DNS      Standard query A ntp.netvision.net.il
 30 6.570551    10.192.0.1            10.2.0.2              DNS      Standard query A ntp.netvision.net.il
 31 7.019219    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Request
 32 7.019401    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Reply
 33 8.070333    10.192.0.1            10.2.0.2              DNS      Standard query A centrex.res.netvision.net.il
 34 8.541665    10.192.0.1            10.2.0.1              DNS      Standard query A centrex.res.netvision.net.il
 35 10.580118   10.192.0.1            10.2.0.2              DNS      Standard query A ntp.netvision.net.il
 36 11.030538   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 37 11.031304   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 38 13.018511   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Request
(...etc)

So what we have here is a successful pppoe establishment. It’s also clear that the adapter got the DNS addresses OK, since it uses them for queries. But alas, no answer is returned, because my firewall rejects packets from any ppp device which are not within a session.

On my computer, the firewall script is run every time a ppp device goes up, by virtue of /etc/ppp/ip-up.local calling the firewall setup script.

In the script, I added the following part:

if [ $PHONEIF ] ; then
 iptables -A INPUT -i $PHONEIF -j droplog
 iptables -A OUTPUT -o $PHONEIF -j droplog

 if [ $EXTIF ] ; then
 iptables -A FORWARD -i $PHONEIF -o $EXTIF -j ACCEPT
 iptables -A FORWARD -i $EXTIF -o $PHONEIF -j ACCEPT
 fi

 # Default rule: Drop forwarded packets from and to adapter
 iptables -A FORWARD -i $PHONEIF -j droplog
 iptables -A FORWARD -o $PHONEIF -j droplog
fi

Where $PHONEIF and $EXTIF are the interfaces (ppp1 and ppp0, usually), as defined previously in the script.

Now everything works properly, packet capture as follows:

No. Time        Source                Destination           Protocol Info
 1 0.000000    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 2 0.000059    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Offer (PADO)
 3 0.000319    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPPoED   Active Discovery Request (PADR)
 4 0.000683    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Session-confirmation (PADS)
 5 0.002173    0.0.0.0               255.255.255.255       DHCP     DHCP Discover - Transaction ID 0x1fa2870d
 6 1.024784    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Request
 7 1.026219    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 8 1.026369    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Reject
 9 1.027837    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Ack
 10 1.028997    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 11 1.029168    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Ack
 12 1.029244    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 13 1.029298    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Request
 14 1.031055    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 15 1.031172    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Nak
 16 1.032347    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 17 1.032473    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Request
 18 1.032544    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Reject
 19 1.034022    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 20 1.035204    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Ack
 21 1.040019    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 22 1.041032    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Ack
 23 1.041705    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Ack
 24 1.042895    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 25 1.046591    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Ack
 26 3.073250    10.192.0.1            10.2.0.1              DNS      Standard query A centrex.res.netvision.net.il
 27 3.089492    10.2.0.1              10.192.0.1            DNS      Standard query response A 82.166.210.6
 28 3.106339    10.192.0.1            82.166.210.6          SIP      Request: REGISTER sip:centrex.res.netvision.net.il
 29 3.131676    82.166.210.6          10.192.0.1            SIP      Status: 200 Ok
 30 11.034587   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 31 11.035375   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 32 21.034734   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 33 21.035485   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 34 31.034857   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
(etc...)

With SIP registration repeated every ~80 seconds, and LCP echoes every 10 seconds. The SIP protocol is defined in its RFC.

Note that there is no authentication whatsoever. If there was, we would have seen the server sending a challenge, to which the phone adapter would respond with an answer. In the case above, the server accepts the connection with no questions asked.

Registration packets in detail

This is a good time to mention, that I’ve replaced my real incoming phone number to 073-6666666 and calling number to 04-8222222. After all, I don’t want my real phone numbers out there. I’m not customer support.

The Register/OK pair above look like this:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 633
 Checksum: 0x1623 (correct)
Session Initiation Protocol
 Request line: REGISTER sip:centrex.res.netvision.net.il SIP/2.0
 Message Header
 From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e90-100c00a-13c4-50029-0-7e817330-0
 To: <sip:200012972736666666@centrex.res.netvision.net.il>
 Call-ID: 100b62d8-100c00a-13c4-50029-0-2a161072-0
 CSeq: 1 REGISTER
 Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-0-2a0-3639eac2
 Max-Forwards: 70
 Supported: replaces,100rel
 Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
 Expires: 1800
 Contact: <sip:200012972736666666@10.192.0.1:5060>
 User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
 Content-Length: 0

and then answer is simply

Session Initiation Protocol
 Status line: SIP/2.0 200 Ok
 Message Header
 From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e90-100c00a-13c4-50029-0-7e817330-0
 To: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=SDq3rh799-
 Call-ID: 100b62d8-100c00a-13c4-50029-0-2a161072-0
 CSeq: 1 REGISTER
 Via: SIP/2.0/UDP 10.192.0.1:5060;received=46.116.190.192;branch=z9hG4bK-0-2a0-3639eac2;rport=5060
 Contact: <sip:200012972736666666@10.192.0.1:5060>;expires=120
 Content-Length: 0

Incoming phone call

An incoming phone call, which isn’t answered looks like this:

 64 83.972831   82.166.210.6          10.192.0.1            SIP/SDP  Request: INVITE sip:200012972736666666@10.192.0.1:5060, with session description
 65 84.194196   82.166.210.6          10.192.0.1            SIP/SDP  Request: NOTIFY sip:200012972736666666@10.192.0.1:5060, with session description
 66 84.339229   10.192.0.1            82.166.210.6          SIP      Status: 180 Ringing
 67 84.581309   10.192.0.1            82.166.210.6          SIP      Status: 200 OK
 68 91.035533   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 69 91.036415   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 70 101.035711  EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 71 101.036461  AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 72 104.484595  82.166.210.6          10.192.0.1            SIP      Request: CANCEL sip:200012972736666666@10.192.0.1:5060
 73 104.485130  82.166.210.6          10.192.0.1            SIP/SDP  Request: NOTIFY sip:200012972736666666@10.192.0.1:5060, with session description
 74 104.508747  10.192.0.1            82.166.210.6          SIP      Status: 200 OK
 75 104.603031  10.192.0.1            82.166.210.6          SIP      Status: 487 Request Terminated
 76 104.624033  82.166.210.6          10.192.0.1            SIP      Request: ACK sip:200012972736666666@10.192.0.1:5060
 77 104.629209  10.192.0.1            82.166.210.6          SIP      Status: 200 O

The INVITE packet’s details are as follows:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 880
 Checksum: 0xe69c (correct)
Session Initiation Protocol
 Request line: INVITE sip:200012972736666666@10.192.0.1:5060 SIP/2.0
 Message Header
 Via: SIP/2.0/UDP 82.166.210.6:5060;branch=z9hG4bKckpn1l30a880su8va740.1
 Call-Id: SDau8fa01-e00a9a6ae9b1553d6117b50ab6a925a6-a0fo130
 From: <sip:048222222@centrex.res.netvision.net.il:5060>;tag=SDau8fa01-10.60.20.110-4294963135-9019
 To: <sip:200012972736666666@centrex.res.netvision.net.il:5060>
 Max-Forwards: 69
 Allow: REGISTER, INVITE, BYE, ACK, CANCEL, REFER, INFO, OPTIONS, SUBSCRIBE, UPDATE
 Session-Expires: 900
 CSeq: 3040744 INVITE
 Contact: <sip:048222222@82.166.210.6:5060;transport=udp>
 Supported: timer
 Content-Type: application/sdp
 Content-Length: 24

Note that the caller’s ID is there!

Outgoing phone call

Picking up the VoIP phone and calling 04-8222222, without the other side answering, yields:

 118 221.029993  10.192.0.1            82.166.210.6          SIP/SDP  Request: INVITE sip:048222222@centrex.res.netvision.net.il, with session description
 119 221.037083  EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 120 221.037885  AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 121 221.056671  82.166.210.6          10.192.0.1            SIP      Status: 100 Trying
 122 221.975013  82.166.210.6          10.192.0.1            SIP/SDP  Status: 183 Session Progress, with session description
 123 222.013388  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 124 222.032360  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 125 222.046283  82.166.210.134        10.192.0.1            UDP      Source port: 16480  Destination port: 5004
 126 222.052372  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 127 222.066521  82.166.210.134        10.192.0.1            UDP      Source port: 16480  Destination port: 5004
 128 222.072316  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 129 222.086438  82.166.210.134        10.192.0.1            UDP      Source port: 16480  Destination port: 5004
(and packets keep flowing...)

As people in the industry know, the voice circuit starts without waiting for the other side to answer in an outgoing call.

The invitation packet in detail is pretty much like the previous one, just the other way around:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 988
 Checksum: 0x196d (correct)
Session Initiation Protocol
 Request line: INVITE sip:048222222@centrex.res.netvision.net.il SIP/2.0
 Message Header
 From: "0736666666"<sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a7b30-100c00a-13c4-50029-da-174234ad-da
 To: <sip:048222222@centrex.res.netvision.net.il>
 Call-ID: 100b2350-100c00a-13c4-50029-da-4cc1ed17-da
 CSeq: 1 INVITE
 Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-da-3561c-7fd18a32
 Max-Forwards: 70
 Supported: replaces,100rel
 User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
 Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
 Contact: <sip:200012972736666666@10.192.0.1:5060>
 Content-Type: application/sdp
 Content-Length: 32

Followed by the Trying packet:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 371
 Checksum: 0x5d52 (correct)
Session Initiation Protocol
 Status line: SIP/2.0 100 Trying
 Message Header
 From: "0736666666"<sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a7b30-100c00a-13c4-50029-da-174234ad-da
 To: <sip:048222222@centrex.res.netvision.net.il>
 Call-ID: 100b2350-100c00a-13c4-50029-da-4cc1ed17-da
 CSeq: 1 INVITE
 Via: SIP/2.0/UDP 10.192.0.1:5060;received=46.116.190.192;branch=z9hG4bK-da-3561c-7fd18a32;rport=506

and then

Session Initiation Protocol
 Status line: SIP/2.0 183 Session Progress
 Message Header
 From: "0736666666"<sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a7b30-100c00a-13c4-50029-da-174234ad-da
 To: <sip:048222222@centrex.res.netvision.net.il>;tag=SD2kim999-10.60.20.110-4294957762-7934
 Call-ID: 100b2350-100c00a-13c4-50029-da-4cc1ed17-da
 CSeq: 1 INVITE
 Via: SIP/2.0/UDP 10.192.0.1:5060;received=46.116.190.192;branch=z9hG4bK-da-3561c-7fd18a32;rport=5060
 Contact: <sip:048222222@82.166.210.6:5060;transport=udp>
 Content-Type: application/sdp
 Content-Length: 199
Session Description Protocol
 Session Description Protocol Version (v): 0
 Owner/Creator, Session Id (o): - 1227677235 2 IN IP4 82.166.210.134
 Owner Username: -
 Session ID: 1227677235
 Session Version: 2
 Owner Network Type: IN
 Owner Address Type: IP4
 Owner Address: 82.166.210.134
 Session Name (s): -
 Connection Information (c): IN IP4 82.166.210.134
 Connection Network Type: IN
 Connection Address Type: IP4
 Connection Address: 82.166.210.134
 Time Description, active time (t): 0 0
 Session Start Time: 0
 Session Start Time: 0
 Media Description, name and address (m): audio 16480 RTP/AVP 18 101
 Media Type: audio
 Media Port: 16480
 Media Proto: RTP/AVP
 Media Format: 18
 Media Format: 101
 Media Attribute (a): rtpmap:18 G729/8000
 Media Attribute Fieldname: rtpmap
 Media Attribute Value: 18 G729/8000
 Media Attribute (a): sendrecv
 Media Attribute (a): rtpmap:101 telephone-event/8000
 Media Attribute Fieldname: rtpmap
 Media Attribute Value: 101 telephone-event/8000
 Media Attribute (a): fmtp:101 0-15
 Media Attribute Fieldname: fmtp
 Media Attribute Value: 101 0-1

I have snipped off the session description protocol parts from the previous packets. The “Media Port” entry is obviously how the sides expose their UDP ports.

July 2014 update

A month ago, or so, the phone suddenly stopped connecting, and the attempt to register was refused flat. I called their support, they told me they will be working on it, and then the phone came back to life. This is the updated UDP packet dump of the registration. Note that it fails first, and then the client tries again, with an improved request.

REGISTER sip:centrex.res.netvision.net.il SIP/2.0
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 1 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-b-2cec-1e8eaaf
Max-Forwards: 70
Supported: replaces,100rel
Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
Expires: 1800
Contact: <sip:200012972736666666@10.192.0.1:5060>
User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
Content-Length: 0

SIP/2.0 401 Unauthorized
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 1 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;received=93.173.36.5;branch=z9hG4bK-b-2cec-1e8eaaf;rport=5060
Content-Length: 0
WWW-Authenticate: Digest realm="NcxSip", nonce="62445575"

REGISTER sip:centrex.res.netvision.net.il SIP/2.0
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 2 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-d-3689-2ec8fad1
Max-Forwards: 70
Supported: replaces,100rel
Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
Expires: 1800
Authorization: Digest username="200012972736666666",realm="NcxSip",nonce="62345345",uri="sip:centrex.res.netvision.net.il",response="26e7f74553dae1131cef72c3c90c5b67",algorithm=MD5
Contact: <sip:200012972736666666@10.192.0.1:5060>
User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
Content-Length: 0

SIP/2.0 200 Ok
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 2 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;received=93.173.36.5;branch=z9hG4bK-d-3689-2ec8fad1;rport=5060
Contact: <sip:200012972736666666@10.192.0.1:5060>;expires=120
Content-Length: 0

As before, numbers and strings that may be specific have been altered in the dump.