my tech blog

Download a Linux distribution for Xilinx’ Microblaze

If you want to get to it, just go to the Xillybus mini-distro for Microblaze’s page.

This distribution is a software kit, which allows you to run a functional Linux system on the SP605 or ML605 hardware evaluation kit (for Spartan-6 and Virtex-6, respectively). All components necessary to build it will be available for download at no cost (partly from Xilinx’ site). The distribution is not a demo kit; it’s a kickstart for a real-life application.

The steps to reaching a functional system include installing pieces of software, and perform specific and rather trivial operations as described in detail. No prior knowledge on either FPGA nor Linux is necessary. Your computer may run either Windows or Linux for running this through.

This distribution will allow you to

• Run a Linux system which is ready for working with, including a DHCP client, telnet, ftp, a small web server, mounting of NFS as well as Windows (SMB) shares. The CompactFlash card will also serve as the local disk which can be written to.
• Easily compile your own user-space applications in with “make” and gcc (a ready-to-go cross-compilation environment will be available for download as well). Compilation is possibly dynamic against libraries which are part of the distro.
• Easily develop your own Linux-reachable peripherals on the FPGA using the Xillybus IP core, and transport data between the FPGA and Linux user space applications with minimal effort.

Required hardware

• A Xilinx SP605 evaluation kit or ML605 ditto.
• A Sandisk CompactFlash card, with at least 512 MB. Let me emphasize this: Make it a Sandisk card, and not anything else. You may get away with using other cards for a one-off loading with data and multiple reads, but the Linux system is going to use this storage as its hard disk. Anything else than Sandisk, including the card which comes with the evaluation kit itself, is most likely to have reliability problems, which will give you a very wrong impression regarding the Linux system’s stability. Believe me, I’ve tried quite a few. Also, try to make the card not bigger than 8 GB, as the SysACE chip’s support of larger cards is questionable.
• A CompactFlash to USB adapter. There is no special requirements here, as long as your operating system recognizes it, and it’s reliable. Which holds true even for those cheap universal card readers available on Ebay.

As mentioned above, this is in the Xillybus mini-distro for Microblaze’s page.

GNUPREFIX=../gnutools/microblazeel-unknown-linux-gnu/bin/microblazeel-unknown-linux-gnu-

CC=$(GNUPREFIX)gcc
AR=$(GNUPREFIX)ar
AS=$(GNUPREFIX)as
CXX=$(GNUPREFIX)g++
LD=$(GNUPREFIX)ld
STRIP=$(GNUPREFIX)strip

CFLAGS=-Wall -I. -O3

APPLICATION=hello

OBJECTS=#somefile.o somefile2.o etc

all: $(APPLICATION)

$(APPLICATION): $(OBJECTS) $(APPLICATION).o
	$(CC) $(CFLAGS) $(OBJECTS) $(APPLICATION).o -o $(APPLICATION)

clean:
	rm -f *~ $(APPLICATION) *.o

# mount -o remount,rw /

# mount -o remount,rw /dev/xsa3

# mount -o remount,ro /

# mount -o remount,ro /dev/xsa3

# halt

#!/bin/sh  

ifconfig eth0 10.12.14.16 netmask 255.0.0.0

/ # chmod a+x /rw/system/userrc

/ # dhcp
udhcpc (v1.17.1) started
Sending discover...
PHY: c0729a0c:07 - Link is Up - 100/Full
Sending discover...
Sending select for 10.12.14.16...
Lease of 10.12.14.16 obtained, lease time 21600
deleting routers
route: SIOCDELRT: No such process
adding dns 10.0.0.1
adding dns 10.0.0.2
/ # eth0: no IPv6 routers present

/ # ifconfig eth0 10.12.14.16 netmask 255.0.0.0

/ # route add default gw 10.11.12.13

# mount -o user=theuser,password=thepass //10.11.12.13/mydisk /mnt/try

# mount 10.11.12.13:/thedir -o nolock,proto=tcp /mnt/try

/ # mount -o remount,rw /

Gnome: Using your own alert sound

As usual with those fancy Gnome themes (as came with Fedora 12 in my case), the basic assumption is that you’re a not-so-clever user, so not too many choices should be left for you. So pick one of those premade alert sounds. Picking your own sound file? Noooo. That’s way too much freedom for a dummy user. So what if that’s possible on Windows? Linux freedom doesn’t mean that end users should get too much control over their computers…

OK, so here’s how to use a custom sound file as your alert. First, generate the desired sound file if necessary, for example, mybeep.ogg, and put is somewhere. Say /usr/share/sounds/ (where there are a lot of other sound clips). Make it a stereo file, possibly WAV (or .ogg) with a sample rate of 44100 Hz.

Then edit /usr/share/gnome-media/sounds/gnome-sounds-default.xml, and add an entry just before the last line (which is </sounds>). It could be something like this:

<sound deleted="false">
 <name>My own beep</name>
 <filename>/usr/share/sounds/mybeep.ogg</filename>
</sound>

All that’s left is to open Preferences > Sound (or gnome-volume-control at the command line) and select the new entry in the list of possible alert sounds.

That’s it. It’s actually simple, once the configuration file is unburied.

Random notes about git

Merely for myself, so I’ll remember how to do it. If I’m doing stupid things, please comment below.

Rule #1

If you’re about to do anything with git being unsure about the consequences, always protect yourself with

$ git branch bettersafe
$ git commit -a
$ git checkout whatever

Assuming that you want to mess on the branch “whatever”

No matter what happens next, you can always return things to where they were by checking out bettersafe. Really. As long as you don’t mess up the bettersafe branch, of course. That is, don’t mention it in any subsequent command, and git won’t touch it. Don’t delete it, don’t use it in merge or rebase commands. Only check it out if things go wrong.

Otherwise, you can rebase, mess around, delete files and squash commits. No commits or other data will be lost, because bettersafe depends on them.

Delete bettersafe at some later time, of course. When you’re sure everything turned out OK.

To keep in mind

• Git manages changes, not versions. Think of a commit as a patch, not a snapshot. Even though a commit’s ID happens to be a representation of the project’s current snapshot.
• A branch is just a pointer to some commit’s ID (possibly base) saying “my next commit will be based on this commit” (the latter is the branch’s HEAD).
• Being on a branch controls where your next commit will go
• Checking out a branch is like checking out the root commit, and running all commits (think patches) up to that branch’s HEAD
• Hence rebasing actually means moving the branch’s first commit’s parent ID.

Backing up and restoring the whole repo

To backup the entire repository:

$ git bundle create mybundle.git --all

This creates the mybundle.git file, which is an efficient storage of the entire repository.

To restore all branches and tags, the following sequence applies:

$ git clone mybundle.git myclone
$ cd myclone/
$ for i in `git bundle list-heads mybundle.git` ; do git checkout ${i##*/} ; done
$ git checkout master

The for-loop produces as lot of output, and eventually leaves the working tree on some random ref. So the final checkout gets you where you want to start, presumably “master” in the example above.

I will be most delighted to know if there exists a single command doing this. As it seems, just “git clone” copies only the commits belonging to the current branch (i.e. HEAD), and any refs that were on its way. This probably makes sense when cloning from a remote repository with gazillions of commits. The concept of a local backup was probably neglected, because of the way git is usually used (that is, in collaboration).

For a partial backup of a certain sequence of commits, say from “master” to “playaround” go

$ git bundle create mybundle.git master..playaround

To use this bundle, “git pull” must be used on the target repository to get the commits. Needless to say, it must already have the commit pointed to by “master” when the bundle was made, or the entire bundle is completely worthless.

Oops. I messed up.

Git doesn’t delete anything except for during periodical (or initiated) garbage collections, so if something was committed in the past, there’s a change it’s still there, only invisible. The trick is to use

$ git reflog

and if the lost commit is found, tag it with

$ git tag rescue_me 7cb12d6

Tagging it makes the commit (and those leading to it) visible in gitk (given that it’s set to show all refs) and also prevents their removal on the next garbage collection.

Git sequence for making a patch for submission

See Documentation/SubmittingPatches in the Linux tree.

First, clone the main git that appears in the MAINTAINERS part for the subsystem the patch is going to. Probably better, add the repo to the existing main Linux repository, initially fetched with

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Remember: The patch must apply (as in “git am”) against the “next” branch of the relevant subsystem’s repository.

Make the changes, verify that the latest version of sparse doesn’t complain with e.g.

$ make M=drivers/staging/mydriver C=2

The C=2 forces as sparse run even on files that need no compilation. C=1 only checks modified files.

Always grab the latest version of sparse and check against the latest possible kernel. Really. I really had new warnings popping up just by checking against a newer kernel.

When all is well, commit. If the patch is a response to a bug report, add a “Reported-by:” header at the end of the commit message manually. Same goes for Tested-by, Suggested-by and Reviewed-by. There is no automatic mechanism for this (like –signoff).

If the patch fixes a bug, add a Fixes: header as well, with a commit ID and title of where the bug first occurs. If the bug was there from the beginning, put the commit ID where the file was inserted. The decision on whether the patch should be applied to stable kernels as well is made based upon this.

Also: Always have something meaningful written in the commit description. Just a title isn’t good enough (Greg rejected a very short patch of mine on the grounds that it had only a subject and no “changelog information”).

$ git commit -a
$ git format-patch master -o patch --signoff

This will create a directory called “patch” and put the patch there. If several commits are made, a different patch file is created for each. That what the commit -a is for.

To get all commits in one patch, use the –to-stdout flag, and of course redirect stdout to a file.

If this is a second submission (version 2 = v2), use

$ git format-patch master -o patch --signoff --subject-prefix="PATCH v2"

It’s also a desired to add a short description on the difference from earlier versions. The place to do it, is under the “—” mark of the signoff. This can be done manually, or with git notes (see below) for creating a note for a commit, and then use the –notes flag with git format-patch. The result in the patch file is something like this:

Signed-off-by: Eli Billauer <eli.billauer@example.com>
---

Notes:
 This is a really horrible patch

Check the patch with (run from kernel tree’s root)

$ scripts/checkpatch.pl --strict patch/0001-name-of-patch

(note that checkpatch.pl has a -f flag, allowing to check a source file rather than a patch)

I marked the –strict flag red, because I forgot to use it in an important occasion. It’s turned on automatically, by the way, when checkpatch detects that the target directory is one of drivers/net, net/ or drivers/staging/. Yes, the directories are hardcoded in the script itself. Perl or not?

Whitespace cleanup:

$ scripts/cleanpatch patch/0001-name-of-patch

Send it through email (the way the kernel guys like it)

git send-email --to 'linux-kernel@vger.kernel.org' patch/0001-name-of-patch

Add a –cc flag to send a copy to someone else than yourself. The mail won’t kick off before confirming it, so don’t worry…

Now, since I’m using my Gmail address as the “From”, the mail must come from a Gmail mail server. In order to relay the mail through their servers (and not my own), there’s a whole story about that. See this post.

git notes

This is a neat mechanism for adding extra information to commits. Its main use is for change log information to patches, so they appear in the email but not in the commit (if and when it’s applied).

To make thing simple, this is the only command needed to maintain notes:

$ git notes edit

It adds and/or edits the notes for the current commit with the selected editor.

As of gitk that goes along with 2.17.1 (2016 edition? It doesn’t say its version), the notes appear at the commit view, below the commit title, after a hard update (Ctrl-Shift-F5). There are also small yellow boxes next to the commit title in the tree view to mark that there are notes related to the commit.

Following the suggestion on this page, add the following lines to .git/config of the relevant repository.

[notes "rewrite"]
    amend = true
    rebase = true
[notes]
    rewriteRef = refs/notes/commits

The problem this solves: Notes refer to commits by their object ID, which changes when the commit is rebased or amended. As a result, the note becomes detached from the commit it related to. This chunk tells git to update the notes’ refs to follow the commits.

It’s also possible to add it to ~/.gitconfig, but if rebasing or commit amending is done on a repository that doesn’t have any notes, one gets a “warning: notes ref refs/notes/commits is invalid”.

Applying a Linux patch into a local project

The path of the files in the Linux project is deeper, so the application is with something like

$ git am -p4 patchdir/0001-this-is-the.patch

or the other way around: Applying a local project’s patch into the Linux kernel:

$ git am --directory drivers/mypath/mydriver/ 0001-this-is-the.patch

Applying a dirty patch

If the patch was made not to be applied directly (e.g. apply changes made in one file to another, by editing the patch file) format-patch can be used to generate the patch, and then go

$ git apply --reject --whitespace=fix patchdir/0001-This_is_the_patch

I think it’s best to remove the header containing the commit ID from which the patch was originally made, but haven’t really tried not doing this.

Preventing diff from working on a binary file

In particular, gitk has a tendency to try diffing *.ngc files and therefore freezing, since they start with text and then go blob. Create (or edit) .gitattributes in the project’s root directory (man gitattributes) to say

*.ngc binary

(as usual with git, the solution is painfully simple) and it probably makes sense to commit this file into the project as well.

Messing around with commits

For a graphical representation of the branches and commits, change directory to where the git commands are run from and simply go

$ gitk &

To see a the complete tree of local branches (recommended) go to View > New View and check “All (local) branches”.

Add files to be watched by git (and hence relevant on the next commit)

$ git add filename filename2 ...

This can also be done as one step of a commit command. Committing all watched files:

$ git commit -a

Oops…? Made some changes which would fit best in the last commit? Want to fix the last commit, and re-edit the commit message? Easy, just go

$ git commit -a --amend

Note that this changes the commit message as well as updating the commit according to the working tree.

And change the date of the commit to “now”:

$ git commit --date "`date -R`" --amend

… but that changes only the authoring date. To set the committing date to the same, go

$ git rebase --committer-date-is-author-date HEAD~1

And of course, the latter command can go deeper into history, depending on what’s instead of “HEAD~1″.

For setting the date to the current one on several commits, use e.g.

$ git rebase --ignore-date origin/master

To commit only part of the changes:

$ git add --all -p
$ git commit

To use an existing commit’s log message (in particular if it’s cherry-picked or even orphaned):

$ git commit -c 77ddd228c8cc26801eb83f421048d30fa1c31564

To move the current branch’s head, so it doesn’t include the latest commit (but leave the changes in the sources), a.k.a. “remove the commit”:

$ git reset HEAD~1

Note that the commit stays in the repository until garbage collected, and its changes remain in the worktree. This actually says “move the branch one step (as in HEAD~1) back. A “git checkout -f” will remove the commit’s effect on the worktree as well.

To really go back one commit (that is, revert its changes in the working tree),

$ git reset --hard HEAD~1

Even though the commit stays in the repository until garbage collected, that may happen sooner than desired. So if the commit may be useful in the future, create a new branch (e.g. “delme”) before backing off the current one, so the commit isn’t lost.  If this is done by mistake, just cherry-pick the reverted commit with gitk (hoping it’s still there).

And check the latest commits with

$ git log

Rebasing your own experiments (branch “foolaround”) on top of the master branch (so your games are on the real thing + your changes)

$ git rebase master foolaround

Note that the third argument, “foolaround” tells git to check out this branch first, and then rebase it to master. It’s otherwise assumed that the current branch is rebased, so this format is somewhat safer. But “git rebase master” is fine as well (and rebasing “master” on itself just says that the branch is up to date, which happens to be always be true).

Move some commit to the top of some other branch, say to master:

$ git rebase 77ddd228c8cc26801eb83f421048d30fa1c31564 master

where that blob in the middle is the commit ID, of course. Unlike “git cherry-pick” which applies the changes only, without changing the tree of commits)

To fool around with the 4 last commits (reorder, remove, squash several commits into one:

$ git rebase -i HEAD~4

Note that when squashing, the commit marked to squash is mixed with the commit one the row above in the list (which was committed later in time). An opportunity to edit the commit message will be given anyhow, so just mark the commits for squashing and go on with it. Of course, several commits can be marked in a row for a multi-squash.

What have I changed since the last commit?

$ git diff

What is the difference between now and some branch?

$ git diff somebranch

What is the commit ID of a given tag, branch or other commit one can refer to (HEAD in this example)?

$ git rev-parse HEAD

What’s the last tag issued in the current branch?

$ git describe --tags --abbrev=0

To check for another branch, add its name as the last argument for the same command above.

Apply changes as if checking out another branch, but stay in place:

$ git checkout -p thatbranch

Each hunk (that is, piece to change) is prompted for. Say “y” to all, and the working tree will be like “thatbranch”. Say “n” to all, and nothing changes. This way or another, you don’t switch branch, only the files changes. This is, in fact, not a checkout.

Find the directory where .git/ sits:

$ git rev-parse --show-toplevel

Untracking files already in index

This recipe works when it’s OK to temporarily remove the files from the worktree.

First, edit the .gitignore so that the relevant files will be ignore in the future. Then commit .gitignore and create a tag on the last commit, say “hold”. Make sure that the files are indeed tracked:

$ git ls-files | less

Now remove the files from the index, possibly with several commands e.g.:

$ git rm --cached useless-file

This changes nothing in the working tree for now, but only marks the files as gone in the index.

And then commit. The files will be removed in the working tree.

$ git commit

To have the files back, patch-checkout the previous commit. HEAD~1 would work instead of “hold”, but the purpose of hold was also to be an anchor for messups:

$ git checkout -p hold

When applying the relevant hunks, git asks as if these will be applied to the index as well, but in fact they don’t go to the index because of the updated .gitignore. So when the process is finished, git remains in sync (unlike it would in a normal checkout -p session).

If all is well, the “hold” tag can be removed.

To get a list of branches so a script can handle the output (as opposed to just “git branch” which isn’t safe):

$ git for-each-ref --format='%(refname:short)' refs/heads/

Cleaning up the directory tree

Use with caution, and think twice before doing this: It deletes all files not tracked in the repo. All those nice private scripts and stuff? Gone.

To clean untracked files (those appearing in “git status”) go first

$ git clean -n

and see what the damage is, and then remove the -n if being sure. To delete all files, regardless of .gitignore (this is “make mrproper” just a little more aggressive):

$ git clean -n -d -fx

Windows: Remove git from Explorer right-click menus

Uninstalling git involves closing explorer.exe (!) and didn’t work even so on my computer, so I took the registry editing route.

Basically removed the HKEY_CLASSES_ROOT\*\shellex\ContextMenuHandlers\Git-Cheetah, HKEY_CLASSES_ROOT\Directory\Background\shellex\ContextMenuHandlers\Git-Cheetah, HKEY_CLASSES_ROOT\Drive\shellex\ContextMenuHandlers\Git-Cheetah registry keys and other places where I found Git-Cheetah under ContextMenuHandlers.

Also removed anywhere the string {ca586c80-7c84-4b88-8537-726724df6929} appeared under something having to do with shell extensions. A bit scary, but harmless.

Setting up my own little git server

A git server is just like any repository, only the data is kept “bare”, that is without the working tree, and with the files usually in the .git subdirectory residing directly on the repo’s root. A simple “git clone” with the repo’s root yields the full git repository, in case of doubt. The gitk utility can also be run from the repo’s root.

First thing first:

# yum install git-daemon

Among others, this adds /etc/xinetd.d/git, which is disabled by default.

Edit the file to read (changed parts in red):

# default: off
# description: The git dæmon allows git repositories to be exported using \
#       the git:// protocol.

service git
{
 disable         = no
 socket_type     = stream
 wait            = no
 only_from       = 10.1.1.0/24 127.0.0.1
 user            = git
 server          = /usr/libexec/git-core/git-daemon
 server_args     = --base-path=/home/git/public_git --export-all --syslog --inetd --verbose --enable=upload-pack --enable=receive-pack
 log_on_failure  += USERID
}

Create a new user “git”, create public_git in its home directory, and then go

# service xinetd restart

And setup a new repo, e.g. (as user “git” in its home directory):

$ mkdir test
$ cd test/
$ git --bare init

And then, as any user, bind the origin. This is not necessary if the repository is cloned from the origin anyhow.

$ git remote rm origin
$ git remote add origin git://localhost/test/

And possibly push and fetch (I don’t like pulling, because the merging can fail). See my note about a problem with “git push” on Windows.

$ git push --all
$ git fetch --all

Note that the the new commit will not necessarily appear after fetching, unless the display mode is set to “all refs”.

To push a certain branch to the remote repo, despite “push –all” failing because some history will be killed, go

$ git push -n origin +master
$ git push origin +master

for the pushing the master branch by force (the -n option is for a dry-run). This is OK in particular after some commit munching on a branch, which no other development entity depends on. Or the other side will have to cherry-pick its latest commits. Or move its local “master” branch to the new one, and rebase from there (losing tags).

To synchronize with the remote repository, either merge or rebase. The remote branch is called origin/branch, e.g.

$ git rebase origin/master

rebases the local changes on the remote ones on the master branch.

To push all tags to the remote server, go

$ git push --tags

but remember not to do this when there are temporary tags (because the only way to get rid of them is from the server).

To turn the remote into an exact copy of the local repo, use

$ git push --mirror

with care: No questions are asked. The remote repo is just overridden. Whatever is gone on the local repo will be gone in the remote one. This is important in particular if commits have been squashed etc. Try “push –all” first. If that fails, it’s likely that the other copies of the repository will have to be re-cloned. Which is fairly OK if they’ve all pushed –all and the current repo has at least fetched –all.

Always make sure there is a “master” branch when pushing. Otherwise attempts to clone will result in “remote HEAD refers to nonexistent ref, unable to checkout” and a useless local copy. This is because HEAD points at “master” by default.

Remote repos

$ git branch --set-upstream localbranch theorigin/master

This makes the local branch track the “master” branch on some remote repository. Unfortunately, for pushing, one still has to be explicit:

$ git push theorigin localbranch:master

This makes git treat “localbranch” as the remote’s “master”, and it’s necessary despite the upstream setting above.

Uploading to github

Definitely, go for ssh authentication. It requires copying the content of ~/.ssh/id_rsa.pub into a text window under Github’s web interface: Click on the icon at the top right, choose Settings and then select “SSH and GPG keys”. Then be sure to define git’s remote as “git@github.com:billauer/theproject.git” and not the https: thing. And from this point on, it’s just like password-less ssh. That is, forget about authentication.

Ah, but it’s not that simple if you have multiple accounts on Github. Because if an ssh key is already used for another user, it’s rejected when trying to apply it on a new one, with the error message “Key is already in use”. The solution is to maintain multiple keys. This is mildly annoying once you get the hang of how to do that. See section named “Having multiple SSH keys” in another post of mine. The trick is to invent a name for a host, say, “sillygit”. Set git’s remote as “sillygit:billauer/theproject.git”. Then set SSH’s config file to recognize “sillygit” as the name of a host, and apply a set of parameters: The real host name, the user name and the SSH key to use.

Don’t confuse this Deploy keys, which are a per-repository SSH keys. These keys allow access to a specific repository only.

Access to multiple users

If the repository should be accessible to multiple users that belong to a group, add this to the config file:

sharedRepository = true

See git’s doc on configuration parameters.

Microblaze + Linux: Sample design of a custom peripheral

Scope

Even though Xilinx supplies a cute wizard for creating peripherals in its EDK (version 13.2 in my case), it’s just enough to work as a demo. For a real-life case there’s no escape from getting down to the system’s guts. As it turns out, things are pretty well organized under EDK’s hood, which makes the attempt to cover it all up with a wizard even more questionable.

This post is a jot-down of the technicalities behind designing a minimal bare-boned peripheral and its Linux driver. With “bare-boned” I mean that it has the absolutely simplest bus interface (AXI4 Lite without even decoding the addresses), and that it’s in pure Verilog. No external IP is used.

This peripheral connects to the SP605′s four LEDs. Any write to its address region updates the LED’s state according to the written value’s four LSBs. Reading back from any of its covered addresses gives the four LSBs back. That’s all.

Sources of information

This post assumes that you’re familiar with running Linux on Microblaze. I have a pretty extensive tutorial on the subject for reference.

These are worth to look at:

• The official cores’ sources (in VHDL) can be found at ISE_DS\EDK\hw\XilinxProcessorIPLib\pcores (path from where ISE is installed). It’s interesting in particular to look at axi_lite_ipif_v1_00_a.
• The AMBA AXI protocol specification, downloaded free from ARM’s site.
• Platform Specification Format Reference Manual (UG642, psf_rm.pdf): Describes the file formats in detail. Necessary when editing the files.
• EDK Concepts, Tools, and Techniques (UG683, edk_ctt.pdf) : The chapter about Creating Your Own Intellectual Property is somewhat helpful to try out the Wizard.

Understanding the process

It looks like the missing link in Xilinx’ documentation is to explain how the whole machinery works with regard to adopting a custom made peripheral. I’ll try to fill in that gap now.

Generally speaking, the minimal core consists of the following files, which should be in a dedicated directory under the “pcores” directory, which is under the EDK project’s top directory:

• data/minimal_v2_1_0.mpd: This file is what EDK looks at when an IP is added to a project. It contains all the information used directly by the EDK. The peripheral’s GUI is set up according to this information, and it’s also used when the EDK generates wrappers and connections for it. Its format is well documented, but it looks like it’s easier to just copy snippets from existing core’s MPD files. It’s also possible to generate this file automatically with PsfUtility from the toplevel source file, but it’s not clear if it’s worth the effort to learn yet another tool.
• data/minimal_v2_1_0.pao: This file supplies EDK with a list of HDL files which need to be synthesized to create the peripheral. It also sets the order of synthesis.
• hdl/verilog/minimal.v: The Verilog file constituting the peripheral. Of course there may be several files, which need to be mentioned in the PAO file.
• Note that “black box” modules (presynthesized netlists) are listed in BBD files, which are not necessary in this example. When used, the MPD file is set to reflect this.

The file names above relate to a peripheral called “minimal”. They change according to the project’s setting and version numbers.

All in all, the flow is pretty simple: Only the MPD file is considered by EDK, and only at platform netlist generation are the HDL files synthesized according to the PAO file. The instantiation and connection depend on the settings within the EDK (actually, the MHS file).

It’s easiest to create just any peripheral with the wizard, see what they do, and then modify the files.

Going from Wizard’s output to minimal peripheral

This is a short outline of the stages. The result is given in the next section.

• Edit the data/*.pao file: Remove all files and insert the single Verilog file, changing the type to verilog.
• In the data/*.mpd file, change OPTION HDL = VHDL to VERILOG too. Also add, under ##ports, PORT minimal_leds = “”, DIR = O, VEC = [3:0] (so that the I/O port is known. Note the =”" part).
• Remove data/*.prj file  so no unnecessary files are included (even though this file seems to be ignored).
• Roughly translate the VHDL file to Verilog. Make it Verilog parameters instead of VHDL generics.
• Rename/remove the devl directory, since its information is not in sync with the new situation, and Custom IP Wizard can’t do any good at this point.
• And finally, in EDK, Project > Rescan User Repositories
• Remove the LED_4bits core from the project, choosing “Delete instance and any connections to internal nets”. This will keep the net names used for connecting to the LEDs, and make them available for connection to the new peripheral. Otherwise, the external net names need to be set, and the system.ucf given at the “project” tab updated to reflect the new nets.
• Add the minimal core to the project, and connect the just released LEDs_4Bits_TRI_O to its minimal_leds port.
• Create bitfile

The synthesis of the peripheral’s HDL takes place during the “create netlist” flow (which is, of course, part of generating bitfile). For example, the synthesis of an instance named minimal_0 will appear as follows in the console

INSTANCE:minimal_0 - C:\tryperipheral\system.mhs line 424 - Running XST
synthesis
PMSPEC -- Overriding Xilinx file
<C:/ise13_2/ISE_DS/EDK/spartan6/data/spartan6.acd> with local file
<C:/ise13_2/ISE_DS/ISE/spartan6/data/spartan6.acd>

And if there are errors in the HDL, they will show up at this point.

Sample files

These are the files used for the minimal peripheral. They are a sloppy adoption of the files generated by the Custom IP Wizard, so they’re very likely to contain unnecessary declarations.

First, the Verilog file:

module minimal #(
 parameter C_S_AXI_DATA_WIDTH = 32,
 parameter C_S_AXI_ADDR_WIDTH = 32,
 parameter C_S_AXI_MIN_SIZE = 'h000001FF,
 parameter C_USE_WSTRB = 0,
 parameter C_DPHASE_TIMEOUT = 8,
 parameter C_BASEADDR = 'hFFFFFFFF,
 parameter C_HIGHADDR = 'h00000000,
 parameter C_FAMILY = "spartan6",
 parameter C_NUM_REG = 1,
 parameter C_NUM_MEM = 1,
 parameter C_SLV_AWIDTH = 32,
 parameter C_SLV_DWIDTH = 32
 )
 (
 input S_AXI_ACLK,
 input S_AXI_ARESETN,
 input [(C_S_AXI_ADDR_WIDTH-1):0] S_AXI_AWADDR,
 input S_AXI_AWVALID,
 input  [(C_S_AXI_DATA_WIDTH-1):0] S_AXI_WDATA,
 input  [((C_S_AXI_DATA_WIDTH/8)-1):0] S_AXI_WSTRB,
 input S_AXI_WVALID,
 input S_AXI_BREADY,
 input  [(C_S_AXI_ADDR_WIDTH-1):0] S_AXI_ARADDR,
 input S_AXI_ARVALID,
 input S_AXI_RREADY,
 output S_AXI_ARREADY,
 output  [(C_S_AXI_DATA_WIDTH-1):0] S_AXI_RDATA,
 output  [1:0] S_AXI_RRESP,
 output S_AXI_RVALID,
 output S_AXI_WREADY,
 output [1:0] S_AXI_BRESP,
 output reg S_AXI_BVALID,
 output S_AXI_AWREADY,

 output reg [3:0] minimal_leds
 );

 assign  S_AXI_RDATA = minimal_leds;
 assign  S_AXI_RRESP = 0; // OKAY on AXI4
 assign  S_AXI_ARREADY = 1; // Always ready for read address
 assign  S_AXI_AWREADY = 1; // Always ready for write address
 assign  S_AXI_RVALID = 1; // Read data always valid (ILLEGAL)
 assign  S_AXI_WREADY = 1; // Always ready to write
 assign  S_AXI_BRESP = 0; // OKAY on AXI4

 // This will not work OK if several "bursts" are sent with no BVALIDs
 // inbetween. Not an expected scenario.

 always @(posedge S_AXI_ACLK)
   if (S_AXI_WVALID)
     begin
        S_AXI_BVALID <= 1;
        minimal_leds <= S_AXI_WDATA;
     end
   else if (S_AXI_BREADY && S_AXI_BVALID) // Active BRESP cycle
     S_AXI_BVALID <= 0;

endmodule

Most of the parameters at the top can be removed, I believe. It appears like they are necessary only when creating the MPD file with PsfUtility.

All ports, except minimal_leds are standard AXI4 lite ports. The implementation of the interface isn’t example for anything except a quick and dirty peripheral which responds to bus requests. The only thing it does actively is to update minimal_leds when necessary, and toggle the AXI_BVALID, so that only one burst response is sent for each write cycle (which is always one clock long in AXI4 lite). It’s OK not to decode the address, since it’s the interconnect’s job to make sure each peripheral gets only what it directed to it.

Holding S_AXI_RVALID high all the time violates the AXI4 spec, since it’s required to be asserted only after ARVALID and ARREADY. But the interconnect tolerated this anyhow.

Now to minimal_v2_1_0.mpd:

BEGIN minimal

## Peripheral Options
OPTION IPTYPE = PERIPHERAL
OPTION IMP_NETLIST = TRUE
OPTION HDL = VERILOG
OPTION IP_GROUP = MICROBLAZE:USER
OPTION DESC = MINIMAL
OPTION LONG_DESC = A minimal peripheral to start off with
OPTION ARCH_SUPPORT_MAP = (others=DEVELOPMENT)

## Bus Interfaces
BUS_INTERFACE BUS = S_AXI, BUS_STD = AXI, BUS_TYPE = SLAVE

## Generics for VHDL or Parameters for Verilog
PARAMETER C_S_AXI_DATA_WIDTH = 32, DT = INTEGER, BUS = S_AXI, ASSIGNMENT = CONSTANT
PARAMETER C_S_AXI_ADDR_WIDTH = 32, DT = INTEGER, BUS = S_AXI, ASSIGNMENT = CONSTANT
PARAMETER C_S_AXI_MIN_SIZE = 0x000001ff, DT = std_logic_vector, BUS = S_AXI
PARAMETER C_USE_WSTRB = 0, DT = INTEGER
PARAMETER C_DPHASE_TIMEOUT = 8, DT = INTEGER
PARAMETER C_BASEADDR = 0xffffffff, DT = std_logic_vector, MIN_SIZE = 0x0, PAIR = C_HIGHADDR, ADDRESS = BASE, BUS = S_AXI
PARAMETER C_HIGHADDR = 0x00000000, DT = std_logic_vector, PAIR = C_BASEADDR, ADDRESS = HIGH, BUS = S_AXI
PARAMETER C_FAMILY = virtex6, DT = STRING
PARAMETER C_NUM_REG = 1, DT = INTEGER
PARAMETER C_NUM_MEM = 1, DT = INTEGER
PARAMETER C_SLV_AWIDTH = 32, DT = INTEGER
PARAMETER C_SLV_DWIDTH = 32, DT = INTEGER
PARAMETER C_S_AXI_PROTOCOL = AXI4LITE, TYPE = NON_HDL, ASSIGNMENT = CONSTANT, DT = STRING, BUS = S_AXI

## Ports
PORT S_AXI_ACLK = "", DIR = I, SIGIS = CLK, BUS = S_AXI
PORT S_AXI_ARESETN = ARESETN, DIR = I, SIGIS = RST, BUS = S_AXI
PORT S_AXI_AWADDR = AWADDR, DIR = I, VEC = [(C_S_AXI_ADDR_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_AWVALID = AWVALID, DIR = I, BUS = S_AXI
PORT S_AXI_WDATA = WDATA, DIR = I, VEC = [(C_S_AXI_DATA_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_WSTRB = WSTRB, DIR = I, VEC = [((C_S_AXI_DATA_WIDTH/8)-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_WVALID = WVALID, DIR = I, BUS = S_AXI
PORT S_AXI_BREADY = BREADY, DIR = I, BUS = S_AXI
PORT S_AXI_ARADDR = ARADDR, DIR = I, VEC = [(C_S_AXI_ADDR_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_ARVALID = ARVALID, DIR = I, BUS = S_AXI
PORT S_AXI_RREADY = RREADY, DIR = I, BUS = S_AXI
PORT S_AXI_ARREADY = ARREADY, DIR = O, BUS = S_AXI
PORT S_AXI_RDATA = RDATA, DIR = O, VEC = [(C_S_AXI_DATA_WIDTH-1):0], ENDIAN = LITTLE, BUS = S_AXI
PORT S_AXI_RRESP = RRESP, DIR = O, VEC = [1:0], BUS = S_AXI
PORT S_AXI_RVALID = RVALID, DIR = O, BUS = S_AXI
PORT S_AXI_WREADY = WREADY, DIR = O, BUS = S_AXI
PORT S_AXI_BRESP = BRESP, DIR = O, VEC = [1:0], BUS = S_AXI
PORT S_AXI_BVALID = BVALID, DIR = O, BUS = S_AXI
PORT S_AXI_AWREADY = AWREADY, DIR = O, BUS = S_AXI
PORT minimal_leds = "", DIR = O, VEC = [3:0]

END

This file is exactly as generated by the Wizard, except for the HDL option in the beginning changed to VERILOG, and the added port minimal_leds at the end. Note its assignment to “”. This file is best created by looking at examples of existing cores.

Now to minimal_v2_1_0.pao:

lib minimal_v1_00_a minimal verilog

which was rewritten to reflect that the peripheral consists of one single Verilog file.

The device tree file

The device tree file needs to be generated as described in one of my posts. The relevant section is given here, since it relates to kernel code presented next:

minimal_0: minimal@7ae00000 {
 compatible = "xlnx,minimal-1.00.a";
 reg = < 0x7ae00000 0x10000 >;
 xlnx,dphase-timeout = <0x8>;
 xlnx,family = "spartan6";
 xlnx,num-mem = <0x1>;
 xlnx,num-reg = <0x1>;
 xlnx,s-axi-min-size = <0x1ff>;
 xlnx,slv-awidth = <0x20>;
 xlnx,slv-dwidth = <0x20>;
 xlnx,use-wstrb = <0x0>;
 }

It’s pretty evident that some of these parameters have no use.

The driver

First, it’s convenient to create a makefile for cross compilation. Even though the correct way is to set the environment variables in the shell, and run the module compilation in the same way the kernel itself is compiled, it’s much more convenient to go just “make” or “make clean” with this makefile. It’s not good for distribution, as the paths to both the kernel tree and cross compiler are hardcoded.

So here’s a dirty, but yet convenient makefile:

export CROSS_COMPILE=/path/to/microblazeel-unknown-linux-gnu/bin/microblazeel-unknown-linux-gnu-
export ARCH=microblaze

ifneq ($(KERNELRELEASE),)
obj-m    := minimal.o

else
KDIR := /path/to/linux-2.6.38.6

default:
 @echo $(TARGET) > module.target
 $(MAKE) -C $(KDIR) SUBDIRS=$(PWD) modules

clean:
 @rm -f *.ko *.o modules.order Module.symvers *.mod.? .minimal.* *~
 @rm -rf .tmp_versions module.target

minimal.ko:
 $(MAKE)
endif

And now to the driver itself, minimal.c:

#include <linux/platform_device.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_device.h>
#include <linux/of_platform.h>
#include <asm/io.h>

/* Match table for of_platform binding */
static struct of_device_id minimal_of_match[] __devinitdata = {
 { .compatible = "xlnx,minimal-1.00.a", },
 {}
};

MODULE_ALIAS("minimal");

static void __iomem *regs;
static struct resource res;

static int __devinit
minimal_of_probe(struct platform_device *op, const struct of_device_id *match)
{

 const int *width;
 int ret;
 int val;

 ret = of_address_to_resource(op->dev.of_node, 0, &res);
 if (ret) {
 printk(KERN_WARNING "minimal: Failed to obtain device tree resource\n");
 return ret;
 }

 printk(KERN_WARNING "minimal: Physical address to resource is %x\n", (unsigned int) res.start);

 if (!request_mem_region(res.start, 32, "mimimal")) {
 printk(KERN_WARNING "minimal: Failed to request I/O memory\n");
 return -EBUSY;
 }

 regs = of_iomap(op->dev.of_node, 0); /* Verify it's non-null! */

 printk(KERN_WARNING "minimal: Access address to registers is %x\n", (unsigned int) regs);

 width = of_get_property(op->dev.of_node, "xlnx,slv-dwidth", NULL);

 printk(KERN_WARNING "minimal: Obtained width=%d\n", be32_to_cpu(*width));

 val = ioread32(regs);
 printk(KERN_WARNING "minimal: Read %d, writing %d\n", val, val+1);

 iowrite32(++val, regs);

 return 0; /* Success */
}

static int __devexit minimal_of_remove(struct platform_device *op)
{
 iounmap(regs);
 release_mem_region(res.start, 32);
 return 0; /* Success */
}

static struct of_platform_driver minimal_of_driver = {
 .probe = minimal_of_probe,
 .remove = __devexit_p(minimal_of_remove),
 .driver = {
 .name = "minimal",
 .owner = THIS_MODULE,
 .of_match_table = minimal_of_match,
 },
};

int __init minimal_init(void)
{
 int ret;
 ret = of_register_platform_driver(&minimal_of_driver);
 return ret;
}

void __exit minimal_exit(void)
{
 of_unregister_platform_driver(&minimal_of_driver);
}

module_init(minimal_init);
module_exit(minimal_exit);

MODULE_AUTHOR("Eli Billauer");
MODULE_DESCRIPTION("Microblaze minimal module");
MODULE_LICENSE("GPL")

It doesn’t do anything special, except for change the state of the LEDs every time it’s loaded. The drivers also reads one of the parameters from the device tree structure. Not fascinating, but keeps the code, well, minimal.

This code should be pretty straightforward to programmers who are familiar with PCI device drivers, with probing and removal working in more or less the same way. I’ve chosen a hardcoded segment of 32 bytes as the requested region. This depends on the peripheral, of course.

A test run

This is the transcript of the session on the UART console, as run on a freshly booted system. LEDs did indeed go on and off as reflected by the numbers.

/ # insmod minimal.ko
minimal: Physical address to resource is 7ae00000
minimal: Access address to registers is c87e0000
minimal: Obtained width=32
minimal: Read 0, writing 1
/ # lsmod
minimal 1978 0 - Live 0xc8056000
ipv6 305961 10 - Live 0xc8763000
/ # cat /proc/iomem
40600000-4060000f : uartlite
40a00000-40a0ffff : xilinx_spi
40e00000-40e0ffff : xilinx_emaclite
7ae00000-7ae0001f : mimimal
/ # rmmod minimal
rmmod: module 'minimal' not found
/ # cat /proc/iomem
40600000-4060000f : uartlite
40a00000-40a0ffff : xilinx_spi
40e00000-40e0ffff : xilinx_emaclite
/ # lsmod
ipv6 305961 10 - Live 0xc8763000
/ # insmod minimal.ko
minimal: Physical address to resource is 7ae00000
minimal: Access address to registers is c8820000
minimal: Obtained width=32
minimal: Read 1, writing 2

Note that rmmod produces an error message, which makes it look as if it failed to remove the module, when in fact all went well.

The physical address was indeed detected correctly (see device tree), and mapped to another kernel virtual address each time.

Linux on Microblaze HOWTO (part IV)

This is part IV of my HOWTO on running Linux on Microblaze. The outline is as follows:

• Part I: Introduction and setting up the Microblaze processor
• Part II: Compiling the kernel
• Part III: Preparing for boot and booting
• Part IV: Compiling user space applications (this page)

Compiling user space applications

We shall now look at how to compile applications for execution under the Microblaze Linux machine. This is pretty straightforward for programs written for the specific environment. The problems may occur when compiling sources which were originally written for fullblown computers, as the build system may not have taken cross compilation into account. And as software projects tend to be hacked to death, with new features added all the time, the code may depend on libraries which are installed on every desktop, but not necessarily on an embedded system. These dependencies are at times a result of a completely offbeat feature, but it’s often simpler to compile the necessary library than to remove the feature. Since there is no single recipe for solving that kind of problems, we’ll stick to the basics of compiling for user space.

Background

Cross compilation of user space applications is actually more difficult than compiling the kernel, mainly because the kernel itself is, after all, a standalone application. There are a few things to take care of:

• Make sure libraries for dynamic linking are in place in the target runtime filesystem, as well as the dynamic linker itself.
• The compilation should be done against the header files corresponding to the libraries present in the target.
• The linked libraries used during compilation should correspond to those in the target.
• The C Runtime stubs (crt1.o, crti.o, crtbegin.o, crtend.o and crtn.o) should fit the Linux user space environment (different files with similar names are used for Microblaze standalones).

This may sound complicated, but most of the job has already been done. So it all boils down to a few simple things to bare in mind, as shown next.

Preparing the target’s file system

In part II it was shown how to download and extract the cross compiler for Microblaze. The same tarball also has the entire package for the target’s root under microblazeel-unknown-linux-gnu/microblazeel-unknown-linux-gnu/sys-root. This directory should be copied into the target’s root as is.

But the target’s root directory is already populated with files as necessary to boot Linux, and run command line utilities with busybox. Some of the files in sys-root, dynamic libraries in particular, already exist in target root, and they’re not identical. But since busybox is statically linked, overwriting the dynamic libraries seems harmless. Overwriting the previous files where applicable is therefore the way to resolve these conflicts, since dynamically linked applications will be compiled against the newer libraries.

All in all, the sys-root directory is ~164 MB, which isn’t too bad when stored on a flash memory. ~129 of these are /lib. The library files include libm, libpthread, libresolv and several other important libraries.

/usr/include takes up ~14 MB, which is probably not necessary on the target system.

Compilation

For cross compilation we use the same compiler used for the kernel. To compile an application:

/path/to/microblazeel-unknown-linux-gnu-gcc --sysroot=/path/to/nfsroot/ -o hello hello.c -lm

where /path/to/nfsroot is the directory which will be the root directory when the executable runs, or a copy of it. The truth is I’m not really sure –sysroot is really necessary, but given the pretty wild search the GNU tools do to find include files and libraries, it looks like a good measure to point directly at where these should be found.

Note that if we omit –sysroot, compiling for Microblaze Linux user space is done simply by using the cross compiler normally, and that works too. This happens because the compiler was configured to look for libraries and includes from its own sys-root. The C runtime stubs are always taken from the compiler’s own.

This is a good time to repeat something said in part II: There is no need to “install” the Microblaze compiler, and neither do the files need to be owned by root. A simple untar anywhere is fine. The compiler uses relative paths to find its resources.

Static linking

To be somewhat safer, static compilation may be preferred, in particular when the whole system’s functionality consists of a single application. The executable file is considerably larger, but doesn’t depend on libraries, so it works even without the sys-root copy mentioned above. Just add the –static flag to gcc. e.g.

/path/to/microblazeel-unknown-linux-gnu-gcc --static --sysroot=/path/to/nfsroot/ -o hello hello.c -lm

The -lm flag is here to demonstrate that libraries should be at the end of the command line. This has no significance when the executable is compiled to be dynamic, but for static linking, failing to put the -lm (and other loadables) at the end will cause misleading errors such as:

/tmp/cckqzZqo.o: In function `main':
: undefined reference to `sin'
collect2: ld returned 1 exit statu

So put the -lm at the end, OK?

Compiling from SDK

Note: I don’t present a working solution in this subject.

Since the BSP, which is generated by the SDK along with the compilation directories, are just include files and libraries, it was appealing to try compiling Linux user space applications from the SDK. The method suggested is to overwrite the BSP created by SDK with include files and libraries for the Linux environment.

The problem I didn’t bother to solve in the end, was that the C runtime libraries used by the linker remained those for a standalone application, so the executable couldn’t run on Linux. Most likely, this can be solved easily, but since I don’t like IDEs myself, I left this issue as is.

Another problem with this method is that the BSP is erased every time the project is rebuilt (but it survives recompilation, of course). So it’s best to keep a copy of the entire BSP directory structure.

In short, this is the procedure, minus the part that makes the linker work with the Linux-related CRTs.

• Create a BSP by creating a C project
• Copy all .a files from sys-root’s  /usr/lib and replace the lib directory in the BSP with a directory containing these (only)
• Replace the BSP’s include directory with sys-root’s /usr/include as is
• In SDK (Eclipse, actually), right-click the C project on the Project Explorer and pick Properties. Under C/C++ Build > Settings > Microblaze gcc linker > Linker script,  remove the linker script given (edit the text, and remove the part saying ../src/lscript.ld)

And again, this almost works. If someone takes this to the finish line, please let me know.

Running a VoIP phone adapter through a Linux machine

The setting

I got an AudioCodes MP202B as a phone line adapter from my Israeli ISP, Netvision. The normal way to connect it is putting it between the computer and the ADSL modem, so it does the “dialing” (sending username and password). This was a no-no for me, because I have a little home network with my own NAT and fake DNSes, so the last thing I wanted was to reconfigure my own network.

My twisted, and not really optimal solution was to let the phone adapter think my own computer is the ISP, so it connects to my computer with pppoe, gets a bogus IP address and DNS details, and then connect to the VoIP network through my computer.

That means, among others, that the phone adapter’s packets undergo NAT just like anything else going through that computer. What about incoming calls, I asked myself. They are initiated by the far end. How will iptables know the that its address needs to be mangled, so it goes to the phone adapter, and not the the host?

The answer, as usual with iptables, is don’t worry, be happy. As it turns out, the phone keeps sending initiation packets on UDP port 5060 periodically, so iptables can easily see the session. The voice packets also find their way. In short, it simply works. As usual.

Note that the bogus IP address the adapter sees is exposed to the VoIP operator in the headers of connection establishment packets. So if the ISP suddenly decides to check if the IP address appearing in these packets is in the valid range, the trick is revealed. Actually, it’s enough to check if the IP address in the headers matches the source address of the UDP packet itself (which has been altered by iptables to the real address given by the ISP). A software update or change in the (security?) configuration in the ISP’s infrastructure can lead to a sudden disconnection of the phone line. But this is not likely to happen: Assuming that the ISP checks the phone number against the ISP’s login name, this leaves no room for malicious tricks. Any extra restrictions are unnecessary, and as any network maintainer knows, add any extra filter, and some manager will shout at you soon.

This solution has an inherent flaw, though: Putting the phone between the computer and ADSL modem allows it to prioritize its own packets over data packets. Without this, voice quality can go down as a result of a massive upload to the web. But running a few tests, I didn’t hear any difference.

Getting the link up

Connect the phone adapter through its WAN jack to the desired Ethernet card (eth3 in my case).

Run a sniffer on the port, and verify that the adapter attempts to start a pppoe session. Something like this:

No. Time        Source                Destination           Protocol Info
 1 0.000000    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 2 0.999256    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 3 2.999042    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 4 6.998632    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)

Set up /etc/ppp/pppoe-server-options, which has the options to pppd, to read something like this:

lcp-echo-interval 10
lcp-echo-failure 2
noauth
ms-dns 10.2.0.1
ms-dns 10.2.0.2

with emphasis on the “noauth” option, since it’s pretty obvious that whoever is connected to the Ethernet jack doesn’t need authentication. Otherwise, the login name and password configured in the phone adapter must be added to pap-secrets or chap-secrets (whichever applies).

The ms-dns option contains DNS addresses for the adapter. These are fake addresses, which are NATed in the hosting machine, so they are real DNSes to the adapter.

Start off the pppoe server (as root, of course) with

# pppoe-server -I eth3 -L 10.192.0.0 -R 10.192.0.1 -N 1

where -N 1 limits the server to only one connections. The -L and -R set the local and remote addresses.

And by the way, to kill the pppoe server along with its connections go:

# killall pppoe-server

The packet capture should now look like:

No. Time        Source                Destination           Protocol Info
 1 0.000000    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 2 0.000369    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Offer (PADO)
 3 0.000650    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPPoED   Active Discovery Request (PADR)
 4 0.001711    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Session-confirmation (PADS)
 5 1.020712    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Request
 6 1.021534    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 7 1.021687    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Reject
 8 1.023117    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Ack
 9 1.024261    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 10 1.024434    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Ack
 11 1.024513    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 12 1.024565    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Request
 13 1.025981    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Request
 14 1.026087    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Reply
 15 1.026740    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 16 1.026854    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Nak
 17 1.028069    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 18 1.028197    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Request
 19 1.028267    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Reject
 20 1.029948    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 21 1.031115    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Ack
 22 1.036625    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 23 1.037321    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Ack
 24 1.038305    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Ack
 25 1.040164    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 26 1.043146    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Ack
 27 1.572454    10.192.0.1            10.2.0.1              DNS      Standard query A ntp.netvision.net.il
 28 3.070230    10.192.0.1            10.2.0.1              DNS      Standard query A centrex.res.netvision.net.il
 29 5.584155    10.192.0.1            10.2.0.1              DNS      Standard query A ntp.netvision.net.il
 30 6.570551    10.192.0.1            10.2.0.2              DNS      Standard query A ntp.netvision.net.il
 31 7.019219    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Request
 32 7.019401    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Reply
 33 8.070333    10.192.0.1            10.2.0.2              DNS      Standard query A centrex.res.netvision.net.il
 34 8.541665    10.192.0.1            10.2.0.1              DNS      Standard query A centrex.res.netvision.net.il
 35 10.580118   10.192.0.1            10.2.0.2              DNS      Standard query A ntp.netvision.net.il
 36 11.030538   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 37 11.031304   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 38 13.018511   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Request
(...etc)

So what we have here is a successful pppoe establishment. It’s also clear that the adapter got the DNS addresses OK, since it uses them for queries. But alas, no answer is returned, because my firewall rejects packets from any ppp device which are not within a session.

On my computer, the firewall script is run every time a ppp device goes up, by virtue of /etc/ppp/ip-up.local calling the firewall setup script.

In the script, I added the following part:

if [ $PHONEIF ] ; then
 iptables -A INPUT -i $PHONEIF -j droplog
 iptables -A OUTPUT -o $PHONEIF -j droplog

 if [ $EXTIF ] ; then
 iptables -A FORWARD -i $PHONEIF -o $EXTIF -j ACCEPT
 iptables -A FORWARD -i $EXTIF -o $PHONEIF -j ACCEPT
 fi

 # Default rule: Drop forwarded packets from and to adapter
 iptables -A FORWARD -i $PHONEIF -j droplog
 iptables -A FORWARD -o $PHONEIF -j droplog
fi

Where $PHONEIF and $EXTIF are the interfaces (ppp1 and ppp0, usually), as defined previously in the script.

Now everything works properly, packet capture as follows:

No. Time        Source                Destination           Protocol Info
 1 0.000000    AUDIO_e5:43:0f        ff:ff:ff:ff:ff:ff     PPPoED   Active Discovery Initiation (PADI)
 2 0.000059    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Offer (PADO)
 3 0.000319    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPPoED   Active Discovery Request (PADR)
 4 0.000683    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPPoED   Active Discovery Session-confirmation (PADS)
 5 0.002173    0.0.0.0               255.255.255.255       DHCP     DHCP Discover - Transaction ID 0x1fa2870d
 6 1.024784    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Request
 7 1.026219    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 8 1.026369    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Reject
 9 1.027837    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Ack
 10 1.028997    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Configuration Request
 11 1.029168    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Configuration Ack
 12 1.029244    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 13 1.029298    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Request
 14 1.031055    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 15 1.031172    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Nak
 16 1.032347    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 17 1.032473    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Request
 18 1.032544    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Reject
 19 1.034022    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 20 1.035204    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Ack
 21 1.040019    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP IPCP PPP IPCP Configuration Request
 22 1.041032    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP IPCP PPP IPCP Configuration Ack
 23 1.041705    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Ack
 24 1.042895    AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP CCP  PPP CCP Configuration Request
 25 1.046591    EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP CCP  PPP CCP Configuration Ack
 26 3.073250    10.192.0.1            10.2.0.1              DNS      Standard query A centrex.res.netvision.net.il
 27 3.089492    10.2.0.1              10.192.0.1            DNS      Standard query response A 82.166.210.6
 28 3.106339    10.192.0.1            82.166.210.6          SIP      Request: REGISTER sip:centrex.res.netvision.net.il
 29 3.131676    82.166.210.6          10.192.0.1            SIP      Status: 200 Ok
 30 11.034587   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 31 11.035375   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 32 21.034734   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 33 21.035485   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 34 31.034857   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
(etc...)

With SIP registration repeated every ~80 seconds, and LCP echoes every 10 seconds. The SIP protocol is defined in its RFC.

Note that there is no authentication whatsoever. If there was, we would have seen the server sending a challenge, to which the phone adapter would respond with an answer. In the case above, the server accepts the connection with no questions asked.

Registration packets in detail

This is a good time to mention, that I’ve replaced my real incoming phone number to 073-6666666 and calling number to 04-8222222. After all, I don’t want my real phone numbers out there. I’m not customer support.

The Register/OK pair above look like this:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 633
 Checksum: 0x1623 (correct)
Session Initiation Protocol
 Request line: REGISTER sip:centrex.res.netvision.net.il SIP/2.0
 Message Header
 From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e90-100c00a-13c4-50029-0-7e817330-0
 To: <sip:200012972736666666@centrex.res.netvision.net.il>
 Call-ID: 100b62d8-100c00a-13c4-50029-0-2a161072-0
 CSeq: 1 REGISTER
 Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-0-2a0-3639eac2
 Max-Forwards: 70
 Supported: replaces,100rel
 Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
 Expires: 1800
 Contact: <sip:200012972736666666@10.192.0.1:5060>
 User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
 Content-Length: 0

and then answer is simply

Session Initiation Protocol
 Status line: SIP/2.0 200 Ok
 Message Header
 From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e90-100c00a-13c4-50029-0-7e817330-0
 To: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=SDq3rh799-
 Call-ID: 100b62d8-100c00a-13c4-50029-0-2a161072-0
 CSeq: 1 REGISTER
 Via: SIP/2.0/UDP 10.192.0.1:5060;received=46.116.190.192;branch=z9hG4bK-0-2a0-3639eac2;rport=5060
 Contact: <sip:200012972736666666@10.192.0.1:5060>;expires=120
 Content-Length: 0

Incoming phone call

An incoming phone call, which isn’t answered looks like this:

 64 83.972831   82.166.210.6          10.192.0.1            SIP/SDP  Request: INVITE sip:200012972736666666@10.192.0.1:5060, with session description
 65 84.194196   82.166.210.6          10.192.0.1            SIP/SDP  Request: NOTIFY sip:200012972736666666@10.192.0.1:5060, with session description
 66 84.339229   10.192.0.1            82.166.210.6          SIP      Status: 180 Ringing
 67 84.581309   10.192.0.1            82.166.210.6          SIP      Status: 200 OK
 68 91.035533   EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 69 91.036415   AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 70 101.035711  EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 71 101.036461  AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 72 104.484595  82.166.210.6          10.192.0.1            SIP      Request: CANCEL sip:200012972736666666@10.192.0.1:5060
 73 104.485130  82.166.210.6          10.192.0.1            SIP/SDP  Request: NOTIFY sip:200012972736666666@10.192.0.1:5060, with session description
 74 104.508747  10.192.0.1            82.166.210.6          SIP      Status: 200 OK
 75 104.603031  10.192.0.1            82.166.210.6          SIP      Status: 487 Request Terminated
 76 104.624033  82.166.210.6          10.192.0.1            SIP      Request: ACK sip:200012972736666666@10.192.0.1:5060
 77 104.629209  10.192.0.1            82.166.210.6          SIP      Status: 200 O

The INVITE packet’s details are as follows:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 880
 Checksum: 0xe69c (correct)
Session Initiation Protocol
 Request line: INVITE sip:200012972736666666@10.192.0.1:5060 SIP/2.0
 Message Header
 Via: SIP/2.0/UDP 82.166.210.6:5060;branch=z9hG4bKckpn1l30a880su8va740.1
 Call-Id: SDau8fa01-e00a9a6ae9b1553d6117b50ab6a925a6-a0fo130
 From: <sip:048222222@centrex.res.netvision.net.il:5060>;tag=SDau8fa01-10.60.20.110-4294963135-9019
 To: <sip:200012972736666666@centrex.res.netvision.net.il:5060>
 Max-Forwards: 69
 Allow: REGISTER, INVITE, BYE, ACK, CANCEL, REFER, INFO, OPTIONS, SUBSCRIBE, UPDATE
 Session-Expires: 900
 CSeq: 3040744 INVITE
 Contact: <sip:048222222@82.166.210.6:5060;transport=udp>
 Supported: timer
 Content-Type: application/sdp
 Content-Length: 24

Note that the caller’s ID is there!

Outgoing phone call

Picking up the VoIP phone and calling 04-8222222, without the other side answering, yields:

 118 221.029993  10.192.0.1            82.166.210.6          SIP/SDP  Request: INVITE sip:048222222@centrex.res.netvision.net.il, with session description
 119 221.037083  EDIMAX_89:ae:12       AUDIO_e5:43:0f        PPP LCP  PPP LCP Echo Request
 120 221.037885  AUDIO_e5:43:0f        EDIMAX_89:ae:12       PPP LCP  PPP LCP Echo Reply
 121 221.056671  82.166.210.6          10.192.0.1            SIP      Status: 100 Trying
 122 221.975013  82.166.210.6          10.192.0.1            SIP/SDP  Status: 183 Session Progress, with session description
 123 222.013388  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 124 222.032360  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 125 222.046283  82.166.210.134        10.192.0.1            UDP      Source port: 16480  Destination port: 5004
 126 222.052372  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 127 222.066521  82.166.210.134        10.192.0.1            UDP      Source port: 16480  Destination port: 5004
 128 222.072316  10.192.0.1            82.166.210.134        UDP      Source port: 5004  Destination port: 16480
 129 222.086438  82.166.210.134        10.192.0.1            UDP      Source port: 16480  Destination port: 5004
(and packets keep flowing...)

As people in the industry know, the voice circuit starts without waiting for the other side to answer in an outgoing call.

The invitation packet in detail is pretty much like the previous one, just the other way around:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 988
 Checksum: 0x196d (correct)
Session Initiation Protocol
 Request line: INVITE sip:048222222@centrex.res.netvision.net.il SIP/2.0
 Message Header
 From: "0736666666"<sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a7b30-100c00a-13c4-50029-da-174234ad-da
 To: <sip:048222222@centrex.res.netvision.net.il>
 Call-ID: 100b2350-100c00a-13c4-50029-da-4cc1ed17-da
 CSeq: 1 INVITE
 Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-da-3561c-7fd18a32
 Max-Forwards: 70
 Supported: replaces,100rel
 User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
 Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
 Contact: <sip:200012972736666666@10.192.0.1:5060>
 Content-Type: application/sdp
 Content-Length: 32

Followed by the Trying packet:

User Datagram Protocol, Src Port: 5060 (5060), Dst Port: 5060 (5060)
 Source port: 5060 (5060)
 Destination port: 5060 (5060)
 Length: 371
 Checksum: 0x5d52 (correct)
Session Initiation Protocol
 Status line: SIP/2.0 100 Trying
 Message Header
 From: "0736666666"<sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a7b30-100c00a-13c4-50029-da-174234ad-da
 To: <sip:048222222@centrex.res.netvision.net.il>
 Call-ID: 100b2350-100c00a-13c4-50029-da-4cc1ed17-da
 CSeq: 1 INVITE
 Via: SIP/2.0/UDP 10.192.0.1:5060;received=46.116.190.192;branch=z9hG4bK-da-3561c-7fd18a32;rport=506

and then

Session Initiation Protocol
 Status line: SIP/2.0 183 Session Progress
 Message Header
 From: "0736666666"<sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a7b30-100c00a-13c4-50029-da-174234ad-da
 To: <sip:048222222@centrex.res.netvision.net.il>;tag=SD2kim999-10.60.20.110-4294957762-7934
 Call-ID: 100b2350-100c00a-13c4-50029-da-4cc1ed17-da
 CSeq: 1 INVITE
 Via: SIP/2.0/UDP 10.192.0.1:5060;received=46.116.190.192;branch=z9hG4bK-da-3561c-7fd18a32;rport=5060
 Contact: <sip:048222222@82.166.210.6:5060;transport=udp>
 Content-Type: application/sdp
 Content-Length: 199
Session Description Protocol
 Session Description Protocol Version (v): 0
 Owner/Creator, Session Id (o): - 1227677235 2 IN IP4 82.166.210.134
 Owner Username: -
 Session ID: 1227677235
 Session Version: 2
 Owner Network Type: IN
 Owner Address Type: IP4
 Owner Address: 82.166.210.134
 Session Name (s): -
 Connection Information (c): IN IP4 82.166.210.134
 Connection Network Type: IN
 Connection Address Type: IP4
 Connection Address: 82.166.210.134
 Time Description, active time (t): 0 0
 Session Start Time: 0
 Session Start Time: 0
 Media Description, name and address (m): audio 16480 RTP/AVP 18 101
 Media Type: audio
 Media Port: 16480
 Media Proto: RTP/AVP
 Media Format: 18
 Media Format: 101
 Media Attribute (a): rtpmap:18 G729/8000
 Media Attribute Fieldname: rtpmap
 Media Attribute Value: 18 G729/8000
 Media Attribute (a): sendrecv
 Media Attribute (a): rtpmap:101 telephone-event/8000
 Media Attribute Fieldname: rtpmap
 Media Attribute Value: 101 telephone-event/8000
 Media Attribute (a): fmtp:101 0-15
 Media Attribute Fieldname: fmtp
 Media Attribute Value: 101 0-1

I have snipped off the session description protocol parts from the previous packets. The “Media Port” entry is obviously how the sides expose their UDP ports.

July 2014 update

A month ago, or so, the phone suddenly stopped connecting, and the attempt to register was refused flat. I called their support, they told me they will be working on it, and then the phone came back to life. This is the updated UDP packet dump of the registration. Note that it fails first, and then the client tries again, with an improved request.

REGISTER sip:centrex.res.netvision.net.il SIP/2.0
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 1 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-b-2cec-1e8eaaf
Max-Forwards: 70
Supported: replaces,100rel
Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
Expires: 1800
Contact: <sip:200012972736666666@10.192.0.1:5060>
User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
Content-Length: 0

SIP/2.0 401 Unauthorized
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 1 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;received=93.173.36.5;branch=z9hG4bK-b-2cec-1e8eaaf;rport=5060
Content-Length: 0
WWW-Authenticate: Digest realm="NcxSip", nonce="62445575"

REGISTER sip:centrex.res.netvision.net.il SIP/2.0
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 2 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;rport;branch=z9hG4bK-d-3689-2ec8fad1
Max-Forwards: 70
Supported: replaces,100rel
Allow: REGISTER, INVITE, ACK, BYE, REFER, NOTIFY, CANCEL, INFO, OPTIONS, PRACK, SUBSCRIBE
Expires: 1800
Authorization: Digest username="200012972736666666",realm="NcxSip",nonce="62345345",uri="sip:centrex.res.netvision.net.il",response="26e7f74553dae1131cef72c3c90c5b67",algorithm=MD5
Contact: <sip:200012972736666666@10.192.0.1:5060>
User-Agent: MP202 B 2FXS/3.0.1_p041_build_19
Content-Length: 0

SIP/2.0 200 Ok
From: <sip:200012972736666666@centrex.res.netvision.net.il>;tag=100a4e40-100c00a-13c4-50029-b-5b9cd2a-b
To: <sip:200012972736666666@centrex.res.netvision.net.il>
Call-ID: 100b6288-100c00a-13c4-50029-a-4e521aee-a
CSeq: 2 REGISTER
Via: SIP/2.0/UDP 10.192.0.1:5060;received=93.173.36.5;branch=z9hG4bK-d-3689-2ec8fad1;rport=5060
Contact: <sip:200012972736666666@10.192.0.1:5060>;expires=120
Content-Length: 0

As before, numbers and strings that may be specific have been altered in the dump.

Linux on Microblaze HOWTO (part III)

This is part III of my HOWTO on running Linux on Microblaze. The outline is as follows:

• Part I: Introduction and setting up the Microblaze processor
• Part II: Compiling the kernel
• Part III: Preparing for boot and booting (this page)
• Part IV: Compiling user space applications

Generating the ACE file

The ACE file is what the System ACE chip reads from, and programs the FPGA accordingly. It consists of a sequence of JTAG operations for each necessary task: Configure the FPGA itself, load the software into memory, set the software execution entry point, and kick the software off. All is done with JTAG commands, which the System ACE generates as it scans through its ACE file.

So let’s get down to business.

Create a directory to gather the relevant files, and copy the following into it:

• The Tcl script for generating ACE file: Found at ISE_DS/EDK/data/xmd/genace.tcl (relative to the path where Xilinx ISE is installed)
• The bitstream (system.bit) file created by the EDK (explained in part I). Found in the ‘hw’ subdirectory in the export bundle from EDK to SDK. Or just under ‘implementation’ in the processor’s working directory. It’s the same file.
• The kernel ELF file (simpleImage.xilinx, or the unstripped simpleImage.xilinx.unstrip) created by the kernel build system (explained in part II), found in arch/microblaze/boot/ in the kernel source tree.

Open a command shell (Project > Launch Xilinx Shell if you like), change to this directory and go:

xmd -tcl genace.tcl -hw system.bit -elf simpleImage.xilinx -ace linuxmb.ace -board sp605 -target mdm

which generates a lot of junk files (.svf most notably, which contain JTAG commands in a portable format), and eventually the linuxmb.ace is created (any file name is OK).

In the example above, I assumed that the target is the SP605 board. Looking at the genace.tcl script reveals easily which boards are supported. If it isn’t, it’s not such a big deal. The only reason the board matters is because the System ACE needs to know which device in the JTAG chain to talk with plus some programming parameters. The -board flags to this scrips allows setting the options in a “genace option file” (whatever that means). I would hack the script, though. It looks easier. See here for more information.

Writing to the Compact Flash

First and foremost: If you have a compact flash which boots anything to the FPGA, don’t format it unless you really have to. The System ACE chip (by Xilinx) which reads from the flash directly is a bit picky about the file system format. Preferably use the card which came with the development kit.

And this too: If you just bought a 2 GB flash or so in a general electronics store, odds are that you’ll need to format it.

I explain how to format the flash in another post of mine.

Assuming that the flash is formatted OK, copy the ACE file to the Compact Flash’ root directory. Make sure that

• there is no other *.ace file in the root directory
• there is no xilinx.sys in the root directory

It is perfectly OK to have unrelated directories on the flash, so if there are some files on the flash already, I’d suggest creating a directory with just any name (say, “prevroot”) and move everything in the root directory into that one. And then copy the desired ACE file (linuxmb.ace in the example above) into the root directory.

That’s it. The Linux kernel should now boot, but it will complain (the kernel will panic, actually) that it doesn’t have any root filesystem. So…

Setting up the root filesystem

Once the kernel is up, it needs something to mount as a root filesystem, in which it expects to find its init executable and quite a few other files. Xilinx supplies an image of this bundle which were downloaded along with the cross compilers (see part II), in the same directory.

You may recall that I chose to mount root over the network, using NFS. So to create a useful root directory to work with, just change directory to whatever is going to be root (in my case, the one exposed via NFS) and go

zcat /path/to/microblaze_v1.0_le/initramfs_minimal_le.cpio.gz | cpio -i -d -H newc --no-absolute-filenames

This bundle includes a practical set of executables (well, it’s actually a lot of symbolic links to busybox) including vi, watch, dd, grep, gzip, tar, rpm, nc and even httpd (a web server…!). There’s also a rootfs.cpio.gz in the kernel sources when downloaded from Xilinx’ git (linux-2.6-xlnx.git in part II) which I haven’t tried out. But it’s opened in the same way.

You may, of course, compile your own programs, which is discussed in part IV.

There’s no “shutdown” executable, though. There’s “halt” instead.

A test run

Well, plug in the Compact Flash card, turn the power on, and hope to see a green LED blinking, which turns to steady green after a few seconds. When the LED is steady, expect some output on the UART. A typical log for SP605 is given at the end of this post.

At times, the SP605 board’s green LED went on, but nothing runs until SYS_ACE_RESET is pressed (the middle button out of three close to the Compact Flash jack). Looks like a powerup issue.

Is it fast? Is it fast?

This is maybe not such a fair comparison, and still the facts speak for themselves:

On Microblaze @ 75 MHz clock (37 BogoMIPS):

# dd if=/dev/zero of=/dev/null bs=1M count=10k
10240+0 records in
10240+0 records out
10737418240 bytes (10.0GB) copied, 1058.304486 seconds, 9.7MB/s
# dd if=/dev/zero of=/dev/null bs=512 count=100k
102400+0 records in
102400+0 records out
52428800 bytes (50.0MB) copied, 9.531130 seconds, 5.2MB/s

The same thing on my own computer  @ 1.2 GHz (5600 BogoMIPS):

$ dd if=/dev/zero of=/dev/null bs=1M count=10k
10240+0 records in
10240+0 records out
10737418240 bytes (11 GB) copied, 0.941238 s, 11.4 GB/s
$ dd if=/dev/zero of=/dev/null bs=512 count=100k
102400+0 records in
102400+0 records out
52428800 bytes (52 MB) copied, 0.0443318 s, 1.2 GB/s

According to the BogoMIPSes, Microblaze should have been 150 times slower, not 1000 times slower!

A typical boot log

early_printk_console is enabled at 0x40600000
Ramdisk addr 0x00000003, Compiled-in FDT at 0xc03c2348
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.38.6 (eli@localhost.localdomain) (gcc version 4.1.2) #19 Fri Aug 5 16:40:02 IDT 2011
setup_cpuinfo: initialising
setup_cpuinfo: Using full CPU PVR support
cache: wt_msr_noirq
setup_memory: max_mapnr: 0x8000
setup_memory: min_low_pfn: 0xc0000
setup_memory: max_low_pfn: 0xc8000
On node 0 totalpages: 32768
free_area_init_node: node 0, pgdat c04f515c, node_mem_map c05ca000
 Normal zone: 256 pages used for memmap
 Normal zone: 0 pages reserved
 Normal zone: 32512 pages, LIFO batch:7
pcpu-alloc: s0 r0 d32768 u32768 alloc=1*32768
pcpu-alloc: [0] 0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 32512
Kernel command line: console=ttyUL0 ip=::::::dhcp rootfstype=nfs root=/dev/nfs rw nfsroot=10.11.12.13:/shared/nfsroot,tcp
PID hash table entries: 512 (order: -1, 2048 bytes)
Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)
Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)
allocated 655360 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Memory: 123204k/131072k available
SLUB: Genslabs=13, HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
NR_IRQS:32
xlnx,xps-intc-1.00.a #0 at 0xc8000000, num_irq=8, edge=0x60
xlnx,xps-timer-1.00.a #0 at 0xc8004000, irq=7
Heartbeat GPIO at 0xc8008000
microblaze_timer_set_mode: shutdown
microblaze_timer_set_mode: periodic
Console: colour dummy device 80x25
Calibrating delay loop... 37.17 BogoMIPS (lpj=185856)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
ns_cgroup deprecated: consider using the 'clone_children' flag without the ns_cgroup.
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
devtmpfs: initialized
NET: Registered protocol family 16
PCI: Probing PCI hardware
bio: create slab <bio-0> at 0
XGpio: /axi@0/gpio@40040000: registered
XGpio: /axi@0/gpio@40020000: registered
XGpio: /axi@0/gpio@40000000: registered
vgaarb: loaded
Switching to clocksource microblaze_clocksource
microblaze_timer_set_mode: oneshot
Switched to NOHz mode on CPU #0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 4096 (order: 3, 32768 bytes)
TCP bind hash table entries: 4096 (order: 2, 16384 bytes)
TCP: Hash tables configured (established 4096 bind 4096)
TCP reno registered
UDP hash table entries: 256 (order: 0, 4096 bytes)
UDP-Lite hash table entries: 256 (order: 0, 4096 bytes)
NET: Registered protocol family 1
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 0 bytes, default 32
Skipping unavailable RESET gpio -2 (reset)
GPIO pin is already allocated
audit: initializing netlink socket (disabled)
type=2000 audit(0.429:1): initialized
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
fuse init (API version 7.16)
msgmni has been set to 240
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 254)
io scheduler noop registered
io scheduler deadline registered
io scheduler cfq registered (default)
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
40600000.serial: ttyUL0 at MMIO 0x40600000 (irq = 6) is a uartlite
console [ttyUL0] enabled
brd: module loaded
loop: module loaded
of:xsysace 41800000.sysace: Xilinx SystemACE revision 1.0.12
of:xsysace 41800000.sysace: capacity: 3980592 sectors
 xsa: xsa1
Xilinx SystemACE device driver, major=254
Generic platform RAM MTD, (c) 2004 Simtec Electronics
xilinx_spi 40a00000.spi: at 0x40A00000 mapped to 0xc8080000, irq=0
of:xilinx_emaclite 40e00000.ethernet: Device Tree Probing
Xilinx Emaclite MDIO: probed
of:xilinx_emaclite 40e00000.ethernet: MAC address is now 00:0a:35:49:b2:00
of:xilinx_emaclite 40e00000.ethernet: Xilinx EmacLite at 0x40E00000 mapped to 0xC80A0000, irq=5
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.19.1-ioctl (2011-01-07) initialised: dm-devel@redhat.com
nf_conntrack version 0.5.0 (1925 buckets, 7700 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Registering the dns_resolver key type
registered taskstats version 1
Sending DHCP requests .
PHY: c0020918:07 - Link is Up - 100/Full
., OK
IP-Config: Got DHCP answer from 10.11.12.13, my address is 10.11.12.155
IP-Config: Complete:
 device=eth0, addr=10.11.12.155, mask=255.255.255.0, gw=10.11.12.13,
 host=10.11.12.155, domain=, nis-domain=(none),
 bootserver=10.11.12.13, rootserver=10.11.12.13VFS: Mounted root (nfs filesystem) on device 0:13.
devtmpfs: mounted
Freeing unused kernel memory: 147k freed
Starting rcS...
++ Mounting filesystem
++ Starting telnet daemon
rcS Complete
/bin/sh: can't access tty; job control turned off
/ # NET: Registered protocol family 10
eth0: no IPv6 routers present

Linux on Microblaze HOWTO (part II)

This is part II of my HOWTO on running Linux on Microblaze. The outline is as follows:

• Part I: Introduction and setting up the Microblaze processor
• Part II: Compiling the kernel (this page)
• Part III: Preparing for boot and booting
• Part IV: Compiling user space applications

Kernel compilation in general

Compiling a Linux kernel traditionally consists of the following steps (some of which are elaborated further below):

• Obtaining a kernel source tree.
• Configure the kernel. Which all in all means to set up a file named “.config” in the kernel source’s root directory.
• Compile actual kernel, ending up with an executable image.
• Compile the post-boot loadable kernel modules.
• Put everything in its place, set up the bootloader
• Pray and boot

When compiling for Microblaze, the process is somewhat different:

• Cross compilation: The compiled binaries run on a processor different from the one doing the compilation.
• Kernel modules are most likely not used at all. They are a bit pointless when the hardware is known in advance, and also add some complexity in setting up the entire system for boot. Besides, modprobe on a Microblaze can take forever.
• The hardware configuration is custom made, and the kernel needs to be informed about it (through the Device Tree Structure)

Downloading kernel sources

Note that all kernels compile for all target architectures. If you download a kernel from Xilinx’ repository, it may have the parts relevant to Xilinx slightly more updated. The emphasis is on “may”.
The “vanilla” kernel (maintained by Linus Torvalds) can be downloaded from the main kernel archive or one of its mirrors. Several other flavors float around, including Xilinx own git

git clone git://git.xilinx.com/linux-2.6-xlnx.git

or Petalogix’ git (after all, they do a lot of maintenance on the Xilinx devices):

git clone git://developer.petalogix.com/linux-2.6-microblaze.git

The question is always which kernel is best. The answer is that it’s a bit of a gamble. It’s usually almost exactly the same piece of software, with git version having the latest changes. That means the latest bug fixes, new drivers, but also the latest, undocumented and undiscovered bugs. Vanilla kernels tend to be more conservative, but the only rule is that there are no rules. So in short, toss a coin and pick one.

Personally, I compiled the kernel which happened to be on my hard disk for other purposes.

Cross compilers

The good news is that there’s no need to compile the GNU tools. As a matter of fact, this part turned out to be surprisingly painless. The cross compiler and binutils binaries + initramfs images can be downloaded with

$ git clone git://git.xilinx.com/xldk/microblaze_v1.0_le.git
$ git clone git://git.xilinx.com/xldk/microblaze_v1.0.git

Choose one, depending on whether you prefer little endian or big endian for your processor. I picked little endian, but there’s one initramfs in the big endian bundle which isn’t there for the little endian set (which only has the “minimal” image).

One of the files fetched by git is microblazeel-unknown-linux-gnu.tar.gz (gzipped tarball) for the little endian version and mb_gnu_tools_bin.tar.bz (bzipped tarball) for big endian. I’ll leave the latter, because I didn’t use it.

There’s no need to install anything, and no need to be root (actually, doing this as root is pretty unwise). Just untar the tarball of your choice in any directory. Tar generates several subdirectories, but we’re after the cross compilers. Or more precisely, to make the kernel build system use them. This boils down to this:

export CROSS_COMPILE=/home/myhomedir/untarred-to/microblazeel-unknown-linux-gnu/bin/microblazeel-unknown-linux-gnu-

First of all, note the dash at the end of the statement. The whole string is a prefix for all compilation commands made by the kernel build system. It is often recommended to set the path to where the compilers are, and then set CROSS_COMPILE to a shorter prefix. I don’t see the point in polluting the overall path. The build environment has no problem with the statement above.

It has also crossed my mind to use the mb-gcc and friends, which are part of the SDK. But that may require another paid-for license, in case different people do the FPGA and software (which usually is the case).

And to wrap this up: If I’ll ever need to build a cross compiler from scratch, I would start with looking at Buildroot (and another page about it) or following this guide (I haven’t tried either, though).

Kernel configuration

Setting this up correctly is a tedious process, and even the most seasoned kernel hackers may not get it right on the first go. If it’s your first time, prepare to spend quite a few hours on this. The less experienced you are with Linux in general, the more time will you need to spend to make an educated guess about your need for each feature offered.

You can try to use my configuration file, or at least start off with it. It was made for against a 2.6.38 kernel, and booted well as shown in part III. Copy the file as .config on the kernel source’s root, and start with oldconfig.

The commands involved are basically (all “make” commands issues at the kernel source’s top directory):

• Clean up everything, including the .config file if present. This is not necessary if you just uncompressed your kernel. It’s actually rarely necessary at all: “make ARCH=microblaze mrproper”. This will delete .config! (I know I just said it).
• Adopt an existing .config file: “make ARCH=microblaze oldconfig”. This is useful in particular when switching to another kernel version or flavor. Only questions about new features are asked. If you downloaded my configuration file, I would suggest not to turn on options that are offered while running oldconfig, unless they are clearly Xilinx related.
• Configure the kernel: “make ARCH=microblaze xconfig”, “make ARCH=microblaze gconfig” or “make ARCH=microblaze menuconfig” (pick one). These applications present the kernel options in a fairly user-friendly manner, and eventually save the result to .config. I recommend xconfig, because it’s graphic and has a search feature, which turns out very useful.

When targeting an embedded platform, the strategy is to enable whatever is necessary in the kernel itself, and not count on kernel modules. A second issue is to eliminate anything unnecessary from the kernel. This is not just a matter of the kernel image’s size and speed, but enabling components which have nothing to do there can cause the kernel compilation to fail, and even worse, the kernel to crash at boot. Each architecture maintains a set of #include headers, and some kernel components may assume certain things that these architecture-dependent parts haven’t caught up with. So the rule that is whatever hasn’t been tested, won’t necessarily work. Enabling an esoteric keyboard driver on a Microblaze processor may very well fail the boot, simply because nobody cares.

In particular, you most likely want to follow these:

• Under Platform Options, set CONFIG_KERNEL_BASE_ADDR to where your DDR RAM begins (0xC0000000 on my processor), the targeted FPGA family as well as the other parameters (USE_*). The USE_* parameters’ correct values can be found in the .dts file. Just copy the values of the processor elements with the same names.
• Also set
  CONFIG_SERIAL_UARTLITE=y
  CONFIG_SERIAL_UARTLITE_CONSOLE=y
  CONFIG_SERIAL_CORE=y
  CONFIG_SERIAL_CORE_CONSOLE=y
• Since we’re not going to use any boot loader, the kernel command line needs to be compiled into the kernel itself: Enable CMDLINE_BOOL (default bootloader kernel argument) and set it to something useful. As for the console, set it to console=ttyUL0, or nothing goes to console after the two first lines sent to console from early_printk_console (CONFIG_CMDLINE_FORCE may be necessary as well. It doesn’t hurt in the absence of a boot loader anyhow)
• Enable CONFIG_MSDOS_FS and CONFIG_VFAT_FS in kernel (not module), so that the SystemACE can be read.
• Enable CONFIG_XILINX_SYSACE
• Enable CONFIG_XILINX_EMACLITE and CONFIG_FB_XILINX
• Disable the FTRACE config option (under kernel hacking, compilation fails) instead of using patch.

And for your own sake, make a copy of the .config file every now and then as you work on it. It’s very easy to delete it by mistake or to mess it up in general.

Setting the Linux boot parameters correctly is very important, because if they’re wrong, kernel recompilation is they only way to fix it in the absence of a boot loader. I’ve chosen to mount the root directory from the network, but note that /dev/sxa is the Compact flash itself (with /dev/sxa1 is the first partition, for example). So it’s fairly simple to add a partition to the flash device, and put a regular root filesystem there. Maybe I’ll do that myself and update this post.

Anyhow, my choice for the Linux boot parameters was

console=ttyUL0 ip=::::::dhcp rootfstype=nfs root=/dev/nfs rw nfsroot=10.11.12.13:/shared/nfsroot,tcp

where “/shared/nfsroot” is the shared NFS directory on the server with IP 10.11.12.13. This command is suitable for getting the root from the network, which is very convenient for development. This setting requires a DHCP server on the LAN. In case you don’t want to configure a DHCP server, use the ip=<client-ip>:<server-ip>:<gw-ip>:<netmask>:::off format instead. Documentation/filesystems/nfs/nfsroot.txt in the kernel sources has more about booting from NFS. I’ve also written a post about booting a diskless PC from network, but it’s a bit of an overkill.

In case you’re interested in how the whole configuration thing comes together, let’s take CONFIG_EARLY_PRINTK for example. In arch/microblaze/kernel/Makefile, one of the lines says:

obj-$(CONFIG_EARLY_PRINTK)    += early_printk.o

On the other hand, in the config file it can say

CONFIG_EARLY_PRINTK=y

So when the Makefile is executed, the target early_prink.o is added to either obj-y, obj-m or obj-n. obj-y is the list of objects to be inserted into the kernel, obj-m is the list of modules, and obj- is the junk list. The configuration rules are given in the Kbuild files, next to the Makefiles.

A small Makefile fix

As of 2.6.38, there is a small error in the arch/microblaze/boot/Makefile, which makes the build system always attempt making an U-Boot image, which is not necessary in our case. This may result in an error message (saying “mkimage” wasn’t found), when everything is actually OK. So in the part saying

$(obj)/simpleImage.%: vmlinux FORCE
 $(call if_changed,cp,.unstrip)
 $(call if_changed,objcopy)
 $(call if_changed,uimage)
 $(call if_changed,strip)
 @echo 'Kernel: $@ is ready' ' (#'`cat .version`')'

remove or comment out the line saying “$(call if_changed,uimage)”.

Compiling the kernel

Before starting: You didn’t forget to set CROSS_COMPILE and copy the updated xilinx.dts file to its place… right?

I prefer cleaning up before compiling:

make ARCH=microblaze clean
rm arch/microblaze/boot/simpleImage.*

This is a good time to ask why the image file isn’t cleaned by “make clean”. To be fixed, I suppose.

And then, the compilation is just

make -j 8 ARCH=microblaze simpleImage.xilinx

Note that the “.xilinx” suffix corresponds to the xilinx.dts file in the arch/microblaze/boot/dts/ directory. If another .dts file should be made effective, change the suffix.

The “-j 8″ means that 8 compilation processes run in parallel, which is suitable for a quad processor with hyperthreading. Skip this option or use another number, depending on your computer, your spare time and your need to see the logic of the events.

The basic UNIX rule is that everything went fine unless an error message appeared. A more explicit confirmation is that it said

OBJCOPY arch/microblaze/boot/simpleImage.xilinx

somewhere close to the end, and that the arch/microblaze/boot/simpleImage.xilinx is indeed there, and has a date stamp that makes sense.

If and when you get errors, well, there’s no simple  recipe to solve that. The easiest way is to eliminate the need to compile that certain file by changing the kernel configuration, if the functionality is indeed unnecessary. Otherwise your best friends are Google and your brain, not necessarily in that order.

As for the Device Tree, it was compiled into a .dtb file (the Device Tree binary blob), which can be found in the same directory as the just generated kernel image. The Device Tree Compiler (dtc) comes with the kernel sources, and can be found in scripts/dtc.

And just to wrap this up: If you insist on seeing all the commands issued instead of the otherwise laconic output, there the KBUILD_VERBOSE flag. For example,

make ARCH=microblaze KBUILD_VERBOSE=1 clean

With a compiled kernel image at hand (which already has the Device Tree built-in), all that’s left is to set up the Compact Flash and boot. Go to part III of this HOWTO.

A few other make statements

For completeness:

• Clean up any compiled binaries: Recommended after a change in .config: “make ARCH=microblaze clean”
• Generate loadable modules: “make ARCH=microblaze modules”. Not necessary if everything needed is compiled into the kernel.
• And then gather the modules in a neat directory (making sure you don’t have a /lib/modules directory with the same version number): “make ARCH=microblaze modules_install”. This will write to /lib/modules on the local machine, so if you happen to compile exactly the same kernel version for your own PC and the embedded target, the kernel modules the PC relies on will be overwritten.

Linux on Microblaze HOWTO (part I)

This is part I of my HOWTO on running Linux on Microblaze. The outline is as follows:

• Part I: Introduction and setting up the Microblaze processor (this page)
• Part II: Compiling the kernel
• Part III: Preparing for boot and booting
• Part IV: Compiling user space applications

Introduction

This HOWTO goes through the procedures for getting a simple Linux system running on a Xilinx Microblaze processor. The examples are given for an SP605 evaluation board, but almost everything here applies for other FPGAs and boards as well. The Xilinx software version used here is 13.2.

There are quite a few variants on how to get the bitstream and Linux kernel into their right places in the FPGA. The approach taken here is to boot up from the Compact Flash alone by writing a file to it. No bootloader is used in this howto; the SystemACE chip is responsible for loading both the FPGA bitstream and Linux kernel image, and it will do so reading one single (.ace) file. The main advantage of this approach is that there’s no need to set up a boot loader, which is yet another piece of software that can go wrong. The main disadvantage is that a bootloader allows some tweaking of the kernel configuration at boot time, which has to be done by recompiling the kernel otherwise.

The root filesystem is mounted from network (NFS) in this HOWTO.

I’m assuming the following prerequisites:

• You have the Xilinx tools set up properly, and have managed to compile and run a simple standalone “Hello, World” application with the EDK/SDK (having loaded the code to the FPGA in any way, we’ll get to that)
• You’ve actually seen the RS-232 console data on a terminal, and feel confident about it (otherwise you may work hard to figure out why everything is stuck, when it’s actually your terminal window’s problem).
• You’re running on one of the evaluation boards, or know how to set up the processor to work with your own (and have that tested already)
• Your board has a systemACE flash chip (recent evaluation boards usually do)
• You have access to a machine running Linux on a computer. Compiling the kernel will require this. The Xilinx tools can be run on whatever’s your cup of tea.
• You have the ability to read and write files to a Compact Flash. This is most easily done with a simple adapter to a PC computer, which should be available in camera or computer accessories shops. Chances are you have one without necessarily being aware of it.

An outline of the steps

So this is what we’ll do:

• Set up a Microblaze processor in the Xilinx EDK so it can run Linux.
• Generate the processor, so an FPGA bitstream is at hand.
• Export the processor to the Xilinx SDK and compile a dummy C application, so that  necessary metadata files are generated
• Generate a Device Tree file (.dts) based upon files created by EDK/SDK, and copy it into the Linux kernel sources, so Linux is in sync with the EDK regarding what it’s running on.
• Configure the kernel and compile it.
• Create a .ace file from the FPGA bitstream and kernel image just compiled.
• Set up the Compact Flash card.
• Boot and hope for good

And of course, certain software tools will need to be downloaded for this. We’ll come to this.

Setting up the processor

If you’re really lazy about this, you can use the minimal processor I’ve generated for the SP605 board. Unzip, double-click system.xmp, and skip to after the bullets below. It will work on that board only, of course.

Otherwise: Start Platform Studio (EDK) and create a new platform, based upon the Wizard’s defaults.

Following a Microblaze on Linux guide, in particular the part regarding minimal hardware requirements, there a need to make sure that the hardware has an MMU with two regions, a timer, an interrupt controller and a UART with an interrupt line. In the platform studio it goes like this:

Starting off with the Wizard’s defaults,

• Double click “microblaze_0″ on the Ports tab, and set the Linux with MMU preset on the Configuration wizard showing up. This will take care of most settings.
• Still in the ports view, add an AXI Interrupt Controller (under Clock, Reset and Interrupt in the IP Catalog). Accept default settings. Make a new connection for its irq output, and connect it to the microblaze_0′s interrupt input pin.
• Pick the RS232_Uart_1 and make a new connection for the interrupt line. Connect that signal to the interrupt controller.
• Add an AXI Timer/Counter, and accept defaults. Make a new connection for the interrupt, and connect it to the interrupt controller.
• Connect the interrupts of the Ethernet lite, SPI Flash, IIC SFP, IIC EEPROM, IIC_DVI, and SysACE cores to the interrupt controller as well.

Then generate bitstream, export to SDK, and run the SDK, adopting this hardware platform. The goal of this is to generate a .mss file, which will be needed later. For this to happen, make a new C project (“Hello World” will do just fine) and compile it.

There is no need to “update the bitstream” like with standalone applications: The Linux kernel can take care of itself, without having its entry address hardwired in the FPGA’s block RAM. We’ll use the system.bit, and not the download.bit (even though the latter works too).

Creating a Device Tree file

The purpose of this stage is to generate a .dts file, which is the format expected by the kernel build environment. It informs the kernel about the structure of the processor and its peripherals. The device tree structure is discusses further here.

If you chose to download and use my processor with no changes whatsoever, you can also get my DTS file. Just copy it to arch/microblaze/boot/dts/ in the to-be compiled kernel source tree.

To make your own .dts file, first create a special directory, and make it the working directory of your shell.

The device tree file is generated automatically with the libgen utility with the help of a Tcl script. As of ISE 13.2, this script needs to be loaded separately with git:

bash> git clone git://git.xilinx.com/device-tree.git

This generates a device-tree directory. Another web page explains how to make SDK recognize the script, but I prefer command line for things like this. Another post of mine explains the device tree further.

Copy the system.xml file from the directory to which you exported to SDK (in the “hw” subdirectory), into the current one. Then copy system.mss from the project’s BSP directory. It will have a name like hello_world_bsp_0.

Edit the copy you made of system.mss, so that the BEGIN OS to END part reads

BEGIN OS
 PARAMETER OS_NAME = device-tree
 PARAMETER OS_VER = 0.00.x
 PARAMETER PROC_INSTANCE = microblaze_0
END

and not “standalone” for OS.

And then run libgen as follows (make sure it’s in the PATH. The easiest way is to launch a “Xilinx shell” from the EDK’s project menu):

libgen -hw system.xml -lp device-tree -pe microblaze_0 -log libgen.log system.mss

Which generates a xilinx.dts in microblaze_0/libsrc/device-tree_v0_00_x. Copy this file to arch/microblaze/boot/dts/ in the to-be compiled kernel source tree. If you can’t find the file there, and libgen didn’t complain about some error, you may have forgotten to edit system.mss as mentioned just above.

Now let’s go on to compiling the kernel, in part II.

The Device Tree for embedded Linux and Xilinx FPGAs

Spoiler

It’s very likely that you don’t need to read this. If all you want is to get a Linux kernel to detect a Microblaze processor on an Xilinx FPGA, the relevant information is in another post of mine. This post goes into the details which are necessary to understand, if you want to write a kernel driver for a device tree mapped peripheral.

Why a device tree is necessary

The main issue with running Linux on an FPGA is that the Linux kernel needs to know what peripherals it has and where it can find them. On PC computers this problem was solved many years ago with the PCI bus: The BIOS detects the peripherals, allocates their addresses and interrupts and tells the operating system what it has and where it can be found. In the embedded world, this information was hardcoded into pieces of the kernel sources, which were written specifically for every board. With many boards out there, the kernel source grew way too fast. This far-from-optimal solution is not feasible with a soft processor, whose peripherals are configured per case. Hacking the kernel sources to match the FPGA is a recipe for bugs, crashes and being stuck with a certain kernel forever.

The elegant solution for this is the Flattened Device Tree. The idea is to create some binary data structure, which is either linked into the kernel image or given to it during boot. This binary blob contains the information about the processor itself and its peripherals, including the addresses, interrupts and several application-specific parameters. So the drivers for these peripherals are written very similar to PCI drivers: They declare what peripherals they support, and obtain their resources from a standard kernel API.

The code for Flattened Device Tree and Open Firmware resides in drivers/of in the kernel tree. The relevant include file is include/linux/of.h.

Generation

Note that at least for Xilinx FPGAs, there is no need to generate the device tree manually. Rather, get a copy of the device tree generator with

bash> git clone git://git.xilinx.com/device-tree.git

which basically consists of a TCL script run by libgen and a configuration file. The device tree generator’s page explains how to make SDK recognize the script, but there’s no reason to play around with SDK for that.

Instead, go

libgen -hw /path/to/system.xml -lp /path/to/device-tree -pe microblaze_0 -log libgen.log system.mss

Which generates a system.dts in microblaze_0/libsrc/device-tree_v0_00_x

The system.mss file is generated as a byproduct when compiling just any a project within SDK, and is found under the directory with the _bsp_n suffix. I still need to find out how to create the file from the command line.

It needs to be modified, so that the BEGIN OS to END part reads

BEGIN OS
 PARAMETER OS_NAME = device-tree
 PARAMETER OS_VER = 0.00.x
 PARAMETER PROC_INSTANCE = microblaze_0
END

and not “standalone” for OS.

To get the system.xml file (which was necessary to create the system.mss), go Project > Export Hardware to SDK in the EDK platform studio. Or

make -f system.make exporttosdk

from the project’s home directory.

The correct setup of the device tree entry can be found in the Documentation/devicetree/bindings directory of the kernel sources. The xilinx.txt file describes the bindings for Xilinx peripherals, and explains how information in the system.mhs file is translated into a xilinx.dts.

As part of a full kernel compilation, the .dts is compiled into a .dtb file (the Device Tree binary blob), which can be found in the same directory as the generated kernel image. The Device Tree Compiler (dtc) comes with the kernel sources, and can be found in scripts/dtc.

A sample entry

The following example is given there for a Uartlite (which we’ll follow on below):

opb_uartlite_0: serial@ec100000 {
 device_type = "serial";
 compatible = "xlnx,opb-uartlite-1.00.b";
 reg = <ec100000 10000>;
 interrupt-parent = <&opb_intc_0>;
 interrupts = <1 0>; // got this from the opb_intc parameters
 current-speed = <d#115200>;     // standard serial device prop
 clock-frequency = <d#50000000>; // standard serial device prop
 xlnx,data-bits = <8>;
 xlnx,odd-parity = <0>;
 xlnx,use-parity = <0>;
 };

It’s recommended to have a look at arch/microblaze/platform/generic/system.dts in the kernel sources for a fullblown file. Or one you’ve generated yourself, for that matter.

Declarations in a kernel module driver

Device tree mapped instances are treated by the kernel very much like PCI devices, only the source of information is the DTB (Device Tree Binary) rather than from the BIOS.

The parallel to PCI’s Vendor/Product IDs is an entry looking like this (taken from uartlite.c):

static struct of_device_id ulite_of_match[] __devinitdata = {
 { .compatible = "xlnx,opb-uartlite-1.00.b", },
 { .compatible = "xlnx,xps-uartlite-1.00.a", },
 {}
};

MODULE_DEVICE_TABLE(of, ulite_of_match)

Which is then bound to a driver with

static struct of_platform_driver ulite_of_driver = {
 .probe = ulite_of_probe,
 .remove = __devexit_p(ulite_of_remove),
 .driver = {
 .name = "uartlite",
 .owner = THIS_MODULE,
 .of_match_table = ulite_of_match,
 },
}

and then, finally, exposed to the kernel with

static inline int __init ulite_of_register(void)
{
 pr_debug("uartlite: calling of_register_platform_driver()\n");
 return of_register_platform_driver(&ulite_of_driver);
}

somewhere at the end of the driver’s code. This format is very similar to the declaration of PCI devices, so if this is unclear, I’d suggest learning how to do it the PCI way, which is by far more documented.

And by the way, when the kernel is configured to support it, the device tree can be viewed in human-readable format in /proc/device-tree.

The of_device_id structure

The structure is defined in include/linux/mod_devicetable.h as

struct of_device_id
{
 char    name[32];
 char    type[32];
 char    compatible[128];
#ifdef __KERNEL__
 void    *data;
#else
 kernel_ulong_t data;
#endif
};

Surprisingly enough, the lengths of the entries are fixed and limited.The three strings, name, type and compatible are compared as strings (with strcmp(), see of/base.c) with the device tree’s node’s data. Everything declared (that is, non-NULL) in the structure must be equal with the node’s info for a match. In other words, NULLs are wildcards.

In the declaration example above, only the “compatible” part was declared, so any device matching the string exactly triggers off a probe on the driver.