After a “find” operation spanning the entire disk, I suddenly had System Monitor (2.28.0 of Fedora 12) telling me that 11.5 GB out of the existing 16 GB were used up, and the applet said 75% was “in use by programs”. Really. Virtually nothing was running on the system.
Just to be sure, I tried out
$ ps aux --sort -rss | less
(list processes sorted by resident memory) and as expected, no memory hog to be seen.
So what does the kernel (a home cooked 2.6.35.4) say for itself?
$ cat /proc/meminfo
MemTotal: 16463436 kB
MemFree: 130568 kB
Buffers: 2818532 kB
Cached: 1387344 kB
SwapCached: 0 kB
Active: 3368176 kB
Inactive: 2246912 kB
Active(anon): 1407872 kB
Inactive(anon): 339276 kB
Active(file): 1960304 kB
Inactive(file): 1907636 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 368 kB
Writeback: 0 kB
AnonPages: 1409264 kB
Mapped: 174032 kB
Shmem: 337932 kB
Slab: 10246976 kB
SReclaimable: 9526744 kB
SUnreclaim: 720232 kB
KernelStack: 5112 kB
PageTables: 47880 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 8231716 kB
Committed_AS: 4241756 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 122564 kB
VmallocChunk: 34359572612 kB
HardwareCorrupted: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 350080 kB
DirectMap2M: 16422912 kB
Hmmm… That’s the kernel eating up ~10 GB, most of which is reclaimable. This page seems to explain the issue: As the disk was scanned, directory and inode metadata was probably cached. So what if that takes nearly 10 GB?
The thing is that it gives a somewhat misleading picture of the computer’s state.
So let’s just tell the kernel to drop all those caches, as suggested in that page:
As root, just go
# echo 3 > /proc/sys/vm/drop_caches
and watch how the memory meter drops. This problem is supposed to be solved in kernel not as ancient as mine.
Fedora 12: It started with some error message about something crashing, and the Workspace Switcher applet was gone.
I tried re-adding it to the panel, but I got an error window saying that the panel encountered a problem while loading “OAFIID:GNOME_WorkspaceSwitcherApplet”. And offered me to delete it from the panel. Or not delete it. Nothing helped. Even not restarting compiz.
I took a look in the panel configuration directory, just to find that there were multiple instances of the same applet:
$ grep -r GNOME_WorkspaceSwitcherApplet ~/.gconf | sort
.gconf/apps/panel/applets/applet_0/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_10/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_11/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_12/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_13/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_15/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_16/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_5/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_6/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_7/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_8/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue>
.gconf/apps/panel/applets/applet_9/%gconf.xml: <stringvalue>OAFIID:GNOME_WorkspaceSwitcherApplet</stringvalue
In a normal setting, there are three lines here (why? Never mind).
The solution was surprisingly simple. Just find the process with the name gnome-panel. And kill it (with a plain kill, no special signal is necessary).
$ killall gnome-panel
The panel disappears reappears promptly. Maybe with duplicate applets, but hey, it finally works!
Trying to insmod a kernel module, which was just compiled against the true headers of the running kernel, I got:
# insmod mymodule.ko
insmod: error inserting 'mymodule.ko': -1 Invalid module format
And this in /var/log/syslog:
mymodule: version magic '3.3.0-xxx SMP preempt mod_unload ARMv7 ' should be '3.3.0-xxx+ SMP preempt mod_unload ARMv7 '
Really? Are you picking on me? Rejecting my module just because of a missing plus sign?!
The plus sign is there in the first place because the kernel was compiled with a git repository, which displayed modifications relative to the official tree. To avoid it, I could have compiled the kernel after renaming the .git directory to something else prior to compiling the kernel. But I forgot to do that.
In fact, I should have turned kernel magic versioning off altogether.
The plus sign is absent in the current kernel headers, because they were generated without the git repository (natively on the target). Hence the difference.
One could use modprobe –force, but isn’t it nicer to just get the version magic right?
This solution holds for the 3.3.0 kernel, but probably works on others as well.
The file you want to edit is /usr/src/kernels/{your version}/include/generated/utsrelease.h and just add the ‘+’ sign to the end of the version number. E.g.
#define UTS_RELEASE "3.3.0-xxx+"
And now all modules compiled against the kernel will load with no issues.
For those interested in the gory details: It’s mymodule.mod.c that is responsible for supplying the version magic. It’s compiled into mymodule.mod.o, and then fused together with mymodule.o to become mymodule.ko. This is what the famous “MODPOST” compilation stage does (in fact, it’s a make -f on scripts/Makefile.modpost, which in turn calls scripts/mod/modpost).
mymodule.mod.c gets the number by including linux/vermagic.h (actually, it includes other similar files as well) which in turn includes generated/utsrelease.h, which supplies the version magic explicitly.
Since daylight saving time is an issue for political dispute in Israel, the time of switching back and forth changes all too often. To keep my computer on track, I need to update the timezone info file every now and then.
There are a lot of tutorials on the web about this, for example these two. Here’s a quick summary:
Check the current situation. Is the timezone file in sync with the real world?
$ zdump -v /etc/localtime | less
Ah, so it isn’t. Go to IANA’s site and download the data file (e.g. tzdata2013d.tar.gz). From within that bundle, fetch the file named “asia”. And go:
$ zic -d tmpdir asia
which creates a new directory, tmpdir, with several zone files inside.
Unfortunately, trying to run zdump on just a file doesn’t work very well. So I went to /usr/share/zoneinfo and removed the file called Israel as well as Asia/Jerusalem and Asia/Tel_Aviv, and the /etc/localtime as well. They were all the same, identical file.
Then copied tmpdir/Asia/Jerusalem into /usr/share/zoneinfo/Asia and
# ln -s /usr/share/zoneinfo/Asia/Jerusalem localtime
Plus, in /usr/share/zoneinfo/Asia, just to be safe:
# ln -s Jerusalem Tel_Aviv
and in /usr/share/zoneinfo:
# ln -s Asia/Jerusalem Israel
And a final check (same as before)
$ zdump -v /etc/localtime | less
A machine reboot may be in place to make all programs catch up with the new data. Or just restart key programs, if it doesn’t matter all that much.
Just to be 100% sure that the change is in effect, a one-liner script can be used. For example how is GMT time vs. local time compare 48 days from now?
$ perl -e '$t=time() + 86400*48; print scalar gmtime($t)." --> ".scalar localtime($t)."\n";'
Side notes: The sources for zic and other utilities are available next to the data file on IANA’s site. But any sane Linux distribution should have these installed already. Or at least at the repository. Also worth to note, that the syntax of these data files is pretty cool: It’s not just dates, but political rules.
The goal: Create spam-harvesting safe mailto links anywhere in the pages, generated on the fly with slightly obfuscated JavaScript, so that harvesting bots don’t get the email addresses. To make it slightly more complicateded, I want a replacement pattern with a parameter, telling the replacement script which email address to inject.
In other words, if I wrote %mail{1}% somewhere in the page’s source, that is replaced by the second email address in a known list of addresses.
Of course there the Token module. But I wanted the replacement to depend on the token string. Globally.
I use arthemia, so this appears everywhere in the paths to files.
Bad attempt #1
I thought that dvessel’s suggestion on a pretty old page would do the job (and it looks like it was obvious to everyone which file to edit, sites/all/themes/arthemia/template.php). But that suggestion was good for Drupal 5.
The theme I used, arthemia, already has an arthemia_preprocess_page() function, but not an arthemia_render_template(). I followed the suggestion, and nothing happened, of course. Drupal 7 doesn’t look in that direction.
Bad attempt #2
There is this golden rule in Drupal, that you don’t edit the core functions. Well, well, it was time to break it again. This time with the core of the core: include/common.inc, modifying drupal_render. Yup, I couldn’t have broken that rule harder. But after a few hours of the regular Drupal ceremony of reading a lot of useless tips on the web, I thought it was best to just get the job done.
The main problem was that all hooks are related to a certain type of elements. There is no hook for “just before the HTML is handed over to the browser”. Or that’s the impression I got.
First, I added the script I wanted included in every page. It would prefer it to be in the <head> section (because that’s usually where global scripts are) and I would prefer it to be a piece of content. But again, hours of looking for the “Drupal” way to do it, I just added the following to the top of sites/all/themes/arthemia/page.tpl.php:
<script language="JavaScript">
<!--
function myfunction(i) {
[ ... ]
}
-->
</script>
It’s an ugly solution, but it gets the job done. I could, in theory, plant it as the content of an unused part of the page, and then mangle the page’s structure as it was being processed. Too much work for the cute shortcut of altering the script from Drupal’s interface.
And now to finding the markups, and make them run the function above. Using the preg_replace_callback() function, render_template was edited to this in the end of the function (just before “return $output”):
// Mangle
$output = preg_replace_callback("|%mail\{(\d+)\}%|",
function ($matches) {
return "<script language=\"JavaScript\">\n<!--\nmyfunction($matches[1]);\n-->\n</script>";
},
$output)
That worked well. Actually, too well. The replacement took place also in editing boxes, so the original markup text couldn’t be edited.
The correct way
OK, so I took one step back. Let’s use the regular hooking mechanism, after all, and define the hooks for the content and the footer. Good enough. And as a side effect, the core is left intact, as I only edit the theme’s page template file.
So instead of the changes above I added this to the top of sites/all/themes/arthemia/page.tpl.php:
<script language="JavaScript">
<!--
function myfunction(i) {
[ ... ]
}
-->
</script>
<?php
function mail_post_render($content, $element) {
$content = preg_replace_callback("|%mail\{(\d+)\}%|",
function ($matches) {
return "<script language=\"JavaScript\">\n<!--\nmyfunction($matches[1]);\n-->\n</script>";
},
$content);
return $content;
}
if ($page['content']) {
$page['content']['#post_render'] = array('mail_post_render');
}
if ($page['footer']) {
$page['footer']['#post_render'] = array('mail_post_render');
}
?>
And that finally worked in a sane manner: The footer and content were mangled, and I was still able to edit the original text. Hurray!
As usual, a simple task took a full day to complete. Welcome to the wonderful world of Drupal.
What happens if a Windows user loses his or her password? No problem, Windows was never meant to be secure. Only appear as if it was.
There are several automatic tools out there. I preferred running my Fedora-based LiveUSB and fix it while actually seeing what I’m doing. The whole thing is about modifying the SAM file used on authentication.
Note that the flow shown below failed. What did work, eventually, was unlocking the Administrator account, which doesn’t have any password to begin with. Why I failed to reset a user’s password is beyond me. Maybe because it had administrator’s privileges? Maybe because it was Windows 8?
Anyhow, first I installed chntpw. It’s an offline registry editor it says somewhere, but I never checked that.
# yum install chntpw
Following this guide, mount the partition and change directory to where the SAM file is (Windows/System32/config, like any NT machine). Note the capital letters in Windows and System32.
Then go (possibly as root)
$ chntpw SAM
chntpw version 0.99.6 080526 (sixtyfour), (c) Petter N Hagen
Hive <SAM> name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 666c <lf>
Page at 0x6000 is not 'hbin', assuming file contains garbage at end
File size 262144 [40000] bytes, containing 5 pages (+ 1 headerpage)
Used for data: 222/17400 blocks/bytes, unused: 14/2920 blocks/bytes.
* SAM policy limits:
Failed logins before lockout is: 0
Minimum password length : 0
Password history count : 0
| RID -|---------- Username ------------| Admin? |- Lock? --|
| 01f4 | Administrator | ADMIN | dis/lock |
| 01f5 | Guest | | dis/lock |
| 03e9 | myself | ADMIN | dis/lock |
---------------------> SYSKEY CHECK <-----------------------
SYSTEM SecureBoot : -1 -> Not Set (not installed, good!)
SAM Account\F : 0 -> off
SECURITY PolSecretEncryptionKey: -1 -> Not Set (OK if this is NT4)
Syskey not installed!
RID : 0500 [01f4]
Username: Administrator
fullname:
comment : Built-in account for administering the computer/domain
homedir :
User is member of 1 groups:
00000220 = Administrators (which has 2 members)
Account bits: 0x0211 =
[X] Disabled | [ ] Homedir req. | [ ] Passwd not req. |
[ ] Temp. duplicate | [X] Normal account | [ ] NMS account |
[ ] Domain trust ac | [ ] Wks trust act. | [ ] Srv trust act |
[X] Pwd don't expir | [ ] Auto lockout | [ ] (unknown 0x08) |
[ ] (unknown 0x10) | [ ] (unknown 0x20) | [ ] (unknown 0x40) |
Failed login count: 0, while max tries is: 0
Total login count: 10
- - - - User Edit Menu:
1 - Clear (blank) user password
2 - Edit (set new) user password (careful with this on XP or Vista)
3 - Promote user (make user an administrator)
4 - Unlock and enable user account [probably locked now]
q - Quit editing user, back to user select
Select: [q] > q
This shows a lot of info, including the list of users. The point is to reset a specific user. Changes made like this will affect all users.
To get just the list of users,
$ chntpw -l SAM
chntpw version 0.99.6 080526 (sixtyfour), (c) Petter N Hagen
Hive <SAM> name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 666c <lf>
Page at 0x6000 is not 'hbin', assuming file contains garbage at end
File size 262144 [40000] bytes, containing 5 pages (+ 1 headerpage)
Used for data: 222/17400 blocks/bytes, unused: 14/2920 blocks/bytes.
* SAM policy limits:
Failed logins before lockout is: 0
Minimum password length : 0
Password history count : 0
| RID -|---------- Username ------------| Admin? |- Lock? --|
| 01f4 | Administrator | ADMIN | dis/lock |
| 01f5 | Guest | | dis/lock |
| 03e9 | myself | ADMIN | dis/lock |
And now let’s modify only one user in the list
$ chntpw -u 'myself' SAM
chntpw version 0.99.6 080526 (sixtyfour), (c) Petter N Hagen
Hive <SAM> name (from header): <\SystemRoot\System32\Config\SAM>
ROOT KEY at offset: 0x001020 * Subkey indexing type is: 666c <lf>
Page at 0x6000 is not 'hbin', assuming file contains garbage at end
File size 262144 [40000] bytes, containing 5 pages (+ 1 headerpage)
Used for data: 222/17400 blocks/bytes, unused: 14/2920 blocks/bytes.
* SAM policy limits:
Failed logins before lockout is: 0
Minimum password length : 0
Password history count : 0
| RID -|---------- Username ------------| Admin? |- Lock? --|
| 01f4 | Administrator | ADMIN | dis/lock |
| 01f5 | Guest | | dis/lock |
| 03e9 | myself | ADMIN | dis/lock |
---------------------> SYSKEY CHECK <-----------------------
SYSTEM SecureBoot : -1 -> Not Set (not installed, good!)
SAM Account\F : 0 -> off
SECURITY PolSecretEncryptionKey: -1 -> Not Set (OK if this is NT4)
Syskey not installed!
RID : 1001 [03e9]
Username: myself
fullname:
comment :
homedir :
User is member of 1 groups:
00000220 = Administrators (which has 2 members)
Account bits: 0x0214 =
[ ] Disabled | [ ] Homedir req. | [X] Passwd not req. |
[ ] Temp. duplicate | [X] Normal account | [ ] NMS account |
[ ] Domain trust ac | [ ] Wks trust act. | [ ] Srv trust act |
[X] Pwd don't expir | [ ] Auto lockout | [ ] (unknown 0x08) |
[ ] (unknown 0x10) | [ ] (unknown 0x20) | [ ] (unknown 0x40) |
Failed login count: 2, while max tries is: 0
Total login count: 1
- - - - User Edit Menu:
1 - Clear (blank) user password
2 - Edit (set new) user password (careful with this on XP or Vista)
3 - Promote user (make user an administrator)
4 - Unlock and enable user account [probably locked now]
q - Quit editing user, back to user select
Select: [q] > 1
Hives that have changed:
# Name
0 <SAM>
Write hive files? (y/n) [n] : y
0 <SAM> - OK
In this case it clears the password (the choice of “1″). In fact, it appears like the password was already cleared when I captured this log, but it did no good.
As mentioned above, the password was still required after this, so the operation failed. I also failed to change the password for this user.
To make a long story short:
$ perl -pi.bak -e 's/this/that/g' `find inhere/ -type f`
or to delete entire lines that contain a certain substring:
$ perl -pi.bak -e 's/^.*?RUNTIME_PARAM.(?:OUTPUT|SHARED)DIR.*?[\n\r]+//gm' $(find . -iname \*.xci)
This will create backup files. Yes, you want them. Really. Otherwise, in particular under Windows, errors like this may show up:
Can't do inplace edit on inhere/example.c: Permission denied.
This may look harmless, but in fact the target file ends up complete deleted. So don’t go just perl -pi -e. It may go horribly wrong.
Another thing to watch out for is having the .git subdirectory included in the “find” statement. In particular, “find .” may lead to this. Messing up the git repository is no fun.
The in-place execution runs separately on each line, so it’s not useful for replacing things that span across more than one line. In that case, a “full” perl script should be used. For example, in order to remove multiple blank lines from C sources,
$ for i in *.c *.h ; do perl -e 'local $/; $a=<>; $a=~s/\n{3,}/\n\n/g; print $a;' < $i > $i.new; mv -f $i.new $i; done
This removes any occurrence of three or more newlines into two. It’s fine to have an empty line, but not more than one.
So there’s this huge pile of source code, with an enormous amount of functions and methods. How can you quickly find where each is defined? IDEs has this functionality built in, but it’s not so easy to push an already existing (and huge) project into an IDE.
The solution is so simple. For example,
$ ctags -f sourcemap -R OpenCV-2.4.0/
which will scan the OpenCV-2.4.0 directory for source file, parse them, and create a textual file, sourcemap, with (probably) all method and function definitions listed in alphabetical order. With the files they were found in. The format isn’t very human-friendly, but still quite easy to find the answer in.
Just a quick summary on how to compile my own cross-compiler in 20 minutes.
I’m trying to find Altera boards that have an embedded ARM (Cortex A9) on them. This is my list so far.
If you know about another board, please comment below. I’ll remove the comment and add the board to the list.
Cyclone V
- Altera’s official Cyclone SoC Development kit with a 5CSXFC6D6F31C8NES.
- The Sockit by Arrow / Terasic with a 5CSXFC6D6F31C8ES.
- The DE-1 SoC board by Terasic, with a 5CSEMA5F31C6, strikingly similar to SoCKit.
- SoCrates by Devboards with a 5CSEBA6U23C7N (how’s your German?)
- Helio by CyTech global (Hong Kong) with a 5CSXFC6C6U23ES
