Some notes on:

Assembly Language, Step by Step, 3rd Edition

by Jeff Duntemann (Wiley, 2009; ISBN: 978-0-470-49702-9)

This is a privately posted and maintained webpage that attempts to provide answers to certain common questions some people have had while following the instructions in Jeff's recent book on assembly language programming. Note: Jeff Duntemann is not personally liable for anything that appears here! That said, I'm currently in contact with him and will post on this site any ideas, fixes, workarounds or other comments that he cares to make. As time allows and a felt need arises, I may expand this site into a forum. Any errors on this page are entirely unintentional. Therefore, use this information at your own risk. If you have any corrections or helpful additions to the tips on this site, please email me at asmhelp(at)ishipaco(dot)com.
—Gary Albers, Shell Beach CA


[Last Update: 2010-10-15]  See this book on Amazon, Assembly Language, Step By Step, where you can read the reviews and comments.   Visit Jeff's official website to download the code, errata and other comments. At the present time, there are two main questions that are being asked in a number of online forums, which I'll try to answer here.

Question One: (or go to the second question, or more tips).

When I follow the command line instructions on page 144-146 [of the book] to assemble and link the example source code, like so:

$ nasm -f elf -g -F stabs eatsyscall.asm
$ ld -o eatsyscall eatsyscall.o

the linker command fails with the error message:

ld: i386 architecture of input file filename.o is incompatible with i386:x86-64 output

Answer: Jeff wrote his book using Linux Ubuntu 8.10 on a 32-bit machine, back before 64-bit CPUs and operating systems were widespread. This error message occurs on computers with a 64-bit CPU and operating system. This problem should not occur on computers with 32-bit CPUs.

What's happening is that nasm is producing an object module with code intended to run on a 32-bit CPU. That's what the -f elf option tells the assembler to do. Note that -f elf is the same as -f elf32—they both yield 32-bit object files. However, the linker (the ld command) on a 64-bit computer has default options set which makes it expect object code assembled to run on a 64-bit CPU. When that's not what it gets during the link (ld) command, it throws a hissy-fit, reports an error, and fails to complete the linking process!

To find out more about the different kinds of object modules nasm can produce, enter the following command at your Linux console command prompt ("$" represents your prompt; don't type it):

$ nasm -hf

Examine the list and note the many different output formats that are available. Also note that one of the formats should be elf64. Using this assembler option instead of elf in the nasm command line will cause nasm to produce 64-bit object code, which the linker on a 64-bit machine wants by default.

Two Solutions: With all this knowledge now in hand, we can see that there are two basic solutions to this problem:

  1. Assemble and link to produce a 32-bit executable. This is recommended for people trying to follow Duntemann's book. 32-bit programs will run on both 32- and 64-bit machines and they involve smaller files than their 64-bit cousins. Most important is that the debugger you'll be using displays 64-bit code, registers and memory addresses quite differently from 32-bit code. Stick with 32-bit programs and the book should be easy to follow:

    $ nasm -f elf -g -F stabs eatsyscall.asm
    $ ld -o eatsyscall eatsyscall.o -melf_i386

  2. Assemble and link to produce a 64-bit executable: There's nothing really wrong with doing that, except … your executable files won't run on the 32-bit machine your best friend has, your object and executable files will be bigger and, most importantly, your debugger will show strange 64-bit register names, and memory locations won't be the same as in a 32-bit program. In short, things in the debugger will be quite unlike what's shown in the book:

    $ nasm -f elf64 -g -F stabs eatsyscall.asm
    $ ld -o eatsyscall eatsyscall.o

In the first solution, we ran the nasm command as it appears in the book, which will produce a 32-bit object file. We had to add the -melf_i386 option to the linker command line. This tells the linker to expect a 32-bit object module and to output a 32-bit executable file.

In the second case we changed the nasm option from -f elf to -f elf64, which will cause nasm to create an object module with 64-bit code. Since that will now please the linker, we didn't need to make any changes to the linker command line.

Warning: Although the eatsyscall (and earlier) programs can be successfully assembled, linked and run as either 32-bit or 64-bit programs, this is not the case with most of the programs later in the book, beginning with the Hexdump3 program starting on pg. 359. This is the first program in the book that uses push and pop instructions. Code containing these instructions will not assemble as 64-bit programs without errors. The reason for this involves a conflict in register sizes. The program code, as written (e.g., push eax), uses references to 32-bit register operands. In other words, what we are trying to do is push a 32-bit register onto a 64-bit stack, which won't work. Of course, the source code could be changed to use 64-bit operands, but that assumes a level of expertise that most readers are not likely to have. The simple solution to this and other such problems is to assemble and link all of the book's programs as 32-bit programs.
(Return to top of page)

Question Two: (or go to the first question, or more tips).

The second problem with which Jeff is currently dealing, is the fact that he was blindsided by the recent release of Linux Ubuntu 10.4. For unknown reasons, the Insight debugger, which Jeff uses throughout his book, was dropped from the Ubuntu 10.4 package. Further, an Insight package is not to be found on any of the standard repositories that Ubuntu 10.4 Synaptic knows about, and I haven't been able to get any Insight packages from other sources to work. However, there are several work-arounds for this problem. In particular, see Solution Three below for a way to actually install Insight on Ubuntu 10.4:

Solution One: Use an earlier version of Ubuntu (e.g., 8.10, 9.04 or 9.10, perhaps on a separate, bootable partition?) which included an Insight package that worked. If you can do that, and understand and heed the above info about 32/64-bit machines and OSs, Jeff's instructions should work. In my case, I installed Ubuntu 9.10 to test how things worked, and they worked fine. It would appear that the problem is limited to the new 10.4 release of Ubuntu. However, the Insight debugger was not installed by default, but was available in a standard repository and installed easily with Synaptic Package Manager.

Warning: If you are currently running Ubuntu 10.4 on your computer and decide to install Ubuntu 9.10 to work with Jeff's book, you might encounter another strange problem. When I did that, the 9.10 install also installed the legacy version of grub (0.97), replacing the grub2 version that my 10.4 had been using. Although the menu list still worked, when I booted into any of my 10.4 menu choices, the Ubuntu splash screens displayed text as unreadable clusters of colored pixels. Also, although the Ubuntu desktop appeared and worked normally, all of the console windows accessed by using ctrl-alt-F(1-6) were also unreadable and unusable because of text being displayed as colored pixels. I strongly suspect this problem might occur after installing any earlier version of Ubuntu, although I haven't yet tested that.

This problem was caused because the boot process was using the newly installed legacy version of grub (0.97), which doesn't understand the ext4 file system used by Ubuntu 10.4. I fixed this by booting into the Ubuntu 9.10 menu selection and then upgrading to grub2. After that, everything worked normally again.

Solution Two: Use the Kdbg debugger with Ubuntu 10.4. Kdbg has been working OK on my Ubuntu 10.4 installation. The faults that Jeff cites in his book seem to have been fixed in the latest version of Kdbg and Ubuntu. The memory window works fine for me. Jeff gives an introduction to Kdbg in his book. Using it instead of Insight shouldn't be difficult. In some cases you might want to experiment using GDB direct from the command line. After all, Kdbg is simply a GUI front end for GDB. Also visit Jeff's official website, where Jeff has posted some other work-around solutions.

Note: To be honest, I have encountered some strange quirks in Kdbg. Occasionally, a program simply won't run in the debugger after being loaded; no error messages, just blank memory and register windows. I've found that quitting Kdbg and adding a simple space character somewhere in the source code and then reassembling and linking will often solve the problem. Sometimes simply "touching" the executable file (i.e., $ touch filename), which does nothing more than update the date for the file, will solve the problem. Why this problem exists and this fix works are a complete mystery to me.

One other quirk in Kdbg I've noticed is: when first running Kdbg, several windows will appear. On my machine, the source code, register, memory and terminal output windows pop up. If I immediately close the output window, which I don't use for these simple programs, Kdbg fails in the way I mentioned above. If, however, I set a breakpoint in the code at the first executable line of code following the first nop instruction and run the code down to the breakpoint, I can then close the output window and Kdbg will continue to run with no problems. Go figure. Try this technique if you experience a similar problem.

Solution Three: Many thanks to Gareth Lapworth for this elegant solution to the Insight problem. Gareth is a lecturer in the Forensics and Security Research Group in the faculty of technology at De Montfort University, Leicester, England. He recently wrote Jeff and provided a way to install Insight on Ubuntu 10.4. He reports that he has successfully implemented this fix on over 400 machines in his lab. I have also had complete success installing Insight on my own system using his method.

Gareth actually provided two methods for the installation. The first, more complex, approach involves compiling from source code. The second method uses a shell script he wrote which handles the entire process transparently. I'll present the second method here, since it works well and doesn't require any knowledge of the many intricacies involved in compiling from source code. If you would like to try the compilation approach, please email me at asmhelp(at)ishipaco(dot)com and I'll be happy to send you his instructions. Here is Gareth's shell script:

#!/bin/sh # Install some packages /usr/bin/apt-get -f --force-yes --yes install tofrodos nasm kate bless kdbg ktechlab kile konsole build-essential xorg-dev libncurses5-dev libpthread-stubs0-dev libpthread-stubs0 # If insight isn't installed, then install it if [ ! -e /usr/local/bin/insight ] then # Make sure we are in the correct directory cd /usr/local/src # Only download the file once. if [ ! -e /usr/local/src/insight-6.8-1-gl.tar.gz ] then /usr/bin/wget -O /usr/local/src/insight-6.8-1-gl.tar.gz fi # Only extract the program once if [ ! -d /usr/local/src/insight-6.8-1 ] then /bin/tar -xvzf /usr/local/src/insight-6.8-1-gl.tar.gz fi cd insight-6.8-1 ./configure make make install fi # If the insight launcher hasn't been installed, then set one up! if [ ! -e /usr/share/applications/insight.desktop ] then echo "[Desktop Entry]" > /usr/share/applications/insight.desktop echo "Version=1.0" >> /usr/share/applications/insight.desktop echo "Encoding=UTF-8" >> /usr/share/applications/insight.desktop echo "Name=Insight Debugger">> /usr/share/applications/insight.desktop echo "Comment=GDB Style graphical debugger" >> /usr/share/applications/insight.desktop echo "Exec=insight" >> /usr/share/applications/insight.desktop echo "Icon=gnome-mnemonic.png" >> /usr/share/applications/insight.desktop echo "Terminal=false" >> /usr/share/applications/insight.desktop echo "Type=Application" >> /usr/share/applications/insight.desktop echo "Categories=Application;Development;" >> /usr/share/applications/insight.desktop fi

Copy this script and paste it into a text editor. Then save it with the filename Open a terminal window, change to the directory where you saved the file, and run the following command from the terminal command line:

$ sudo sh ./

If you have any problems with this method, you can download the original script here. That's all there is to it. Insight should be installed and ready to run!

Note: Gareth mentions that this installation downloads a custom archive from his personal website at the university and this will be maintained for some time. It is important to note that this version of Insight won't necessarily be the latest version. He will update to the latest versions when he can. However, I've found that this version is entirely adequate to work with all the instructions in Jeff's book. Be aware that this is a major installation which involves downloading about ninety files. The compilation process also takes considerable time, so be patient. To quote Gareth's explanation:

"Insight isn't just a front end to GDB, instead it is complete port of the software. Therefore we will need to compile GDB and in order to do so, we will need to download a whole bunch of libraries."

Again, many thanks to Gareth for this final fix. After running the shell script and completing the installation, it should be obvious that this was never going be a "simple" fix.
(Return to top of page)

More Tips:

Configuring the Konsole terminal in Kate using the Gnome desktop: Jeff gives instructions for configuring the Konsole terminal for use with the Kate editor on pg. 192 of his book. Konsole is the default terminal for the KDE desktop but, if you are using the Gnome desktop, the default terminal is different and you need to add an extra step to configure Konsole. Assuming you've already installed Konsole, first open a terminal window in the Gnome desktop (Applications/Accessories/Terminal). This will launch the default Gnome terminal. Then run a Konsole terminal from the command line: ($ konsole). This will open a Konsole terminal window. Then follow Jeff's directions for configuring a new profile for using the Konsole terminal in Kate.
(Return to top of page)