System Call


A system call is how a program requests a service from an operating system's kernel that it does not normally have permission to run. System calls provide the interface between a process and the operating system. Most operations interacting with the system require permissions not available to a user level process, e.g. I/O performed with a device present on the system, or any form of communication with other processes requires the use of system calls.

Generally, systems provide a library that sits between normal programs and the operating system, usually an implementation of the C library(libc), such as glibc. This library exists between the OS and the application, and increases portability. On Unix, Unix­like and other POSIX­compatible Operating Systems, popular system calls are open, read, write, close, wait, exec, fork, exit, and kill. Many of today's operating systems have hundreds of system calls. For example, Linuxhas 319 different system calls. Similarly, FreeBSD has almost 500.

Under Linux the execution of a system call is invoked by a maskable interrupt or exception class transfer, caused by the instruction int 0x80. We use vector 0x80 to transfer control to the kernel. This interrupt vector is initialized during system startup, along with other important vectors like the system clock vector. After the switch to kernel mode, the processor must save all of its registers and dispatch execution to the proper kernel function.


Note: This is implemented in Debian Lenny with 2.6.26 kernel. The method differs in different distributions/kernel. The system call is named mycall and it takes two integers as input and return their sum.

1) Add “.long sys_mycall” at the end of the list in the file syscall_table.S.

Full path for the file syscall_table.S is /usr/src/linux­source-2.6.26/arch/x86/kernel/syscall_table_32.S.

2) Add system call along with the number in unistd_32.h

Add “ #define __NR_mycall 327 ” at the end of the list. 327 is the last system call number.

Full path for the file unistd_32.h is /usr/src/linux­source­2.6.26/include/asm­x86/unistd_32.h

3) Add the following line at the end of the file syscalls.h:

asmlinkage long sys_mycall(int i, int j);

All system calls are marked with the asmlinkage tag, so they all look to the stack for arguments.

Full path of syscalls.h is /usr/src//linux­source­2.6.26/include/linux/syscalls.h.

4) Add mycall/ to core­y += in Makefile.

The line in the end shall look like:

core­y += kernel/ mm/ fs/ ipc/ security/ crypto/ block/ mycall/

Full path for Makefile is /usr/src//linux­source­2.6.26/Makefile.

5) Create a new directory in /usr/src//linux­source­2.6.26/ and name it mycall

6) Create a new file called mycall.c in /usr/src//linux­source­2.6.26/mycall. Contents of the file shall be as follows:

#include <linux/linkage.h>
asmlinkage long sys_mycall(int i, int j)

7) Create Makefile in /usr/src//linux­source­2.6.26/mycall. Makefile shall be like:

obj­y := mycall.o

8) Compile the kernel througg the normal procedure

9) Create the following userspace program to test your system call and name it testmycall.c. The contents of this file shall be:

#include <unistd.h>
#include <stdio.h>
#define __NR_mycall 327
long mycall(int i, int j)
return syscall(__NR_mycall, i, j);
int main()
printf("%d\n", mycall(12,10));
return 0;

10) Compile it to test whether the system call works

root@jestinjoy­desktop:/# gcc mycall.c

root@jestinjoy­desktop:/# ./a.out