__ __ _____    ____                                        __  
   / //_/|__  /   / __ \ ___   _____ ___   ____ _ _____ _____ / /_ 
  / ,<    /_ <   / /_/ // _ \ / ___// _ \ / __ `// ___// ___// __ \
 / /| | ___/ /  / _, _//  __/(__  )/  __// /_/ // /   / /__ / / / /
/_/ |_|/____/  /_/ |_| \___//____/ \___/ \__,_//_/    \___//_/ /_/
/ K³ Research Documentation Project / Blog

Every Journey Starts with a FAIL

...or Understanding Syscall Conventions for Different Platforms

February 15, 2018

Introduction

Not long ago I started looking into FreeBSD kernel exploitation. There are only a few ressources but probably the best starting point is argp’s Phrack article from 2009. And while he does only provide one technique, I wanted to understand it and port it to a modern FreeBSD release before describing new, own researched techniques.

Well, at least this was my plan. In reality I ended researching how different operating systems resp. the same operating system but for different architectures implement syscalls. Hence, new exploiting methods have to wait for another post. In this one I want to describe my personal FAIL while porting argp’s exploit example to a FreeBSD 11.1-RELEASE running on a 64bit processor. Maybe this will give other people interested in kernel stuff some insights they didn’t know before. If you already know how syscalls work on 32bit and 64bit *BSD because you are an experienced exploit or kernel developer, you will probably want to search for something else to read. Moreover, some of the debugging stuff can look laborious because I wanted to show the steps I have done while attacking my problem instead of showing a simple walkthrough to the solution.

The Problem

argp described in his article vulnerable code consisting of a loadable kernel module which exposes a syscall to the userland. Because it was written around the time when FreeBSD 8-RELEASE came out and because he has written himself that the code needs smaller adjustments to work with this version (it was written for FreeBSD 7) I thought I will first port it to FreeBSD 11.1-RELEASE. Moreover it was written for an Intel 32bit processor architecture as we can see from his shellcode examples. Hence, I wanted to go right away the harder way and modify it to work on an 64bit processor.

The stripped down example which will be enough for this post looks like this:

struct argz
{
	char *buf;
	u_int len;
	int op;
	u_int slot;
};

static int
trigger(struct thread *td, struct argz *uap)
{
	uprintf("%p\n", uap->buf);
	uprintf("%u\n", uap->len);
	uprintf("%d\n", uap->op);
	uprintf("%u\n", uap->slot);

    return 0;
}

The userland caller looks like this:

struct argz
{
	char *buf;
	u_int len;
	int op;
	u_int slot;
};

int
main(int argc, char *argv[])
{
	int sn, modid;
	struct module_stat mstat;
	struct argz vargz;

	vargz.len = 256;
	vargz.buf = calloc(vargz.len + 1, sizeof(char));

	if(vargz.buf == NULL)
	{
		perror("calloc");
		exit(1);
	}

	mstat.version = sizeof(mstat);
	if((modid = modfind("sys/trigger")) == -1)
	    err(1, "error in modfind");
	if(modstat(modid, &mstat) != 0)
	    err(1, "error in modstat");

	sn = mstat.data.intval;
	vargz.op = 3;

	vargz.slot = 1;
	printf("%p\n", (void*)vargz.buf);
	printf("%u\n", (void*)vargz.len);
	printf("%d\n", (void*)vargz.op);
	printf("%u\n", (void*)vargz.slot);
	if(syscall(sn, vargz) != 0)
	    err(1, "error in syscall");

	free(vargz.buf);
	return 0;
}

So, after loading the kernel module with ‘kldload trigger’ and executing the caller code, we would expect that the userland program prints the values of the members of ‘struct argz’ and after the the kernel code does the same, wouldn’t we? In other words, something like this:

root@freebsd64:trigger/ # kldload trigger
trigger loader at 210
root@freebsd64:trigger/ # ./trigger
0x800e16000
256
3
1
0x800e16000
256
3
1
root@freebsd64:trigger/ #

However, what we get in reality looks like this:

root@freebsd64:trigger/ # kldload trigger
trigger loaded at 210
root@freebsd64:trigger/ # ./trigger
0x800e16000
256
3
1
0xd2
1
32767
4294961816
root@freebsd64:trigger/ #

What the heck has just happened here? Moreover, we can easily verify that the code works as expected if we run it on a 32bit processor. If you think that this is not interesting at all, let’s remove the printf calls from the userland code. We can see something like this:

root@freebsd64:trigger/ # ./trigger
0x7fffffffeab0
256
3
210
root@freebsd64:trigger/ #

What? Removing the printf calls modified the pointer of our character buffer and some other variables, too. That is, we have a syscall that behaves differently depending on other code in userland before the syscall execution. Now, you should be baffled because an imperative programming language like C should behave deterministic - well, it still does but if you are not used to syscall conventions (like me before writing this) it does look undeterministic ;)

But more on this later, first let us debug this a little bit.

Debugging

The first thing we could do is using truss(1), a tool from FreeBSD base which shows executed syscalls and corresponding arguments while a program runs. Attaching it to our trigger program will result in the following output (uninteresting syscalls are removed):

root@freebsd64:trigger/ # truss ./trigger
...	
modstat(0x1f1,0x7fffffffeab0)			        = 0 (0x0)
0x7fffffffeab0
256
3
210
lkmnosys(0x7fffffffeab0,0x300000100,0xd2,0x140) = 0 (0x0)
...

lkmnosys is our syscall. This name is hard coded into the kernel and corresponds to the syscall offset which was printed during the execution of kldload (’trigger loaded at 210’). The modstat(2) call before has the offset 301, for example.

Two things should catch our eyes:

  1. Both, modstat(2) and our syscall, are called with the same stack adress which is printed out later.
  2. The printed out arguments are in another order passed to the syscall - The 3 is passed via the higher bits of the second argument while the 210 is passed via the third argument. The fourth argument is not printed out at all.

That is, the arguments are already transferred wrong into the kernel space. Some online research led me to read about syscall conventions which are differently defined for different platforms. For 64bit FreeBSDs the arguments are passed to the kernel via the CPU registers RDI, RSI, RDX, RCX, R8 and R9. We can verify this by looking into the function ‘cpu_fetch_syscall_args’ in ‘sys/amd64/amd64/trap.c’:

int
cpu_fetch_syscall_args(struct thread *td, struct syscall_args *sa)
{
...
frame = td->td_frame;
reg = 0;
regcnt = 6;

params = (caddr_t)frame->tf_rsp + sizeof(register_t);
...
    reg++;
    regcnt--;
....
argp = &frame->tf_rdi;
argp += reg;
bcopy(argp, sa->args, sizeof(sa->args[0]) * regcnt);
... // At this point syscall parameters are handled in cases
... // one needs more than 5 parameters

}

argp points to a ‘struct trapframe’ which holds the values of the registers. args, which is referenced by ‘sa->args’ is an array of eight register_t variables which are defined as int64. Hence, a syscall on a 64bit FreeBSD can have up to eight parameters.

Now, we know how the syscall convention on a 64bit FreeBSD is defined. But we still don’t know how to use this as we expected it to work as already used.

Redemption

Well, stubborn as I am and as pointed out by a friend I was just using syscall(2) wrong. If we look in the manpage of syscall(2), we can find this function prototype:

int syscall(int number, ...);

In other words, syscall awaits a variable number of arguments which are passed to the kernel code. The correct syscall(2) usage should look like this:

syscall(sn, vargz.buf, vargz.len, vargz.op, vargz.slot);

And now we get the expected output:

root@freebsd64:trigger/ # ./trigger
0x800e16000
256
3
1
0x800e16000
256
3
1
root@freebsd64:trigger/ #

Why the Original Code Worked While It Was Wrong

aAs written above, the syscall convention for the 32bit architecture is different from the one for the 64bit architecture. Indeed, a syscall on a 32bit FreeBSD system passes the arguments via the stack while the syscall offset is stored in the EAX register. The transfer into the kernel address space is done in ‘cpu_fetch_syscall_args’ in ‘sys/i386/i386/trap.c’.

int
cpu_fetch_syscall_args(struct thread *td, struct syscall_args *sa)
{
    ...
    frame = td->td_frame;

    params = (caddr_t)frame->tf_esp + sizeof(int);
    sa->code = frame->tf_eax;
	
	...
	
    if (params != NULL && sa->narg != 0)
	    error = copyin(params, (caddr_t)sa->args,
            (u_int)(sa->narg * sizeof(int)));
    else
	...
}

That is, ‘params’ points to ESP+4 bytes offset. Later, the arguments are copied into the kernel space which is referenced by ‘sa->args’. ‘args’ is an array of eight ‘register_t’ which is defined as ‘int32_t’ on the 32bit platform in comparison to the 64bit platform. And as ‘struct args’ only consisted of integers they got copied into the syscall arguments which are given to the trigger function inside the kernel module. We could verify this by changing ‘int op’ to ’long long op’ in the kernel module and in trigger.c. We get the following output:

root@freebsd64:trigger/ # ./trigger
0x28414000
256
3
1
0x28414000
256
4294967295
2
root@freebsd64:trigger/ #

To bring this to an end: argp’s version only worked for his special choice of arguments and only on 32bit. On 32bit FreeBSD platforms the arguments are transferred into kernel space by 4 byte integers, hence it will only work for intergers anyway. On 64bit FreeBSD platforms we have to use syscall(2) in the intended way.

NB: Linux on 32bit platforms does use the CPU registers and not the stack to transfer syscall arguments into the kernelspace. That is, just use a struct to transfer arguments doesn’t work.

Misc.

I would like to thank fs (@__schulze) for giving me the hint that I used syscall(2) wrong and for pushing me to finally start a blog :)