I just posted a new version of the Leaf framework. So I thought this might be a good time to blog on how to write and use a hit tracer using Leaf. Even though it is mostly a static analysis tool, the data it collects during this process is really useful to a debugger. I wanted a debug API and I wanted it fast, so a few versions ago I wrote a quick wrapper to Ptrace and put it into Leaf. It currently has only been tested for x86 Linux so there's work to be done in making it support BSD. I am always looking into other ways to make the debugging API cleaner, more useful and easier to code with, so please send any suggestions. So lets look at the steps needed to write a plugin that implements a hit tracer.
Here is what my basic hit tracer, 'lhit' (included with Leaf) implements:
1. LEAF_init() - a mandatory function that must be present in all plugins. You can use it to initialize any private data structures your plugin may need, or you can leave its function body blank.
2. LEAF_interactive() - this is the plugin hook a debugger would want to call. Ideally you only want *one* plugin calling this, it doesnt make sense to have more then one. If your plugin implements this hook it will be called after all other static analysis if finished, consider it your debugger plugins main()
3. LEAF_attach(pid_t) - takes a pid_t as its only argument and will attach the debugger to your target process.
4. LEAF_set_hittracer(pid_t, breakpoints_t, int) - this is where it gets slightly tricky. Your plugin must declare a structure somewhere of type breakpoints_t. Pass the targets pid, the breakpoints structure and flag (ON/OFF) to this function and Leaf will automatically use the vector of function addresses it collected during static analysis and set breakpoints on each of them. There is no need for your plugin to manage any of this. There is also another function called LEAF_set_breakpoint, which takes a pid_t, a breakpoints_t structure, and the address you want to break on, you can use this for any other manual breakpoints you want to set.
5. LEAF_cont(pid_t) - this one is pretty self explanitory, it takes a pid_t as its only argument, and instructs the traced program to continue. At this point Leaf will handle calling wait() for you. All you have to do is inspect and handle the signal it returns. If you had used LEAF_set_hittracer and you hit one of the breakpoints it set then you will want to call LEAF_reinstall_breakpoint and Leaf will take care of putting the old instruction back, single stepping and reinstalling the breakpoint for you.
6. LEAF_get_regs(pid_t, user_regs_struct) - this will retrieve the processes registers for you.
7. LEAF_detach(pid_t) - will detach Leaf from your process.
8. LEAF_cleanup() - another mandatory plugin hook which you can use to free memory or close file descriptors, or you can leave it blank.
You will find an example hit tracer (lhit) which implements all of this in version 0.0.15 of Leaf here. Its not the best hit tracer in the world but it does the job. The debugger internals will be getting an overhaul soon, but the API should stay the same.
This new version of Leaf also contains my experimental LeafRub plugin which embeds a Ruby interpreter for scripting capabilities. An example LeafRub.rb script is also included, but I'll blog more about that later.
