Eliminating Copies in C-FFI

We've run into a situation fairly regularly when wrapping C libraries where we'd like to avoid making extra copies of data when passing it between Dylan and C code via the C-FFI. Related to this, we want to be able to use multiple types from the Dylan side of things such as <buffer>, <byte-vector> or <byte-string> without having to have separate code paths for each of them. Each of these classes can store raw byte data that we may want to share with other code.

An example that we'll work with in this discussion comes from bindings (in simplified and excerpted forms) for the LevelDB library:

extern void leveldb_put(
    leveldb_t* db,
    const leveldb_writeoptions_t* options,
    const char* key, size_t keylen,
    const char* val, size_t vallen,
    char** errptr);


The easiest way to bind leveldb_put would be to use <C-string>:

define C-function %leveldb-put
  input parameter db_ :: <leveldb-t*>;
  input parameter options_ :: <leveldb-writeoptions-t*>;
  input parameter key_ :: <C-string>;
  input parameter keylen_ :: <size-t>;
  input parameter val_ :: <C-string>;
  input parameter vallen_ :: <size-t>;
  output parameter errptr_ :: <char**>;
  c-name: "leveldb_put";

However, this would mean that every key and value would have to be copied from an existing byte container (like <buffer>, <string> or <byte-vector>) to ...

read more »

There are comments.

Binding nanomsg with melange

For those who haven't heard about it, nanomsg is a great library from Martin Sustrik. It provides high performance messaging with a simple API similar to that of BSD sockets. It bears some similarity to what someone might do with ZeroMQ and is licensed under the liberal terms of the MIT license. The nanomsg library is currently in a pre-alpha state.

I have recently written bindings for it for use with Dylan which are available as nanomsg-dylan.

In Dylan, we have a couple of options when writing bindings. We have a low level direct-c-ffi and a higher level C-FFI. Using the lower level interface is fairly tedious and verbose while using C-FFI is fairly convenient. But writing a binding using either involves a lot of work and hand-translation of the C APIs into the right Dylan definitions. (Currently, the direct-c-ffi system is not documented.)

This is where the melange tool is very useful. Melange can parse C headers and automatically generate the C-FFI bindings. While doing this code generation, it also handles details like automatically translations the C names into the correct Dylan names following the Dylan conventions.

Melange now has some preliminary documentation.

Using Melange

Melange generates a ...

read more »

There are comments.

Getting Started with OpenDylan 2012.1

There are many exciting changes in the 2012.1 release of OpenDylan, but to me, some of the most exciting changes are the ones that make it easier to get a new project going and building. We'll be writing more about that in the future, but for now, let's look at how to get OpenDylan installed in the first place!


Linux and Mac OS X

Installing OpenDylan is pretty easy on Linux and Mac OS X.

  1. First, you'll want to make sure that you have the basic developer tools like gcc installed.

  2. On Gentoo, OpenDylan is available via the package system, so go ahead and install it that way. Packages for other operating systems are coming in the future and help is appreciated.

  3. Otherwise, download and install a build of OpenDylan for your operating system. Put OpenDylan in a directory of its own, like /opt/opendylan-2012.1.

  4. Make sure that the bin directory under the installation path is on your PATH. You can do this by placing a line in your .bashrc in your home directory like this:

    export PATH=/opt/opendylan-2012.1/bin:$PATH

    You can enter that command in your current shell as well ...

read more »

There are comments.