A New IDE Approach

We've previously written about the state of the Open Dylan IDE. In short, it is currently only available on Windows and looks like an application from the 1990s (because it is). Simply getting it running on Mac OS X or Linux would be a lot of work (an attempt was made to port the underlying GUI framework), not counting modernization and adding new features.

In late September, we had a seemingly crazy idea: What if we built a new IDE on an existing IDE framework. We've previously worked with Eclipse and didn't want to do that again. IntelliJ, however, officially supports IDEs for Java, Python, Ruby, PHP, and Objective C. Unofficially, it has support for a host of other languages including Erlang, Lua, Scala, HaXe and many others.

So, we wondered, what would it take to get some basics running for Dylan? In the end, the answer was that within 10 hours of first looking at IntelliJ plug-in authoring, we had it recognizing file types, doing some basic parsing for LID files and some other minor things.

This led to the creation of DeftIDEA, a new Dylan Environment For Tools running inside of IntelliJ IDEA.

Why did ...

There are comments.

Beginning to Rethink Streams

Dylan's current streams library has served us moderately well over the years. However, it has some issues which can be addressed by a new design, expanding the range of problems for which it is suited.

How things are now

According to the current streams library's documentation, the design goals were:

A generic, easy-to-use interface for streaming over sequences and files. The same high-level interface for consuming or producing is available irrespective of the type of stream, or the types of the elements being streamed over.
Efficiency, especially for the common case of file I/O.
Access to an underlying buffer management protocol.

One of the things it was explicitly not designed to handle was, again, according to the design goals in the documentation:

A comprehensive range of I/O facilities for using memory-mapped files, network connections, and so on.

Unfortunately, the primary interface to our current network library is based on these very streams for which network connections were not a design goal. While this works in practice, it imposes some important limitations on our networking code. The biggest of these is that sockets can not be non-blocking as it is expected that reads and writes will complete ...

There are comments.

Binding nanomsg with melange

For those who haven't heard about it, nanomsg is a great library from Martin Sustrik. It provides high performance messaging with a simple API similar to that of BSD sockets. It bears some similarity to what someone might do with ZeroMQ and is licensed under the liberal terms of the MIT license. The nanomsg library is currently in a pre-alpha state.

I have recently written bindings for it for use with Dylan which are available as nanomsg-dylan.

In Dylan, we have a couple of options when writing bindings. We have a low level direct-c-ffi and a higher level C-FFI. Using the lower level interface is fairly tedious and verbose while using C-FFI is fairly convenient. But writing a binding using either involves a lot of work and hand-translation of the C APIs into the right Dylan definitions. (Currently, the direct-c-ffi system is not documented.)

This is where the melange tool is very useful. Melange can parse C headers and automatically generate the C-FFI bindings. While doing this code generation, it also handles details like automatically translations the C names into the correct Dylan names following the Dylan conventions.

Melange now has some preliminary documentation.

Using Melange

Melange generates a ...

There are comments.

Deepening LLDB Integration

A previous post covered the subject of the initial LLDB and Dylan integration.

While much of this post discusses what we did to improve Dylan integration, the overall techniques are broadly useful for many languages. In fact, if anyone is interested, I am available for consulting on this topic. Just email me.

Catching Up

In the previous post, we talked about changing the base representation from void* to uintptr_t to work around a bug in LLDB. This was, in part, due to our goal to try to have the integration work with the currently shipping version of LLDB.

Unfortunately, this work won't be completed and merged. In separate developments, it has emerged that it can be unsafe to treat a pointer as an integer when working with a garbage collector. The details of this are quite interesting, so out of an abundance of caution, we will not be going down this route.

Expanding Objects

We are able to use the "pointer depth" parameter to commands like expression and frame variable to show the "expanded" form of objects:

(lldb) frame variable -P 1 T15
(dylan_value) T15 = 0x017e8000 {<buffer>} {
  [buffer-next] = 0x00000001 {<integer>: 0}
  [buffer-end] = 0x00000001 {<integer>: 0}
  [buffer-position] = 0x00000001 {<integer>: 0 ...

There are comments.

Dylan and Gradual Typing

As we look to the future and what we would like for Dylan to become and investigate how we would like for Dylan to evolve, it is helpful to look at some of the current work and how Dylan compares, where Dylan falls down and whether or not we can improve it.

One of those areas is in the guarantees offered by the type system. While Dylan is seen as a dynamic language, it has a number of features that help provide optional static type checking. As we'll see, there is a lot of room for improvement in this area.

In this post, using a missing compile-time warning as the driver, we'll walk through some details of the Dylan type system and then see how it differs from a gradually typed system. We'll see that type annotations are interpreted very differently under a gradual typing regime versus the Open Dylan compiler.

I've previously written a Type System Overview which may be useful, but hopefully this post can stand on its own.

One particularly interesting body of work is that on Gradual Typing. From What is Gradual Typing:

Gradual typing is a type system I [Jeremy Siek ...

There are comments.

Early Thoughts on Atom

I got a beta of the Atom editor soon after its initial release. The first thing that I decided to do was to add support for the Dylan language.

As we had a Textmate bundle available, this was an easy process:

apm init -p ~/.atom/packages/language-dylan \
  -c https://github.com/textmate/dylan.tmbundle

This was the start of a pleasurable experience with Atom. Creating and publishing a new package was easy and the command line tools were easy to use (and documented). After just a couple of minutes, I had a package that provided basic syntax highlighting and code folding for Dylan. It also worked on any Dylan file that I opened, unlike using our Textmate bundle in SublimeText which would occasionally hang on some files (while Textmate was fine). How's that for extensibility? :)

Next up, I decided to improve upon our new Dylan package. So I added support for auto-indenting and code snippets. This too was straight forward and easy.

As a comparison, I've also worked in the past with some other Dylan hackers on a plugin for IntelliJ to provide Dylan language support. While many things in IntelliJ are relatively easy to do, they all ...

There are comments.

Eliminating Copies in C-FFI

We've run into a situation fairly regularly when wrapping C libraries where we'd like to avoid making extra copies of data when passing it between Dylan and C code via the C-FFI. Related to this, we want to be able to use multiple types from the Dylan side of things such as <buffer>, <byte-vector> or <byte-string> without having to have separate code paths for each of them. Each of these classes can store raw byte data that we may want to share with other code.

An example that we'll work with in this discussion comes from bindings (in simplified and excerpted forms) for the LevelDB library:

extern void leveldb_put(
    leveldb_t* db,
    const leveldb_writeoptions_t* options,
    const char* key, size_t keylen,
    const char* val, size_t vallen,
    char** errptr);

<C-string>

The easiest way to bind leveldb_put would be to use <C-string>:

define C-function %leveldb-put
  input parameter db_ :: <leveldb-t*>;
  input parameter options_ :: <leveldb-writeoptions-t*>;
  input parameter key_ :: <C-string>;
  input parameter keylen_ :: <size-t>;
  input parameter val_ :: <C-string>;
  input parameter vallen_ :: <size-t>;
  output parameter errptr_ :: <char**>;
  c-name: "leveldb_put";
end;

However, this would mean that every key and value would have to be copied from an existing byte container (like <buffer>, <string> or <byte-vector>) to ...

There are comments.

Function Types and Dylan 2016

Moving towards Dylan 2016, the Dylan community would like to address some weaknesses in the language specification and what can be readily expressed in Dylan code. In this post, we'll look at function types as well as provide a brief introduction to some details of the type system implementation within the Open Dylan compiler.

Function Types

One of the big holes in the Dylan type system is the inability to specify function types. What this means is that you can only say that a value is of type <function> and can't indicate anything about the desired signature, types of arguments, return values, etc. This is unfortunate for a number of reasons:

Poor static type safety. The compiler can verify very little involving a function value. It can't warn when the wrong number of arguments or the wrong types of arguments are passed.
Less clear interfaces. The type signature of a function must be documented clearly rather than being expressed clearly within the code.
Optimization is more difficult. Since the compiler can't perform as many checks at compile time, more checks need to be performed at run-time, which limits the amount of optimization that can be performed ...

There are comments.

Getting Started with OpenDylan 2012.1

There are many exciting changes in the 2012.1 release of OpenDylan, but to me, some of the most exciting changes are the ones that make it easier to get a new project going and building. We'll be writing more about that in the future, but for now, let's look at how to get OpenDylan installed in the first place!

Installation

Linux and Mac OS X

Installing OpenDylan is pretty easy on Linux and Mac OS X.

First, you'll want to make sure that you have the basic developer tools like gcc installed.
On Gentoo, OpenDylan is available via the package system, so go ahead and install it that way. Packages for other operating systems are coming in the future and help is appreciated.
Otherwise, download and install a build of OpenDylan for your operating system. Put OpenDylan in a directory of its own, like /opt/opendylan-2012.1.
Make sure that the bin directory under the installation path is on your PATH. You can do this by placing a line in your .bashrc in your home directory like this:
```
export PATH=/opt/opendylan-2012.1/bin:$PATH
```
You can enter that command in your current shell as well ...

There are comments.

Hello, World!

define method say-hello () => ()
  format-out("Hello, world!\n");
end;

say-hello();

As part of the work that DataFueled is doing, we are sponsoring work in the Dylan programming language. This work is being done under the umbrella of the Dylan Foundry.

The Dylan Foundry is where we'll be blogging about the work that we're up to, what our future plans might be, and the libraries that we've developed.

The Dylan Foundry will also be assisting with issues surrounding the Dylan development tools as well as deploying Dylan applications into production in a server environment.

There are comments.