Dylan and Gradual Typing

As we look to the future and what we would like for Dylan to become and investigate how we would like for Dylan to evolve, it is helpful to look at some of the current work and how Dylan compares, where Dylan falls down and whether or not we can improve it.

One of those areas is in the guarantees offered by the type system. While Dylan is seen as a dynamic language, it has a number of features that help provide optional static type checking. As we'll see, there is a lot of room for improvement in this area.

In this post, using a missing compile-time warning as the driver, we'll walk through some details of the Dylan type system and then see how it differs from a gradually typed system. We'll see that type annotations are interpreted very differently under a gradual typing regime versus the Open Dylan compiler.

I've previously written a Type System Overview which may be useful, but hopefully this post can stand on its own.

One particularly interesting body of work is that on Gradual Typing. From What is Gradual Typing:

Gradual typing is a type system I [Jeremy Siek ...

There are comments.

Function Types and Dylan 2016

Moving towards Dylan 2016, the Dylan community would like to address some weaknesses in the language specification and what can be readily expressed in Dylan code. In this post, we'll look at function types as well as provide a brief introduction to some details of the type system implementation within the Open Dylan compiler.

Function Types

One of the big holes in the Dylan type system is the inability to specify function types. What this means is that you can only say that a value is of type <function> and can't indicate anything about the desired signature, types of arguments, return values, etc. This is unfortunate for a number of reasons:

Poor static type safety. The compiler can verify very little involving a function value. It can't warn when the wrong number of arguments or the wrong types of arguments are passed.
Less clear interfaces. The type signature of a function must be documented clearly rather than being expressed clearly within the code.
Optimization is more difficult. Since the compiler can't perform as many checks at compile time, more checks need to be performed at run-time, which limits the amount of optimization that can be performed ...

There are comments.

HTTP and Stream Processing

As we discussed in the previous post, we are thinking about a new design and implementation for the streams library in Open Dylan.

While the examples in this post are in Dylan and are using code from our HTTP server, these issues exist in HTTP frameworks in other languages. The code should be clear enough that little to no Dylan knowledge is required to understand the points being made here.

What does this have to do with HTTP? There are several pain points in our HTTP stack as it is currently written:

Requests are read in their entirety into memory, so a large request (such as a file upload) takes a significant amount of memory.
Responses often buffer their entire output in memory as well.
Because of the use of the existing streams library, we don't handle non-blocking sockets and require a thread per socket.
We don't have a good model for handling long-lasting connections such as might be used with Server Sent Events or WebSockets without tying up a thread for the duration of the socket being open.

We don't know yet what the new streams API will look like, but we can look at our ...

There are comments.

Beginning to Rethink Streams

Dylan's current streams library has served us moderately well over the years. However, it has some issues which can be addressed by a new design, expanding the range of problems for which it is suited.

How things are now

According to the current streams library's documentation, the design goals were:

A generic, easy-to-use interface for streaming over sequences and files. The same high-level interface for consuming or producing is available irrespective of the type of stream, or the types of the elements being streamed over.
Efficiency, especially for the common case of file I/O.
Access to an underlying buffer management protocol.

One of the things it was explicitly not designed to handle was, again, according to the design goals in the documentation:

A comprehensive range of I/O facilities for using memory-mapped files, network connections, and so on.

Unfortunately, the primary interface to our current network library is based on these very streams for which network connections were not a design goal. While this works in practice, it imposes some important limitations on our networking code. The biggest of these is that sockets can not be non-blocking as it is expected that reads and writes will complete ...

There are comments.