Rewrite SMLserver

[melsman]

The aim of this task is to make it possible to rewrite SMLserver to make it work with native x64 machine code. Such a rewrite allows for removing the KAM backend, which is currently not maintained (it does not work on 64-bit architectures). The strategy is to remove SMLserver entirely from the MLKit source repository and instead make it possible to craft an efficient web-server outside of the MLKit code base.

Design

Here is a possible design that will make it possible to craft an efficient web server in C:

1. Add the following functionality to an o-file generated by the MLKit compiler:

typedef struct {
  uintptr_t *sp_top, *sp_bot; 
  Region topregion;       // top-most region
  uintptr_t *exn_ptr;     // exn-pointer 
  char *exn_msg;          // NULL if no exception
} *Context;
typedef struct {..} *Snapshot; 
Context eval_libs (void);                // Evaluate all libraries in MLB-files (except last or the 
                                         //  multiple scripts in script...end constructs)
Context eval (Context c, char *punit);   // Evaluate a program unit (last entry in MLB
                                         //  file or entry in script...end construct) 
char **punits (int **n);
Snapshot snap (Context c);
Context resnap (Snapshot s, Context c);

With the above functionality, we can write a web-server in C that is multi threaded, makes use of pre-evaluated libraries (using the snapshot functionality), and pools database connections.

Features

1. The functionality can be tested in isolation.
2. The functionality can perhaps be used to write a REPL (given dynamic linking).
3. Contexts can be built for different thread stacks and each thread can snapshot the thread-specific context for the benefit of increased performance.
4. The snapshot functionality can be left out initially.
5. The web-server code (or the REPL code) can be held outside of the MLKit repository.

Tasks

• Remove the KAM (bytecode) backend and the apache2 module code.
• Set aside a register to hold the execution context pointer throughout execution. When the top-region is manipulated or accessed, it can be done through the context pointer; the context pointer can also be used for accessing the current exception pointer. Finally, it can be used for the parallelism mechanism (thread context).

[Munksgaard]

Does this need to be a part of the compiler? Couldn't it just be a separate library that allows you to build HTTP servers?

[melsman]

I'm investigating various designs. Let's discuss. It would be great if we could leave such a development outside of the mlkit repository. One problem is, however, that we need to provide guarantees that after a response has been served (for a request), memory is restored to before the request was received. I'm looking at various region-resetting strategies that we can expose as mlkit-functionality and then make use of to write high-performing web-servers. We also want the possibility of writing multi-threaded servers that can serve multiple requests simultaneously (using a thread pool). To begin with, however, we can live with a pre-forking version and we can also live with enabling the garbage collector.