ServerKit

Overview

ServerKit is an SDK for building server programs on Linux. The focus is to help developers quickly build stable, efficient, scalable, and uniformly configured, installed, and executed server programs in the C programming language.

It consists of:

Server

A core program, which loads application-specific modules to perform unique server duties. This program is also responsible for all configuration parsing and creation of database connection pools, which are made available to the modules.

Libserver

A shared library, which provides base functionality for the server program, in addition to implementing the ServerKit API.

API

A light interface provided by libserver supplying the following (at this time):

Server program

Basic operation

The server program is very simple, it performs the following steps when executed:
  1. Acquire a personality specified via the SERVER_PERSONALITY_PATH environment variable.
  2. Open any modules found within ${SERVER_PERSONALITY_PATH}/modules serially, adding any per-module configuration options to the parser vocabulary under a section named after the modules name. Upon opening, the construct method of each module is invoked so any required early initialization can be performed.
  3. Open and parse the configuration file found at ${SERVER_PERSONALITY_PATH}/c11n using libconfuse and a vocabulary combining both core server program configuration options and module-specific options imported in the previous step.
  4. Check the process' UID, if it is root check that the configuration set executes_as_root to true. If not, abort, otherwise continue.
  5. Modify the process' argv[0] to reflect the configured identity.
  6. Create all database pools configured in the c11n file, making them available for lookup via their configured identifier by the loaded modules.
  7. Invoke the prestart method of each loaded module serially, note that this occurs before the server program makes any attempt to switch users. If a configuration section was found whose name matched the module being prestarted, the configuration data from that section is provided to the modules prestart method in the form of a libconfuse cfg_t type.
  8. If the configuration specified uid and/or gid settings, attempt to switch the uid and gid respectively, aborting on failure.
  9. Create an operator thread for each loaded module serially, the thread function used is the operator function in each module.
  10. Wait indefinitely for signals. Upon receipt of SIGUSR1 process-wide statistics are reported on stderr.

Note that the server program does not perform a "daemonize" step, this is due to being generally used under DJB's daemontools. A daemonize configuration option may be added in the future if there is sufficient demand.

In the final step process-wide statistics can be reported upon delivery of the SIGUSR1 signal.

Sample output upon delivery of SIGUSR1 to a personality running shttp with 30 threads during an apachebench run with 1000 concurrency:

ServerKit status report

 Thread pools
  Label                  MinThreads MaxThreads StackSize  ActiveBusy ActiveTotal
  ------------------------------------------------------------------------------
  shttp_workers          1          30         32768      30         30         
                  WaitCount   Channel
                  -------------------
                  30          recv
                  0           send
                  0           send headers
                  0           send autoidx
                  0           send constant response
                  0           sendfile
                  0           open
                  0           opendir
                  0           readdir

 Database connection pools
  Identifier             MinConns   MaxConns   ActiveBusy ActivTotal Errors
  ------------------------------------------------------------------------------

 Heaps
  Label             MaxUnits   Opts ActChnkSiz ActivUnits ActivChnks AllocUnits
  ------------------------------------------------------------------------------
  shttp_sessions    2000       ES-  640        1260       6          894        
  log_entry_heap    1000       ES-  10         0          0          0          


When run under daemontools this output would be logged locally presumably via multilog.

Utilizing the vtstats module these statistics can be displayed periodically through a virtual terminal on the system console as well.

Configuration

The configuration for a server instance (referred to as a personality) is stored in the c11n file located within the directory referenced by the SERVER_PERSONALITY_PATH environment variable.

Core configuration options

Name Type Description
database section A section which configures a database pool.
bypass_libc_syslog boolean Set to true to enable the direct udp-based remote syslog logging.

Defaults to false.

libc_syslog_bypass_address string Set to the address of the syslog collection host when bypass_libc_syslog is set to true.
executes_as_root boolean Set to true if this personality is to be started by root, if you enable this you should probably also use the uid & gid options. If left false and run by root server will abort with an error.

Defaults to false.

uid integer The uid to switch to after completing the prestart phase of startup, this will require root so executes_as_root needs to be true for this setting to work.

Defaults to false.

gid integer Same as uid but switches the gid.
identity string This setting gets appended to "server." in the argv[0] of the server process, so if you set this to say "pop", ps auxc will show "server.pop" for this personality. This is less important if you only run a single ServerKit personality, but when running multiple personalities on the same system it makes it simple to identify which process belongs to which personality.

Database section configuration options

Note: the title you provide for the database section is used as the database identifier, modules will use that identifier for looking up the database.
Name Type Description
user string User name for database authentication.
password string Password for database authentication.
name string Name of database to use.
host string Hostname or address of database server.
pool_min integer Minimum database connections to keep open in the database pool.
pool_max integer Maximum database connections to allow in the database pool.

Sample configuration of a theoretical POP3 server personality

/etc/server/pop/c11n (SERVER_PERSONALITY_PATH=/etc/server/pop)
identity = pop
bypass_libc_syslog = true
libc_syslog_bypass_address = 192.168.2.100

#Since pop binds to port 110 it needs root, binding to the port is part of
#the pop module prestart routine, so we setup a mail uid and gid for server
#to switch to post-prestart.  If you omitted the uid and gid lines, server
#would just continue running as root, a bad idea.
executes_as_root = true
uid = 30000
gid = 30000

database FOOFOO {
	name = maildatabase
	user = mailuser
	password = sekret
	host = localhost
	pool_max = 30
	pool_min = 2
}

#pop section vocabulary is added by a pop module, note FOOFOO is supplied
#to the pop module so it can lookup the database pool defined above.
pop {   
	database = FOOFOO 
	port = 110
	address = 230.120.11.4
	backlog = 32
	min_sessions = 10
	max_sessions = 2000
	max_threads = 30
	min_threads = 1
	maildrop_query_format_string = select maildrop from EMailUsers where EMailUsers.password="PASSWORD(%p)" and EMailUsers.user="%u"
}

API - the libserver library

Still working on the API documentation, there is complete but rough ASCII version available here. This is also included in the source archive referenced below, at doc/api.txt.

There is also an unfinished HTML version of the API reference available here


Source archives

Release date Tar.gz MD5 checksum
08-21-2014 serverkit-2.1.2.tar.gz c150cdf4f0ce6c6973e44cae80aef8bd
10-09-2008 serverkit-2.1.1.tar.gz 7e0c63d795b82b0052035e2b77deb1b5
02-11-2008 serverkit-2.1.0.tar.gz dd8913f783b071fac9472b541acb26c4
12-27-2007 serverkit-2.0.1.tar.gz aeea6da0a806413f3130ce690fa47465
12-04-2007 serverkit-2.0.0.tar.gz fddab8c9616fe78273cda25e227aa675
11-26-2007 serverkit-1.2.0.tar.gz d32773e1ac325fd69ce514b62fc5eefa

License

The current ServerKit release is licensed under the GNU Lesser General Public License version 2.1, a copy of which can be found here.

Previous releases have been licensed under the more restrictive GNU General Public License version 2, and before that the even more restrictive Honest Public License version 1.1.


Modules

httpx
HTTP/1.1 host-specific httpd switch
Postfix-policy
High-performance policy server for Postfix
R2d2
High-performance POP3 server
Vtstats
Personality statistics reporting module which outputs to a virtual terminal

Contributed modules

Shttp
Small and fast web server by Yingyuan Cheng

Contribute

I accept patches or feature requests, simply email me at the email address contained within the source tree. I am also always looking for testers so if you're willing to test new ServerKit modules or features, let me know.

Module authors interested in having their module(s) hosted here, or referenced off-site in the modules section should email me.

If you would like to contribute to the ServerKit project monetarily, please donate via the paypal button below.


External links

© 2008 - Vito Caputo