Fun with embeddability
Embeddability? I am quite certain this is in fact an english word, but you probably won’t find it among the other -ilities of qualities, software might want to address. Definitely among it, there are some boring and pretty self-explanatory ones like maintainability, even some dogmatic ones like correctness[1], but fortunately also funnier ones like extensibility.
And like to often, how a certain quality is best achieved depends on a plethora of things, but according to Wikipedia, one to archive extensibility is to use scripting languages and everything finally comes together: They can be quite embeddable.
So in case you have some time to kill, join me on a lengthy journey through 20+ years of personal FOSS-history. We are having a look at different approaches of embeddings and also see why this is always great idea - plus there are memes.
Barebone &
Unbeknownst to my past self, I made my first experience with this kind of extensibility in 2004, when I started my long journey with Xlib. During that time I started a project called deskbar with the lofty goal to print system information like cpu load, battery usage etc. directly onto the root window of the X session. There were plenty of alternatives like GKrellM readily available, but who in their right mind prefers pre-built stuff over rolling your own[2]?
The initial idea was just to include everything in one binary, but I quickly discovered the ergonomics of re-compiling and shipping everything together are annoying and I switched to a simple plugin system.
Screenshots first &
I would have loved to show some screenshots of deskbar in action here, but unfortunately after messing with the infamous Autotools and trying to compile old C-code with a modern compiler this is as far as I got[3]:
$ ./configure && make
deskbar 0.1
-----------------
Build with ZLIB support.......: yes
Build with PNG support........: yes
Plugins:
Common Plugins................: Clock CPU Date
Battery Plugin................: no
XMMS Plugin...................: no (1)
BMP Plugin....................: no (2)
Debug Plugin..................: no
The binary will be installed in /usr/local/bin,
the lib in /usr/local/lib and the plugins
in /usr/local/lib/deskbar.
Try make now, good luck!
make all-recursive
make[1]: Entering directory '/home/unexist/build/deskbar-0.1'
# --- %< --- snip --- %< ---
/bin/bash ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -I/usr/include -I/usr/include -MT htable.lo -MD -MP -MF .deps/htable.Tpo -c -o htable.lo htable.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -I/usr/include -I/usr/include -MT htable.lo -MD -MP -MF .deps/htable.Tpo -c htable.c -fPIC -DPIC -o .libs/htable.o
In file included from htable.c:2:
/usr/include/string.h:466:13: error: storage class specified for parameter 'explicit_bzero'
466 | extern void explicit_bzero (void *__s, size_t __n) __THROW __nonnull ((1)) (3)
| ^~~~~~~~~~~~~~
/usr/include/string.h:471:14: error: make[2]: *** [Makefile:457: htable.lo] Error 1
make[2]: Leaving directory '/home/unexist/build/deskbar-0.1/libdeskbar'
make[1]: *** [Makefile:479: all-recursive] Error 1
make[1]: Leaving directory '/home/unexist/build/deskbar-0.1'
make: *** [Makefile:374: all] Error 2storage class specified for parameter 'strsep'
471 | extern char *strsep (char **__restrict __stringp,
| ^~~~~~
/usr/include/string.h:478:14: error: storage class specified for parameter 'strsignal'
478 | extern char *strsignal (int __sig) __THROW;
| ^~~~~~~~~
# --- %< --- snip --- %< ---
make[2]: *** [Makefile:457: htable.lo] Error 1
make[2]: Leaving directory '/home/unexist/build/deskbar-0.1/libdeskbar'
make[1]: *** [Makefile:479: all-recursive] Error 1
make[1]: Leaving directory '/home/unexist/build/deskbar-0.1'
make: *** [Makefile:374: all] Error 2| 1 | X Multimedia System (XMMS) |
| 2 | I can only guess what is was supposed to do, since the plugin is just an empty stub that returns NULL |
| 3 | Yes, oh well… |
Nevertheless, this output clearly proves there has been a plugin system with conditional compilation, which bases solely on linking magic, and we have to move on.
I dug a bit further and stumbled upon my old project page on SourceForge, which luckily still provides sftp access to the project page:
And with even more luck, although the page is a bit unfinished, the file listing included screenshots:
Runtime loading &
Everything in C is a bit more complicated, so let us ignore the scary memory handling and just talk about the two interesting calls dlopen and dlsym:
DbPlugElement *element = NULL;
element = (DbPlugElement *) malloc (sizeof (DbPlugElement));
snprintf (buf, sizeof (buf), "%s/%s.so", PLUGIN_DIR, file);
element->handle = dlopen (buf, RTLD_LAZY); (1)
if ((err = dlerror ())) (2)
{
db_log_err ("Cannot load plugin `%s'\n", file);
db_log_debug ("dlopen (): %s\n", err);
free (element);
return;
}
/* Get entrypoint and call it */
entrypoint = dlsym (element->handle, "db_plug_init"); (3)
element->data = (*entrypoint) (); (4)| 1 | Load the named shared object from path |
| 2 | There is apparently a third call, but rarely mentioned at all |
| 3 | Find the address of a named entrypoint |
| 4 | Execute the entrypoint for profit |
The entrypoint here is quite interesting, since the main application cannot know what is included
in the plugin or even what is exported.
Following the idea of Convention-over-configuration, the defined contract here
expects a symbol named db_plug_init inside a plugin, which is called on load and must return
a pointer to an initialized struct of type DBPlug:
static DbPlug plugin =
{
"Battery", /* Plugin name */
battery_create, /* Plugin create function */
battery_update, /* Plugin update function */
battery_destroy, /* Plugin destroy function */
&data, /* Plugin data */
NULL, /* Plugin format */
3600 /* Plugin update interval */
};
DbPlug *
db_plug_init (void)
{
plug = &plugin;
return (&plugin); (1)
}| 1 | Pass the local address back to the main application |
Once loaded the plugin is called in the given interval and can exchange data with the main application.
void
battery_update (void)
{
int capacity = 0,
percent = 0;
char buf[100], state[20];
/* Get battery info */
if (!fd1)
{
snprintf (buf, sizeof (buf), "/proc/acpi/battery/BAT%d/state", bat_slot); (1)
fd1 = fopen (buf, "r");
memset (buf, 0, sizeof (buf));
}
else
fseek (fd1, 0, SEEK_SET);
/* --- %< --- snip --- %< --- */
}| 1 | Here the battery plugin checks the battery values from the ACPI interface |
Allowing contribution this way is really easy and powerful, but like so often comes with a catch. Segmentation faults, the bane of software engineering, don’t make halt inside plugins like they should, but they wipe the board and kill the entire application.
I think Torvalds nailed it perfectly and I agree this should never happen:
Mauro, SHUT THE FUCK UP!
WE DO NOT BREAK USERSPACE!
https://www.shutupmauro.com/
I am kind of surprised how far I went in trying to keep problems in the plugin at bay. The original project included memory management[4] for plugins and also applied the next two calls I’d like to demonstrate next.
Error handling &
Handling segmentation faults properly is really difficult and the common sense is normally catch them and exit gracefully when possible. Still, there are cases when faults can be safely ignored and a plugin interface is a paragon for this.
This can be done with the pair of setjmp and longjmp, which behave for most practical senses like a goto on steroids:
static int¬ 26 save_call (DbPlugElement *element,¬
save_call (DbPlugElement *element,
DbPlugFunc plugfunc
const char *name)
{
if (plugfunc)
{
if (setjmp (env) == 0) (1)
plugfunc ();
else
{
db_log_mesg ("Ayyyee! Segmentation fault in plugin %s!\n", element->data->name); (2)
db_log_debug ("Call to %s () failed\n", name);
db_plug_unload (element);
return (1);
}
}
return (0);
}| 1 | Save stack and instruction pointer for later use when it is for the first time; otherwise ditch the plugin |
| 2 | Well, different times back then.. |
When the application receives the bad signal SISEGV, it checks if there are stored stack
and instruction values and rewinds the stack accordingly:
static void
sig_handler (int sig)
{
switch (sig)
{
case SIGSEGV:
longjmp (env, 1); (1)
db_log_debug ("Something went wrong! Segmentation fault!\n");
db_sig_destroy ();
abort ();
break;
/* --- %< --- snip --- %< --- */
}| 1 | Check the values and pass control if necessary; otherwise just bail out |
Scripting languages &
Three years later in 2007 I continued on building upon my Xlib skills and started my long-lasting project subtle.
Over the years there have been many major breaking changes, from the initial design to the state it currently is in. Two of the post-related changes were the integration of the scripting language Lua and its later replacement with Ruby after a few years in this glorious issue #1.
Integrating Lua &
I am not entire sure where I picked Lua up, but I never played WoW so probably from somewhere else and I can only talk about the state and API from back then.
Adding a scripting language solves quite a few problems:
-
File loading and parsing can be offloaded to the language core
-
The language itself comes with a basic subset of things you can do with it
-
Bonus: Config handling can also be directly offloaded
Time for Screenshots &
My attempt of trying to compile the project and provide an actual screenshot this time ended quickly as well:
$ ./configure && make
# --- %< --- snip --- %< ---
subtle 0.7b
-----------------
Binary....................: /usr/local/bin
Sublets...................: /usr/local/share/subtle
Config....................: /usr/local/etc/subtle
Debugging messages........:
Try make now, good luck!
make all-recursive
make[1]: Entering directory '/home/unexist/build/subtle-0.7b'
Making all in src
# --- %< --- snip --- %< ---
if gcc -DHAVE_CONFIG_H -I. -I. -I.. -I.. -g -O2 -I/usr/include/lua5.1 -g -O2 -MT subtle-event.o -MD -MP -MF ".deps/subtle-event.Tpo" -c -o subtle-event.o `test -f 'event.c' || echo './'`event.c; \
then mv -f ".deps/subtle-event.Tpo" ".deps/subtle-event.Po"; else rm -f ".deps/subtle-event.Tpo"; exit 1; fi
event.c: In function ‘subEventLoop’:
event.c:352:57: error: implicit declaration of function ‘subSubletSift’; did you mean ‘subSubletKill’? [-Wimplicit-function-declaration]
352 | subSubletSift(1);
| ^~~~~~~~~~~~~
| subSubletKill
make[2]: *** [Makefile:310: subtle-event.o] Error 1
make[2]: Leaving directory '/home/unexist/build/subtle-0.7b/src'
make[1]: *** [Makefile:233: all-recursive] Error 1
make[1]: Leaving directory '/home/unexist/build/subtle-0.7b'
make: *** [Makefile:171: all] Error 2This is kind of embarrassing for an official release and really have to question the quality in retrospect, but this won’t stop us now.
After a dive into the code there were some obviously problems and also blatant oversights and if you are interested in the shameful truth here is silly patch:
And without further ado here is finally the screenshot of the scripting part in action, before we dive into how this is actually done under the hood:
Runtime loading &
Starting with the easy part, offloading the config handling was one of the first things I did and this made a config like this entirely possible:
-- Options config
font = {
face = "lucidatypewriter", -- Font face for the text
style = "medium", -- Font style (medium|bold|italic)
size = 12 -- Font size
}
-- Color config
colors = {
font = "#ffffff", -- Color of the font
border = "#ffffff", -- Color of the border/tiles
normal = "#CFDCE6", -- Color of the inactive windows
focus = "#6096BF", -- Color of the focussed window
shade = "#bac5ce", -- Color of shaded windows
background = "#596F80" -- Color of the root background
}
-- --- %< --- snip --- %< ---Essentially the C API of Lua is a stack machine and the interaction with is through pushing and popping values onto and from the stack.[6]
I’ve removed a bit of the fluff and checks upfront, so we can have a quick glance at the config loading and jump further into nitty-gritty details:
/* --- %< --- snip --- %< --- */
subLogDebug("Reading `%s'\n", buf);
if(luaL_loadfile(configstate, buf) || lua_pcall(configstate, 0, 0, 0)) (1)
{
subLogDebug("%s\n", (char *)lua_tostring(configstate, -1));
lua_close(configstate);
subLogError("Can't load config file `%s'.\n", buf);
}
/* --- %< --- snip --- %< --- */
/* Parse and load the font */¬
face = GetString(configstate, "font", "face", "fixed"); (2)
style = GetString(configstate, "font", "style", "medium");
size = GetNum(configstate, "font", "size", 12);
/* --- %< --- snip --- %< --- */| 1 | Internal calls to load the config file and just execute it in a safe way pcall |
| 2 | Once everything is stored inside configstate we fetch required values |
#define GET_GLOBAL(configstate) do { \ (1)
lua_getglobal(configstate, table); \ (2)
if(lua_istable(configstate, -1)) \
{ \
lua_pushstring(configstate, field); \ (3)
lua_gettable(configstate, -2); \
} \
} while(0)
/* --- %< --- snip --- %< --- */
static char *
GetString(lua_State *configstate,
const char *table,
const char *field,
char *fallback)
{
GET_GLOBAL(configstate);
if(!lua_isstring(configstate, -1)) (4)
{
subLogDebug("Expected string, got `%s' for `%s'.\n", lua_typename(configstate, -1), field);
return(fallback);
}
return((char *)lua_tostring(configstate, -1)); (5)
}| 1 | Blocks in C macros require this fancy hack; probably best to skip over it |
| 2 | We check and fetch a table[7] |
| 3 | Push the string onto the current stack |
| 4 | Pull the value with index -2 from the stack |
| 5 | And convert it to our desired format |
Error handling &
Loading of plugins at runtime is basically the same as loading the config upfront, so let us just move on to error handling, which is slightly more interesting. It is probably no surprise, but the API is quite rudimentary and the handling of the stack and calls in case of an actual error is up to person to embed the engine.
Before we can see how this is done, let us quickly check how our battery plugin evolved from the arcane version in C to the Lua glory. First of all, plugins have been rebranded to sublets[9] and it (at least to me) became a bit more readable:
-- Get remaining battery in percent
function battery:meter() (1)
local f = io.open("/proc/acpi/battery/BAT" .. battery.slot .. "/state", "r")
local info = f:read("*a")
f:close()
_, _, battery.remaining = string.find(info, "remaining capacity:%s*(%d+).*")
_, _, battery.rate = string.find(info, "present rate:%s*(%d+).*")
_, _, battery.state = string.find(info, "charging state:%s*(%a+).*")
return(math.floor(battery.remaining * 100 / battery.capacity))
end| 1 | The : here is used as a kind of namespace separator and should be read as a global table
called battery with the entry meter. |
Once the sublet is loaded and initialized we can just call it analogue to our save_call from
before:
void
subLuaCall(SubSublet *s)
{
if(s)
{
lua_settop(state, 0); (1)
lua_rawgeti(state, LUA_REGISTRYINDEX, s->ref);
if(lua_pcall(state, 0, 1, 0)) (2)
{
if(s->flags & SUB_SUBLET_FAIL_THIRD) (3)
{
subLogWarn("Unloaded sublet (#%d) after 3 failed attempts\n", s->ref);
subSubletDelete(s);
return;¬
}
else if(s->flags & SUB_SUBLET_FAIL_SECOND) s->flags |= SUB_SUBLET_FAIL_THIRD;
else if(s->flags & SUB_SUBLET_FAIL_FIRST) s->flags |= SUB_SUBLET_FAIL_SECOND;
subLogWarn("Failed attempt #%d to call sublet (#%d).\n",
s->flags & SUB_SUBLET_FAIL_SECOND) ? 2 : 1, s->ref);
}
switch(lua_type(state, -1)) (4)
{
case LUA_TNIL: subLogWarn("Sublet (#%d) does not return any usuable value\n", s->ref); break;
case LUA_TNUMBER: s->number = (int)lua_tonumber(state, -1); break;
case LUA_TSTRING:
if(s->string) free(s->string);
s->string = strdup((char *)lua_tostring(state, -1));
break;
default:
subLogDebug("Sublet (#%d) returned unkown type %s\n", s->ref, lua_typename(state, -1));
lua_pop(state, -1);
}
}
}
}| 1 | A bit stack setup and retrieval via upfront |
| 2 | Here we call lua_pcall, which abstracts and hides the nasty setjmp and longjmp
handling from us |
| 3 | Looks like I discovered bitflags there and utilized it for error handling |
| 4 | Type handling for a more generic interface |
Integrating Ruby &
Moving fast-forward with subtle, I’ve replaced Lua with Ruby after a while and this is an entirely different way of integration, but let us just stick to our recipe here and do one mistake after another.
Time for Screenshots &
This time we can keep it short and simple, since I am using it on a daily on several devices and can easily provide screenshots without messing with outdated and broken builds[10].
Runtime loading &
So when we finally start subtle everything comes together, and we see known pieces from other projects before, which is more or the less entirely the same.
Just feel free to skip the next few listings and join us later and for the ones remaining..
Just kidding, here is the promised triplet of loading info, config and the battery thingy:
$ subtle -d :2 -c subtle.rb -s sublets
subtle 0.12.6606 - Copyright (c) 2005-present Christoph Kappel
Released under the GNU General Public License
Compiled for X11R0 and Ruby 2.7.8
Display (:2) is 640x480
Running on 1 screen(s)
ruby: warning: already initialized constant TMP_RUBY_PREFIX
Reading file `subtle.rb'
Reading file `sublets/battery.rb'
Loaded sublet (battery)
Reading file `sublets/fuzzytime.rb'
Loaded sublet (fuzzytime)The config looks a bit different, mainly because we are now using a custom DSL, but we are going to cover this part in detail shortly, promised.
# Style for all style elements
style :all do (1)
foreground "#757575"
background "#202020"
icon "#757575"
padding 0, 3
font "-*-*-*-*-*-*-14-*-*-*-*-*-*-*"
#font "xft:sans-8"
end
# Style for the all views
style :views do (2)
# Style for the active views
style :focus do
foreground "#fecf35"
end
# --- %< --- snip --- %< ---
end| 1 | Ruby is famous for metaprogramming and we obviously make have use of it here |
| 2 | Styles are a CSS-like way of configuring colors in subtle - batteries and inheritance included |
And lastly, a quick glimpse into the battery sublet, which naturally also makes use of the mentioned DSL:
on :run do |s|
begin (1)
now = IO.readlines(s.now).first.to_i
state = IO.readlines(s.status).first.chop
percent = (now * 100 / s.full).to_i
# --- %< --- snip --- %< ---
# Select icon for state
icon = case state (2)
when "Charging" then :ac
when "Discharging"
case percent
when 67..100 then :full
when 34..66 then :low
when 0..33 then :empty
end
when "Full" then :ac
else :unknown
end
s.data = "%s%s%s%d%%" % [
s.color_icon ? s.color : s.color_def, s.icons[icon],
s.color_text ? s.color : s.color_def, percent
]
rescue => err # Sanitize to prevent unloading
s.data = "subtle"
p err
end
end| 1 | Ruby comes with exception handling and this eases the whole scripting part greatly |
| 2 | Aww, this kind of reminds of Rust <3 |
So when we talk about metaprogramming, what exactly is different here? If you have a closer look at the previous examples, we mostly defined data structures and methods there, which were later collected during load and/or actually called by the host application. In other words our scripts defined an API according to the rules of the host application, which then runs it. With metaprogramming now, we turn this around and define methods and provide an API for our scripts to let them call it.
The Ruby integration in subtle is quite vast, and there are many cool things I’d like to show, but time is precious, as is our attention span and sobriety is in order. So we have to cut a few corners here and there and follow loads of indirection abstraction, but I think we better stay with the styles excerpt from above.
Loading styles from the config consists of following basic building blocks:
void subRubyInit(void) {
VALUE config = Qnil, options = Qnil, sublet = Qnil;
/* --- %< --- snip --- %< --- */
config = rb_define_class_under(mod, "Config", rb_cObject); (1)
/* Class methods */¬
rb_define_method(config, "style", RubyConfigStyle, 1); (2)
/* --- %< --- snip --- %< --- */
}| 1 | Define a holding class for our method definition |
| 2 | Define the actual method style and bind it to RubyConfigStyle |
void subRubyLoadConfig(void) {
VALUE klass = Qnil;
/* Load supplied config or default */
klass = rb_const_get(mod, rb_intern("Config")); (1)
config_instance = rb_funcall(klass, rb_intern("new"), 0, NULL);
rb_gc_register_address(&config_instance); (2)
if (Qfalse == RubyConfigLoadConfig(config_instance,¬
rb_str_new2(subtle->paths.config ? subtle->paths.config : PKG_CONFIG))) { (3)
subSubtleFinish();
exit(-1);¬
} else if (subtle->flags & SUB_SUBTLE_CHECK) {
printf("Syntax OK\n");
}
/* --- %< --- snip --- %< --- */
}| 1 | Call back our config class and create a new instance |
| 2 | Take care, that the internal garbage collector doesn’t get rid of it |
| 3 | Wrap it again and continue in the next snippet |
static VALUE RubyConfigLoadConfig(VALUE self, VALUE file) {
/* --- %< --- snip --- %< --- */
printf("Reading file `%s'\n", buf);
/* Carefully load and eval file */
rargs[0] = rb_str_new2(buf);
rargs[1] = self;
rb_protect(RubyWrapEvalFile, (VALUE) &rargs, &state); (1)
if (state) {
subSubtleLogWarn("Cannot load file `%s'\n", buf);
RubyBacktrace();
return Qfalse;
}
return Qtrue;
} /* }}} */¬| 1 | Ruby uses its own version of setjmp and longjmp, so wrap everything up and pass it over |
/* RubyWrapEvalFile */
static VALUE RubyWrapEvalFile(VALUE data) {
VALUE *rargs = (VALUE *) data, rargs2[3] = {Qnil};
/* Wrap data */
rargs2[0] = rb_funcall(rb_cFile, rb_intern("read"), 1, rargs[0]); (1)
rargs2[1] = rargs[0];
rargs2[2] = rargs[1];
rb_obj_instance_eval(2, rargs2, rargs[1]); (2)
return Qnil;
} /* }}} */| 1 | Then we use the internal symbol rb_cFile to call File#read on our arguments |
| 2 | And then a final eval - see we adhere to the motto! |
Error handling &
Actually we covered this already in the previous section, so nothing to be done here and we better hurry.
Integrating JavaScript &
During the 2020s lots of weird things happened and I was forced into my own sort of crisis being stuck with and on a macOS for some years. Needless to say the window management there totally annoyed me and I started another highly ambitious project aptly named touchjs.
There, I tied the new[11] Touch Bar, basic window management via Accessibility API and a JavaScript integration based on duktape together.
Time for Screenshots &
Unfortunately we are back at build problems: Somehow and totally unexplainable to me, I forgot to check in some essential headers to the project which led to a full-halt:
$ make
clang -c -mmacosx-version-min=10.12 -x objective-c src/touchjs.m -o src/touchjs.o
src/touchjs.m:17:10: fatal error: 'delegate.h' file not found
17 | #include "delegate.h"
| ^~~~~~~~~~~~
1 error generated.
make: *** [src/touchjs.o] Error 1$ make
clang -c -mmacosx-version-min=10.12 -x objective-c src/touchbar.m -o src/touchbar.o
src/touchbar.m:57:23: error: use of undeclared identifier 'kQuit'
57 | [array addObject: kQuit];
| ^
src/touchbar.m:149:21: warning: class method '+presentSystemModalTouchBar:systemTrayItemIdentifier:' not found (return type defaults to 'id') [-Wobjc-method-access]
149 | [NSTouchBar presentSystemModalTouchBar: self.groupTouchBar
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
150 | systemTrayItemIdentifier: kGroupButton];
| ~~~~~~~~~~~~~~~~~~~~~~~~
src/touchbar.m:150:39: error: use of undeclared identifier 'kGroupButton'; did you mean 'kGroupIcon'?
150 | systemTrayItemIdentifier: kGroupButton];
| ^~~~~~~~~~~~
| kGroupIcon
# --- %< --- snip --- %< ---Fixing something that isn’t there is quite difficult, and it took me some time and reading reference manuals to understand what I actually have to restore. When I made the first progress there, I suddenly remembered I have in fact a backup of the MacBook Pro from back then.
Although I really had fun playing with it, there has never been a real usage of the project. Luckily I already worked test-driven, so I can show off these test scripts written in JavaScript[12] along with some resulting shots of the Touch Bar:
/* WM */
var wm = new TjsWM(); (1)
tjs_print("wm: trusted=" + wm.isTrusted());
/* Events */
wm.observe("win_open", function (win) {
tjs_print("Open: name=" + win.getTitle() + ", id=" + win.getId() + ", frame=" + win.getFrame()); (2)
});| 1 | Highly ambitious as I’ve promised |
| 2 | Well, just print some details of windows in the normal state |
And some more with actual UI elements:
var b = new TjsButton("Test")
.setBgColor(255, 0, 0)
.bind(function () {
tjs_print("Test");
});
/* Attach */
tjs_attach(b);
/* --- %< --- snip --- %< --- */
var b4 = new TjsButton("Exec")
.setBgColor(255, 0, 255)
.bind(function () {
var c1 = new TjsCommand("ls -l src/");
tjs_print(c1.exec().getOutput());
});
var s1 = new TjsSlider(0)
.bind(function (value) {
tjs_print(value + "%");
rgb[idx] = parseInt(255 * value / 100);
l1.setFgColor.apply(l1, rgb);
});
var sc1 = new TjsScrubber()
.attach(b1)
.attach(b2)
.attach(b3)
.attach(b4);
/* Attach */
tjs_attach(l1);
tjs_attach(sc1);
tjs_attach(s1);
Runtime loading &
We could go into details here how the loading process and error handling works in Obj-C, but I ultimately replaced Obj-C with Rust and later on also got rid of the macbook, so interested in how this can be done in Rust? Bet you are!
Around 2023 I started another pet project under the nice moniker rubtle. I can only guess what my plans for it were, but might have been glimpse into the future, but more on that later, when we talk about the last project of this blog post. Whatever the plans were, I didn’t spend too much time on it and rubtle isn’t polished in any sense.
So why do I mention it at all you might ask? Within rubtle I followed a different approach we haven’t covered so far. Instead of inventing an own API, I created bridge[14] and allowed the scripts to interact directly with the underlying engine:
fn main() {
let args: Vec<String> = env::args().collect();
if 1 < args.len() {
let contents = fs::read_to_string(&args[1]); (1)
let rubtle = Rubtle::new();
init_global(&rubtle);
init_rubtle(&rubtle); (2)
match contents {
Ok(val) => rubtle.eval(&val),
Err(_) => eprintln!("File read failed"),
}
} else {
println!("Usage: {}: <JSFile>", args[0]);
}
}| 1 | Just file loading, no surprises here yet |
| 2 | Now it is getting exciting - off to the next listing! |
fn init_rubtle(rubtle: &Rubtle) {
#[derive(Default)]
struct UserData {
value: i32,
};
let mut object = ObjectBuilder::<UserData>::new() (1)
.with_constructor(|inv| {
let mut udata = inv.udata.as_mut().unwrap();
udata.value = 1;
})
.with_method("inc", |inv| -> CallbackResult<Value> { (2)
let mut udata = inv.udata.as_mut().unwrap();
udata.value += 1;
Ok(Value::from(udata.value))
})
/* --- %< --- snip --- %< --- */
.build();
rubtle.set_global_object("Rubtle", &mut object); (3)
}| 1 | Using the builder pattern was really a fight back then to me |
| 2 | Here we are assembling an object by adding some values and methods |
| 3 | And we register this as a global object |
Compiled, ready and armed we can feed this fancy test script into it:
var rubtle = new Rubtle();
rubtle.set(5);
assert(5, rubtle.get(), "Damn"); (1)
rubtle.inc();
assert(6, rubtle.get(), "Damn");
print(rubtle.get()) (2)| 1 | Seriously no idea.. |
$ RUSTFLAGS=-Awarnings cargo run -- ./test.js
Compiling rubtle-duktape v0.1.0 (/home/unexist/projects/rubtle/rubtle-duktape) (1)
Compiling rubtle-lib v0.1.0 (/home/unexist/projects/rubtle/rubtle-lib) (2)
Compiling rubtle v0.1.0 (/home/unexist/projects/rubtle/rubtle)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.03s
Running `target/debug/rubtle ./test.js`
<JS> "6" (3)Error handling &
Inside rubtle-lib is lots of scary stuff and I don’t want to scare away my dear readers, so the next excerpt is boiled down and absolutely safe to handle:
impl Rubtle {
/* --- %< --- snip --- %< --- */
///
/// Set value to context and assign a global reachable name
///
/// # Arguments
///
/// `name`- Name of the value
/// `rval` - The actual value
///
/// # Example
///
/// use rubtle_lib::{Rubtle, Value};
///
/// let rubtle = Rubtle::new();
/// let rval = Value::from(4);
///
/// rubtle.set_global_value("rubtle", &rval);
///
pub fn set_global_value(&self, name: &str, rval: &Value) {
unsafe {
let cstr = CString::new(to_cesu8(name));
match cstr {
Ok(cval) => {
self.push_value(rval); (1)
ffi::duk_require_stack(self.ctx, 1); (2)
ffi::duk_put_global_lstring(
self.ctx,
cval.as_ptr(),
cval.as_bytes().len() as u64,
);
}
Err(_) => unimplemented!(),
}
}
}
/* --- %< --- snip --- %< --- */
}
unsafe extern "C" fn fatal_handler(_udata: *mut c_void, msg: *const c_char) { (3)
let msg = from_cesu8(CStr::from_ptr(msg).to_bytes())
.map(|c| c.into_owned())
.unwrap_or_else(|_| "Failed to decode message".to_string());
eprintln!("Fatal error from duktape: {}", msg);
process::abort();
}| 1 | Did I mention duktape is also a stack machine and exposes this type of API? |
| 2 | This is a similar handling of the stack like we’ve seen in Lua |
| 3 | And we essentially provide a error handler to trap errors when they occur |
Recap &
| Ease of use | Richness of API | Language agnostic | Error handling | Performance |
|---|---|---|---|---|
Low to complex; depends on the chosen language |
You provide the API, can be full-fledged interface but also just a simple bridge |
Absolutely, there usually are many bindings and they can also be created with FFI |
Depends a bit on the language, but ranges from easy to complex |
Another thing that depends on the embedder and the embeddee[15] |
Webassembly &
I think Webassembly is one of the more interesting topics from the web technology cosmos. It allows to create binaries from a plethora of languages and to run them mostly at full speed directly inside stack-based[16] virtual machines. Originally meant for embedding in the web, it can also be utilized in other types of software and provide more flexibility when required, but also raw speed on execution.
There is lots of movement and things might break change quite often, but frameworks provide stability here where required. Extism is such a framework and also the one used in my latest project subtle-rs as a re-write in Rust and the spiritual successor of subtle.
Screenshots first &
subtle-rs is under active development and therefore a piece of a cake to demonstrate it:
Runtime loading &
In contrast to the other projects, subtle-rs doesn’t use a scripting language as its config, but relies on a simple TOML file. Therefore it doesn’t make sense to go into detail here. If you still are curious just check the repository: https://github.com/unexist/subtle-rs/blob/master/subtle.toml
Startup and loading the four existing plugins works like a charm:
$ cargo run -- -d :2 --config-file ./demo.toml
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.12s
Running `target/debug/subtle-rs -d ':2' --config-file ./demo.toml`
[2026-01-25T15:48:20Z INFO subtle_rs] Reading file `"./demo.toml"'
[2026-01-25T15:48:20Z INFO subtle_rs] subtle-rs 0.1.0 - Copyright (c) 2025-present Christoph Kappel <christoph@unexist.dev>
[2026-01-25T15:48:20Z INFO subtle_rs] Released under the GNU GPLv3
[2026-01-25T15:48:20Z INFO subtle_rs] Compiled for X11
[2026-01-25T15:48:20Z INFO subtle_rs::display] Display (:2) is 640x480
[2026-01-25T15:48:20Z INFO subtle_rs::plugin] Loaded plugin (time) (1)
[2026-01-25T15:48:20Z INFO subtle_rs::plugin] Loaded plugin (fuzzytime) (2)
[2026-01-25T15:48:20Z INFO subtle_rs::plugin] Loaded plugin (mem) (3)
[2026-01-25T15:48:20Z INFO subtle_rs::plugin] Loaded plugin (battery) (4)
[2026-01-25T15:48:20Z INFO subtle_rs::screen] Running on 1 screen(s)| 1 | Written in Zig - https://github.com/unexist/subtle-rs/tree/master/plugins/time |
| 2 | Written in Go - https://github.com/unexist/subtle-rs/tree/master/plugins/fuzzytime |
| 3 | Written in JavaScript - https://github.com/unexist/subtle-rs/tree/master/plugins/mem |
| 4 | Written in Rust - https://github.com/unexist/subtle-rs/tree/master/plugins/battery |
Under the hood the integration works a bit different to the embeddings before. The plugins run alone and isolate in their virtual machine and all capabilities beside the ones provided by the language and the wasm target must be exported by the embedding host. On the other side, the plugin can also define methods, export them and they can in turn be called by the host.
Creating such exports and load a plugin is quite easy with Extism:
/* --- %< --- snip --- %< --- */
host_fn!(get_battery(_user_data: (); battery_idx: String) -> String { (1)
let charge_full = std::fs::read_to_string(
format!("/sys/class/power_supply/BAT{}/charge_full", battery_idx))?; (2)
let charge_now = std::fs::read_to_string(
format!("/sys/class/power_supply/BAT{}/charge_now", battery_idx))?;
Ok(format!("{} {}", charge_full.trim(), charge_now.trim()))
});
/* --- %< --- snip --- %< --- */
pub(crate) fn build(&self) -> Result<Plugin> {
let url = self.url.clone().context("Url not set")?;
// Load wasm plugin
let wasm = Wasm::file(url.clone());
let manifest = Manifest::new([wasm]);
let plugin = extism::PluginBuilder::new(&manifest) (3)
.with_wasi(true)
/* --- %< --- snip --- %< --- */
.with_function("get_battery", [PTR], [PTR],
UserData::default(), Self::get_battery) (4)
.build()?;
debug!("{}", function_name!());
Ok(Plugin {
name: self.name.clone().context("Name not set")?,
url,
interval: self.interval.unwrap(),
plugin: Rc::new(RefCell::new(plugin)),
})
}
/* --- %< --- snip --- %< --- */| 1 | The macro host_fn! allows us to define functions for our webassembly guest |
| 2 | Funny how the path of the acpi interface has changed over the years |
| 3 | Extism also provides an easy-to-use loader |
| 4 | Time to register our host function |
And just to complete the usual triplet again, here is what the battery plugin actually does:
#[host_fn("extism:host/user")]
extern "ExtismHost" {
fn get_battery(battery_idx: String) -> String; (1)
}
#[plugin_fn] (2)
pub unsafe fn run<'a>() -> FnResult<String> {
let values: String = unsafe { get_battery("0".into())? }; (3)
info!("battery {}", values);
let (charge_full, charge_now) = values.split(" ") (4)
.filter_map(|v| v.parse::<i32>().ok())
.collect_tuple()
.or(Some((1, 0)))
.unwrap();
Ok(format!("{}%", charge_now * 100 / charge_full))
}| 1 | This imports the function from the host |
| 2 | Mark this function for export to the host |
| 3 | Sadly the unsafe here is required… |
| 4 | Pretty straight forward - parse and convert with a bit error checking - one line |
Error handling &
Due to the isolation of the plugins the error handling happens inside the virtual machine:
/* --- %< --- snip --- %< --- */
impl Plugin {
pub(crate) fn update(&self) -> Result<String> {
let res = self.plugin.borrow_mut().call("run", "")?; (1)
debug!("{}: res={}", function_name!(), res);
Ok(res)
}
/* --- %< --- snip --- %< --- */
}| 1 | Just a quick call and result check of the plugin function |
Recap &
| Ease of use | Richness of API | Language agnostic | Error handling |
|---|---|---|---|
Depends on the language, but you can pick from the list of supported ones |
All noteworthy API must be provided by the host, like time |
Yes, the list of supported language is quite nice |
Extism offers easy integration and error checking |
Conclusion &
Time for a conclusion after such a marathon through many ideas, languages and projects, so we can call this a day. We have seen different approaches of providing an API to essentially shape what a guest or plugin can do in your application. And we have also covered error checking and seen how it can range from being arcane and nasty to be handled entirely by your framework.
I think taken with care the integration of scripting languages can be a great way to ease the hurdle of providing new feature sets. It can also allow different audiences not familiar with the host language or host domain to enrich it. And additionally approaches like webassembly allow to combine the raw processing speed of compiled languages with the ease-of-use flow of scripting.
The list of examples is quite long, but please help yourself:
mem.c if you are curious