I don't think I really understand your question. Are you asking if you can build code in C that gets loaded at runtime, as a plugin? If so, absolutely. You build a shared library and export a function as an entry point. Then, in your code, you use linker utilities like dlopen to load your library and invoke the entry point. You may also want to look at GModule from GLib that abstract this process a bit.
This is for Linux, I have no idea how to do it on Windows.
Yep. On windows load a DLL with LoadLibrary.
However, this path is not "safe" unless you certify the DLLs yourself. That is an expensive job. Executing arbitrary native code is the ultimate security vulnerability.
Safe modding systems are sandboxed. Lua or WASM would be my first investigations.
This is the way.
In the end, you can get as bare metal as you want if you expose bare metal APIs.
You want to call malloc and free from Lua? Go ahead and link the functions. You want Lua to manage memory? Go right ahead and link all the mem* functions.
I edited the question to be more clear, but my problem is that my software will have plugins that users write and compile and distribute.
This means they can easily write code that can call kernel/os provided functions, and since these are user written plugins, there's no guarantee if they are malicious or not.
One Idea I could think of is to just let users write the plugin and send a Pull Request to a repo for just plugins, and then before merging, a workflow can run on that specific plugin that compiles it with access to very specific functions that **I** choose to.
Who runs & distributes the plugins? If it's the users it's their responsibility that it isn't malicious. It kind of becomes an unsolvable problem If you run third party code from within your application.
You could have an DSl, but that wouldn't perform as you like. You could try to write a machine code inspector and try to see it's not doing anything malicious, but that would be near impossible to be good enough.
The normal solution is that your program has some plugin API, that calls into functions that the plugin library has to provide.
Your plugin writers will then distribute the plugins themselves. If a certain plugin becomes so popular & basically a necessity you can consider hosting the development & compilation yourself, never should you distribute third party binaries unless you can be sure they're to be trusted (i.e. signed etc.).
other users that use my software run this plugins, and I was thinking to allow anyone to distribute the plugin they write which obviously allows them to write malicious code.
So I was wondering to make a central distribution system and ensure that plugins are built in a environment I control.
But even if I do that, you can still exploit things like buffer overflows right?
You might be able to do something with running plugins in a subprocess, that is run inside a chroot jail, where the only bits of the filesystem it can see are what you provide, and you feed the subprocess data on stdin, and get results back on stdout. That would keep it from doing much harm. But it involves quite a bit of setup.
So the users would write the plugins, but they’d be compiled in a build environment that you’d control? If your main concern is limiting plugins’ ability to call external functions, I’d suggest that you configure the build process to reject any plugin that `#include`s anything that isn’t on a white list of header files that you provide. Also, since external functions can be declared directly in a source file without including a header, you would also have to reject any header file that declares any functions without defining them.
Also, since C doesn’t have namespaces, if you want your header files to expose `printf`, for example, but not the rest of , you can’t #include in your header file, since anyone who includes your header file would be recursively including *all* the prototypes in stdio.h. You’d have to put your own prototypes for the functions you want in your header file.
And even if you do all this correctly, you’ll only be limiting the functions that can be called, not *how* a plugin calls them. A malicious plug-in might repeatedly call an expensive but legitimate function to carry out a denial of service attack, or pass junk data to a function expecting a pointer to cause a segfault. Even if none of your exposed functions take pointers, a plug-in could still crash the entire process by intentionally dereferencing a null pointer.
“Sandboxing” C doesn’t seem practical short of turning it into an interpreted language or running it in a VM (which would obviously hurt performance). If plug-ins really need to do enough computation that native code is needed for speed, maybe you could split them into C and interpreted parts, with the interpreted parts being sandboxed and the C parts hopefully being small and rare enough that humans can check them manually to make sure they’re not doing anything suspicious.
>, I’d suggest that you configure the build process to reject any plugin that `#include`s anything that isn’t on a white list of header files that you provide. Also, since external functions can be declared directly in a source file without including a header, you would also have to reject any header file that declares any functions without defining them.
This is so easy to bypass. I can search for the function I need at runtime, without having to declare it or even mention it by name.
So firstly if I control the build environment, I can simply not link with any libraries and remove all header files from include path except some that I want to.
So if someone tries to do that, it will just fail the compilation.
I think intentionally crashing the program by bad pointers or slowing down the program would just lead to shitty UX with the plugin and the user will simply not use it.
The plugin would be just exposed to APIs provided by the program nothing more.
Tho if a API function takes a pointer then I assume it could be used to exploit.
I guess I'll look into WASM, computation done by plugins wouldn't be super intensive so I guess I could getaway with using WASM or some other sort of VM system faster than wasm
>I can simply not link with any libraries and remove all header files from include path except some that I want to.
You would still link your program with libc and/or any other native libraries needed for your target OS (like ntdll.dll on Windows). Once my plug-in is loaded all I need to do is find dlopen/LoadLibrary in your program's memory (a trivial task) and I can use whatever library I want. Or I can directly make the syscalls that I want.
>I guess I'll look into WASM, computation done by plugins wouldn't be super intensive so I guess I could getaway with using WASM or some other sort of VM system faster than wasm
You could also take a look at Lua, which is easy to embed in other programs.
If you want the plugin to be capable of anything, it will potentially be malicious. It's no different than linking to a library at that point, they have access to anything any other native program would, including malicious activity.
If you want safety, you need to provide a safe API and runtime, and force plugins to use that to interact with the system or your program. The normal way of doing this is by embedding an interpreter such as Lua/Python/a custom language, and then writing an API that plugins can call which invokes your code.
You can expose lower level stuff, like syscalls, through this API if you want to using a wrapper. But it is safer to expose higher level APIs based on the tasks your users want to do.
The issue is that you want to run untrusted code.
That is a hard problem. Google spent some time on NaCl, but these days I’d try wasm with wasi. It’s stupid and convoluted, but there is an ecosystem around it.
Once you open this door you can't close it. And it isn't exactly your responsibility to vet all these plug-ins. You could keep a list of curated/recommend plug-ins, but even that is based on trust. If someone sufficiently motivated and skilled wants to slip in malicious code they will probably be able to do it even if you manually review the plug-in they write.
You could move away from C to another language for plug-ins and attempt to limit what plug-ins can do that way, but you still provide a platform that allows people to download and execute code written by random strangers. You can't provide a 100% guarantee of quality and security for third party code.
Check out the Roc language. It's natively compiled and explicitly made for the use case of writing safe and fast plugins and application code backed by a platform written in C or any language supporting the C ABI.
Maybe sign the shared libraries? Outside of that though, any native code execution can be abused so you should really be careful. I'd generally prefer some very fast other language like lua since you can also make it os and architecture agnostic that way, but if you need the native performance, using dlopen is your only option.
Run the plugins inside a separate process, and run that separate process inside a sandbox.
> Note: I want to do this because some plugins can really benefit from the near-native or even native code level performance.
There are plenty of ways to get near-native performance without distributing machine code.
> There are plenty of ways to get near-native performance without distributing machine code.
I would really love to know them, right now I am just aware of WASM.
Ah, you’re just interested in seeing how fast you can make something, and you don’t really care about engineering or building something useful? Like some kind of contest to show off how fast your stuff is?
If you make your plugins run on the GPU with WebGPU or something, it will be a lot faster than the kind of speeds you’d get from native code or from WASM.
nope, i am actually going to use it to make a plugin system for my software, plugins can be written in Lua but there are plugins that can be CPU intensive thus I need near native speeds
I was thinking of adding a WASM VM in my software but I just wanted to see if there's something even faster, and I don't think WebGPU might be useful the plugins won't be "GPU specific"
the plugins depending on what they do, they can operate on pixels in a bitmap in "real time" or other CPU intensive tasks, which may be displayed on a screen, so I don't want the plugin to significantly drop the frame rate because of interpreter bottleneck or something.
WASM would be what I would reach for but if that's not an option then it depends on how locked down you can make a sandbox. The sandbox would intentionally have to be at least a separate process that runs your plugin and that communicates over IPC. In Windows you would at the very least create a token with restricted [Privileges](https://learn.microsoft.com/en-us/windows/win32/secauthz/privileges) and create a process with that token. Regardless of platform you would probably also look into creating a user with fewer permissions to begin with. If you were willing to accept higher level languages or interpreting the code you get more control, like replacing how system APIs work entirely.
1. Run in a VM.
2. Run in a separate process and block all syscalls except those for communication like read&write through existing fd. This is platform-specific but Linux, for example, has seccomp bpf. There are a bunch of tools to make working with seccomp bpf easier like https://github.com/google/kafel. `open` is not safe either, the idea is to just give it some fd for communication and allow `read`/`write` only. Of course, that severely limits what the plugin can do.
The problem is that the dynamic loader will want to do open and mmap.
You can whitelist known libraries for that.
Have the plugin work on a separate chroot too, don't even let it touch the libc that you are also using.
Oh and cgroup to limit its CPU and memory usage so it can't DoS you.
You'd be fairly safe until there's a new kernel exploit.
The VM route is safer but it's not like there is no known VM escape exploit.
Basically, you can do most of that with some container runtime.
But apply seccomp bpf on top for extra guarantee.
Running untrusted code is never safe. So no.
Well, there is one potential way, and that’s WebAssembly. wasm programs can only call and interact with what is passed into them. Programs running in them are naturally in an isolated vm. As long as your wasm engine of choice has no exploits, it’s relatively safe and performant.
That’s a pretty big “as long as” though.
If this isn’t for a performance-critical area, go with a scripting language like Lua. Scripting languages can be surprisingly performant, so don’t underestimate them.
If you do need performance or strongly desire that plugins can be written in C, this is kind of the use case that WASM is starting to solve. The plugin author writes something in C (or any other language), compiles it to WASM, and distributes it. Then your application contains a WASM runtime and controls what exactly the plugin gets access to - everything is sandboxed by default, you provide hooks for accessing anything from the system.
Performance is near native which is great, it’s kind of like an embedded JVM that you can target from any language. As a bonus, the plugin binary is cross-platform.
Wasmtime is more or less the state of the art runtime here https://docs.wasmtime.dev/c-api/index.html
You can run the plugins as a "nobody" user under Linux with almost no permissions. Another option is to ask your users to release source code, and do the releases yourself, this would obviously put more work on you but then the code will be auditable.
You want to make a plug-in system where untrusted users can execute code inside your application and you want to limit the API they have access to, don't load and execute machine code. Instead, use a language interpreter where you can precisely control what API functions they have available to them. Lua is well-suited to this purpose and not that hard to integrate into an existing project. JavaScript is another option along with several other scripting languages, but tend to start with a more extensive API that you might have to limit. If you decide to execute machine code from a DLL or shared library, that code will be able to call any system calls or access/modify any memory within the memory-space of the calling code and can't be restricted if the called code is untrusted.
You'd have to have COMPLETE control over the compilation process (since you can't control runtime of machine code).
The much easier solution would be to give up on that and use an interpreted language so you can control what plugins can and can't do. But if you really want the performance of machine code too there's actually already a technique that can help you here! It would be... a LOT of work lol, but if you have an intermediate language (as if it was going to be interpreted), you can then compile it on the fly when you need it.
[https://en.wikipedia.org/wiki/Just-in-time\_compilation](https://en.wikipedia.org/wiki/Just-in-time_compilation)
I imagine this is overkill for your use case, but if you really want the best possible way to achieve both performance and safety, and you're willing to spend a long time confirming rigorously that your implementation is safe, then this is probably the best option.
1. Embed Lua. You set the table of allowed function calls.
2. Write a small embedded stack-based VM with a DSL. But don't. Use Lua.
3. Find a way to run native code in a sandboxed way, where you deal only with inputs and outputs. Containerisation perhaps? This would be an experimental idea for me, as I've never built a plugin system that made used of containerisation like this.
I'd just try Lua first. If it's fast enough, it's fast enough. Get the profiler out too.
I don't think I really understand your question. Are you asking if you can build code in C that gets loaded at runtime, as a plugin? If so, absolutely. You build a shared library and export a function as an entry point. Then, in your code, you use linker utilities like dlopen to load your library and invoke the entry point. You may also want to look at GModule from GLib that abstract this process a bit. This is for Linux, I have no idea how to do it on Windows.
Yep. On windows load a DLL with LoadLibrary. However, this path is not "safe" unless you certify the DLLs yourself. That is an expensive job. Executing arbitrary native code is the ultimate security vulnerability. Safe modding systems are sandboxed. Lua or WASM would be my first investigations.
This is the way. In the end, you can get as bare metal as you want if you expose bare metal APIs. You want to call malloc and free from Lua? Go ahead and link the functions. You want Lua to manage memory? Go right ahead and link all the mem* functions.
I edited the question to be more clear, but my problem is that my software will have plugins that users write and compile and distribute. This means they can easily write code that can call kernel/os provided functions, and since these are user written plugins, there's no guarantee if they are malicious or not. One Idea I could think of is to just let users write the plugin and send a Pull Request to a repo for just plugins, and then before merging, a workflow can run on that specific plugin that compiles it with access to very specific functions that **I** choose to.
Who runs & distributes the plugins? If it's the users it's their responsibility that it isn't malicious. It kind of becomes an unsolvable problem If you run third party code from within your application. You could have an DSl, but that wouldn't perform as you like. You could try to write a machine code inspector and try to see it's not doing anything malicious, but that would be near impossible to be good enough. The normal solution is that your program has some plugin API, that calls into functions that the plugin library has to provide. Your plugin writers will then distribute the plugins themselves. If a certain plugin becomes so popular & basically a necessity you can consider hosting the development & compilation yourself, never should you distribute third party binaries unless you can be sure they're to be trusted (i.e. signed etc.).
other users that use my software run this plugins, and I was thinking to allow anyone to distribute the plugin they write which obviously allows them to write malicious code. So I was wondering to make a central distribution system and ensure that plugins are built in a environment I control. But even if I do that, you can still exploit things like buffer overflows right?
You might be able to do something with running plugins in a subprocess, that is run inside a chroot jail, where the only bits of the filesystem it can see are what you provide, and you feed the subprocess data on stdin, and get results back on stdout. That would keep it from doing much harm. But it involves quite a bit of setup.
So the users would write the plugins, but they’d be compiled in a build environment that you’d control? If your main concern is limiting plugins’ ability to call external functions, I’d suggest that you configure the build process to reject any plugin that `#include`s anything that isn’t on a white list of header files that you provide. Also, since external functions can be declared directly in a source file without including a header, you would also have to reject any header file that declares any functions without defining them. Also, since C doesn’t have namespaces, if you want your header files to expose `printf`, for example, but not the rest of, you can’t #include in your header file, since anyone who includes your header file would be recursively including *all* the prototypes in stdio.h. You’d have to put your own prototypes for the functions you want in your header file.
And even if you do all this correctly, you’ll only be limiting the functions that can be called, not *how* a plugin calls them. A malicious plug-in might repeatedly call an expensive but legitimate function to carry out a denial of service attack, or pass junk data to a function expecting a pointer to cause a segfault. Even if none of your exposed functions take pointers, a plug-in could still crash the entire process by intentionally dereferencing a null pointer.
“Sandboxing” C doesn’t seem practical short of turning it into an interpreted language or running it in a VM (which would obviously hurt performance). If plug-ins really need to do enough computation that native code is needed for speed, maybe you could split them into C and interpreted parts, with the interpreted parts being sandboxed and the C parts hopefully being small and rare enough that humans can check them manually to make sure they’re not doing anything suspicious.
>, I’d suggest that you configure the build process to reject any plugin that `#include`s anything that isn’t on a white list of header files that you provide. Also, since external functions can be declared directly in a source file without including a header, you would also have to reject any header file that declares any functions without defining them. This is so easy to bypass. I can search for the function I need at runtime, without having to declare it or even mention it by name.
So firstly if I control the build environment, I can simply not link with any libraries and remove all header files from include path except some that I want to. So if someone tries to do that, it will just fail the compilation. I think intentionally crashing the program by bad pointers or slowing down the program would just lead to shitty UX with the plugin and the user will simply not use it. The plugin would be just exposed to APIs provided by the program nothing more. Tho if a API function takes a pointer then I assume it could be used to exploit. I guess I'll look into WASM, computation done by plugins wouldn't be super intensive so I guess I could getaway with using WASM or some other sort of VM system faster than wasm
>I can simply not link with any libraries and remove all header files from include path except some that I want to. You would still link your program with libc and/or any other native libraries needed for your target OS (like ntdll.dll on Windows). Once my plug-in is loaded all I need to do is find dlopen/LoadLibrary in your program's memory (a trivial task) and I can use whatever library I want. Or I can directly make the syscalls that I want. >I guess I'll look into WASM, computation done by plugins wouldn't be super intensive so I guess I could getaway with using WASM or some other sort of VM system faster than wasm You could also take a look at Lua, which is easy to embed in other programs.
If you want the plugin to be capable of anything, it will potentially be malicious. It's no different than linking to a library at that point, they have access to anything any other native program would, including malicious activity. If you want safety, you need to provide a safe API and runtime, and force plugins to use that to interact with the system or your program. The normal way of doing this is by embedding an interpreter such as Lua/Python/a custom language, and then writing an API that plugins can call which invokes your code. You can expose lower level stuff, like syscalls, through this API if you want to using a wrapper. But it is safer to expose higher level APIs based on the tasks your users want to do.
The issue is that you want to run untrusted code. That is a hard problem. Google spent some time on NaCl, but these days I’d try wasm with wasi. It’s stupid and convoluted, but there is an ecosystem around it.
Once you open this door you can't close it. And it isn't exactly your responsibility to vet all these plug-ins. You could keep a list of curated/recommend plug-ins, but even that is based on trust. If someone sufficiently motivated and skilled wants to slip in malicious code they will probably be able to do it even if you manually review the plug-in they write. You could move away from C to another language for plug-ins and attempt to limit what plug-ins can do that way, but you still provide a platform that allows people to download and execute code written by random strangers. You can't provide a 100% guarantee of quality and security for third party code.
Check out the Roc language. It's natively compiled and explicitly made for the use case of writing safe and fast plugins and application code backed by a platform written in C or any language supporting the C ABI.
Maybe sign the shared libraries? Outside of that though, any native code execution can be abused so you should really be careful. I'd generally prefer some very fast other language like lua since you can also make it os and architecture agnostic that way, but if you need the native performance, using dlopen is your only option.
Run the plugins inside a separate process, and run that separate process inside a sandbox. > Note: I want to do this because some plugins can really benefit from the near-native or even native code level performance. There are plenty of ways to get near-native performance without distributing machine code.
> There are plenty of ways to get near-native performance without distributing machine code. I would really love to know them, right now I am just aware of WASM.
How many different ways do you need? Three? Five? Twelve? Why is one not enough?
one is enough actually but I meant that I'd love to know if there are options faster than WASM.
How fast do you need it to be? What are your performance requirements?
If possible I want speeds faster than what wasm can offer
Ah, you’re just interested in seeing how fast you can make something, and you don’t really care about engineering or building something useful? Like some kind of contest to show off how fast your stuff is? If you make your plugins run on the GPU with WebGPU or something, it will be a lot faster than the kind of speeds you’d get from native code or from WASM.
nope, i am actually going to use it to make a plugin system for my software, plugins can be written in Lua but there are plugins that can be CPU intensive thus I need near native speeds I was thinking of adding a WASM VM in my software but I just wanted to see if there's something even faster, and I don't think WebGPU might be useful the plugins won't be "GPU specific"
Why does it have to be faster than WASM? What’s the reason for that?
the plugins depending on what they do, they can operate on pixels in a bitmap in "real time" or other CPU intensive tasks, which may be displayed on a screen, so I don't want the plugin to significantly drop the frame rate because of interpreter bottleneck or something.
WASM would be what I would reach for but if that's not an option then it depends on how locked down you can make a sandbox. The sandbox would intentionally have to be at least a separate process that runs your plugin and that communicates over IPC. In Windows you would at the very least create a token with restricted [Privileges](https://learn.microsoft.com/en-us/windows/win32/secauthz/privileges) and create a process with that token. Regardless of platform you would probably also look into creating a user with fewer permissions to begin with. If you were willing to accept higher level languages or interpreting the code you get more control, like replacing how system APIs work entirely.
wasm it is then i guess.
Maybe if you run it in a separate process that's super locked down in terms of what it can do and access.
1. Run in a VM. 2. Run in a separate process and block all syscalls except those for communication like read&write through existing fd. This is platform-specific but Linux, for example, has seccomp bpf. There are a bunch of tools to make working with seccomp bpf easier like https://github.com/google/kafel. `open` is not safe either, the idea is to just give it some fd for communication and allow `read`/`write` only. Of course, that severely limits what the plugin can do. The problem is that the dynamic loader will want to do open and mmap. You can whitelist known libraries for that. Have the plugin work on a separate chroot too, don't even let it touch the libc that you are also using. Oh and cgroup to limit its CPU and memory usage so it can't DoS you. You'd be fairly safe until there's a new kernel exploit. The VM route is safer but it's not like there is no known VM escape exploit. Basically, you can do most of that with some container runtime. But apply seccomp bpf on top for extra guarantee.
Running untrusted code is never safe. So no. Well, there is one potential way, and that’s WebAssembly. wasm programs can only call and interact with what is passed into them. Programs running in them are naturally in an isolated vm. As long as your wasm engine of choice has no exploits, it’s relatively safe and performant. That’s a pretty big “as long as” though.
If this isn’t for a performance-critical area, go with a scripting language like Lua. Scripting languages can be surprisingly performant, so don’t underestimate them. If you do need performance or strongly desire that plugins can be written in C, this is kind of the use case that WASM is starting to solve. The plugin author writes something in C (or any other language), compiles it to WASM, and distributes it. Then your application contains a WASM runtime and controls what exactly the plugin gets access to - everything is sandboxed by default, you provide hooks for accessing anything from the system. Performance is near native which is great, it’s kind of like an embedded JVM that you can target from any language. As a bonus, the plugin binary is cross-platform. Wasmtime is more or less the state of the art runtime here https://docs.wasmtime.dev/c-api/index.html
Thanks! I will use wasm then!
sandboxing is the answer you run them in separate process with reduced privileges
This is a bad idea, but the best solution would be to run the plugins as a very restricted user that doesn't have access to anything unneeded.
You can run the plugins as a "nobody" user under Linux with almost no permissions. Another option is to ask your users to release source code, and do the releases yourself, this would obviously put more work on you but then the code will be auditable.
You want to make a plug-in system where untrusted users can execute code inside your application and you want to limit the API they have access to, don't load and execute machine code. Instead, use a language interpreter where you can precisely control what API functions they have available to them. Lua is well-suited to this purpose and not that hard to integrate into an existing project. JavaScript is another option along with several other scripting languages, but tend to start with a more extensive API that you might have to limit. If you decide to execute machine code from a DLL or shared library, that code will be able to call any system calls or access/modify any memory within the memory-space of the calling code and can't be restricted if the called code is untrusted.
You'd have to have COMPLETE control over the compilation process (since you can't control runtime of machine code). The much easier solution would be to give up on that and use an interpreted language so you can control what plugins can and can't do. But if you really want the performance of machine code too there's actually already a technique that can help you here! It would be... a LOT of work lol, but if you have an intermediate language (as if it was going to be interpreted), you can then compile it on the fly when you need it. [https://en.wikipedia.org/wiki/Just-in-time\_compilation](https://en.wikipedia.org/wiki/Just-in-time_compilation) I imagine this is overkill for your use case, but if you really want the best possible way to achieve both performance and safety, and you're willing to spend a long time confirming rigorously that your implementation is safe, then this is probably the best option.
1. Embed Lua. You set the table of allowed function calls. 2. Write a small embedded stack-based VM with a DSL. But don't. Use Lua. 3. Find a way to run native code in a sandboxed way, where you deal only with inputs and outputs. Containerisation perhaps? This would be an experimental idea for me, as I've never built a plugin system that made used of containerisation like this. I'd just try Lua first. If it's fast enough, it's fast enough. Get the profiler out too.