It’s been a long time since I posted anything here. Sorry for my inconsistency.
But today is a special day, because with this post, you’ll be able to do magical things. Yes, I really mean it.
Previously, in an other post I showed you how you could have a « hot reload » feature in your project (specifically using the raylib framework, but that’s the same principle for whatever C/C++ project anyway) in C.
Today, I’m going to show you something much more simpler, with potentially the same capabilities: make a program which can read C code and compile & run it on the fly. So to be shorter: how to use C as a « scripting » language for your project.
After that, you’ll be able to do something pretty similar as below. It’s pure C, no hot reload via dynamic library, everything is recompiled (at high speed, 9x faster than gcc) & rerun at runtime whenever the file is modified. Pretty cool no?
To be able to do that, we will use the amazing tcc compiler. It’s a really small C compiler (C99 mostly, there are just few of the C11 features supported), which has been made by Fabrice Bellard, an amazing french (cocorico, I’m french too 🙂 ) programmer.
Actually, you can compile this tiny C compiler in a library. So… You’ve got the point. You can embed this library and compile and run code 9x faster than GCC, on the fly, during runtime.
To do so, first, we are going to download the source code of TCC, which are available here, on the official website: https://download.savannah.gnu.org/releases/tinycc/
Then we need to extract this tar.bz2 file. You can use a tool like winrar for that, or the tar command line, which is included with a bash terminal (like gitbash).
Inside the folder where you extracted the TCC sources, run from the same bash terminal:
Then type mingw32-make libtcc.a to create the libtcc static library which we will link against in our program.
Now, we are going to create a new C project. Create a new directory for your project, and create 3 new directories inside it: src for the source code, include for the headers, and lib for the static libraries. Or do whatever you want, but classicly, any C project has something similar to that folder organization, and I will refer to that for the rest of this article.
Copy the libtcc.a static library from the TCC directory, that you created with the previous command line, inside your fresh created lib directory.
You can copy as well the libtcc.h, and the tcclib.h files, which are at the root of this directory, and every files which are inside the include directory (at least stdarg.h and stddef.h) inside your include directory.
libtcc.h is the header we will include in our main program to be able to call the libtcc functions, which will compile and run C code at runtime.
tcclib.h and the other headers or necessary to give the minimal C standard to the C code we will compile at runtime.
Now everything is setup, we need to do one more thing. We need to compile the whole TCC to create the libtcc1.a static library, which will not be linked staticly to our program, but to the C code we want to compile and run at runtime. Actually this library contains every definitions of the tcclibh, stdarg.h, and stddef.h include files.
So go to your TCC directory, and simply run:
Copy this new libtcc1.a static library you just created with this cmd to your lib folder.
Now every thing is set. We are going to create a basic C program, which will create a tcc context, and compile & execute some code at runtime. That’s the exciting part!
Go to your project directory, inside src create a C file. I will simply call it main.c, and add these lines to it:
You can compile this main.c file with:
gcc main.c -I./include -L./lib -ltcc -o main.exe
And if you execute this main.exe file, you should get this output inside your terminal:
So we have a warning, but we have our result! We compiled successfully and run C code, which is contained inside the program variable, at runtime!
Now, I’m going to explain to you what each line actually does, and what we could do to remove this anoying warning we get from the libtcc compilation step (for those of you who are experimented in C or C++, you actually might already know why we have this warning).
We got it because the program we compiled with libtcc doesn’t know what is the declaration of the printf function (even if that’s worked). To get rid of this warning, we simply need to include tcclib.h.
After compiling again, no more warning!
So now, it’s time to get a little bit of an explaination about what’s happening here.
The first thing we do in our program is creating a TCC context, which set everythings up for the compilation at runtime. Of course, you know if you are familiar in C/C++ code that every time a function gives you a pointer to something in return, that means it allocate some memory and you need to test if this memory is correclty allocated by testing the adress of the pointer.
Because we allocated some memory, we need to free it when we don’t need it anymore, in our case, after the compilation step. This is done by the tcc_delete function call, at the end.
Now we have our tcc context, we are going to choose which « ouput » our program should compile for with the tcc_set_output_type.
If you take a look at the libtcc.h header file, you’ll see that there are several ouputs forms, and that gives you a hint for when you should call this function:
So you see that if you want to be able to compile & run C code direclty inside your program, the output needs to be in memory.
But you could also compile code at runtime, and create a static library, or an executable file from it.
The next step is the most important one. It’s the compilation step. It’s at this step the magic happens.
tcc_compile_string is quite explicit… It takes a string, try to compile it, and return 0 if something was wrong during the compilation step. If everything correclty compiled, the machine code is loaded into memory.
Now we move to the next step. We need to call the tcc_relocate function to be able to call the tcc_get_symbol function, which gives us back a symbol from the compiled code that we can use later (in our case, the foo function).
Again, it’s well explained in the libtcc.h header file:
So with that information, we know we need this line at this step, and let libtcc manage the memory internally for us:
Then, and only then, we can get the function back with the tcc_get_symbol call, which actually works exactly like symbols from a dll file. We can then store it inside a pointer to a const function, which I simply called with the exact same name foo here.
Of course, you need to define this function pointer with the exact same parameters you defined inside the program variable.
You could add an additional step after assigning this function pointer, to check if it’s null or not of course.
After getting this function back inside our foo variable, we can call it like any regular function.
And… That’s all actually!
You see, nothing complicated, the libtcc API is well designed and super easy to use. Now you are able to compile C code and run it at runtime.
You can do whatever you want with that, like hot reloading (see the gif from above for example), by watching when a file is modified, and recompile it (just look inside the libtcc.h header file to get the right function, in replacement of tcc_compile_string, to be able to compile an entire C file, and not just a char*).
Just a hint: on linux, you could use the stat system call function to get file informations 🙂
Now it’s your turn! Don’t hesitate to open the libtcc.h header file to see the functions you can use from the API. Everything is clearly commented/documented inside. For example, you could call a special function you created when an error occured during compilation (I let you find how to do that, just look for how to pass your function callback to libtcc).
I hope this tutorial will help you create awesome stuff. I’m looking forward to see how you implement this in your project, and for what purpose. Don’t hesitate to tell me in the comments below!