The launch of the new version of the project "Cosmopolitan 2.0", which develops a standard C library and a universal executable file format that can be used to distribute programs for different operating systems without the use of interpreters and virtual machines.
The container to generate universal executable files relies on combining specific segments and headers from different systems operating systems into one file, combining several different formats used in Unix, Windows, and macOS.
To ensure that a single executable run on systems Windows and Unix, a trick is used to encode Windows PE files as a shell script, taking advantage of the fact that the Thompson Shell does not use the "#!" script marker.
To create programs that include multiple files (combining all resources into a single file), it is supported to generate an executable file in the form of a specially crafted ZIP archive. Outline of the proposed format (example hello.com application):
The qemu-x86_64 call is provided for additional portability and allows code compiled for the x86_64 architecture to run on non-x86 platforms, such as Raspberry Pi boards and Apple devices equipped with ARM processors. The project can also be used to create stand-alone applications that work without an operating system (bare metal). In such applications, a bootloader is attached to the executable file, and the program acts as a bootable operating system.
Main novelties of Cosmopolitan 2.0
In this new version it is highlighted that the schema for accessing internal resources has been changed in from a zip file (when opening files, the usual /zip/… paths are now used instead of using the zip:.. prefix). Similarly, to access disks in Windows, it is possible to use paths like "/c/…" instead of "C:/…".
A new APE loader is proposed (Actually Portable Executable), which defines the format of universal executable files. The new bootloader uses mmap to allocate the program to memory and no longer changes the content on the fly. If necessary, the universal executable can be converted into regular executables tied to individual platforms.
On Linux, it is possible to use the binfmt_misc kernel module to run APE programs. It is noted that the use of binfmt_misc is the fastest startup method, in addition to An implementation of the functionality of the pledge() and unveil() system calls, developed by the OpenBSD project, as well as providing an API for using call data in C, C++, Python, and Redbean programs, as well as a utility from promise.com to isolate arbitrary processes.
We can also find that for Linux, the performance of the clock_gettime and gettimeofday calls has been increased up to 10 times due to the use of the vDSO (Virtual Dynamic Shared Object) mechanism, which allows transferring the system call handler to user space and bypassing the switch context.
The build uses Landlock Make, an edition of GNU Make with stricter dependency checking and the use of the Landlock system call to isolate the program from the rest of the system and improve caching efficiency. As an option, compiler capability and the usual GNU Make are preserved.
On the other hand, it also highlights the implemented functions for multithreading: _spawn() and _join(), which are universal bindings over specific APIs for different operating systems. Work is also underway to implement support for POSIX threads.
Of the other changes that stand out:
- Added support for the “–ftrace” and “–strace” options to executable files to send information about all function calls and system calls to stderr.
- Added support for the closefrom() system call compatible with Linux 5.9+, FreeBSD 8+ and OpenBSD.
- Mathematical functions for working with complex numbers have been moved from the Musl library.
- Many math functions have been sped up.
- The nointernet() function is proposed, which disables network capabilities.
- Added new functions to efficiently append strings: appendd, appendf, appendr, appends, appendw, appendz, kappendf, kvappendf, and vappendf.
- A protected version of the kprintf() family of functions, designed to work with elevated privileges, has been added.
- Significantly improved performance of SSL, SHA, curve25519, and RSA implementations.
Finally, if you are interested in learning more about it, you should know that the project codeor is distributed under the ISC license (simplified version of MIT/BSD).