Systems Programming

Concepts of Programming Languages

Sebastian Macke

Rosenheim Technical University

Last lecture

2

What is Systems Programming?

Definition

"Systems programming involves designing and writing computer programs that allow the computer hardware to interface with the programmer and the user, leading to the effective execution of application software on the computer system.

Typical programs include the operating system and firmware, programming tools such as compilers, assemblers, I/O routines, interpreters, scheduler, loaders and linkers as well as the runtime libraries of the computer programming languages."

3

Systems Programming Today

4

Programming Languages based on which Programming languages

Also most the compiling programming languages incorporate assembler

5

Calling C Code from Go with cgo

6

C code example

#include<stdlib.h>
#include<stdio.h>

void main() {
    srandom(1);
    printf("random int is %li\n", random());
}

run with 

gcc random.c -o random
./random
random number is ...
7

Calling C Code from Go with cgo

package main

// #include<stdlib.h>
// #include<stdio.h>
// void Cmain() {
//   srandom(1);
//   printf("random int is %li\n", random());
// }
import "C"

func main() {
    C.Cmain()
}
8

Calling C Code from Go with cgo

Import the pseudo package C in your Go source and include the corresponding C header files (comment) to call a C function. 

package rand

// #include <stdlib.h>
import "C"
func Random() int {
    var r C.long = C.random() // calls "int rand(void)" from the C std library
    return int(r)
}

func Seed(i int) {
    C.srandom(C.uint(i))
}
9

Calling C Code from Go with cgo

//go:build linux
// +build linux

package main

// #include <stdlib.h>
import "C"
import "fmt"

func Seed(i int) {
    C.srandom(C.uint(i))
}

func Random() int {
    var r C.long = C.random() // calls "int rand(void)" from the C std library
    return int(r)
}

func main() {
    Seed(1)
    fmt.Printf("random int from C is %d\n", Random())
}
10

Controlling Linking with Build Flags

Build flags for the cgo command can be set in the Go source file as comments: 

// #cgo CFLAGS: -DPNG_DEBUG=1
// #cgo amd64 386 CFLAGS: -DX86=1
// #cgo LDFLAGS: -lpng
// #include <png.h>
import "C"

When calling functions from a library outside libc, the Go programm must be linked with that library (here: png.lib/png.so).

This can be done by adding cgo build flags to the comment section.

11

Mapping the C namespace to Go

12

Conversion between C and Go strings

The C package contains conversion functions to convert Go to C strings and vice versa
Also: opaque data (behind void *) to []byte 

// Go string to C string; result must be freed with C.free()
func C.CString(string) *C.char

// C string to Go string
func C.GoString(*C.char) string

// C string, length to Go string
func C.GoStringN(*C.char, C.int) string

// C pointer, length to Go []byte
func C.GoBytes(unsafe.Pointer, C.int) []byte

golang.org/cmd/cgo/

13

Exercise 1.1

Write a Cgo program that reads the hostname from the libc function gethostname and prints it to the console.
The corresponding C function is defined in unistd.h and can be used as follows: 

#include<unistd.h>
#include<stdio.h>

int main() {
    char buffer[256];
    gethostname(buffer, 256);
    printf("%s\n", buffer);
    return 0;
}
14

Exercise 1.2

Write a Cgo program that creates a new SQLITE database file and executes the following statements: 

create table if not exists students (id integer not null primary key autoincrement, name text);
insert into students (name) values ('Your Name');
insert into students (name) values ('Another Name');

Do not use any GitHub libraries, but write your own Cgo wrapper.
Verify the content of the database using any SQLITE inspector (e.g. IntelliJ or https://sqlitebrowser.org/).

github.com/s-macke/concepts-of-programming-languages/blob/master/docs/exercises/Exercise8.md

15

Exercise 1.2

#include <stdio.h>
#include <stdlib.h>
#include <sqlite3.h>
// compile with gcc sqlite.c -lsqlite3

void Exec(sqlite3 *db, char *sql) {
    int rc = sqlite3_exec(db, sql, NULL, NULL, NULL);
    if (rc != SQLITE_OK) {
        printf("sqlite3_open returned code %i", rc);
        exit(1);
    }
}
int main() {
    sqlite3 *db;
    int rc = sqlite3_open("test.db", &db);
    if (rc != SQLITE_OK) {
        printf("sqlite3_open returned code %i", rc);
        return 1;
    }
    Exec(db, "create table if not exists students (id integer not null primary key autoincrement, name text);");
    Exec(db, "insert into students (name) values ('Your Name');");
    Exec(db, "insert into students (name) values ('Another Name');");
    sqlite3_close(db);
    return 0;
}
16

Platform/OS Programming with Go

17

Platform independent programming with Go

The packages os and runtime encapsulate platform dependent APIs 

// Copyright 2018 Johannes Weigend
// Licensed under the Apache License, Version 2.0
package main

import (
    "fmt"
    "os"
)

func main() {
    pid := os.Getpid()
    fmt.Println("process id: ", pid)
}
18

Use signals to perform some tasks before exiting the program

func cleanup() {
    fmt.Println("cleanup")
}

func main() {
    c := make(chan os.Signal)
    signal.Notify(c, os.Interrupt, syscall.SIGTERM)
    go func() {
        <-c
        cleanup()
        os.Exit(1)
    }()

    for {
        fmt.Println("sleeping...")
        time.Sleep(10 * time.Second)
    }
}
19

The OS Package

Access to the standard I/O streams 

os.Stdin
os.Stdout
os.Stderr

Functions for executing processes (os/exec) 

cmd := exec.Command(path, args)

Access to the environment 

os.Getenv("FOO")
os.Setenv("FOO", "1")

Access to files, directories, pipes, ... 

f, err := os.Open("/tmp/dat") // f.Read() / f.Write() / f.Seek()

golang.org/pkg/os/

20

The Runtime Package

Package runtime contains operations that interact with Go's runtime system, such as functions to control goroutines. It also includes the low-level type information used by the reflect package 

func main() {
    println("OS: ", runtime.GOOS)
    println("ARCH: ", runtime.GOARCH)
    println("ROOT: ", runtime.GOROOT())
    println("Number of CPUs: ", runtime.NumCPU())
    println("Current number of goroutines: ", runtime.NumGoroutine())
    var rtm runtime.MemStats
    runtime.ReadMemStats(&rtm)
    rtmAsBytes, _ := json.MarshalIndent(rtm, "", "  ")
    fmt.Println(string(rtmAsBytes))
}

golang.org/pkg/runtime/

21

Platform dependent programming with Go

//go:build linux || darwin
// +build linux darwin

// Copyright 2018 Johannes Weigend, Johannes  Siedersleben
// Licensed under the Apache License, Version 2.0
package main

import (
    "fmt"
    "golang.org/x/sys/unix"
)

func main() {
    pid, _, _ := unix.Syscall(unix.SYS_GETPID, 0, 0, 0)
    fmt.Println("process id: ", pid)
}

golang.org/pkg/syscall/

godoc.org/golang.org/x/sys

22

Platform dependent programming with Go

func main() {
    path := "mydir"
    dir, _ := syscall.BytePtrFromString(path)
    mode := 0777
    _, _, e := unix.Syscall(unix.SYS_MKDIR, uintptr(unsafe.Pointer(dir)), uintptr(mode), 0)
    if e != 0 {
        log.Fatal("Error code", e)
    }
}
23

Fun with the File System

24

What's a File System?

25

What is FUSE?

26

Writing a FUSE Filesystem

github.com/s-macke/concepts-of-programming-languages/blob/master/src/system/fuse/main.go

An inode (index node) descripes a file-system object such as a file or directory.

27

What do people do with FUSE?

28

Building a Container from the Scratch

29

What is a Container?

30

What is a Container?

31

The grandfather of containerization: Change Root (chroot, year 1982)

32

Demo: chroot

# mkdir -p /home/user/jail
# chroot /home/user/jail /bin/bash
  => chroot: failed to run command ‘/bin/bash’: No such file or directory

Change Root can not start because there is no /bin/bash inside the new root!
The solution is to copy bash inclusive dependent libraries inside the directory jail tree 

$ mkdir /home/user/jail/bin
$ cp /bin/bash /home/user/jail/bin
33

Many Applications need Dependent Libraries to Run

List dependent libraries for /bin/bash 

ldd /bin/bash
linux-vdso.so.1 =>  (0x00007fff11bff000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x0000003728800000)
libdl.so.2 => /lib64/libdl.so.2 (0x0000003d56400000)
libc.so.6 => /lib64/libc.so.6 (0x0000003d56800000)
/lib64/ld-linux-x86-64.so.2 (0x0000003d56000000)

Copy libraries to jail root 

mkdir /home/user/jail/lib64
$ cp /lib64/{libtinfo.so.5,libdl.so.2,libc.so.6,ld-linux-x86-64.so.2} /home/user/jail/lib64

Start /bin/bash with new root 

# chroot /home/user/jail
# pwd => /
# ls => /bin, /lib64

The bash shell can only access the jail file system!

34

Linux CGroups (Year 2007)

35

CGroups Features

Resource limiting
groups can be set to not exceed a configured memory limit, which also includes the file system cache

Prioritization
some groups may get a larger share of CPU utilization or disk I/O throughput

Accounting
measures a group's resource usage, which may be used, for example, for billing purposes

Control
freezing groups of processes, their checkpointing and restarting

36

Linux Namespaces

Examples of resource names that can exist in multiple spaces:

37

How to build a Container in Go

First we need a program to check our changes in visibility

stats.go 

func main() {
    fmt.Println("User ID: ", os.Getuid())
    fmt.Println("Group ID: ", os.Getgid())
    fmt.Println("Process Id: ", os.Getpid())
    PrintHostname("Current")
    /*
        syscall.Sethostname([]byte("container"))
        PrintHostname("New")
    */
}

func PrintHostname(pre string) {
    hostname, _ := os.Hostname()
    fmt.Println(pre, "Hostname: ", hostname)
}
38

How to build a Container in Go

Second, we need a way to run the program inside go 

func main() {
    cmd := exec.Command("./stats")
    cmd.Stdin = os.Stdin
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    err := cmd.Run()
    if err != nil {
        panic(err)
    }
}

we just fordward

to the default terminal streams

39

Clone flags

40

Change user id and group id

    cmd.SysProcAttr = &syscall.SysProcAttr{
        Cloneflags: syscall.CLONE_NEWUSER,
        UidMappings: []syscall.SysProcIDMap{
            {
                ContainerID: 0,
                HostID:      os.Getuid(),
                Size:        1,
            },
        },
        GidMappings: []syscall.SysProcIDMap{
            {
                ContainerID: 0,
                HostID:      os.Getgid(),
                Size:        1,
            },
        },
    }

    err := cmd.Run()
    if err != nil {
        panic(err)
    }
}
41

Change Process ID and allow to change the hostname

        Cloneflags: syscall.CLONE_NEWPID | syscall.CLONE_NEWUSER | syscall.CLONE_NEWUTS,
        UidMappings: []syscall.SysProcIDMap{
            {
                ContainerID: 0,
                HostID:      os.Getuid(),
                Size:        1,
            },
        },
        GidMappings: []syscall.SysProcIDMap{
            {
                ContainerID: 0,
                HostID:      os.Getgid(),
                Size:        1,
            },
        },
    }

    err := cmd.Run()
    if err != nil {
        panic(err)
    }
}
42

How to build a Container in Go

The next changes can't be made before startup, but have to be done in the started process.
One way to do this is to run itself, but with different parameters.
The code shows the main functionality of the "docker run" command: 

// go run main.go run <cmd> <args>
func main() {
    switch os.Args[1] {
    case "run":
        run()
    case "child":
        child()
    default:
        panic("help")
    }
}
43

How to build a Container in Go

The run() function starts a new process (a clone) with a unique namespace
The child() function function is called in the new process and configures the new namespace (setHostname ...) 

func run() {
    log.Printf("Running %v \n", os.Args[1:])

    cmd := exec.Command("/proc/self/exe", append([]string{"child"}, os.Args[2:]...)...)
    cmd.Stdin = os.Stdin
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    cmd.SysProcAttr = &syscall.SysProcAttr{
        // Does not compile on OSX
        Cloneflags:   syscall.CLONE_NEWUTS | syscall.CLONE_NEWPID | syscall.CLONE_NEWNS,
        Unshareflags: syscall.CLONE_NEWNS,
    }

    must(cmd.Run())
}
44

The Child Process Initializer

func child() {
    log.Printf("Running %v \n", os.Args[1:])

    cg() // Initialize CGroup limits

    cmd := exec.Command(os.Args[2], os.Args[3:]...)
    cmd.Stdin = os.Stdin
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr

    // We are in an independent clone -> Setting hostname is safe!
    must(syscall.Sethostname([]byte("container")))
    must(syscall.Chroot("/opt/alpinefs")) // container local file system (alpine linux)
    
    must(os.Chdir("/"))
    must(syscall.Mount("proc", "proc", "proc", 0, "")) // support for the ps command
    must(cmd.Run())

    must(syscall.Unmount("proc", 0))
}
45

CGroup Initialization limits Resource Consumption of Container

func cg() {
    cgroups := "/sys/fs/cgroup/"
    pids := filepath.Join(cgroups, "pids")
    os.Mkdir(filepath.Join(pids, "container"), 0755)
    must(ioutil.WriteFile(filepath.Join(pids, "container/pids.max"), []byte("20"), 0700))
    // Removes the new cgroup in place after the container exits
    must(ioutil.WriteFile(filepath.Join(pids, "container/notify_on_release"), []byte("1"), 0700))
    must(ioutil.WriteFile(filepath.Join(pids, "container/cgroup.procs"), []byte(strconv.Itoa(os.Getpid())), 0700))
    ...
}
46

See also the Video of Liz Rice

www.youtube.com/watch?v=Utf-A4rODH8

47

Summary

48

Exercise

github.com/s-macke/concepts-of-programming-languages/blob/master/docs/exercises/Exercise8.md

49

Thank you

Tags: go, programming, concurrent, go routines, channels

Sebastian Macke

Rosenheim Technical University

Sebastian.Macke@th-rosenheim.de

https://www.qaware.de

Use the left and right arrow keys or click the left and right edges of the page to navigate between slides.
(Press 'H' or navigate to hide this message.)