Posts on dmitry_vk's notes

Travel Charger for my new laptop

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 24 Dec 2025 14:00:00 +0100

I recently purchased a new laptop - HP ZBook Ultra G1a 14. It's a great laptop that has a powerful CPU/GPU, a big battery and is still light-weight.

Charging issues

I usually don't use original laptop power supplies due to their bulkiness and general inconvenience.

The new laptop is quite picky when it comes to charging it (caveat: the laptop comes in different variants with different TDPs, and variants with lower wattages might be less picky about power supply).

All the laptops that I had before could be charged using a small travel-size multi-port 65W USB-C charger, which could even charge multiple device simultaneously with the laptop. This is super convenient when traveling as I can take just one charger and use it to charge everything.

Unfortunately, the new laptop would not charge using any of the chargers that I had. 65W chargers did not work at all, and the 100W charger only worked in a single-port configuration.

Initially I purchased a UGREEN 160W charger for around 60EUR. Although it technically can charge the laptop, I wasn't fully satisfied with it. The two major issues with it for me were:

The charger is bulkier and heavier than I would like.
Even though the total available power (160W) is more than enough for my charging needs, the way that it splits the power across its output ports is not ideal. When connecting 3 devices, it allocates too little power to the laptop for it to start charging. This forces me to shuffle and re-connect my devices to keep them charged.

After some research I ended up buying Anker 160W charger for around 100EUR. Its main strengths for me were:

It is compact and lightweight. Not as compact as 65W chargers are, but still tolerable.
Power allocation across ports is not fixed and can be configured, which enables to adapt the charger to my laptop.
According to tests done by others, it should be more efficient and produce less heat, though I haven't had the chance to test it out. My previous 65W charger could get pretty warm when charging at its maximum power.

In the end I am pretty happy with the Anker charger am I going to keep it.

Charging experiments

During the course of solving the charging issues, I've experimented with multiple chargers and collected some data.

Charging in OFF state

The process of charging the laptop is very different depending on whether it's off or on. When it's off, its battery can be charged using any charger at all, even the most basic 5W (5V 1A) USB-A charger. It would charge very slowly, but at least I know that in a pinch it's possible to charge it anywhere. Apparently it can convert input voltage to battery voltage, so I assume that its DC-to-DC converter is too weak to be able to drive the CPU off of it.

Charging in the ON state

After testing several chargers and several configurations of their output ports, I came to the conclusion that the only requirement is that the charger allocates at least 85W to the laptop. If the charger has multiple port, then the port that the laptop uses must be able to provide 85W, otherwise it won't charge at all. The laptop can use either 20V or 28V for charging (its original charger is 28V).

All chargers (except for Anker) have a fixed power allocation between ports. Anker's charger supports several modes:

“AI” mode, which tries to guess power requirements of each connected device
C1 Priority, which allocates the largest portion of power to the first port, and the remaining power is split between ports 2 and 3
Dual Laptop, which allocates 70W to first 2 ports and 20W to the remaining port
Custom mode, which allows editing exactly how many watts is allocated to each port.

As far as I can see (and I confirmed it with the Anker Support), the voltages on each port are independent of each other, so it is possible to mix and match devices with different voltage requiements.

The following table is a summary of the chargers and configurations I've tried.

Picture	Charger	Port configuration	Outcome
Mode	Port 1	Port 2	Port 3	Port 4 (USB-A)
		Belkin 65W USB-C PD Wall Charger with PPS Model number: WCH013vf		laptop	-	-	-	❌
		UGREEN 65W 3-port USB-C GaN Fast Charger Model number: X556		laptop	-	-	-	❌
		UGREEN 100W GaN Fast Charger Model number: CD226		laptop	-	-	phone	✅
	laptop	-	phone	-	❌
		UGREEN 160W 4-port GaN Fast Charger Model number: X762		laptop	tablet	-	phone	✅
	laptop	tablet	phone	-	❌
		Anker Prime 160W, 3 ports, smart display Model number: A2687	"AI"	laptop	tablet	phone		✅
"AI"	laptop	other laptop	phone		❌
C1 priority	laptop	tablet	phone		✅
C1 priority	laptop	other laptop	phone		✅
Dual laptop	laptop	phone			❌
Custom, 84W	laptop	-	-		❌
Custom, 85W+35W+35W	laptop	phone	tablet		✅

After the experiments, I've settled on the following custom mode:

Some unsafe Rust tricks

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 25 Jun 2025 00:00:00 +0200

I'm doing a hobby project in Rust and I have used a couple of unsafe tricks that I'd like to share.

Trick #1: Lifetime transmute

Let's imagine the following:

we have a struct that is shareable across threads
some of struct's fields are protected by a mutex
we want to lock the mutex and send the lock guard across a channel

Normally, a lock guard has a lifetime parameter (e.g. struct MutexGuard<'a, T> with lifetime parameter 'a) that is used to guarantee that the lock guard cannot outlive the lock. The way this guarantee is enforced prevents us from sending an owned lock guard across a channel.

Here's an example of a compilation error:

error[E0597]: `buf` does not live long enough
  --> src/main.rs:16:17
   |
13 |     let buf = Arc::new(Buffer {
   |         --- binding `buf` declared here
...
16 |     let guard = buf.data.blocking_lock();
   |                 ^^^ borrowed value does not live long enough
...
19 |         guard,
   |         ----- this usage requires that `buf` is borrowed for `'static`
20 |     });
21 | }
   | - `buf` dropped here while still borrowed

for the following code:

fn producer(tx: Sender<LockedBuffer>) {
    let buf = Arc::new(Buffer {
        data: Mutex::new(vec![1,2,3]),
    });
    let guard = buf.data.blocking_lock();
    tx.send(LockedBuffer {
        buf: buf.clone(),
        guard,
    });
}

struct LockedBuffer {
    buf: Arc<Buffer>,
    guard: MutexGuard<'static, Vec<u8>>,
}

struct Buffer {
    data: Mutex<Vec<u8>>,
    // ....
}

(full code)

The borrow check is preventing us from doing something potentially unsafe, as it does not understand that we're packing buf and guard into a single struct, and guard references inside buf. Rust's borrow check is just a local analysis, so it's natural that it fails to see that.

We can easily work around by extending the lifetime to 'static with std::mem::transmute:

    let guard = buf.data.blocking_lock();
    
    // SAFETY: the `guard`'s referent `buf` is kept alive for the lifetime of `guard`,
    // because it's stored in `LockedBuffer`, and `LockedBuffer` is not destructured.
    let guard: MutexGuard<'static, _> = unsafe { std::mem::transmute(guard) };

    tx.send(LockedBuffer {
        buf: buf.clone(),
        guard,
    });

(full code)

The trick works because Rust uses lifetimes only for borrow checking, and lifetimes are completely erased after borrow-checking. That means that as long as we don't anything bad at runtime, having a “wrong” lifetime does no harm.

In general, transmute can be dangerous, but in this case its use is sound, since we take additional measures to ensure that the lifetime constrains between a lock and its guard are satisfied.

Note that there is a yoke crate that uses a similar technique (but is much more advanced and safe).

Trick #2: controlling access to shared data with external locks

Normally, Mutexes and RwLocks in Rust act as containers that wrap the protected resource. This design leads to an ergonomic and safe API, but can be restrictive at times.

I have a case where I have a node-based data structure and I want to use a single lock to control access to set of fields of all nodes instead of having a mutex inside of each node.

This access pattern can be implemented with UnsafeCell, which requires some unsafe code around it. In this specific case, the use of UnsafeCell is pretty hard to make safe in general. But it is still useful and can be sound.

It works like this:

the owning struct has a lock (which will be used to control access to child nodes). If we don't have any other state to protect, we can just add a empty unique type.
```
struct Buffers {
    lock: Mutex<BuffersLockToken>,
    buffers: Vec<Buffer>,
}
struct BuffersLockToken;
```
the internal struct has an UnsafeCell around the protected data:
```
struct Buffer {
    flags: UnsafeCell<u32>,
}
```
we also need to implement Sync for this struct (by default, UnsafeCell is not shareable across threads):
```
unsafe impl Sync for Buffer {}
```

we provide method to access the resource. The method access a reference to the lock guard for the owner to serve as an evidence that that lock is held. This method must be unsafe as it's possible to call this method multiple times on the same instance and obtain multiple mutable references (which is UB in Rust).

impl Buffer {
    // SAFETY: `flags_mut` should not be called multiple times on the same `Buffer` with the same `lock_evidence`
    unsafe fn flags_mut<'a>(&'a self, _lock_evidence: &'a MutexGuard<'a, BuffersLockToken>) -> &'a mut u32 {
        // SAFETY: the access is protected by the lock on owning `Buffers`
        // this is not completely safe, but safe enough.
        unsafe {
            &mut *self.flags.get()
        }
    }
}

and it's used like so:

let buffers = Buffers { ... };
let guard = buffers.lock.blocking_lock();
*buffers.buffers[0].flags_mut(&guard) = 123;

(full code)

Even though this usage requires unsafe at the call site (as the user of flags_mut must ensure some of the safety invariants), it is still totally worth it in my project.

I had a wrong mental model of Rust's thread-local variables performance

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 25 Dec 2024 23:22:00 +0200

I had always assumed that accessing thread-local variables is somewhat expensive. This made me reluctant to use them in my code.

However, it turns out that thread-local values in practice have a similar cost of accessing them as normal global variables.

I've made a quick-and-dirty benchmark and here are its (non-scientific) results:

bin::do_static_read: 1.28 ns/iter
bin::do_thread_static_read: 1.28 ns/iter
lib::do_static_read: 1.28 ns/iter
lib::do_thread_static_read: 1.28 ns/iter

Of course, this is a (probably flawed) microbenchmark, but it still shows that I should not be worrying too much about the cost of Rust's tracing crate (which uses thread-locals in several places).

Here's the relevant disassembly (on x86-64 machine, as reported by cargo asm):

# accessing a static variable in an executable
mov rax, qword ptr [rip + VAR]

# accessing a static variable in a library
mov rax, qword ptr [rip + VAR]

# accessing a thread-local variable in an executable
mov rax, qword ptr fs:[VAR::VAL@TPOFF]

# accessing a thread-local variable in a libary
lea rdi, [rip + VAR::VAL@TLSLD]
call __tls_get_addr@PLT
mov rax, qword ptr [rax + VAR::VAL@DTPOFF]

For static variables, Rustc was able to use PC-relative addressing mode (which is quite efficient), but thread-local variables are a bit more interesting.

Apparently, thread-local storage is implemented differently when defining it in an executable or a (shared) library. This StackOverflow comment and the OsDev article seem to have some details regarding it. What seems to be happening is:

when defining a thread-local in a binary, the compiler is able to use the FS segment register to reference the thread-local area
when defining a thread-local in a library, the compiler must use a less efficient way of obtaining the base address of the thread-local area via calling __tls_get_addr
Why the benchmark results are the same for bin and lib:
- I think Rustc does cross-library inlining (to a certain degree), so the actual assembly being executed (after inlining) is probably a bit different than cargo asm has reported
- If I add #[inline(never)] to a function that contains the reading code, thread-local access latency increases to 1.58 ns/iter.

Constructing DST pointers in Rust

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 13 Oct 2024 22:37:00 +0200

Recently I was working on a Rust library that implements several data structures. One of the things I needed to do was to allocate variable-length structs. I didn't immediately find a good way to do this (even in Rustonomicon), so I think it's worth documenting how I ended up implementing it.

I found some very useful hints in the code of slice-dst and zerocopy crates.

Background

Rust has several kinds of dynamically sized types (DSTs):

slices: [T]
trait objects: dyn Trait

structs that end with a DST:

struct LenPrefixedBuf {
  len: usize,
  data: [u8]
}

In this post, I'll ignore trait objects and focus on the other two kinds of DSTs: slices and structs with slices.

One way to create a pointer to a DST is to create a pointer to a statically sized type and cast it to a pointer to a compatible DST. For my use case, this was not possible because the size of slices has to be dynamic.

Constructing pointers to slices

Pointers to slices can be created using std::ptr::slice_from_raw_parts:

let n: usize = 10;
let layout = Layout::array::<i32>(n).unwrap();
let allocated: *mut u8 = unsafe { alloc(layout) };

let slice_ptr: *mut [u8] = slice_from_raw_parts_mut(allocated, n);

Constructing pointers to variable-length structs

Constructing pointers to a variable-length struct is more complicated, and there is no straightforward way to do it. There's an unstable feature that should make it easier, but I want to avoid using unstable features unnecessarily.

The way to construct a pointer is as follows:

First, make a slice pointer with the address of a struct and the correct number of elements. The element type of a slice does not matter.
Then cast the slice pointer to a struct pointer.

This works because of the way pointer casting is specified for DSTs (see https://doc.rust-lang.org/reference/expressions/operator-expr.html#pointer-to-pointer-cast):

If T and U are both unsized, the pointer is also returned unchanged. In particular, the metadata is preserved exactly.

#[repr(C)]
struct BufWithLen<T> {
    header: BufWithLenHeader,
    data: [T],
}
#[repr(C)]
struct BufWithLenHeader {
    len: usize,
}
impl<T> BufWithLen<T> {
    fn layout(n: usize) -> Layout {
        Layout::new::<BufWithLenHeader>()
            .extend(Layout::array::<i32>(n).unwrap())
            .unwrap()
            .0
            .pad_to_align()
    }
}

let n: usize = 10;
let layout = BufWithLen::<i32>::layout(n);
let allocated: *mut u8 = unsafe { alloc(layout) };

// 1) First create pointer with the right address and the right metadata
let dummy_slice: *mut [u8] = slice_from_raw_parts_mut(allocated, n);
// 2) Then cast pointer to the correct type
let buf_ptr: *mut BufWithLen<i32> = dummy_slice as *mut _;

References to DSTs

A pointer can be converted to a reference by dereferencing it:

let p: *const [T] = ...;
let r: &[T] = unsafe { &*p };

Of course, the unsafe block requires establishing the appropriate safety guarantees.

Using TortoiseMerge in Linux and Mac OS

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 28 Sep 2024 19:05:00 +0200

When I was using Tortoise SVN many years ago, I got used to its integrated merge conflict resolution tool TortoiseMerge. Since then I have tried many other merge tools, but I still consider it to be the best. I find that it can handle even complex merges and its merge results are very trustworthy (which I cannot say about most other merge tools).

That's why I continue using it, even though it was originally only available for Windows. With the help of Wine, it works well on Linux and Mac OS. In this post I would like to describe how to install and integrate it with Git.

Wine

TortoiseMerge can be run on Linux or MacOS using Wine (which is an API compatibility layer for running Windows applications on Unix-like operating systems).

Wine should be installed together with winetricks (which is a set of scripts to help install Windows apps on Wine). On Arch Linux, this can be done with:

sudo pacman -S wine winetricks

On MacOS it can be done with Homebrew:

brew install wine
brew install winetricks

Wine prefix

Wine has a concept of “prefixes” which are isolated environments for Windows apps. Using a separate prefix for each app ensures that applications don't affect each other, and it makes it much easier to remove apps.

There is a default prefix (~/.wine), but I like to make separate prefixes for every application that I run via Wine. I usually create Wine prefixes in a per-application directory under ~/opt.

mkdir -p ~/opt/tortoisemerge/wineprefix

To use the prefix, the WINEPREFIX environment variable should be set when running wine-related commands. This variable contains the path where the prefix should be stored. Wine will automatically create and populate if necessary.

Configuring Wine

I usually change some Wine's defaults right after initializing it. I especially don't like that it defaults to 96 DPI which makes fonts too small for modern monitors.

To fix it:

Run winecfg

WINEPREFIX=~/opt/tortoisemerge/wineprefix winecfg

In the Graphics tab of the Wine configuration window change the DPI to a more appropriate values. E.g., 2x scale equals 192 DPI.

Installing TortoiseMerge on Linux and Mac OS

The latest version of TortoiseMerge is distributed as a part of TortoiseSVN.

Install some required Windows libraries that are missing from Wine:
```
WINEPREFIX=~/opt/tortoisemerge/wineprefix winetricks uiribbon
```
Download the installer from https://tortoisesvn.net/downloads.html. I'm using the x86-64 version.
Install it in Wine (set the filename to the actual installer filename):
```
WINEPREFIX=~/opt/tortoisemerge/wineprefix wine ~/Downloads/TortoiseSVN-1.14.7.29687-x64-svn-1.14.3.msi
```
This will pop up the typical Windows installer, just press “Next” a couple of times.

Check that everything works after installation:

WINEPREFIX=~/opt/tortoisemerge/wineprefix wine ~/opt/tortoisemerge/wineprefix/drive_c/"Program Files"/TortoiseSVN/bin/TortoiseMerge.exe

The TortoiseMerge window should appear:

Integration with Git

TortoiseMerge works, now we need to tell Git to use it as a diff and merge tool.

git config --global mergetool.tortoisemerge.cmd 'WINEPREFIX=~/opt/tortoisemerge/wineprefix wine ~/opt/tortoisemerge/wineprefix/drive_c/"Program Files"/TortoiseSVN/bin/TortoiseMerge.exe -base "$BASE" -mine "$LOCAL" -theirs "$REMOTE" -merged "$MERGED"'
git config --global difftool.tortoisemerge.cmd 'WINEPREFIX=~/opt/tortoisemerge/wineprefix wine ~/opt/tortoisemerge/wineprefix/drive_c/"Program Files"/TortoiseSVN/bin/TortoiseMerge.exe -mine "$LOCAL" -theirs "$REMOTE"'

git config --global merge.tool tortoisemerge
git config --global diff.tool tortoisemerge

After that Git will use TortoiseMerge as a default merge and diff tool.

Efficient batching

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 25 Jan 2024 23:58:00 +0300

There is a popular misconception that introducing batching sacrifices latency to improve throughput.

I want to share a technique that enables batching without incurring any additional latency. I call it the “non-blocking batching” but this name isn't super popular.

I first read about it in an article by Lorenzo Alberton but I haven't seen much use of this technique in the wild.

Batching

Batching is a common technique to improve the efficiency of processing a queue of tasks. It is performed by gathering multiple tasks into one batch and processing multiple tasks at once instead of one by one.

When processing tasks in a queue, there are two metrics that we care about:

Throughput - How many tasks per second we can process
Latency - How much time has passed between enqueueing the task and processing it

Batching improves throughput by amortizing the overhead of expensive operations that can be done once per batch instead of once per task.

Examples of expensive operations:

database transaction commit
write to a file on a disk
network request
locking a mutex.

How batching is usually done

Usually, batch processing is implemented like this:

Wait for a new task in a queue (blocking indefinitely)
Add the task to a new batch
Wait until a specified timeout has passed, adding new tasks to the batch
- If the maximum batch size was reached before the timeout, stop waiting
Process all tasks of the batch
Loop to the beginning

For a random task distribution (governed by a Poisson random process), the batch processing looks as follows:

On this diagram, there are two timelines:

The upper timeline is the arrival of new tasks into the queue.
The bottom timeline is the processing. The yellow portion represents the waiting for new tasks in the batch, green is the batch processing.
Lines connect queue tasks with the batch that they were processed in.

Analysis of batching efficiency

We have two parameters for batching:

Maximum batch size.

Batch processing time is non-linear in the batch size, and after some threshold, batching stops being effective. Limiting the batch size prevents this from happening.
Batch waiting time.

Increasing batch waiting time allows more tasks to accumulate into a single batch.

But simultaneously this is a wasted time that increases the latency of every task by approximately half of the batch waiting time.

Usually, the batch waiting time is set to around half of the acceptable latency or just by eyeballing it.

But, counterintuitively, it is best to set this time to zero. That is, we have the following batching algorithm:

Wait for a new task in a queue (blocking indefinitely)
Get up to MaximumBatchSize tasks from the queue without waiting
Process the tasks
Loop back to the beginning

Let's analyze the behavior of this batching in several scenarios.

Scenario 1. The rate of arrival of new tasks is slow (much slower than processing time). In this case, every task will be processed immediately with no additional latency. Additional waiting time would bring no benefit and just increase the latency.
Scenario 2. Tasks arrive at regular intervals, but the average interval length is smaller than the processing time of one task.

We get full utilization of processing resources with no additional latency. The batch size automatically adapts to the speed of new tasks’ arrival and the processing time.
Scenario 3. New tasks arrive irregularly, in bursts. When idle, the first task of a burst gets processed immediately and the rest gets into one batch.

This might be the only case when waiting might make sense since it will give a change for the entire burst to be processed as one batch.

Benchmarks

I did a simulation of different cases and found out that the non-blocking batching beats the conventional batching every time.

Here's how two batching algorithms compare in different scenarios.

Scenario	Average latency with non-blocking batching	Average latency with blocking batching
Slow rate of new tasks	1.55s	1.94s
Fast rate of new tasks	5.13s	7.1s
Bursty new tasks	2s	2.06s
New tasks arrive at the same as they can be processed	5.14s	7.12s

We can see that in all scenarios it is better to do non-blocking batching compared to conventional blocking batching.

In all cases, the following parameters were used:

Cost of processing: 1s per batch + 0.07 s per task in batch
Batch waiting time: 0.5s

The code for the benchmark is available at https://github.com/dmitryvk/nowait-batch-modeling.

Conclusion

The non-blocking batching is the superior batching algorithm. It gives a better latency, has fewer edge cases, and does not require complicated tuning.

It should be the default when developing systems that have a queue somewhere.

Some further notes

Although non-blocking batching improves latency and throughput, it also in many cases increases the utilization (that is, the fraction of a time when the processor is active). If we have a low rate of arrival of new tasks and we want to minimize the utilization of the processor, then it is beneficial to introduce artificial delays.
This batching easily generalizes to multiple workers. Each worker would do batching independently.

On the usefulness of topological sort

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 27 May 2023 21:16:00 +0300

The topological algorithm is very useful, yet it is quite easy to understand and implement. I've implemented it quite frequently and want to share some of the examples where it was useful for solving practical tasks.

Topological sort

Topological sort can be defined as follows.

First, we have a graph G(V, E) that consists of a set of vertices V and a set of edges E.

For the purposes of the topological sort, an edge A→B models a dependency of vertex A on vertex B. That is, we want the processing of vertex A to happen only after the processing of B is completed. The topological sort of the graph is the permutation of its vertices (i.e., every vertex must be used only) such that a vertex is listed after all its dependencies.

For a given graph, the topological sort might not exist (if and only if there are cyclical dependencies) or there might be many topological sorts (as is usually the case).

There are two possible topological sorts for this graph:

4, 5, 3, 1, 2
5, 4, 3, 2, 1

I won't recite the algorithm here, but here are some links that have detailed descriptions of several algorithms for topological sorting:

Uses of topological sort

Whenever we have a task that requires ordering items or actions based on some criteria, topological sorting can be useful. Here are some cases where I have used it in real-world projects.

Verifying the correctness of workflows in business process management software.

I've been making a workflow system in which users would edit their workflows via the GUI.

Workflows consist of several steps. Some of the steps can be performed in parallel while some are constrained to be performed one after another. When a user finishes editing a workflow, it has to be verified that all of the steps could be completed in some order. Topological sort of workflow steps does exactly that - if topological sort exists then the workflow is valid.
Output ordering for code generation.

The order of definitions and statements in generated code oftentimes matters.

For example, to produce an SQL code to create tables, indices, and constraints, we have to ensure that:
- an index is created only after the corresponding tables and columns
- a foreign key constraint is created only after the corresponding tables, columns, and unique constraints
Another example is the bundling of javascript code. Before the introduction of ES6 classes and modules, we had to rely on libraries to emulate classes. Class definitions using those libraries were inherently order-dependent: class could only be defined after its superclass was defined. This necessitates ordering bundled modules in the order of topological sorting.
Code initialization order

I was designing a modular system that loads and initializes plugins and modules. The initialization of a module could invoke components provided by another module and register its components, as well as do some side effects. This means that modules depend on each other, and the order of module initialization is specified by the dependencies of modules.
Type inference in nested data queries

In several BI systems, we have let users define custom data queries that could refer to other data queries. To infer the types of result data table columns, queries had to be processed in the order specified by their interdependencies.
Database migrations

I was designing a database migration framework for a larger framework for building web apps. It was designed for modularity: web apps are composed of modules (some are pre-built, some are custom-built) built by different teams.

The framework was designed with one big shared database in mind, and each module was responsible for its own set of tables in the shared database. To manage changes to the database schema, we employed migrations but quickly ran into the problem of ordering the execution of migrations. The complexity of database schema management was exacerbated by the concurrency of development: different teams released versions of their modules on different schedules without too much coordination.

Since all modules were in the same database, they executed DDL statements (such as CREATE TABLE or ALTER TABLE) or DML statements (such as INSERT INTO or UPDATE) that referred to the tables owned by other modules. Essentially, each migration is required to be executed before or after some other migrations (that created or dropped the corresponding tables or columns).

By specifying this ordering in the migration itself, we could then find an appropriate order for database migrations.

Issues with topological sort

Some fine points must be considered when employing topological sort:

When a graph contains a cycle, the toposort algorithm just gives up. But in some cases, we want to process the graph anyway, even if some parts of it cannot be sorted
The topological sort definition is ambiguous. A graph may have multiple topological sorts. This issue comes in two dimensions:
- Different algorithms may produce different order
- Subtle changes in the graph may produce drastically different topological order

Programming contests are good for me (and probably for you too)

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 06 May 2023 23:31:00 +0300

Criticism of programming contests

I have come across numerous instances of criticism of competitive programming and programming contests. To summarize, the criticism boils down to these points:

Competition-style tasks during job interviews are the wrong way to filter candidates;
Competitive programming does not give a relevant experience in software engineering;
Competitive programming is detrimental to software engineering work.

I must say upfront that I agree with the first point. The problems that are used in programming contests are quite different from the tasks that are normally encountered in day-to-day work.

But it is the other points that I want to address. Programming contests give a lot of skills and experience that are usually quite hard to obtain elsewhere. The programming contests were transformative for my programming skills.

Typical structure of a contest

Programming contests are designed to test and challenge the skills of designing and implementing algorithms. They are relatively short timewise (usually around 5 hours). In some contests, teams compete while in others individuals do.

During the contest, the participants are offered a small number of problem statements to solve. Participants may solve tasks in any order.

A problem statement

A problem statement is a non-ambiguous description of a task to solve.

The solution to the problem is a source code for a program that reads input data (that describes a specific instance of a problem) from STDIN and outputs the solution STDOUT.

The time and memory that the solution may use are limited. The constraints on time and memory are always present in the problem statement. Since time and memory are constrained, the problem statements also include limits on possible input data such as the number of elements, maximum values for numeric values, and so on.

The problem statement also includes several instances of input data with the correct corresponding output data (collectively called “a test case”) to serve as examples.

Scoring and submissions

Having coded the solution, a participant submits it for grading. The grading is automated and the result is available in a short time (usually in about a minute). The grading system compiles the source code and runs the program on many test cases and checks its output. The result of a run on some test is either “OK”, “Wrong Answer”, “Runtime Error”, “Time Limit Exceeded” or “Memory Exceeded” (result names are self-explanatory). In case of a failure, the participant will only know the number of the first failing test case without any other specifics.

There are 2 distinct methods to score solutions:

If the solution passes all tests, it gets a score.

The time to solve is also taken into account. So the participants are ranked first by the number of solved tasks (or by the sum of scores) and then by the sum of submission times. Many competitions, including the ACM ICPC, use this scoring style.
Each passed test case gains some amount of score.

This style is not as popular, though some beginner-friendly competitions do use such a style.

The specific skills that contests help level up

Here's a list of skills that are required for successful participation in a programming competition.

Data Structures and Algorithms

This is the obvious one.

The problems are primarily designed to practice the knowledge of algorithms and the skill to combine them or invent new ones on the spot.
Attention to Details

Many problems and algorithms have numerous edge cases that must not be missed. Sometimes a tiny detail might mean a difference between either a trivial or a very complicated algorithm.

All participants are trained to read the task descriptions carefully and not miss any important details.
Selection of appropriate algorithm based on the scale and complexity of a problem.

Oftentimes it is possible to solve a problem using either of several different algorithms, and one needs to pick an appropriate algorithm. It takes a certain skill to be able to pick an algorithm based on a task description. Some factors that affect algorithm selection are:
- The ease of the implementation of an algorithm.
  
  It makes no sense to use an advanced algorithm (or a data structure) when a simpler one would do. The complexity theory, big-O notation together with some well-known scale constants help here.
- Time and memory constraints, input size, and specifics of the input data.
  
  The bigger the input size, the more efficient the algorithm has to be.
  
  The amount of memory and time that the algorithm takes depends on the size of an input (e.g., number of elements in the input), limits on numeric values (e.g., this is very important for dynamic programming algorithms), or some properties of an input (e.g., the number of distinct values, number of graph components, presence of cycles, etc.).
- Algorithm assumptions and special properties of input data.
  
  Some algorithms may only be applied if an input satisfies some criteria. E.g., Dijkstra's algorithm won't work on graphs with negative edges, and heap-based algorithms won't work if we need to preserve the order of elements.
Thorough testing of code and algorithms

The strict grading procedure quickly teaches to only submit tested solutions, since the code that has not been thoroughly tested does not have a chance to be accepted.

Since the tasks and algorithms are quite complex, manual example-based testing does not suffice, and more advanced testing techniques are required.

Some of the commonly employed techniques are:
- randomized and property-based testing
- assertions for the invariants
- oracle-based testing. Some common oracles are:
  - alternative (usually more trivial) implementation of an algorithm
  - generating the input data together with the correct answer or in such a way the correct answer is known
Staying focused in high-stress conditions

Competitions are always stressful. Contests can be quite intense, especially when the time is running out.

It is natural to do “shotgun debugging” when under stress, i.e. non-systematically trying some random ideas without first acquiring an understanding of the nature of the bug in hopes that it fixes the bug. Instead, we must be methodical and keep using proven working techniques since there is no other way.

How the structure of contests helps acquire skills

Rapid feedback

We learn and gain experience by doing something and seeing if it works or not. This requires that the feedback loop is quick and gives good and actionable feedback.

The feedback loop in competitions is orders of magnitude faster and more precise compared to typical software engineering projects. Commercial software projects are lengthy. It might take months or years to see all the latent bugs that may be lurking in the codebase. Competitions provide near-instantaneous feedback - one quickly learns to test the code and which testing techniques are quick and reliable.

Commercial projects are long, complicated, and have a lot of parts and participants. It might be hard to learn from experience on real-world projects since:

It usually takes a significant time for the code to get from a developer to exercising in production of all the functionality including the edge cases One may even not work on this project anymore.
Mistakes are costly.

Therefore projects have a lot of redundancy and fail-safety mechanisms that mask some problems in the software.

It is very beneficial for real-world software to have mechanisms such as retries, self-healing, and periodic reboots. But for education, the software should fail loudly and explicitly – so we can learn not to make the same mistakes again.

In programming competitions, such solutions simply won't be accepted.
It is hard to attribute the success or failure of a project to specific parts of the software or to the work of specific individuals or teams.

Strict solutions checking with tight time and memory constraints

The test suites for problems have very good coverage and it is almost unbelievable how well they can find bugs and flaws in solutions.

It is virtually impossible to have a buggy or inefficient solution accepted since the grading system checks correctness and efficiency.

Hiding of failed tests, penalties for failing submissions

Since a participant does not see the failing test case, the attempts to brute-force the solution are infeasible.

Penalties for faulty submissions encourage participants to not rely on any provided tests. Instead, participants must implement their own testing. Since the contest time is limited, this testing must also be effective and must not take too much time.

Conclusion

Based on all of the above,

competitive programming is not the same as real-world software engineering;
nonetheless, the training for programming competitions can immensely improve some of the software engineering skills;
To learn efficiently, it is very useful to have good feedback loops.

The faster and more accurate they are, the easier it is to learn. Contests offer good feedback loops for some of the kills, for some of which there are a few alternatives.

How to implement async stream in Rust without resorting to macros

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 07 Jan 2023 10:35:00 +0300

I published the async-fn-stream crate which is a lightweight alternative to async-stream.

Async iteration

Rust provides support for async iteration via the Stream trait. It allows to decouple producer of data from the consumer, just like an Iterator allows the same for blocking code. This is immensely useful if the producer is expected to produce a large (or infinite) sequence.

Both Stream and Iterator are quite easy to use:

// Iterator
for x in produce_iter() {
    println!("{x}");
}

fn produce_iter() -> impl Iterator<Item = i32> { ... }

// Stream
let stream = produce_stream().await;
pin_mut!(stream);
while let Some(x) = stream.next().await {
    println!("{x}");
}

async fn produce_stream() -> impl Stream<Item = i32> { ... }

We can see that the usage of Stream has somewhat more boilerplate code but it's still quite OK. Most likely Rust will in the future gain language support for using Streams in for loops.

The `async-stream` crate

Currently, it's not too ergonomic to hand-code implementations of either Stream or Iterator. The Iterator::next() method might sometimes be cumbersome to implement, but usually, it's doable without too much effort.

But Stream::poll_next() is much harder to implement. Often this necessitates storing (and swapping) futures inside of Stream implementation struct which is quite hard to do without unsafe code. To tackle this issue, Tokio team has written a great crate async-stream which I recommend checking out.

This macro lets us create an async stream with ease:

use std::path::Path;

use async_stream::stream;
use futures::{pin_mut, Stream, StreamExt};
use tokio::{
    fs::File,
    io::{AsyncBufReadExt, BufReader},
};

async fn read_lines(path: &Path) -> std::io::Result<impl Stream<Item = std::io::Result<String>>> {
    let file = File::open(path).await?;
    let mut file = BufReader::new(file);

    let result = stream! {
        let mut buf = String::new();
        loop {
            buf.clear();
            match file.read_line(&mut buf).await {
                Ok(0) => break,
                Ok(_) => yield Ok(buf.clone()),
                Err(error) => yield Err(error),
            }
        };
    };

    Ok(result)
}

And it even runs and provides correct output!

Why I don't like to use `async-stream` in production code

So, is the problem solved? Maybe so, but for me, unfortunately, this solution leaves a lot to be desired.

And I must say upfront that this is not a fault of async-stream - those are general problems associated with macros in Rust. async-stream just makes it more apparent because stream! { ... } often encompasses large chunks of code.

Let's look closely at this code.

The first that I notice is that rustfmt does not even try to reformat the code inside stream! macro. For example, if I mess up the formatting of this code like so:

let result = stream! {
    let mut buf = String::new();
    loop
    
    
    {
                    buf.clear(); match file.read_line(&mut buf).await {
            Ok(0) => break,
                Ok(_  )   => yield Ok(buf.clone()), Err(error) => yield Err(error),
        }
    };
};

then rustfmt will leave this code as-is. And if you use rustfmt --check in the CI pipeline to ensure that formatting stays consistent, it also won't notice any inconsistencies in formatting.

The second and much more important downside is that some features of the IDE do not work as well as they should. Here's what happens if I try to use autocomplete in Visual Studio Code with rust-analyzer:

An alternative to `async-stream`

Having found this solution unsatisfactory for my needs, I went ahead and published my take on it. Meet the async-fn-stream crate.

It's the same idea as async-stream but implemented without resorting to macros.

Here's what the code looks like and the summary of differences:

async_stream::stream! { ... } is replaced with async_fn_stream::fn_stream(|emitter| async move { ... }) (there's also a try_fn_stream which is similar to try_stream!);
yield X is replaced with emitter.emit(X).await.

use std::path::Path;

use async_fn_stream::fn_stream;
use futures::{pin_mut, Stream, StreamExt};
use tokio::{
    fs::File,
    io::{AsyncBufReadExt, BufReader},
};

async fn read_lines(path: &Path) -> std::io::Result<impl Stream<Item = std::io::Result<String>>> {
    let file = File::open(path).await?;
    let mut file = BufReader::new(file);

    let result = fn_stream(|emitter| async move {
        let mut buf = String::new();
        loop {
            buf.clear();
            match file.read_line(&mut buf).await {
                Ok(0) => break,
                Ok(_) => emitter.emit(Ok(buf.clone())).await,
                Err(error) => emitter.emit(Err(error)).await,
            }
        }
    });

    Ok(result)
}

And now, code completion works fine:

If you've also tried using async-stream and met similar issues, I recommend checking my crate out.

Small example of async GUI code with Gtk-rs: alert() function

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 29 Mar 2020 20:29:45 +0300

Let's suppose that I want to write the following function in Rust with Gtk-rs:

/// Display the message in the dialog window and wait for it to be dismissed by the user.
pub async fn alert(title: &str, text: &str);

After some experimentation, I found that one good way to implement such a function is to use one-shot channels. Currently, futures crate provides them on stable releases — so I'll use those. Note that Rust's async ecosystem still evolves and things might change in the future.

Oneshot channels are created with oneshot::channel() function and have two ends: sender and receiver. The key property is that the receiver end is a Future. We will use this to build our functions.

In this article, I'll walk us through the implementation of alert() function.

That's a function epilogue and some use's which we'll need later. Note that the function is declared as async which means that it actually returns an anonymous implementation of Future<Output=()>.

/// Display the message in the dialog window and wait for it to be dismissed by the user.
pub async fn alert(title: &str, text: &str) {

    use gtk::prelude::*;
    use std::rc::Rc;
    use std::cell::RefCell;
    use futures::channel::oneshot::channel as oneshot_channel;

We will need a channel that will hold the result of our function. Since the result is an empty unit value, this channel will act as an event or barrier on which further code will wait.

    let (result_sender, result_receiver) = oneshot_channel();

Next, we will have to deal with some memory management details. Here are a few important facts:

oneshot::Sender::send() method consumes Self
Gtk-rs's accepts bare Fn(&Self) functions as callbacks (e.g., see DialogExt.connect_response). These may be potentially called multiple times and hence they capture their environments as immutable &-borrows.

These two facts combined together means that we need to wrap result_sender in several wrappers:

Option. We will use Option::take() method to be able to consume the sender
RefCell. Since our callback is an Fn(), the captured environment is immutable and we need interior mutability to be able to send into the channel.
Rc. result_sender will be shared between our main async stack frame and callback.

    let result_sender = Rc::new(RefCell::new(Some(result_sender)));

Then we are ready to build our UI widget.

    let dialog = gtk::MessageDialogBuilder::new()
        .buttons(gtk::ButtonsType::Ok)
        .message_type(gtk::MessageType::Info)
        .text(text)
        .modal(true)
        .title(title)
        .type_(gtk::WindowType::Toplevel)
        .build();

Then we should attach handler for response signal which is emitted when user responds to dialog (e.g., by closing it or by click on OK button). In our case, we don't care what was the user's response - we just need to known when user dismissed the dialog.

The Rc with the sender is cloned; in this case, it's not strictly necessary, but in general, it is usually required to clone objects that go into callback's captured environment.

    let result_sender2 = result_sender.clone();
    dialog.connect_response(move |_, response| {

In the response signal handler, we need to notify the channel that we are done. First we unwrap all layers of wrappers and then send the unit value into the channel.

        println!("Response={}", response);
        let result_sender = result_sender2.borrow_mut().take().unwrap();
        result_sender.send(()).unwrap();
    });

After the signal handler is connected, we should show the dialog.

    dialog.show_all();

And then we await from the receiver end of the channel. Since we are awaiting asynchronously, Gtk+ event loop keeps processing events concurrently while we are waiting. And that's very important — otherwise, our dialog which we just created would not be able to show itself on a screen and accept user input.

    result_receiver.await.unwrap();

After the response signal, the dialog window will stay open and be visible to the user. We need to close the window with .destroy() method.

Note: Gtk+ terminology of close, delete and destroy is pretty confusing. I've had to re-read documentation and examples multiple times to double-check my code.

    dialog.destroy();

The dialog window now is not displayed but the corresponding Gtk+ object still exists. Rust's RAII (Drop) will clean it up automatically at this point.

And now, we just need to run this function. The current version of Gtk-rs has all the necessary pieces to run async GUI code.

The following code will initialize Gtk+ and then run async code to completion and then will quit the program.

async fn ui_main() {
    alert("Message", "Hello, world").await;    
}

fn main() {
    gtk::init().expect("Failed to initialize Gtk+");

    glib::MainContext::default().spawn_local(async move {
        ui_main().await;
        gtk::main_quit();
    });

    gtk::main();
}

Note: Some time ago I released a gtk-future-executor crate which could be used to execute the future on Gtk+ event loop. This crate should be considered obsolete.

The whole example is available as a gist.

Note: this post was written assuming the following versions:

Gtk-rs 0.8.1
Async-std 1.5.0
Glib-rs 0.9.3
futures 0.3.4

Notes on guaranteed shutdown of test containers

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 08 Dec 2019 03:15:07 +0300

One use for containers is to provision dependencies for automated software testing.

This approach is supported by at least several libraries for several languages. These libraries are usually easy to set up and easy to use.

Usually, one would start the container in setUp and stop the container in tearDown. This approach usually works fine but it does one quite big drawback: if a test crashes then you leak the container.

In my prior experiences with provisioning dependencies for automated tests, it is very easy to exhaust resources when debugging tests or when tests fail to clean up after themselves. I believe that tests should be as robust as possible so I find this situation unacceptable.

I had not seen any mention of this in documentation so I decided to share what I use to prevent this issue.

I had tried different techniques to overcome this issue and I had settled on the following technique:

All containers are started with autoremove option to ensure that storage is cleaned up when the container is stopped. But this is not enough - we need to ensure that the container will actually be stopped.
When the container starts, the watchdog is injected which monitors whether there is an active user of the container.
If watchdog detects that test software has unexpectedly disconnected from container then it forcibly terminates the container.
One of the easy and robust ways of implementing watchdog is the following:
1. A shell script is injected into a running container that reads from standard input until it is closed.
```
# this loop will keep running while tests are attached to the container
# when docker detects client disconnect the loop will be terminated
while read VAR; do
  true;
done;

kill -9 1 # or some other command that will terminate the container
```
2. Background task inside the test process starts and attaches to the injected script.

This reliably ensures that tests don't leave unused containers behind.

Some container management libraries do not even try solve this issue (e.g., popular .NET library TestContainers-dotnet).

TestContainers (a java library) uses different approach for ensuring guaranteed shutdown and release of all used resources. It starts a separate container (called ryuk) and establishes persistent connection to it; ryuk then uses this connection to monitor presence of its client. I haven't used it extensively, but apparently it works really well. Note that this is not mentioned in TestContainers’ documentation and I even missed this very important feature on my first review of such libraries.

(Updated on 2019-12-19 to mention ryuk)

Why Rust instead of C++

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Fri, 17 Nov 2017 22:13:00 +0300

The talk Curiously Recurring C++ Bugs at Facebook by Louis Brandy at CppCon 2017 demonstrates several examples of when Rust might be better than C++.

ICFPC 2012 is coming

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Tue, 03 Apr 2012 23:36:00 +0300

I can't help but note that ICFPC is due to start on the 13th of July.

Notes on threads and spaghetti code

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Mon, 13 Feb 2012 23:45:00 +0300

Recently there were a lot of talks going along the lines of “threads and blocking code is bad and callbacks-based asynchrony is good”.

TL;DR

event-driven concurrency is pretty much equivalent to thread-based concurrency
it is possible to implement threads with a low overhead
threads allow writing natural sequential code
therefore threads are more useful than asynchrony based on events and callbacks

A little bit of theory

Both synchronous blocking computation processes and async non-blocking computation processes have some state associated with them. For example, for an HTTP server that state might contain:

network socket and associated kernel resources
buffer in userspace for socket IO
HTTP protocol parser state
HTTP headers and body
HTTP request body parser state
response buffer
other state that is necessary for HTTP handler state (e.g., state and buffer for database query processing)

This state is always present independent of the style that the code is written in - whether it uses callbacks or is represented as a state machine or if it's sequential blocking code running in its own thread.

Proponents of some fashionable and hyped technologies sometimes present the idea that the only way to write “web scale” apps is to ditch threads and blocking calls, and switch to a style that I call “callback spaghetti”. A more formal name for “callback spaghetti” is the continuation-passing style. Transforming code to CPS (continuation-passing style) is a mechanical task, but when done manually it becomes quite tedious and error-prone and it also balloons the size of the code.

Continuation-passing style by passing into each function a callback which is to be called by the inner function the result is available. Furthermore, composing functions involves composing callbacks. It is possible to see that the resulting callback chain is equivalent to a call stack, albeit represented as a linked list instead of a contiguous array.

My argument for sequential code running in separate threads is as follows: such sequential code is easier to read, and flow control structures are implemented more or less easily. Such code is more reliable since there are simple and reliable mechanisms for error handling; there are exceptions; stack traces contain useful information; it is easier to ensure the timely release of resources.

In practice

The main argument against thread-based concurrency is that threads occupy a lot of resources, hence we can't create too many of them. For native threads this might be true: they have relatively large creation overhead, they occupy a significant amount of memory for stack (usually several megabytes) and thread-local storage, and they have a lot of overhead for context switching. But OS-native threads are just one way to implement threads: threads can be implemented with lower overhead.

Let's consider Linux glibc threads as an example. Here's what is contained inside a thread:

per-thread OS task scheduler state inside a kernel
stack space
CPU context (saved CPU and FPU registers)
signal mask
thread return value
synchronization objects

The main areas of overhead for such threads are:

thread switching always happens from the OS kernel, i.e. for each thread switch there is a transition from userspace to kernel and back. This is bad for performance if such switches happen often
thread scheduler, depending on its implementation, might visibly slow down if the number of threads gets too large
threads have quite a large stack by default - around several megabytes. In my opinion, this is the main reason why we cannot have too many native threads.

POSIX defines several means to manage contexts and switch them - makecontext, getcontext, setcontext, swapcontext. A context in POSIX contains the entire thread state except for kernel objects. With the help of the contexts, we can make our own threads. Such threads that are created in userspace are called green threads. When using green threads, we have to provide the stack memory ourselves. This lets us have control over and minimize memory usage by threads.

The largest downside of POSIX contexts is that a context also contains the signal mask which entails a syscall to change it in order to maintain context switch semantics defined by POSIX. Oftentimes we don't have a particular need to manage a signal mask. If we implement swapcontext manually (which would take us a dozen of assembly instructions per CPU architecture) then the overhead would reduce significantly.

As a simple optimization, we can skip saving unused CPU registers. As a further optimization, we could employ liveness analysis to avoid saving local variables that are not used while a thread is not being scheduled.

In conclusion, the usage of green threads allows us to write the same good old straightforward sequential code and at the same time have a lot of threads running. For this reason, I much prefer using green threads for network IO-related tasks and native OS threads for everything else.

Looking for a new maintainer for CL-GTK2

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 02 Feb 2011 09:09:00 +0300

I haven't been writing anything for Lisp for some time, and my projects are slowly getting unmaintained. Cl-gtk2 is one of them, unfortunately. But I would hate to see cl-gtk2 stop evolving since it is truly useful.

That's why I'm looking for someone who would take cl-gtk2 from me and keep maintaining it. I would provide any kind of support if necessary.

SBCL-1.0.45 with Windows threads

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 19 Dec 2010 23:40:00 +0300

I've merged the patch from Anton Kovalenko to use TLS slot 63. There was some minor hiccup - I merged one commit but totally missed the next commit in the series and lost some trying to untie the know.

I've then reabased my branch onto SBCL-1.0.45 and published the binary to https://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.45-threads.msi.

Couple of words regarding FS:0x14

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 21 Nov 2010 09:20:00 +0300

The Apache Harmony JVM on Win32 also uses the Arbitrary Data slot of TIB (link).

And also this slot is used as a TLS pointer in a GML library (OpenGL Multithreading Library) inside the Spring project (Sprint is a game engine that can run games such as Total Annihilation).

A useful article on codegen techniques

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Tue, 16 Nov 2010 10:32:00 +0300

I've stumbled upon a reference to the article One-pass Code Generation in V8 in mr-aleph's post (though it's more of a slide deck than an article). I found the article to be quite good regarding both the contents and the style. It describes the code generation architecture that employs several techniques to generate code without “silly” instruction sequences such as push eax; pop eax; without the use of peephole optimizations but instead by providing more information to code generation routines for primitive operations.

At first sight, this looks quite similar to the code generation architecture of the SBCL compiler.

It is a pure joy to study using articles like that.

cl-sqlite-0.2 released

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Mon, 15 Nov 2010 06:45:00 +0300

I've released the new version 0.2 of cl-sqlite.

I've added support for named parameters in queries. For error signaling, now sqlite-error class is used instead of cl:error.

The link to the new version: http://common-lisp.net/project/cl-sqlite/releases/cl-sqlite-0.2.tar.gz.

SBCL 1.0.44 with Windows threads

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 06 Nov 2010 23:39:00 +0300

I've updated the windows-threads branch to SBCL-1.0.44. Installer is available at https://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.44-threads-1.msi.

SBCL Windows threads implementation notes, part 2

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Fri, 05 Nov 2010 22:28:00 +0300

Suspension

Thread suspension is implemented via safepoints. Safepoint is implemented with a read of the special memory location (‘GC poll address’ which is located in ‘GC poll page’).

In the first phase, the ‘master’ thread unmaps the GC poll page. After this, other threads will at some time get page faults. Several issues must be dealt with:

The reaction to unmapping is not immediate - the thread must reach the safepoint
Some threads will not reach safepoint soon (if a thread is executing foreign code or a blocking system call)
Even if a thread has reached a safepoint, it does not mean that GC can start. The thread may be inside a WITHOUT-GCING section, for example. In this case, the thread may not be resumed with the GC poll page unmapped.

We can draw some conclusions:

Every thread that can reach safepoint (if it's not in foreign code or in blocking syscall) must reach it before GC can proceed.
1. Every thread that can not reach safepoint must not interfere with GC if it suddenly returns to a lisp code
After all the threads have reached safepoint, we must wait for all threads to be ready for GC.

This implies the two-phase suspension.

Phase 1:
1. GC poll page is remapped as unreadable
2. The master thread checks each thread: if it's running lisp code, wait until it reaches a safepoint. Thread is considered to reach a safepoint when its state is STATE_SUSPENDED_BRIEFLY.
Phase 2:
1. GC poll page is mapped again so that threads can run until they are ready for GC
2. The master thread waits for every thread to be ready for GC. This is achieved by waiting for the state of every thread to become STATE_SUSPENDED (except for threads that are ready for GC)

Thread is ready for GC if:

thread_state(thread) == STATE_SUSPENDED; or
thread_is running foreign code and it is not inside WITHOUT-GCING or WITHOUT-INTERRUPTS, and blockable signals are unblocked

For this, the thread-local variable *GC-SAFE* is introduced - it tracks the current readiness for GC of a thread. It is guaranteed that when *GC-SAFE* changes from NIL to T, the thread checks if GC is in progress and enters the suspended state.

Thread interruption

Thread interruption is similar, but we don't need to wait for all threads to reach a safepoint - it is only necessary for the interrupted thread to reach a safepoint.

Phase 1:
1. The GC poll page is remapped as unreadable
2. The master thread checks interrupted thread: if it's running lisp code, wait until it reaches a safepoint. Thread is considered to reach a safepoint when its state is STATE_SUSPENDED_BRIEFLY.
Phase 2:
1. GC poll page is mapped
2. All threads that have reached a safepoint are released

Safepoint code

The safepoint code is called to check whether the thread has something to do related to SBCL internal working. It is called:

When a thread reaches a safepoint and the GC poll page is unmapped
When leaving and entering a foreign code
On other occasions.

Safepoints have several responsibilities.

If there is a GC or thread interruption in progress, the thread has to notify the master thread that it has reached a safepoint. Safepoint does this by changing the state to STATE_SUSPENDED_BRIEFLY and waiting for the state to be changed by the master thread. When it resumes, the thread checks whether it should suspend or interrupt.
If a thread should suspend, it is checked whether it can be suspended. If thread is suspendable, it changes its state to STATE_SUSPENDED; otherwise, it sets STOP_FOR_GC_PENDING (and sets pseudo_atomic_interrupted)
If a thread should interrupt, it either sets INTERRUPT_PENDING and pseudo_atomic_interrupted or executes interruption.
If GC is pending and the thread can do GC, it runs the GC
If an interrupt is pending and a thread can execute it, the thread executes it.

On some occasions, the runtime is in a very fragile state and can not really do anything that safepoint must do (e.g., change thread state, execute GC, execute interruption). These are e.g. using lisp thread synchronization primitives. To control this, the *DISABLE-SAFEPOINTS* variable is used.

GC code is run inside a safepoint, and safepoint code is not reentrable. GC code itself has safepoints (since SUB-GC is a normal lisp function, it calls lisp synchronization routines and does several switches to/from foreign code). To prevent the reentering of a safepoint code, the *IN-SAFEPOINT* variable is used.

Some manuals are factually wrong

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 04 Nov 2010 17:49:00 +0300

I've bought a new Logitech mouse, and the manual for it says this:

Horizontal scrolling (feature may depend on software). This function is not available on Linux

But the funn thing is that it works without a hitch.

SBCL Windows threads implementation notes, part 1

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 04 Nov 2010 14:08:00 +0300

Since I've Windows threads patches to SBCL for review, I'm publishing my implementation notes.

Thread-local storage

For each thread, Windows allocates a Thread Information Block (ref. 1, ref. 2, ref. 3). Thread can access its own Thread Information Block through the FS register (e.g., %FS:0 is the first field in TIB). TIB contains frequently used thread-specific data such as Last Error Number, pointer to SEH (structured exception handling) frame, thread id, and TLS backing store. This structure is documented in ref. 2 and ref. 4. TIB has an ‘Arbitrary’ field which is described as:

The 14h DWORD pvArbitrary field is theoretically available for applications to use however they want. It's almost like an extra thread-local storage slot for you to use, although I've never seen an application use it.

This sound just like what is needed for implementing thread-local storage - a memory location that we can freely use and that is easily accessible. Unfortunately, there is a possibility that some libraries would also use this field for their purposes (and there are examples of such libraries). And when some other conflicting code will run, it will be hard to detect this.

So clearly, a better way of storing the TLS pointer is needed. TIB also contains the first 64 slots for Windows thread-local storage. There are several options for implementing TLS with that:

Windows allows executables to preallocate TLS slots. We can take, e.g., TLS slot 0, and all access to lisp's TLS will go through TLS at a fixed offset. But it would somewhat complicate the initialization of lisp runtime and the build process.
We can grab some other fixed slot number by successively allocating TLS slots (TlsAlloc) until we get the slot we want. After that, we free the slots we allocated that we don't need. This way, we can use TLS slot 63 and lisp's TLS pointer will be at a known offset. It is quite safe to use this slot; libraries on Windows commonly use no more than one TLS slot (and ‘system’ libraries don't even use TLS - they have their own fields in TIB).
We can allocate the TLS slot in a normal Windows way and store it in a global variable. This way, we don't need to do any weird things, but TLS access will have one more indirection. This would further complicate SBCL's TLS access because macros and code generation assume that TLS does not require any indirection. This is the best way to go but it requires more changes.

Currently, TLS is implemented as option 2. On initialization, we take slot 63 (or fail if we couldn't - but I can't imagine the situation where this can happen). When the windows-threads branch will be merged to SBCL, we should consider option 3 because if implemented now, the change would be more than necessary for Windows threading support.

TLS notes:

This only applies to Win32. Win64 is very different
Clozure CL, on the other hand, uses ES segment register and undocumented Win32 functions to allocate a segment and store it in a segment register. This is problematic because WOW64 (the Windows subsystem to run Win32 applications in Windows 64) does not preserve the value of ES register during context switches (including thread preemption). This is the reason why the 32-bit version of Clozure CL does not work on Windows 64.

Thread suspension and interruption

Lisp code in SBCL runs in a managed environment – SBCL needs to be able to safely suspend threads (because it uses the stop-the-world garbage collector) and interrupt them (i.e., call some function in a specific thread).

On Unix-like systems, suspending and interrupting threads is simple (but correctly synchronizing threads and writing code that is tolerable to asynchronous interruptions is not simple). This is achieved by using POSIX signals.

Windows API, on the other hand, does not provide equivalent asynchronous interruptions. They have to be emulated. I'll list several ways of emulating them:

‘Thread hijacking’. Windows lets stop a thread, examine its context, modify it, and restart a thread. This way, another thread may modify the stack and registers in such a way that thread will execute some other function and then return to where it was.

This might seem to be the way to emulate signals, but it is not. This is racy with Windows internals in several ways (ref. 5).
QueueUserAPCEx. Windows has support for Asynchronous Procedure Calls (APCs), but only ‘kernel APCs’ can interrupt a thread when it is in user mode. ‘kernel APCs’ are available from drivers and there is the QueueUserAPCEx project (ref. 6) that provides support for them.

This has a very big drawback in that it requires a kernel-mode driver to be installed to function correctly. Because of that, I haven't tried it. By the way, the pthreads_win32 (ref. 7) project uses QueueUserAPCEx if it's available.
Thread polling. Thread periodically checks if it should suspend or interrupt. This does not require any special support from an operating system. But it requires injecting polling into the code and to tolerate potentially infinite delays (they may occur when a thread executes a blocking operation).

Among the noted approaches, the only one that really works is the last one. So it's clear that we have no other choice but to implement polling in SBCL threads. Luckily, I didn't even need to do this myself – Paul Khuong implemented ‘gc safepoints’ for SBCL (ref. 8, ref. 9). The safepoints are implemented by reading a specific memory location and discarding the result (e.g., with test %eax, GC_POLL_PAGE_ADDR instruction). In normal conditions, this instruction will not affect the running code. But if the memory page is read-protected or unmapped, this will cause an exception and we will land in the exception handler where we can analyze what has happened and what to do next.

In the next post, I'll write about how threads are synchronized for timely suspension and interruption.

SBCL 1.0.43 with Windows threads

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 30 Oct 2010 05:59:00 +0300

I've rebased the Windows threads branch against the current version of SBCL: https://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.43-threads-g002bdc7.msi.

Some linguistic discoveries

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 30 Sep 2010 22:22:00 +0300

Sometimes I ponder whether inaccurate use of terminology causes us to misunderstand things as our perception is distorted by various associations.

For example, there is the term “pattern”. It is used in different contexts, and two uses are particularly useful - “design pattern” and “pattern recognition”. In Russian, this term is usually translated as “шаблон” which literally means “template”. Being a Russian, I've gotten used to perceiving this term as a “template” and I believe that had caused me to not understand properly several papers.

But then it dawned on me that “pattern” rather means “закономерность” (“pattern”) instead of “шаблон” (“template”). A template is an object that defines the properties of other objects, while a template is an object that represents the properties of other objects.

And the weird thing is that dictionaries don't always help to properly understand words. For example, Lingvo translates the term “pattern” in pretty much the same way - as a “template”.

As far as I can see, it's not just me that is susceptible to such biases. For example, the title of the famous GoF book “Design Patterns” is translated into Russian as “Шаблоны проектирования” (“Design Templates”) rather than “Закономерности проектирования” (“Design Patterns”) or “Часто используемые приемы проектирования” (“Common Design Techniques”). And I believe that this perpetuated the belief that the GoF book is about some ready-to-use templates for software architecture or rules to be followed to get a good architecture.

SBCL 1.0.42 with Windows threads

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 25 Sep 2010 15:31:00 +0300

I've rebased my patches for Windows threading against SBCL 1.0.42 and made a release: https://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.42-threads.msi

Alpha version of SBCL with Windows threads is ready for testing

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 16 Sep 2010 22:07:00 +0300

I've published a new binary build of SBCL with Windows threads support at https://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.40-threads-2.msi.

I would greatly appreciate it if you would run the tests on your machines (the tests that I've used for debugging are at http://gist.github.com/582848) and report back the results.

The tale of one deadlock in SBCL on Windows

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 15 Sep 2010 12:48:00 +0300

I keep discovering new bugs.

I recently finished the debugging session which took several weeks of my time. The SBCL process was hanging while loading cl-gtk2 from SLIME. But this hanging magically fixed itself if I would type anything into the *inferior-lisp* Emacs buffer, after which the loading would continue without any troubles. If I loaded cl-gtk2 from the console, all would go well.

At first, I was blaming it on some kind of a deadlock somewhere deep inside SBCL. I've tried various options for printing debug logs but it all did not help.

Normal native debuggers are unable to debug Lisp code in SBCL since SBCL uses interrupt instructions (int 3) to process interrupts, and debuggers just keep stopping all the time. Although recent SBCL versions gained the support for using UD2 instruction which is not supported to break inside the debugger. The only thing to help debug was a stack trace of all threads at the moment of a hang. Of course, native stack traces are useless for debugging Lisp code so I didn't even know in which of the high-level code did the hang happen.

By bisecting over cl-gtk2 code I could find the code that hung - which was a call to the gtk-init-check function. When the hang happened, the stack trace contained a call to GdiPlus.dll!__DllMainCRTStartup which had called kernel32.dll!_LoadLibraryExW. This suggested to me that the hang was due to the loading of a GdiPlus.dll dynamic library.

To check this hypothesis, I've run SBCL from the console, and the call (load-shared-object "GdiPlus.dll") completes without errors, but in REPL it hangs.

The hang itself happened inside the GetFileType function. MSDN documentation for it is quite harmless:

Retrieves the file type of the specified file.

What instead was usable is the comment inside MSDN that GetFileType can sometimes hang. Searching on the web for GetFileType hangs reveals this nugget:

For instance, GetFileType on a pipe hangs if there is a pending read request, which, BTW, causes the DLL load to hang sometimes since the C runtime startup in the DLL calls GetFileType on all known handles while setting up the file descriptor table for open/fopen.

Well, I'm used to documentation being quite spotty (especially when it comes to Microsoft documentation), but it's impossible to prepare for something like this in advance.

Fortunately, this bug is not caused by my code and that means that I'm ready to publish the SBCL fork with Windows threads for wider testing.

More fixes for SBCL on Windows

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 18 Jul 2010 15:17:00 +0300

Apparently, I still haven't nailed down the thread suspension for SBCL.

Some newly discovered problems are related to calling a Lisp code from the C function funcall. The thread state was changing incorrectly: thread enters Lisp code which means that the gc_flag should be unset, but it is not the case. And then the garbage collector skips such a thread and keeps on collecting the garbage while the mutator thread is doing its own business. Obviously, this causes memory corruption.

Also, the thread state is messed up during stack unwinding. It looks like that gc_safe should be moved out of the thread struct and made into a proper symbol. This should ensure correct tracking of thread state during stack unwinding and other thread state changes as the Lisp runtime maintains the dynamic bindings stack.

I've also tackled sockets under Windows. And they are very weird. E.g., one can not simply write, read and poll the socket from different threads. If the read or poll operation has started earlier than a socket write then the write will just block until the read completes. This causes SLIME to not work in multithreaded mode since it uses different threads for writing to and reading from a socket. And I really don't want to go the async IO path.

Some fixes for Windows threads in SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Fri, 09 Jul 2010 00:47:00 +0300

I've made several more fixes for the support of Windows threads in SBCL:

The code is ported to SBCL 1.40 from 1.36;
The code received some cleanup to ease merging;
Virtual memory page protection attributes are used to implement “gc safepoints”; This has brought the performance back to on par with SBCL. Earlier the following code took 1.529 seconds but now it takes 0.089 seconds
```
(loop repeat (expt 10 8))
```
SB-THREAD:INTERRUPT-THREAD is now supported to be able to interrupt threads;
Fixes for various errors in thread synchronization. Not everything has been fixed yet since the tests still sometimes fail with hard-to-reproduce crashes.

The source code is now on github, and the binary is at http://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.40-threads-1.msi.

A better replacement for DebugView

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 04 Jul 2010 21:13:00 +0300

I've been using DebugView to debug SBCL code but I've kept bumping into its limitations.

But now I've found a replacement for it - it's the DbgDump. Its main distinction from DebugView is that it's not a GUI program but rather a CLI program that writes into either file, console, or a network socket (using the syslog protocol). Therefore when the debuggee generates thousands of log messages per second, I don't see such a noticeable slowdown.

More on Windows threads in SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 30 Jun 2010 23:35:00 +0300

I've had to redo interrupt handling in SBCL/Win32, and also rewrite thread suspension and interruption once more. Having done that, now my tests pass. With this in mind, thread support is getting closer to completion. But the volume of changes is getting bigger and bigger which reduces my confidence in swiftly merging this into SBCL HEAD. But it's still manageable as a fork.

I haven't released the code yet since it requires more polishing and some more testing.

On inline assembly in SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 24 Jun 2010 23:06:00 +0300

Introduction

There recently was a discussion on http://lisper.ru/ about inline assembly in SBCL, and here's a guide on including inline assembly in SBCL.

SBCL (as well as in other compilers) contains primitives - language constructs that are not lowerable to other constructs. SBCL implements these intrinsics both in the IR1 (which represents the code as a control/data flow graph) and in the IR2 (virtual machine). At the IR1 level, constructs such as let and lambda are implemented, while simpler “intrinsics” (such as arithmetics, memory allocation, etc.) are implemented as virtual machine instructions that have associated code generation procedures.

These instructions are called VOPs in the SBCL parlance. SBCL is architected in such a way that the set of VOPs is extensible rather than fixed in stone. For example, the sb-cga project (a computer graphics algebra library) implements fast matrix and vector multiplication using the SSE2 instruction set by adding a couple of VOPs.

Example

Let's dissect a simple example - let's add an inline assembly code that XORs two numbers together.

Abstractly, the assembly for this looks like the following (using the Intel assembly syntax):

mov result, arg1
xor result, arg2

result is the registry which will contain the result;
arg1 and arg2 are the registers that contain arguments.

In order to invoke this sequence of assembly instructions from Lisp code, we have to do several things:

Define a new primitive function (we'll call it my-xor)
Define a new VOP corresponding to this primitive
Define a normal function my-xor

Defining a new primitive function

By doing the first two operations, SBCL will be able to compile expressions like (my-xor a b).

The macro SB-C:DEFKNOWN is responsible for defining a primitive. It accepts types of arguments and types of returned value(s). In this example, we will use only short integers (the FIXNUM type).

(sb-c:defknown my-xor (fixnum fixnum) fixnum)

Defining a new VOP

The macro SB-C:DEFINE-VOP registers a new VOP. This macro has a lot of parameters related to codegen (such as register selection, argument types, clobbered registers, and lifetimes of arguments), optimization (the “cost” of an operation), and some others. For this example, the VOP definition looks like this:

(sb-c:define-vop (my-xor)
  (:translate my-xor)
  (:args (a :scs (sb-vm::any-reg))
         (b :scs (sb-vm::any-reg)))
  (:arg-types fixnum fixnum)
  (:results (c :scs (sb-vm::any-reg)))
  (:result-types fixnum)
  (:policy :fast-safe)
  (:generator
   0
   (sb-c::inst sb-vm::mov c a)
   (sb-c::inst sb-vm::xor c b)))

Let's dissect parts of this definition

(sb-c:define-vop (my-xor) defines the VOP name. In general, the VOP name might be different than the function name. It is often the case that one function has several corresponding VOPs for different argument types (such as multiple variants of +)
(:translate my-xor) specifies that the VOP being defined implements the primitive MY-XOR. This part of the definition specifies that this VOP should be used to translate the (my-xor a b) expression
(:args ...) and (:results ...) defines the names of the arguments and result values, and the argument passing ABI.
- :scs specified the set of allowed storage classes. We may consider that a storage class is either a set of registers, the CPU stack, or the FPU stack. The sb-vm::any-reg storage class corresponds to any of the general-purpose registers.
(:arg-types ..) и (:result-types ...) specifies the types of arguments and return values.
(:policy :fast-safe) specifies the set of circumstances in which to invoke this VOP (when there is ambiguity). Allowed values are:
- :small - when a smaller code size is desired
- :fast - when faster (but maybe unsafe) code is desired
- :fast-safe - when it is desired to produce code that is both safe and fast
(:generator ...) defines the meat of the VOP - the code generation procedure. This procedure may be an arbitrary Lisp code that should invoke some special macros to emit instructions.

Inside this procedure, we use the SB-C::INST macro that implements an assembly syntax similar to the Intel assembly language. Arguments for instructions may be:
- numeric constants
- symbolic references to arguments, results, or temporaries
- references to specific registers. For example, SB-VM::EAX-TN is a reference to EAX register
- memory address references. For example, the following references a 4-byte word at an address eax + edi * 2 + 3: (sb-vm::make-ea :dword :base sb-vm::eax-tn :index sb-vm::edi-tn :scale 2 :disp 3)
- references to labels. The labels are created with gen-label and emit-label:
```
(let ((l (gen-label)) (inst jmp l) (emit-label l))
```
- references to various objects. For example, (sb-vm::make-fixup "strlen" :foreign) is the reference to the external function strlen

Defining a wrapper function

So, having defined a VOP, it remains for us to do just a small thing - define a wrapper function MY-XOR around the primitive MY-XOR:

(defun my-xor (a b)
  (my-xor a b))

This would allow us to not just compile expressions such as (my-xor a b) but also take a reference to normal Lisp function #'MY-XOR.

At a first glance, this definition of my-xor is nonsensical: function is defined as a plain recursion without the base non-recursive case. But there is a catch: the function MY-XOR calls the primitive MY-XOR. That's why the compiler generates not a function call but rather generates the code for the VOP corresponding to the primitive MY-XOR.

Let's make sure that this understanding of this code is correct:

(disassemble 'my-xor)
=>
; disassembly for MY-XOR
; 02DED843:       488BD1           MOV RDX, RCX               ; no-arg-parsing entry point
;       46:       4831FA           XOR RDX, RDI               ; <-- beginning of our code
;       49:       488BE5           MOV RSP, RBP
;       4C:       F8               CLC
;       4D:       5D               POP RBP
;       4E:       C3               RET
;       4F:       CC0A             BREAK 10                   ; error trap
;       51:       02               BYTE #X02
;       52:       18               BYTE #X18                  ; INVALID-ARG-COUNT-ERROR
;       53:       54               BYTE #X54                  ; RCX
;       54:       CC0A             BREAK 10                   ; error trap
;       56:       02               BYTE #X02
;       57:       08               BYTE #X08                  ; OBJECT-NOT-FIXNUM-ERROR
;       58:       95               BYTE #X95                  ; RDX
;       59:       CC0A             BREAK 10                   ; error trap
;       5B:       04               BYTE #X04
;       5C:       08               BYTE #X08                  ; OBJECT-NOT-FIXNUM-ERROR
;       5D:       FED501           BYTE #XFE, #XD5, #X01      ; RDI

This listing has 2 instructions starting at offset 0x02DED843 that are exactly the output of just defined VOP. And the rest is the standard prolog and epilogue, and the code to check arguments’ types.

Conclusion

Let's also try to use the newly defined primitive inside a normal function:

(defun foo (a b)
  (declare (type fixnum a b)
           (optimize (speed 3) (safety 0)))
  (my-xor (1+ a) (1+ b)))

And let's have a look at its disassembly:

; disassembly for FOO
; 0325A727:       48FFC2           INC RDX                    ; no-arg-parsing entry point
;       2A:       48FFC7           INC RDI
;       2D:       48C1E203         SHL RDX, 3
;       31:       48C1E703         SHL RDI, 3
;       35:       488BD2           MOV RDX, RDX               ; <-- the code generated for MY-XOR
;       38:       4831FA           XOR RDX, RDI               ; <--
;       3B:       488BE5           MOV RSP, RBP
;       3E:       F8               CLC
;       3F:       5D               POP RBP
;       40:       C3               RET

In this listing, we can see that the SBCL generated exactly the code that we wanted.

The code to generate such a simple inline assembly got quite verbose. This is because we utilize the full capabilities of the compiler, not just a simpler inline assembly.

ICFPC has finished

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Tue, 22 Jun 2010 23:13:00 +0300

Well, the ICFPC is now finished in which I, technically, participated. “Technically” is because I skipped the beginning of its and half of the second day. And on the third day I tried to get some rest, and for the last half a day I was at work. So, it wasn't productive during this year's contest. I'm hoping for the next year.

More fixes for SBCL Windows threads

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 06 Jun 2010 01:28:00 +0300

With a tip from Anton Vodonosov I've found and fixed two more threading-related bugs that lead to handle and memory leak.

The new version of SBCL is available at http://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.36.16-threads-3.msi.

Fixes for FFI callbacks in SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Wed, 02 Jun 2010 23:53:00 +0300

After adding threads into the Windows port of SBCL there were some errors in the way Lisp callbacks worked when called from the C code: when running Lisp code in a callback the thread had its gc_safe flag set. This flag means that this thread is in a section of code that is safe regarding the garbage collector and that it promises not to touch the Lisp heap. But this invariant is clearly violated - so callbacks should clear this flag and check whether the thread should suspend to ensure safe garbage collection. And similarly, when returning to the C code we should set this flag and check the thread suspension predicate.

The fixes are in github and the fixed binary is at http://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.36.16-threads-2.msi.

The next thing on my list is to fix the read-char-no-hang function for Windows sockets.

Pre-alpha version of SBCL with Windows threads

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 30 May 2010 22:34:00 +0300

I've uploaded the first version of SBCL with support for Windows threads to http://sites.google.com/site/dmitryvksite/sbcl-distr/sbcl-1.0.36.16-threads.msi. So I would like to ask fellow lispniks who are reading this and are using Windows to test this and report any bugs. In particular, I'm interested in whether it works under different versions of Windows (XP, Vista, 7, 2003, 2008) and with different CPUs (x86 and x86-64).

In the released version, threads and threading primitives should basically work with the exception of the sb-thread:interrupt-thread and sb-thread:terminate-thread functions. Note that the performance of this version may be significantly worse than the unithreaded version since a lot of things are made in a quick-and-dirty manner without any regard for performance. As an example, a simple loop such as (loop repeat (expt 10 8)) runs about 10 times longer.

SLIME is working although it sometimes behaves weirdly.

I've tried to run the hunchentoot web server as a test and it didn't work. It happens due to the lack of non-blocking read in usocket library, which is predicated on the lack of support for non-blocking socket read and polling in the Windows port of SBCL.

Thanks in advance to whoever will test and report the results!

PS. A couple of words on how to report bugs in a way that would most help debug them. There is a small DebugView utility that shows the debug output coming from the OutputDebugString function. If you attach this output to your bug report it would help enormously.

Safepoints for thread suspension

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 27 May 2010 00:25:00 +0300

It was significantly easier to implement thread suspension on Windows using the safepoints rather than using asynchronous signal processing. The lack of them on Windows (which is a controversial topic when comparing Windows APIs to Unix APIs) made it significantly harder. The implementation of safepoints for SBCL was independently done by Paul Khuong (http://repo.or.cz/w/sbcl/pkhuong.git/shortlog/refs/heads/gc-safe-points) which I gladly took as a basis. The GC-polling code in the safepoints is not yet optimal, though.

For now, the garbage collector does not correctly support having code regions that don't participate in the garbage collection process - that is, code that blocks or calls the foreign code.

And, of course, the optimizations are to be done.

A better terminal emulator for Cygwin

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 23 May 2010 10:23:00 +0300

I've discovered the mintty – a great terminal emulator for Cygwin.

Compared to the built-in Windows terminal, Mintty is like day and night. Its UI is resizable, I can set any font, I can select, copy and paste text, and it is fast.

More troubles in Windows thread suspension

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 20 May 2010 00:06:00 +0300

For now, my work on threading support in SBCL is stalled since I still can not implement thread suspension for garbage collection. The main problem is that Windows API does not offer asynchronous signals (which are available on all other platforms and are surfaced as a pthread_kill function in POSIX Threads API).

I've tried several options:

The thread in pseudo-atomic sections suspends on its own, and in all the rest cases uses the SuspendThread. But it's fraught with problems in many places: threads sometimes just hung, there were errors inside the garbage collector and I was getting spurious page faults. I later understood that this was due to stopping the threads in the wrong place; SBCL uses not just pseudo-atomic sections for non-interruptible code, but also the thread signal mask.
Then I tried adding a signal mask to a thread and considered this mask when suspending the thread: first, we suspend the thread using SuspendThread, and if the thread has masked the SIG_STOP_FOR_GC signal, the thread is resumed using ResumeThread and we sleep for a while and then repeat. Having done this, threads have stopped hanging, but now I got different behavior: as soon as garbage collection completes and the threads resume, some threads discover invalid virtual memory protection flags. I attribute this to the thread suspending inside the exception handler and so it observes the exception handler context instead the proper thread context.

With all of that, I conclude that I won't be able to achieve asynchronous non-cooperative thread suspension.

One of the things that I've researched was the way the .NET garbage collector handles thread suspension (http://msdn.microsoft.com/en-us/library/678ysw69.aspx). .NET CLR tracks suspension requests for each thread. And compiler inserts special instructions - gc safepoints - so that the thread would check from time to time whether it should suspend. Specific CPU instructions used for the safepoints are not as important. These instructions could be, for example, a read from the special region of memory which is unmapped when garbage collection is started, causing page fault and exception in the reading thread and giving it a chance to react to garbage collection. When the thread leaves the bounds of CLR-managed code (that is, performs a blocking operation or calls out to foreign code), it sets the flag that signals that this thread will not touch the GC heap. And when the thread returns back into CLR-managed code, it checks whether it should stop and wait for GC to complete. And there is a weird quirk - if a thread did not stop within 250 milliseconds it will be forcefully stopped using SuspendThread. A thread might not stop timely if did not reach the safepoint within a long time. In the case of .NET, this might happen if the thread is running a long loop that does not call other methods or perform memory allocations. In this regard, the .NET garbage collector is much less careful than the JVM which inserts safepoints not just to method calls but also to backward jumps (any loop is implemented as a backward jump) which guarantees that thread will eventually reach a safepoint.

I shall try to employ a similar technique.

Safepoints will be placed in several kinds of places:
- exit from pseudo-atomic section
- change of thread signal mask
- handler for trap CPU instructions
- return from foreign code
- return from blocking operations
When entering a foreign code (such as a C function call) or invoking blocking operations, a thread will set a flag to signify that it will not do anything with the GC-managed heap, so this thread may be ignored for the purpose of suspending all mutator threads during GC cycle
As I don't want to go too deep into the compiler, I will try to implement GC as follows:
- First, all threads are suspended
- GC enumerates all threads and checks their state:
  - if the thread is already safe for garbage collection, it is resumed
  - if the thread is in a pseudo-atomic section or has masked SIG_STOP_FOR_GC, then it is resumed and GC awaits while the thread leaves this section
  - If the thread is running a Lisp code, put a breakpoint on the closest jmp or call instruction by replacing it with a trap instruction and resume the thread. Soon enough the thread will reach the breakpoint and enter the exception handler. The exception handle will restore the proper instruction and suspend the thread.
  - So far I haven't come up with what to do if the thread runs neither a Lisp code nor a foreign code, is not in a pseudo-atomic section and its signals are unmasked. We could probably resume the thread, wait for a little and then try again.

More progress in Windows threading for SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 02 May 2010 11:59:00 +0300

There is some progress on Windows threads for SBCL.

I am using the code shown below as a test. As of now, it does run without hanging. That is, concurrent memory allocation in multiple threads with concurrent garbage collection requests is now working. I will check it more thoroughly on a beefier 4-core computer for multiple hours to ensure that the approach is right, but for now it seems to be working fine.

(defun cons-lot (stream char)
  (sleep 2)
  (loop
    (loop repeat 10
      for ar = (make-array (* 1024 1024 1/2) :initial-element 0)
      do (setf (aref ar 1000) 112))
    (format stream "~A" char)
    (finish-output stream)))

(defun threaded-cons-lot (n)
  (loop
    with stream = *standard-output*
    repeat n
    do (sb-thread:make-thread (lambda () (cons-lot stream #\.)))))

(threaded-cons-lot 4)

After all, lack of asynchronous IPC in Windows API is noticeable. Correct thread suspension is achievable in POSIX (it takes to carefully manipulate signal mask to ensure that the signal does not arrive at unfortunate moments), while it becomes very fragile on Windows. There are not a lot of options for asynchronous thread communication: basically, we can use SuspendThread and analyze and manipulate the thread state (which is very non-trivial). Another option is to implement cooperative thread suspension in which the thread itself should on its own periodically check whether it should suspend or interrupt. I don't claim to be an expert in this area, but it seems to be a significant drawback of Windows APIs in the IPC area.

(On the other note, thread synchronization in Windows is quite tricky and fragile due to internals of asynchronous inter-thread communication. For example, asynchronous procedure call may cause a thread waiting on a synchronization event to miss a notification from PulseAll function.)

In SBCL, garbage collection is implemented in the following manner:

The thread discovers that it is time to do a garbage collection cycle. The thread may discover this either while performing an allocation (in which case the thread is inside a pseudo-atomic section with signals masked) or due to explicit call to SB-EXT:GC (in which case the thread is not in the pseudo-atomic case).
The thread locks the *ALREADY-IN-GC* mutex whils having its signals unmasked. This is required in order for multiple threads to be able to invoke the garbage collection: one thread will take the lock and other threads will be stoppable (since the signals are unmasked).
The thread that has locked the mutex sends the SIG_STOP_FOR_GC signal to all other threads and awaits for them to actually stop. When the thread stops, it set its thread.state field to STATE_SUSPENDED and signals the thread's state_cond condition variable.
The threads receive the signals and suspend themselves. The key point is that the thread will receive the signal only when it is ready to handle it. For example, the thread is able to handle the signal and suspend while it is blocked trying to lock the *ALREADY-IN-GC* mutex.
When all threads are suspended, a garbage collection is performed. While one thread performs the garbage collection, all threads are waiting on the state_cond conditional variables.
Threads have their state set to STATE_RUNNING and are resumed.

In Win32 API there are significant challenges with the 4th item. There is no way to have the thread code execute some code in such a way that would be safe for the thread without having the thread be in some special state. If we ignore safety then there is the SetThreadContext function that lets us redirect the thread to arbitrary code. If we limit the thread only to some special state, then there is the QueueUserAPC function, but APCs (async procedure calls) are invoked only while the thread is in some special state - that is, while it's blocked on SleepEx, WaitForSingleObjectEx or WaitForMultipleObjectsEx.

As an alternative, there a different technique to achieve thread notification which is used in the JVM. A special code sequence is inserted in some locations of a program that somehow checks whether the thread should suspend or interrupt or do something special. This request can be communicated via the memory protection attributes of a special memory page so that accessing it is normally a no-op but in special circumstances it would redirect the thread to the signal handler. This is a viable approach but it requires more work and requires the thread to periodically do something.

So my plan is as follows. Since I can't make the thread stop in safe way, then I will have to resort to more complicated way of stopping threads using SuspendThread. For garbage collection, there are two hazards - if another thread locks something that is needed for a garbage collector, and if the garbage collector witnesses an object in the middle of its initilization. That's why all the locks used by the garbage collector must be taken right at the start of garbage collection cycle. An incomplete object initialization corresponds to a pseudo-atomic section; therefore we can set the thread's pseudo-atomic-interrupted flag so that the thread would know that it should stop as it leaves the pseudo-atomic section. If a thread is inside a pseudo-atomic and tries to invoke the garbage collection then it must grab the GC mutex - therefore we use ptread_mutex_trylock instead of pthread_mutex_lock so that one thread would successfully grab the mutex and the other threads would know to stop. And the thread a normal state is stopped with the SuspendThread with all of its locks kept held which means that the garbage collector must not take the same locks to avoid deadlocking. And the starting and stopping of the GC is done with a pair of AutoResetEvents: gc_suspend_event and gc_resume_event.

Now I need to verify the proper functioning of the thread stopping and come up with a way to interrupt running threads which is, for example, used by SLIME to interrupt long-running processes.

Progress in Windows threading support in SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Fri, 09 Apr 2010 01:15:00 +0300

I've moved a little bit forward with Windows thread support in SBCL. Now I can the thread to start, and even the garbage collection may run when multiple threads are running. That's not entirely true since stopping of threads is not fully implemented. For example, gc_stop_the_world would if fail if the thread was in a pseudo-atomic section but it should be easy to fix. At least I got simple examples to work.

But now I got to the point that I'm not so sure whether it's right to stop the thread by pushing a new frame into its call stack. While the thread is being hijacked by a new call stack frame, it might be holding some important system resources (that are not designed with such a case in mind) that might be necessary for the garbage collector to run.

If this is the case then I would have to come up with other ways of stopping the threads. For example, it's possible to avoid stopping the thread if the Lisp-side of its stack and the Lisp heap will be untouched by that thread. But I'll leave that option for a later time in case the simpler ways of stopping will not work somehow.

Windows threads progress for SBCL

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Mon, 29 Mar 2010 00:31:00 +0300

I am continuing to hack on SBCL to have working thread support for Windows.

The Thread-Local Storage area is stored in the Arbitrary Data Slot of TIB (Thread Information Block) which is mapped in every thread to the address FS:0x14. Some of the VOPs now take more instructions and use more registers and do more memory accesses since in other ports SBCL can directly address the TLS area, but instead, in Windows, we have to load the pointer to the TLS area into a register. So far is too hard to quantify the performance effects of this.

I'm still kind of slow at this low-level stuff. I've spent a week debugging an issue caused by my inattentiveness. Incidentally, this made me get to know the GDB better. GDB is a brilliant debugger. This might be hard to see behind its lowly command-line UI. One of the nice things is that GDB has macros that help implement application-specific debugging functionality. Before that, I mostly used the Visual Studio debugger and didn't expect the GDB to be so good.

As for SBCL, today I had a first working build that would run (earlier it wouldn't get past the cold init). TLS is apparently also working.

Now I need to work on the runtime to teach it to create threads and suspend them for garbage collection purposes.

For the thread suspension, I'm planning to inject a stack frame for a fake call to the function that will await the resumption signal (as in http://github.com/dmitryvk/testw32/blob/master/plant_stack.c). This should allow for correctly suspending the threads that are performing blocking syscalls as well as threads that running userspace code.

How to access thread-local data in Windows

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Thu, 18 Mar 2010 00:21:00 +0300

I have stopped liking the idea to utilize the GS segment register for storing the pointer to a thread-local storage. Despite my efforts, the GS register still sometimes ends up containing zero.

Apart for the GS registers there are different options for accessing the thread-local data:

The ES register. Clozure CL compiler for Windows uses this register to access thread-local storage and it also avoids emitting “string” instructions (such as CMPS) since those instructions use the ES register. Apparently, they are not used since in Clozure CL only EAX register may contains an untagged value which also helps to make the garbage collection precise. Consequently, this approach would require SBCL to not generate string instructions and to save/restore the ES register while calling foreign code. I consider such tradeoffs unacceptable.
Inside the Thread Information Block in the Arbitrary Data Field. This field is always available for any thread at the adress %FS:0x14. Using this approach will entail changing in just some places related to handling the current thread. Having skimmed the code, it seems that there are just a couple of such places and they are pretty straightforward.

For now I'm betting on using %FS:0x14.

Microsoft's documentation style

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Mon, 08 Mar 2010 08:53:00 +0300

MSDN is in general a nice documention collection but sometimes reading it makes me sad.

Today's exhibit the SignalObjectAndWait function. It contains the following note:

Note that the “signal” and “wait” are not guaranteed to be performed as an atomic operation. Threads executing on other processors can observe the signaled state of the first object before the thread calling SignalObjectAndWait begins its wait on the second object.

Everywhere (except the MSDN) it is written that the SignalObjectAndWait performs the signal and wait operations atomically (even the Jeffrey Richter's book agrees) whilst MSDN states the opposite - that it's not atomic.

I'm really confused about this and don't know whether I should rely on this function anymore. This a pretty fundamental synchronization primitive and it would be bad to avoid using it.

And the funny thing is that the older version of the documention also states that it's an atomic operation.

Windows APC won't be of use to me

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 07 Mar 2010 02:46:00 +0300

I was happy to see that Windows has support for Asynchronous Procedure Calls (APCs). APC seems to have the same functionality that the signals have, i.e. the Windows API allows to run some code in the context of another thread. But there's a catch - APCs are only invoked during specific moments.

Hence I had to resort to calling code in the context of another thread by manipulating the stack and the context of a thread in order to “plant” a call to the function. At least, that's what I've been able to conjure as a hack. I still have to find out how blocking calls would work in the interrupted thread but simple tests show that evertything should be OK with it.

How to use the GS register in Windows

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sat, 06 Mar 2010 06:32:00 +0300

Thanks to a hint, I've been able to understand how to use the %GS register. So the plan to access the TLS is as follows: when a thread starts it saves the pointer to its state (more specifically, a segment selector for a new segment whose base is a pointer to a structure with thread-specific data and with no limit) at the address %FS:0x14 (which corresponds to the Arbitrary Data Slot of the Windows Thread Information Block).

And at certain moments, that selector is copied into the %GS register:

when a thread starts
after each call to a foreign function
when entering the “planted” function for hijacking a thread

At first, I thought that the %GS is being reset every time the scheduler preempts the thread. But experiments have shown that %GS is only being reset when calling Windows API functions. I was confused by the fact that I was unable to set this register via the SetThreadContext at all – that's why I thought that this register is not persisted in the thread context.

Implementing this approach would require changing the way foreign functions are called and to take it into account when implementing thread hijacking via stack frame planting.

How to adapt SBCL's garbage collection and threading to Windows

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Sun, 28 Feb 2010 21:00:00 +0300

I'm trying to implement threading support in the Windows version of SBCL. I've been looking at the source code and I was quite surprised that this had not been implemented yet. I guess the SBCL community is quite small.

I did a preliminary analysis of what needs to be done to support threads and how it can be done.

The SBCL sources are quite interesting. There are a lot of places which hadn't seen attention for quite a long time. And several places in the source code are quite involved to hard to wrap my head around – especially code that deals with signals and signals masking. The documentation is quite scarce (especially when it comes to introductory materials) but on the whole, it's approachable. The old CMUCL papers and articles are quite useful (since SBCL is based on CMUCL) and are linked from the sbcl-internals.

SBCL uses the OS API in nontypical ways. I was particularly paying attention to things that are required to make SBCL work on Windows.

The primary feature of Lisp that made SBCL developers go to great lengths and employ various tricks and hacks is, of course, the garbage collector. SBCL performs garbage collection if either successive allocation fails or periodically (the trigger is based on the number of bytes since the last collection). The garbage collection is performed in the calling thread and it suspends all other threads. When all threads are suspended, it starts to scan the garbage collection roots (that is, stacks of all threads and global variables). Having completed the garbage collection, all stopped threads are resumed.

As memory allocation is a frequently invoked operation in Lisp, it should run as fast as possible. For this, every thread in SBCL has a per-thread memory region in which allocation happens without locking by just a simple bump allocator.

But memory allocation is not an atomic operation and consists of several steps:

Increase the pointer to free memory by the allocation size
Compare the pointer with the bounds of the free region
Initialize the object in the allocated space.

Now, suppose the garbage collector stopped the thread in the middle of the allocation. It would observe an inconsistent state in which the GC invariants are violated and would corrupt the memory if it would try to do collection in such a case.

That's why SBCL employs a technique named “pseudo-atomic sections”. These sections are short segments of code that are handled specially by the garbage collector - the garbage collector ensures that no thread is suspended inside a pseudo-atomic section.

The pseudo-atomic sections are implemented as follows:

Upon entering the pseudo-atomic section the thread sets the flag to signify that it's in such a section
The garbage collector checks this flag for each thread and if it's set, sets another flag to signify that the garbage collector is waiting for this thread and unblocks this thread
Upon exiting the pseudo-atomic section, the thread checks for this flag and suspends itself if necessary.

While starting the collection, the garbage collector ensures that the collection is not running already and if this is the case, it waits for the collection to complete and then retries the memory allocation.

When running under Linux, the SBCL uses signals delivered to specific threads in order to stop or resume their execution. It uses real-time signals for that purpose since normal signals are not queued: the thread will skip a signal if already has a signal queued for it while that signal is masked. For correct signal handling, threads manipulate their signals mask.

On Windows, it makes no sense to use signals since Windows does not have signals (not even talking about real-time signals). But it supports suspending threads. After the thread is suspended, it is possible to query or set the thread context. This lets us, for example, imitate a function call. This lets us implement thread interruption by invoking a special code inside it. Also, this is one of the ways to synchronize pseudo-atomic sections with the garbage collector.

Every thread has an area that is local to the current thread. On Linux, the pointer to it is stored in either the %FS register on x86 or the %R12 register on x86-64. Windows lets each thread store an arbitrary pointer-sized value at the address %FS:0x14. And some runtimes use the %GS register to store the base pointer to the thread-local data area. Using the functions NtQueryInformationProcess/NtSetInformationProcess we can set the base and limit for the memory segment, and then using the GetThreadContext/SetThreadContext it is possible to set the selector for this segment in a segment register (%GS for Windows x86 since %FS is already taken for Thread Execution Block). A segment register stored the segment selector which is defined in the LDT (Local Descriptor Table). It is only possible to have 8192 segment selectors which limits the possible number of threads to just 8192.

In a per-thread area we would be able to store the following data:

dynamic binding stack
control stack
thread flags
thread synchronization objects
current dynamic values of symbols
per-thread allocation area

Let's see how the GENCGC garbage collector allocates memory. Memory is allocated from the per-thread memory region in a pseudo-atomic code section. The allocation sequence is as follows:

The thread enters a pseudo-atomic section
If the per-thread region has sufficient free space to satisfy the allocation request, the allocation happens from this region
Otherwise, a new region is allocated (this might trigger garbage collection)

Now, about the dynamic bindings for special variables. SBCL stores in the symbol structure the following:

global value
TLS index for the per-thread current value

Also, there is a per-thread binding stack that stores the pairs consisting of a symbol and the symbol's value. When a code dynamically binds a special variable to a new value, it pushes the symbol and its old value to the bindings stack and writes a new value into a thread-local area using the TLS index. Upon leaving the scope of dynamic binding, everything is done in reverse order.

The Lisp stack (also named the control stack) is separated from the foreign code stack (named the numeric stack) and from the dynamic bindings stack. The difference between the Lisp stack and the foreign code stack is that the Lisp stack only contains pointers to boxed Lisp objects and is scanned by the garbage collector while the foreign code stack is not inspected by the garbage collector at all.

The SB-THREAD package defines a set function to control threads:

make-thread creates a new thread
join-threads waits for thread completion
interrupt-thread interrupts a thread. Interruption is analogous to UNIX signals: thread stops doing whatever it was doing and invokes the specified function. If the thread is already being interrupted then the new interrupt is queued rather than interrupting the current interrupt handler. In Linux, this is implemented with the help of a signal. On Windows, it might be possible to implement using the combination of SuspendThread and SetThreadContext to manipulate the thread state.
terminate-thread forcibly terminates a thread. Thread termination is based on thread interruption: a thread is interrupted with a special function that unwinds the stack and exits from the thread.

Also, the timers are based on interrupts. A timer is a facility to periodically call a function in a specified thread (or in a new thread).

The main synchronization object used by SBCL is the futex. On Linux, it is implemented as a native Linux futex and is called a “lutex”. In a nutshell, futex is a combination of a mutex with a conditional variable, i.e. it protects the data from concurrent access and allows to wait for changes in the data. Waiting for changes is used, for example, inside the SBCL to wait for the thread state to change. On Windows, there are no conditional variables (or rather, they are available but only in the newest versions of Windows), so we will need to implement futexes based on the primitives that we have (for example, a critical section and an event). One peculiarity of SBCL is that the futexes themselves are not exposed, but rather the synchronization functions accept the pointer to the data being protected by the futex. This allows not dumping the futexes when saving the image to the disk and in general to disregard their initialization and finalization which somewhat simplifies their implementation.

SLIME is getting better

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Tue, 09 Feb 2010 07:06:00 +0300

SLIME is getting better. Today I discovered that it shows the arguments for the local functions defined with flet.

I.e., given the following code:

(defun foobar ()
  (flet ((bar (x y z) (+ x y z)))
    (bar )))

if I put the cursor after the (bar , the minibuffer will display (bar x y z).

Summary of recent changes in cl-gtk2

Kalyanov.Dmitry@gmail.com (Dmitry Kalyanov) — Tue, 26 Jan 2010 00:28:00 +0300

I've been able to make some time to work on cl-gtk2 and as a result, I've published a few changes. Below is a short summary of them.

Load-time checking for minimum versions of libraries

During loading, cl-gtk2 now checks versions of the libraries and generates a load-time error if the version is too old. This should help to improve error diagnostics in such cases.

Support for several versions of Gtk+

Added rudimentary support for multiple Gtk+ versions. During loading cl-gtk2 adds into the *features* the symbols that correspond to the versions of glib and Gtk+, and the rest code may use them for conditional compilation of classes, functions, and methods. So if you try to load cl-gtk2 with, say, a Gtk+-2.18 installed, you will be able to access classes from Gtk+-2.18. But if later you upgrade to the later Gtk+ version you will have to recompile the cl-gtk2.

Improved demo program

I've slightly improved the demonstration program gtk-demo:demo included in the cl-gtk2. Now it looks like a text page with links to various examples:

Working with the application's main loop

Now the functions ensure-gtk-main, leave-gtk-main, join-gtk-main, and the macro with-main-loop work similarly in both the single-threaded and multithreaded Lisp implementations.

They should be used in the following manner.

The main function would contain the code similar to the following:

(defun run ()
  (within-main-loop
    (your-application-code) ;; at some point, the application will call leave-gtk-main
  ))
(defun main ()
  (run)
  (join-gtk-main)
  (quit))

Then the main function will return when the application invokes the leave-gtk-main function and the application will exit. The main function like this can be used as a top-level function when saving the image or when running a script, and the run function may be used to invoke the application from the REPL during development - it will work in the background and you may incrementally add the code to the application.

Thread-safe finalization of GBoxed instances

I've made the finalization of GBoxed instances thread-safe and fixed a couple of bugs related to that.

Posts on dmitry_vk's notes

Travel Charger for my new laptop

Charging issues

Charging experiments

Charging in OFF state

Charging in the ON state

Some unsafe Rust tricks

Trick #1: Lifetime transmute

Trick #2: controlling access to shared data with external locks

I had a wrong mental model of Rust's thread-local variables performance

Constructing DST pointers in Rust

Background

Constructing pointers to slices

Constructing pointers to variable-length structs

References to DSTs

Using TortoiseMerge in Linux and Mac OS

Wine

Wine prefix

Configuring Wine

Installing TortoiseMerge on Linux and Mac OS

Integration with Git

Efficient batching

Batching

How batching is usually done

Analysis of batching efficiency

Benchmarks

Conclusion

Some further notes

On the usefulness of topological sort

Topological sort

Uses of topological sort

Issues with topological sort

Programming contests are good for me (and probably for you too)

Criticism of programming contests

Typical structure of a contest

A problem statement

Scoring and submissions

The specific skills that contests help level up

How the structure of contests helps acquire skills

Rapid feedback

Strict solutions checking with tight time and memory constraints

Hiding of failed tests, penalties for failing submissions

Conclusion

How to implement async stream in Rust without resorting to macros

Async iteration

The async-stream crate

Why I don't like to use async-stream in production code

An alternative to async-stream

Small example of async GUI code with Gtk-rs: alert() function

Notes on guaranteed shutdown of test containers

Why Rust instead of C++

ICFPC 2012 is coming

Notes on threads and spaghetti code

TL;DR

A little bit of theory

In practice

Looking for a new maintainer for CL-GTK2

SBCL-1.0.45 with Windows threads

Couple of words regarding FS:0x14

A useful article on codegen techniques

cl-sqlite-0.2 released

SBCL 1.0.44 with Windows threads

SBCL Windows threads implementation notes, part 2

Suspension

Thread interruption

Safepoint code

Some manuals are factually wrong

SBCL Windows threads implementation notes, part 1

Thread-local storage

Thread suspension and interruption

SBCL 1.0.43 with Windows threads

Some linguistic discoveries

SBCL 1.0.42 with Windows threads

Alpha version of SBCL with Windows threads is ready for testing

The tale of one deadlock in SBCL on Windows

More fixes for SBCL on Windows

Some fixes for Windows threads in SBCL

A better replacement for DebugView

More on Windows threads in SBCL

On inline assembly in SBCL

The `async-stream` crate

Why I don't like to use `async-stream` in production code

An alternative to `async-stream`