Blargh

I fixed shell pipes

Thu, 18 Jun 2026 00:00:00 +0000

In a previous post I made pipes in unix shells more reliable. Well, it had some drawbacks. I’ll summarize the problem, the failed previous version, and then show the new and improved one.

Problem summary

Downstream processes in a unix shell pipe cannot know if the upstream finished successfully, or exited with an error. This means that it can’t know if it should “commit” the data it received.

Example uses:

$ pg_dumpall | xz -9 | google_cloud_storage_upload gs://bucket/path/postgres.dump
$ generate_data | psql --single-transaction

In both of these cases you want the right hand side to STOP, and not finalize the upload or commit the transaction.

The previous version

$ goodpipe <<EOF
[
  ["gsutil", "cat", "gs://example/input-unsorted.txt"],
  ["sort", "-S300M", "-n"],
  ["gzip", "-9"],
  ["gsutil", "cp", "-", "gs://example/input-sorted-numerically.txt.gz"]
]
EOF

This works fine for simple cases, but doesn’t support tee or per-command environment variables very well.

And I don’t want to invent a complex language, so my replacement took a different path.

wp — Wrap Pipe

wp on github.

wp instead wraps the input and/or output with a very minimal encapsulating protocol. This allows normal data to pass through, but still allows the downstream to get EOF as metadata.

If the data stream ends before receiving the EOF marker, then do not commit. The wrapped downstream child process sees this as stdin remaining open, and instead it’s getting terminated with a signal.

wp can either encapsulate when it wraps something that outputs data, with wp -o, or decapsulate and receive the EOF marker when it’s handling input data, or both.

Examples

$ wp -o pg_dumpall | wp -io xz -9 | wp -i google_cloud_storage_upload gs://bucket/path-postgres.dump
$ wp -o generate_data | wp -i psql --single-transaction

Quick install, if you have `cargo`

cargo install --locked wp-cli

Quake demos raytraced again

Sun, 14 Jun 2026 00:00:00 +0000

This is a follow-up to a previous post about raytracing Quake demos.

But first, the money shot:

And flat shaded and textured videos. Youtube is Very Aggressive™ with its compression, so the quality there is not good. For pixel quality the above images showcase it better.

A new raytracer

One of my original reasons for creating the quake demo povray files is that it was a good source of data for 3D experiments. POV-Ray is a great raytracer, though entirely CPU (no GPU) and no longer state of the art.

POV-Ray has plenty of built in options, but takes forever to render the 30-60fps demos I want to play with.

Also POV-Ray is AGPL now, so nope nope nope nope nope. That’s a dead end.

Another AI detour

We live in interesting times. We could be living in a time when no two people are running the same email client, or music player, or shell. There used to be a barrier to writing these things custom. I know people who wrote their own shell and use it as a daily driver. I wrote my own email client, and use that.

There are many people out there, me included, who are perfectly able to write their own shell, but don’t. For the shell, my need is not above my threshold of putting in the effort. But now? I could, if I ran into more annoyances with Bash.

But do you not like Bash? Just ask the AI to write one exactly the way you want it. If it breaks, well you’re the only user and your fingers are trusted input. “It should be fine” (famous last words)

If the static site generator I use for this blog (Jekyll) gives me any trouble, like some Ruby dependency troubleshooting, I’ll replace it with a custom one in a heartbeat.

So let’s write a new raytracer

I don’t have to “find” the best renderer, anymore. I can just have AI write a custom one. Oh, but do I have to modify QPov (the demo-to-pov converter) to write a new file format? I could ask AI to add other output format. Or I could have AI write a converter.

Nah, I’ll just have AI load the existing POV-Ray files full of includes and macros. Remember, I’m not writing “the perfect general purpose raytracer”. This is not reusable code. I’m just turning my data into raytraced files.

The process

I got my initial result in under half an hour. I didn’t save the exact prompts, but they were as short and vague as this:

Write a raytracer in rust for the files in this directory.
The output is a garbage image. Fix it.
Make it parallel using the rayon crate.
Add optional textures as specified in the input files.
I think you got the texture coordinates upside down.
Frame 302 has rendering errors. Fix the renderer.
Add adaptive antialiasing.
It’s a bit slow. Optimize it.
Switch to outputting PNG files using the png crate. Maximum lossless compression.
Add some rendering metadata to the output PNG.

Some notable impressive feats:

It fixed the initial garbage by “realizing” that it could render with POV-Ray, and compare the output. After the initial working version it no longer needed to do that, and didn’t.
When the output was no longer garbage it said that it could “see” a hallway now, which made it realize it was done.
For the frame 302 rendering bug, it rendered 301 and 303 to compare, and could see a wall disappearing from the rendered output.
When working on the problem, it would render an image, and then do image recognition. Yeah, that’s what I’d do too.

The result

It’s not povray, but it’s fast. The example 4K frame with antiaalias from earlier took 30 seconds on my laptop (5m39s CPU time). POV-Ray (though admittedly with much more advanced effects) would probably take days.

I can now iterate on other things, such as a better way to render the sky and water/lava/slime, and add special effects.

I can… but it depends on when I have an itch to continue on this project.

AI solving problems

Sat, 13 Jun 2026 00:00:00 +0000

I’ve been able to find some time, lately, to work on my project backlog. And because it’s 2026, I’ve been using AI as a diligent intern.

I’ve ranted before about seccomp, but still used it for a project or two. But then, rarely, it triggered an unexpected openat. That’s exactly the kind of I do want to detect and kill the binary for, so I don’t just want to allow it. I want to know where it’s coming from.

strace showed it’s trying to read /proc/sys/vm/overcommit_memory.

It’s certainly not my code. But just the Rust transitive dependency tree is quite a few crates:

$ cargo tree | sed -r 's/^[^a-z]+//;s/ .*//' | sort -u | wc -l
236

Step 1 was to run it in gdb, and reproduce the problem. But it’s a bit trickier than that, because seccomp fully kills the process, so no backtrace. And setting breakpoints requires a few more syscalls to work, just for the process to work under gdb (e.g. sigaltstack).

And turns out some calls fail with EINTR if running under a debugger.

Yes, I can fix all these things. But why not put the AI intern on it?

AI coding detour

I’ve previously vibe coded a raytracer just with the prompt “make a raytracer for the files in this directory”. The files in question were my POV-Ray files converted from old Quake demos.

It worked very well that time (flat shaded and textured), so why not?

(youtube’s compression is Very Aggressive™. The originals look crisp)

Could the AI solve it?

Skipping to the end, yeah:

openat(AT_FDCWD, "/proc/sys/vm/overcommit_memory", O_RDONLY|O_CLOEXEC)
    = -1 EACCES (Permission denied)

This is now an error instead of killing the binary.

Could I have solved it?

Sure. But it reminds me of the well publicised Google interview question “how many piano tuners are there in New York city?”.

The best answer I’ve heard is “is it important for the company? — Yes — Then I’d look it up”. That’s the right answer, of course. There’s another “more right” answer to the “why are manhole covers round?” question.

Is it important to know which of my transitive dependencies try to read this file? Not really. I could blanket allow that file to be read. I know what that file is, and I actively don’t care if an attacker can read it.

So without AI I would not have spent more than 30 seconds trying to find out. And I spent a couple of seconds doing rg overcommit_ ~/.cargo, only showing references in jemalloc, which this project doesn’t transitively depend on.

So it’s likely somewhere in some C code. Or in the Rust standard library.

So instead I spent 9 minutes reading up on the QUIC protocol for a couple of my other projects, while the AI went brrr.

The AI session

Prompt

Sometimes the code tries to read /proc/sys/vm/overcommit_memory readonly and crashes because seccomp. Why does this happen?

Output

[… thinking out loud and searching the web before the final answer …]

• It’s glibc malloc, not our code.

  glibc’s Linux malloc has a lazy check_may_shrink_heap() path. When it
  considers shrinking a malloc heap, it checks whether Linux is in strict
  overcommit mode by opening:

  /proc/sys/vm/overcommit_memory

  readonly with O_RDONLY | O_CLOEXEC, reading one byte, then caching the
  result.  Source: glibc malloc-sysdep.h
  (https://codebrowser.dev/glibc/glibc/sysdeps/unix/sysv/linux/malloc-sysdep.h.html#34),
  called from heap shrink logic in arena.c
  (https://codebrowser.dev/glibc/glibc/malloc/arena.c.html#506).

  It is “sometimes” because that path is lazy and allocation-pattern dependent.
  With multiple worker threads, glibc creates per-thread malloc arenas; later
  frees/trims can hit heap_trim -> shrink_heap -> check_may_shrink_heap. If the
  first time that happens is after seccomp is loaded, our default KillProcess
  action kills the process on openat.

  Best fix is probably not to broadly allow file reads. Add a targeted seccomp
  rule that returns EACCES/EPERM for openat instead of killing, ideally only for
  readonly/CLOEXEC flags. glibc treats open failure as “can’t check, assume no
  strict shrink requirement” and caches that, while Landlock still prevents
  actual filesystem access. Warming this before seccomp is possible but brittle
  because the triggering path is allocator-version and arena-state dependent.

─ Worked for 3m 42s ──────────────────────────────────────────────────────────────

Prompt

Sounds good. Change it to do that.

Output

⚠ Heads up, you have less than 25% of your 5h limit left. Run /status for a breakdown.

[uh oh. But to be clear this was far from my first task of the 5h window. In
 fact I have another unrelated task running concurrently]

• Changed src/privs.rs:120 so readonly openat returns EACCES instead of killing
  the process under seccomp.

  The rule is conditional on openat argument 2, masked with O_ACCMODE, being
  O_RDONLY. That covers glibc’s lazy /proc/sys/vm/overcommit_memory probe while
  still leaving write-capable opens under the default KillProcess action. No
  read syscall was added.

  Verified with:

  - cargo check --bins
  - cargo test --bin tarweb-h3
  - cargo build --bin tarweb-h3
  - Live tarweb-h3 --threads 2 + curl -sk --http3 https://[::1]:18083/README.md success
    ful.

─ Worked for 5m 39s ──────────────────────────────────────────────────────────────────

Final patch

Yeah. Not bad. I can work with this.

diff --git a/src/privs.rs b/src/privs.rs
index 19b6307..cce5ef6 100644
--- a/src/privs.rs
+++ b/src/privs.rs
@@ -1,6 +1,6 @@
 use anyhow::{Context, Result, anyhow};
 use caps::CapSet;
-use libseccomp::{ScmpAction, ScmpFilterContext, ScmpSyscall};
+use libseccomp::{ScmpAction, ScmpArgCompare, ScmpCompareOp, ScmpFilterContext, ScmpSys
    call};
 use tracing::{info, trace, warn};

 /// Drop privileges to bare minimum.
@@ -127,18 +127,32 @@ fn drop_caps() -> Result<()> {
     ] {
         f.add_rule(ScmpAction::Allow, ScmpSyscall::from_name(name)?)?;
     }
+    // glibc malloc may lazily probe /proc/sys/vm/overcommit_memory.
+    f.add_rule_conditional(
+        ScmpAction::Errno(libc::EACCES),
+        ScmpSyscall::from_name("openat")?,
+        &[ScmpArgCompare::new(
+            2,
+            ScmpCompareOp::MaskedEqual(libc::O_ACCMODE as u64),
+            libc::O_RDONLY as u64,
+        )],
+    )?;

     if with_rustls {
         // Rustls does some memory allocation.

Follow-up

So, now it works. But I’d want to adjust it a bit, to not merely return EACCES for any probing of files such as /etc/passwd or TLS certificates. I want that to terminate the binary.

For merely denying filesystem access on a per path prefix basis I use landlock, but that doesn’t kill the binary.

I’m not aware of a way for seccomp to block specific paths (again, seccomp sucks), but maybe the AI knows? But oh no:

╭─────────────────────────────────────────────────────────────────────────────────╮
     […]
│  5h limit:             [░░░░░░░░░░░░░░░░░░░░] 0% left (resets 19:26)            │
│  Weekly limit:         [██████████░░░░░░░░░░] 48% left (resets 16:12 on 18 Jun) │
╰─────────────────────────────────────────────────────────────────────────────────╯

So maybe it’s time to go outside in the nice weather? “Touch grass”, as the kids say.

RustRadio UI improved

Tue, 26 May 2026 00:00:00 +0000

This is just a short followup to the last RustRadio post. If you came for more rants about C, you’ll be disappointed.

I’ve never been that interested in writing UI code, including HTML. You can see the “programmer art” in the screenshots linked from www.habets.pp.se.

And then the slightly different tech section, that doesn’t serve much of a purpose now that we have github.

I’ve not been happier with GTK, QT, and the others either.

But RustRadio needs a UI.

I feel like the browser is the most stable and portable UI. So I’d already decided on that. So now I have to manually do a bunch of DOM manipulation, to create an interactive UI? Or worse, learn the React/Angular/Whatever flavor of the day, that will be obsolete by tomorrow afternoon? Gag me with a spoon.

LLM to the rescue

For now I’m just continuing to focus on the SDR and architectural parts of RustRadio, and I’m letting the LLM-written code do the HTML manipulation.

Yeah, it’s kinda vibe coding. But doesn’t use unsafe, and it demonstrably outputs what I want (I mean, sure it may require some follow-up prompts), so who cares?

The vibe coding is isolated to the files doing the drawing. If I want to artisanally craft better code in the future, that’s the file that needs to be rewritten. Until then, it works.

The demo, in all its glory

You can run the demo too.

See the quick start instructions in the ruwasm repo for how to run this UI live with an RTL-SDR.

Everything in C is undefined behavior

Tue, 19 May 2026 00:00:00 +0000

If he had been a programmer, Cardinal Richelieu would have said “Give me six lines written by the hand of the most expert C programmer in the world, and I will find enough in them to trigger undefined behavior”.

Nobody can write correct C, or C++. And I say that as someone who’s written C and C++ on an almost daily basis for about 30 years. I listen to C++ podcasts. I watch C++ conference talks. I enjoy reading and writing C++.

C++ has served us well, but it’s 2026, and the environment of 1985 (C++) or 1972 (C) is not the environment of today.

I’m definitely not the first to say this. I remember reading a post by someone prominent about a decade ago saying that a good case can be made that use of C++ is a SOX violation. And while I was not onboard with the rest of their rant (nor their confusion about “its” vs “it’s”), I never disagreed about that point.

With time I found it to be more and more true. WAY more things are undefined behavior (UB) than you’d expect.

Everyone knows that double-free, use after free, accessing outside the bounds of an object (e.g. array), and accessing uninitialized memory is UB. After all, C & C++ are not memory safe languages. And yet we as an industry seem to be unable to stop making even those mistakes over and over.

But there’s more. More subtle. More illogical.

It’s not about optimizations

Some people seem to think that as long as they don’t compile with optimizations turned on, undefined behavior can’t hurt them. They believe that the compiler is somehow being deliberately hostile, going “AHA! UB! I can do whatever I want here!”, and without optimizations turned on it won’t.

This is incorrect.

UB doesn’t mean that the compiler can take advantage of your sloppiness. UB means that the compiler can assume that your code is valid. It means that the intention of your code that’s oh so obvious when read by a human, doesn’t even have a way to be expressed between compiler stages or modules.

UB means that the compiler doesn’t even have to implement some special cases in its code generation, because they “can’t happen”.

The compiler, and really the underlying hardware too, is playing a game of telephone with your UB intentions. It may end up with what you wanted, but there’s no guarantee for now or in the future.

UB is everywhere

The following is not an attempt at enumerating all the UB in the world. It’s merely making the case that UB is everywhere, and if nobody can do it right, how is it even fair to blame the programmer? My point is that ALL nontrivial C and C++ code has UB.

Accessing an object which is not correctly aligned

As an example of this, take this code:

int foo(const int* p) {
   return *p;
}

If this function is called with a pointer not correctly aligned (probably meaning on an address that’s a multiple of sizeof(int), but who knows), this is UB. C23 6.3.2.3.

On Linux Alpha, in some cases this would merely trap to the kernel, which would software emulate what you intended. In other cases it would (probably) crash your program with a SIGBUS.

On SPARC it would cause a SIGBUS.

Sure, on x86/amd64 (henceforth just “x86”) this is likely fine. Hell, it’s probably even an atomic read. x86 is famously extremely forgiving about cache coherency subtleties.

So here we have three cases:

kernel gave a helping hand (Alpha for some loads)
crash (other Alpha loads, and SPARC)
not a problem (x86)

What about ARM, RISC-V, and others? What about future architectures? A future architecture could even have special int-pointer registers that do not populate the lowest bits, because such pointers cannot exist.

Even if it works, maybe the compiler one day changes from using one load instruction to another, and suddenly that’s no longer fixed up by the kernel.

Because the compiler is not obligated to generate assembly instructions that work on unaligned pointers. Because it’s UB.

Or how about this:

void set_it(std::atomic<int>* p) {
        p->store(123);
}
int get_it(std::atomic<int>* p) {
        return p->load();
}

Is this operation atomic when the object is not correctly aligned? That’s the wrong question to ask. Mu, unask the question. It’s UB. (but also yes, in practice this can easily be an atomicity problem)

If you want to get even more convinced, you can try thinking about what happens if an object you thought you were reading atomically spans pages. But don’t think too much about it, or you may conclude that “it’s fine”. It’s not. It’s UB.

Actually, it was UB even before that

Don’t blame the foo() function, above. The act of dereferencing the pointer wasn’t the problem. Merely creating the pointer was enough to be a problem.

Example:

bool parse_packet(const uint8_t* bytes) {
        const int* magic_intp = (const int*)bytes;   // UB!
        int magic_raw = foo(magic_intp);  // Probably crashes on SPARC.
        int magic = ntohl(magic_raw); // this is fine, at least.
        […]
}

That cast is the problem, not foo().

It’s perfectly valid for the compiler to assign specific meaning, such as garbage collection or security tagging bits, to the lower bits of an int*.

`isxdigit()` on `char` input

bool bar(char ch) {
        return isxdigit(ch);
}

isxdigit() is a simple function that takes a character and returns 1 if it’s a hex digit. 0-9 or a-f. It can also take the value EOF. Uh, ok. What value is EOF? Per C23 7.4p1 we know it’s an int, and we can infer that it’s not representable by unsigned char.

isxdigit() therefore takes an int, not a char. All values of char fit inside int, so we should be fine. Casting from char to int fits, so per section 6.3.1.3 we’re fine, right?

No. Because if bar() is called with a value other than 0-127, and on your architecture char is signed (implementation defined, per 6.2.5, paragraph 20 in C23), then the integer value ends up negative.

And the following is a valid implementation of isxdigit(), that would cause a read of who-knows-what memory. It could even be I/O mapped memory, triggering things to happen that is more than merely getting a random value or crash. It could cause the motor to start. Less likely in an application running in a desktop operating system than in an embedded system, sure. But there are user space network drivers (for performance), so even user space won’t protect you.

int isxdigit(int c) {
        if (c == EOF) {
                return false;
        }
        return some_array[c];
}

Casting from `float` to `int`

int milliseconds(float seconds) {
        int tmp = (int)(seconds * 1000.0); /* WRONG */
        return tmp + 1; /* WRONG separately (signed overflow is UB) */
}

When a finite value of real floating type is converted to an integer type[…]If
the value of the integral part cannot be represented by the integer type, the
behavior is undefined.
— 6.3.1.4

And, by omission, it’s also UB if the float is a non-finite value.

So how do you compare a float to INT_MAX? Do you cast the float to int? No, that’s the UB you want to avoid. So you cast INT_MAX to float? How do you know it can be represented exactly? Maybe casting INT_MAX to float rounds to a value not representable in int, and your comparison becomes non-representative?

Maybe the following works? You’ll miss out on representing some really high values, but maybe that’s OK?

int milliseconds(float seconds) {
        const float ftmp = seconds * 1000.0f;
        if (!isfinite(ftmp)) {
                // or other error reporting.
                return 0;
        }
        if ((float)(INT_MIN + 1000) > ftmp) {
                // or other error reporting.
                return 0;
        }
        if ((float)(INT_MAX - 1000) < ftmp) {
                // or other error reporting.
                return 0;
        }
        // Now safe to convert.
        const int tmp = (int)ftmp;
        if (INT_MAX == tmp) {
                // or other error reporting.
                return 0;
        }
        // Now safe to add.
        return tmp + 1;
}

I just wanted to convert a float to an int. :-(

I bet there’s lots of code out there that take a value in seconds, and convert it to integer milliseconds, by just multiplying and casting.

Object at address zero

Most programmers won’t have to deal with this, but I don’t think there’s any C standards compliant way in practice to put an object at address zero. This can come up in OS kernel and embedded coding.

By 6.3.2.3 an integer constant zero (which is convertible to a pointer) and nullptr are the “null pointer constant” (which I’ll just call NULL). C doesn’t specify that the actual pointer NULL points addr machine address zero, because the C standard only talks of the C abstract machine, not about hardware.

All C guarantees is that if you compare NULL to zero you’ll see them equal. But for all you know that’s because the zero is converted to the native platform’s NULL, which happens to be 0xffff.

It also explicitly says that dereferencing a null pointer, no matter what the value, is undefined behavior. It’s the example of UB under 3.4.3.

This also means that you can’t assume that memset(&ptr, 0, sizeof(ptr)); will create a NULL pointer! You cannot initialize your structs this way and assume member pointers are NULL! And this does apply to most programmers.

And yes, some historic machines used non-zero NULL pointers.

But let’s say you have a modern machine, where NULL is a pointer to address zero, and you actually have an object there.

Again, C 6.3.2.3 says that NULL compares unequal to “any object or function”. So this is UB:

void (*func_ptr)() = NULL;
func_ptr();

C says “there is no function there”. For all you know the compiler has no internal way to even express your intention here. You may argue that “but surely it’ll just emit a call instruction to the bit pattern of all zeroes? Nothing else seems reasonable.

What is “all zeroes”, though? On 16bit x86, is it 0000:0000? Is it CS:0000?

Variable arguments and types (e.g. printf with `%ld` instead of `%lld`)

This is UB:

execl("/bin/sh", "sh", "-c", "date", NULL);     /* WRONG */
execl("/bin/sh", "sh", "-c", "date", 0);     /* WRONG */

This is not:

execl("/bin/sh", "sh", "-c", "date", (char*)NULL);

Because the argument needs to be a pointer, and the NULL macro may be misinterpreted as an integer zero.

Similarly, this is UB:

uint64_t blah = 123;
printf("%ld\n", blah);  /* WRONG */

It needs to be:

uint64_t blah = 123;
printf("%"PRIu64"\n", blah);

So how do you print an uid_t? Well, you could cast them to uintmax_t and print them using PRIuMAX. But is uid_t even unsigned? Oh well, worst case you get a nonsense value printed instead of -1, I guess.

Divide by zero is UB

Sure, you probably knew this. But did you consider the security aspects of it? It’s not rare for the denominator to come from untrusted input.

And there’s so much more. The C23 standard contains 283 uses of the word “undefined”. And that’s not even including the things that are undefined by omission.

Bonus non-UB

Nobody can apply integer promotion rules at code skimming speeds. Nobody.

This post is already long enough, but as a start:

unsigned char a = 0xff;
unsigned char b = 1;
unsigned char zero = 0;
bool overflowed = (a + b) == zero;
// overflowed is set to zero, not one.

unsigned char a = 0x80;
uint64_t b = a << 24;     // Bonus UB(?)
// b is now 18446744071562067968 (ffffffff80000000), not 2147483648 (0x80000000).
// even with all our variables unsigned.

LLMs are better than us at this

Point an LLM at ANY C code, asking it to find UB, and it will. And it’ll be right almost all the time, nowadays.

I felt a bit bad after it correctly found ones in my code, so I thought I’d point it at the mature and pedantically written OpenBSD. I just picked the first tool I could think of, find, and it spit out a bunch.

I sent the project a patch for an out of bounds write (and also for a non-UB logic bug). I didn’t send them patches for the UB that was left and right, partly because the OpenBSD project has not been very receptive in the past for bug reports, my sense of “this is probably fine, in practice”, and that if OpenBSD wants to weed out UB from their code base, then that’s a major project that should be done in a better way than me just being the middle man between the LLM and them for a patch here and there.

I’ve seen several people complain that “nobody knows how to code C except me”. And they’re only wrong by one person.

So what do we do now?

We can’t just throw away our C and C++ code bases. But leaving them inherently broken is also not an option.

We need some way of fixing UB at scale, without committing AI slop nor overwhelming human reviewers.

This too is not a new opinion, nor a great revelation.

But yes, writing C or C++ in 2026 without an LLM supervising you for UB should probably be seen as a SOX violation, and just plain irresponsible. If OpenBSD people can’t find these problems given 30+ years, what chance do the rest of us have?

It may not scale to large code bases, but for my own projects I’ve asked the LLM to find UB, if necessary explain it, and fix it. And then stare at the output until I can confirm the issue and the fix.

A problem with this is that in order to confirm the findings, you’ll need an expert human. But generally expert humans are busy doing other things. This is janitor work, but too subtle to leave to the junior programmers who have traditionally been assigned janitor work.

This blog post was discussed on hackernews.

Quantum safe amateur radio secure shell

Mon, 11 May 2026 00:00:00 +0000

I’ve previously pointed out that the AX.25 implementation in the kernel is pretty poor. It’s not really being maintained, and even when it gets fixes after I reported it, with people running LTS OSs it can take like 5 years before before the fix actually reaches users, if ever. So when writing applications, you still have to work around kernel bugs from a decade ago. This makes it kind of pointless to upstream patches.

The exception is security patches, and reading between the lines of why the AX.25 code is now being removed from the kernel, it sounds like maybe some LLM (like the looming “Mythos” and the related Glasswing) may have found some severe problems. But even if there aren’t any known security problems yet, having code is now more of a liability than ever. Code needs to be removed, or taken responsibility of. (tangent about ffmpeg at the bottom of this post)

With the kernel code removed, say goodbye to the old walkthrough.

The new API

Well, not “new”, per se, but “replacement”.

With the socket based API about to be gone, we need some other way for applications to send packets and manage connections.

For sending raw packets to and from the modem there’s KISS. I have no real complaints about it. Not much to get wrong about sending frames. It’s implemented by most modems, like the software modem Dire Wolf and by some radios like the Kenwood TH-D75, so it’s not going anywhere.

For connected mode (streams of in order data, like with TCP) the biggest contender seems to be AGW. Dire Wolf implements it, and I’ve made a messy implementation of an AGW client in Rust. The async Rust API works, as we’ll see, but the code needs some refactoring and cleanup due to it being written exploratorily while I was deciding what it should even do, and how.

The AGW protocol is not super amazing, but it gets the job done. One can build a connection API on top of it, as I have, and never have to think about the AGW protocol ever again.

There’s another protocol called RHP, specified here and here. It came out of the XRouter project. Since XRouter is closed source, I have a strong aversion to it. It seems both counter to how I see amateur radio, and anachronistic, for it to be closed source. It’s bad enough that VARA and Winlink are closed source. And people are definitely working on replacing VARA with various other modes because of it.

tl;dr: I’m going with AGW for now. If someone writes a Rust crate for RHP exposing a compatible AsyncRead/AsyncWrite API, I certainly wouldn’t mind adding that dependency to optionally use.

I have not yet implemented AGW (or RHP) in my own AX.25 stack, but I plan to. For now that means I’ll use Dire Wolf.

axsh

Link to code.

The previous implementation

My previous axsh implementation, since deleted, had some problems:

it was implemented in C++, and not only do I prefer Rust, how could I even call something written in C++ “secure”? (a blog post for another day)
used the kernel API, so that needs rewriting,
used SEQPACKET, which proved to be a bit “weird” when interoperating with some other APIs, and
used crypto primitives vulnerable to quantum computers.

So with everything but terminal management needing a rewrite, this is a reason to rewrite the whole thing.

Requirements

Don’t use kernel AX.25 sockets — this means use AGW.
Use Rust.
Also work on TCP (mainly for debugging) — This means using an internal framing protocol.
Be quantum safe — Use ML-DSA+ed25519 dual signed for authentication of server and client.
Be efficient — This means don’t use ML-DSA for per packet signatures (they are huge), at the cost of some quantum safety (see the README).

Non-requirement: Encrypt — This would violate the amateur radio license. And then, why not just use SSH?

Example

If you have an AGW server, such as Dire Wolf, then it’s easy to run axsh. Just start a server:

axshd \
    -k server.key \
    -v debug
    -a authorized_keys \
    --agw-addr localhost:8010 \
    -l M0QQQ-1

Then log in:

axsh \
    -k client.key \
    -s M0QQQ-2 \
    --agw-addr localhost:8000 \
    M0QQQ-1

Then wait like 30-40 seconds for the handshake to complete. The reason for the wait is the large ML-DSA signatures used in the handshake.

It can’t be the same Dire Wolf instance, since Dire Wolf only shuffles packets between the radio and AGW clients, not from one AGW client to another. In my case I had one Dire Wolf connected to an ICom 9700, and another to a Baofeng UV5R using an AIOC (all in one cable). AIOC is highly recommended for experimentation over the air.

So yeah my test is between just about the cheapest VHF/UHF radio that exists, and maybe the most expensive one.

Dire Wolf setup

ICom 9700

In addition to running rigctld -m 3081 -r /dev/ttyUSB0 -s 19200:

$ cat direwolf-9700.conf
# Identified with `aplay -l`.
#
# To get pulseaudio to get its dirty hands off of the device, I turned it "Off"
# under "Configuration" in `pavucontrol`.
ADEVICE plughw:2,0
PTT RIG 2 localhost:4532
CHANNEL 0
MYCALL M0QQQ-3
AGWPORT 8010
KISSPORT 8011
MODEM 1200
PACLEN 256
$ direwolf -t 0 -c direwolf-9700.conf

AIOC

$ cat direwolf-aioc.conf
# Identified with `aplay -l`.
#
# To get pulseaudio to get its dirty hands off of the device, I turned it "Off"
# under "Configuration" in `pavucontrol`.
ADEVICE plughw:1,0
ARATE 48000
PTT /dev/ttyACM0 DTR -RTS
CHANNEL 0
MYCALL M0THC-8
MODEM 1200
PACLEN 256
$ direwolf -t 0 -c direwolf-aioc.conf

Why not just run this over TCP/IP?

With KISS providing packet support (and AGW providing a higher level API on top, if preferred), why not just run TCP/IP, and let the very stable OS TCP implementation take care of everything?

TCP is definitely more modern, stable, and maintained, but it doesn’t scale down to slow speeds very well. A TCP+IPv4 header is at least 40 bytes, and if you don’t want to be some sort of caveman, IPv6 is another 20 bytes. At 1200bps that would be 267-400ms overhead for every packet¹. Checking a random TCP data packet on my laptop I see that with TCP options TCP/IPv4 is actually 52 bytes, or 350ms.

Counting the air time (milliseconds, not just bytes) makes this overhead problem more obvious.

And because of amateur radio license reasons TCP would still need to identify the callsign, you probably have to add 17 bytes (113ms) as a surrounding header.

That leaves TCP with 69 or 89 bytes overhead per packet, meaning 460ms or 593ms. And since you don’t want to tie up the RF channel for too long (only for the whole packet to be dropped due to interference), you won’t want to send packets that are too large.

Of course it’s 4x as slow if you want to do something like Bell 103 on HF.

AX.25 connected mode takes that down to 19 bytes (126ms) overhead (if using Mod 128 mode) per data packet.

Because of the AX.25 segmenter, for bulk data TCP is not as bad as it may have sounded. For a 1500 byte TCP segment, fitting in just under 8 200 byte AX.25 frames (totalling 17*8+69=205 bytes of overhead), this means 1367ms overhead instead of plain AX.25 (at 19*8=152 bytes) 1013ms. A 1500 byte payload takes 10 seconds to send, so that’s an overhead of 13.7% instead of 10.1%.

But for interactive use cases, worst case a single payload packet, it’s 467ms vs 133ms. And that’s only counting the data frames, not the acknowledgments. A TCP ACK is at a minimum 17+40=57 bytes, or 380ms. An AX.25 RR is 18-19 bytes, or 120-127ms.

That makes TCP about three times less efficient, compared to AX.25.

A bigger problem with TCP, especially untweaked, is resend timers and window sizes. At 1200bps you don’t actually want too big a window size, since you don’t want to tie up the RF channel for several minutes if the other end has gone away. So a bunch of airtime tweaks are needed. And at best you’ll end up with the numbers above.

Maybe you could tweak TCP to be more friendly to lower speeds, and find the other overhead acceptable. If so, then you’ll be happy to hear that axsh supports running on TCP as well.

Why not QUIC?

Well first, it inherits the same problems from TCP/IP. Sure, the UDP header is smaller than the TCP header, but then on top of that there’s the QUIC header.

The second problem is that QUIC is meant to be encrypted. Ripping out encryption, while staying secure, seems more dangerous that keeping it simple and just working from the requirements. Probably the whole handshake would have to be redesigned.

FFmpeg & Google

AX.25 being removed from the Linux kernel reminds me of LLM finding that bug in ffmpeg, causing all that drama.

I have no dog in this fight, but in my opinion ffmpeg is in the wrong, here. Their argument seems to be all about how this particular encoder is rarely used, is just a hobby project, etc.. Ok, but it’s in your code base. Even if disabled by default, why would you want to ship a security footgun? Maybe some hobbyists out there build ffmpeg with all encoders enabled. Do you want them to be vulnerable to someone’s virus?

So Google should either keep quiet, or give a patch? Well, keeping quiet because the codec is rarely used is not really an option. That’s borderline negligent and morally culpable, for when someone eventually gets hacked.

So Google “should” always provide a patch in these cases? Perhaps, depending on the meaning of the word “should”. Google is rich, so “should” be morally forced to contribute to your software, just because Google (presumably, via youtube) is a heavy user of ffmpeg?

Well, that just sounds like the the (non-)problem with open source software (or free software) in general. The license permits use and profit without contribution. If you wanted a tithe then you should have put that in the license. Sounds like you want everyone to be free only to do what you want. That’s not how that works.

This is also why I don’t like the AGPL license. It’s not free software if it binds me in your serfdom.

Footnotes

Actually, it’s a tiny bit more, because of the occasional bit stuffing ↩

You cannot sell AI written software

Tue, 05 May 2026 00:00:00 +0000

You may have seen it too. This trend of “I wrote some software to solve a problem. I think it’s pretty great. Does anyone have any feedback?”. Maybe it’s a budget app. Or some company management thingy, tracking sales. Or invoicing.

Maybe you take a look. It looks pretty slick. But then you get a feeling of uncanny valley. It’s just not right. Maybe you can’t even put your finger on it.

I’m not an accountant, so when I see some accounting software do something in a different way, it’s interesting. Why is it that way? What can I learn from the fact that a professional thinks it should be this way?

You already know what’s weird about it, if nothing else because of the title of this post. The software works this way because the LLM wrote it that way. There’s no reason. It’s not even wrong.

How do you give “feedback” on that? My feedback would be that you don’t understand the problem you’re trying to solve, and have shown no sign you intend to understand it, so how could you possibly think you can solve it?

You’re not asking for feedback. You’re asking for someone else to do everything.

A house analogy

Analogies sometimes work, sometimes not. I’m not sure if this one will work.

You have a house with stairs that go nowhere, because “houses have stairs, right?”. You didn’t put the stairs there, and you have no sense for why there should be stairs in any particular place, so sure, why not there?

Contrast this with an actual architect. They know why they put the stairs there. They know why they didn’t put the stairs on the other side of the room. It’s not (just) that they have actual knowledge, it’s also that they actually had to think about “how do you get to upstairs, and what’s the experience of walking up those stairs?”. They had to make many small decisions, each of which required thinking, and required that the choices add up to a coherent whole.

The AI put the stairs somewhere, and you said “sure, stairs”. They’re in the kitchen, but since you only had to OK the stairs, not actively choose which room they go in, it didn’t even occur to you that room selection was a thing.

You don’t need architect credentials to do this right. You could have selected a room. It probably would have been a fine room. Maybe a professional could give you feedback on that choice. But if you don’t make choices then you’re not actually contributing to the thing you supposedly built.

You made no choices, so you don’t understand the problem you supposedly solved.

Jurassic park phrased it well

“If I may… Um, I’ll tell you the problem with the scientific power that you’re using here, it didn’t require any discipline to attain it. You read what others had done and you took the next step. You didn’t earn the knowledge for yourselves, so you don’t take any responsibility for it. You stood on the shoulders of geniuses to accomplish something as fast as you could, and before you even knew what you had, you patented it, and packaged it, and slapped it on a plastic lunchbox, and now you’re selling it”

Except you didn’t even “read what others had done”. Not even that bare minimum.

Artisanally crafted custom solutions

I use my own email client, written in artisanally hand crafted code, like back in the stone age. It solves a problem for me. I have my own youtube UI. And my own RSS reader. For a while I used my own SSH. I know people who wrote their own shell, and that is their daily driver. I plan to move this blog to my own webserver behind a custom SNI router.

Those can get feedback. I’m not saying they’re great, but even if the feedback is “why would you trust OpenSSL more than OpenSSH?”, it’s useful.

But honestly, if someone wants an email client kind of like mine, but customized to their preferences, then they may be better off asking an LLM to write their personal email client. That was not true 5 years ago. I don’t want to use email threading. Maybe you do. It’d be annoying to support both.

AI can make useful custom stuff

I was having some glitches with my video conference, and wanted to troubleshoot. I couldn’t find a tool that did what I wanted exactly. gping came close, but not quite.

I wanted something that pings regularly and frequently, summarizing these many pings about once a second, to illustrate subsecond network outages. And using Linux ICMP sockets, so that it doesn’t require root.

I spent about as much time looking for the perfect tool as I then spent telling an LLM to write it.

I had it write blipfinder, and it solved my problem. I could have had this tool written for me before my meeting had even ended.

Two relevant points about this:

I am, actually, a network expert. I know right from wrong in this space.
I am not, however, vouching for this tool. It solved my problem, and it’s amazing that custom throwaway tooling can be generated like this.

Just for fun, I had the LLM write it again. That version was also acceptable. It made some different choices. Which choices were best? Meh, nobody gave it any thought.

Why can you not sell it?

You weren’t forced to solve the problem, so you weren’t forced to actually understand the problem. You cannot identify a correct solution. Your software will never be fit for purpose for anybody but you.
Anybody who’s seen your software can ask an LLM to write it from scratch for them. If you aren’t adding any value by knowing right from wrong, correct from incorrect, then your potential customers are unable to buy expertise from you.

It can still be the right custom software for you. In fact custom software is a huge win for LLM. But selling that is not.

5 years ago there was no software in this uncanny valley. It was either obvious crap, or it was professional. 5 years ago your potential customers couldn’t replace you and your software with a prompt.

But now? If you sell this slop then you don’t have customers, you have “marks”.

Rustradio - SDR framework now also in the browser, with Wasm

Sat, 11 Apr 2026 00:00:00 +0000

I’ve previously blogged about RustRadio, my GNU Radio like framework for writing software defined radio applications in Rust. And now there’s more progress of an interesting kind.

Anything that tries to do something similar to GNU Radio needs a few things:

Core framework.
SDR components (filters, clock recovery, multipliers, etc).
A user interface.

In addition to these, GNU Radio also has the excellent GNU Radio Companion for interactive creation of flowgraphs, but I’m not tackling that yet.

I have a core framework, and some components (blocks). But the UI has been a bit lacking.

I’ve played around with TUI applications, but I always knew I also wanted to support having a UI in the browser. I’m not as interested in adding support for QT or Windows native UI. The browser will do fine.

There are two ways to get the UI in the browser:

Have the browser talk to a backend that’s running the actual DSP.
Running the DSP in the browser, with no need for a backend.

While I’ll want (1) eventually, and have some ideas about that, this post is about running everything in the browser, using Wasm.

Another great benefit of using Wasm for DSP and UI is that no installation is required. Users can just point their browser and run it immediately.

I know that this is just scratching the surface on some aspects of this, but I hope I at least didn’t get anything wrong.

Wasm, in short

There have been various technologies to running code in the browser. ActiveX, VBScript, Java, and Flash come to mind. They’ve all gone away. Only Javascript remains. Typescript is a better Javascript, but it’s still basically Javascript.

Wasm allows you to compile programming languages that normally compile to native code, like C, C++, and Rust, to a portable binary that the browser can execute. This is better than compiling (transpiling) them to Javascript, because it doesn’t require the overhead of guaranteeing Javascript behavior while actually executing another language’s compiled code.

Great. I’m not a fan of Javascript, so this should mean I’ll be able to do web coding without writing even a single line of Javascript. Well, aside from the line that goes “load the Wasm”.

More importantly for this project, it means I’ll be able to run RustRadio in the browser.

Web workers

Javascript is single threaded. That’s not a problem in itself, since RustRadio is perfectly happy running single threaded. But if something gets stuck in a loop then you’ll get the “Page is not responding” dialog, making you and your users unhappy. And even without that, while some heavy computation is happening in the main UI thread, the page will be unresponsive.

Javascript will likely never have threads in the traditional sense (some frameworks fake it, as I’ll get to).

So if you want to avoid locking up the main UI thread, then you need to spin off a “Worker”. This worker runs in its own thread, and has very limited access to the outside world, including other workers. In particular, it does not have access to the DOM (the web page). Only the main UI “thread” has access to that.

What you can do is post messages between the worker and the other worker or main thread that spawned it. You can also create a shared buffer, but I’ve not done that yet, so ignoring that for now.

Any heavy CPU work that’s expected to take a few milliseconds should be considered for offload to a Worker instead of running on the main UI thread. And keep in mind low end devices like phones, when estimating “a few milliseconds”.

The sandboxing of a Worker is also nice, in that if something runs in a worker, then you know that it can’t affect the DOM.

It’s apparently possible to run threads with Wasm in some setups, but it’s not been standardized, so it’s not something I’m going to build for. In particular, it doesn’t work on Apple phones, which is a big enough target that it’s a showstopper.

So for my purposes the only way to run a separate thread is to run a separate worker.

Computing and receiving messages on the same thread

A worker receives a message by having its onmessage handler called. But since the worker is just the one thread, it can’t interrupt whatever heavy computation was currently happening. It can’t even do something POSIX signal like, and redirect execution into a new stack. No, it just patiently waits until the worker finishes what it’s doing before calling that callback.

There are two ways of doing this. Either the worker snapshots its computation, and hopes it’ll get a message later, resuming it, or (technically equivalently) you use async functions. The latter seems better in my limited experience. Because async is just fancy syntax for returning a future, it’s not actually different from returning and resuming; It’s just automatic. It also means that having returned, any other onmessage handler for more messages have a chance to run.

async->sync->async

Sometimes async code needs to send a sync function to some API (e.g. register a sync callback). This handler is then called outside of async, and can’t await on any async. onmessage and button click handlers are sync, yet may need to await. This is part of that whole “what color is your function” thing.

It seems that the core library of Rust Wasm, wasm_bindgen, helpfully provides a function to register more futures for execution, with spawn_local(). This makes things easy. The handler then just registers the async call, and returns.

My example: An AX.25 1200bps decoder, in the browser

I’ve blogged about AX.25 before. For this post all you need to know is that it’s a data packet sent over radio, and that my example is able to find and decode the packet from a recording.

The live site is here if you want to try it. Or just enjoy this screenshot and video if you don’t. :-)

My first memory error in Rust

Rust is a memory safe language. JS is too. So how is this the first time I’ve managed to get memory corruption with Rust? I get this when I try a sufficiently large file (50MB+ tends to trigger it).

Notice in this screenshot how the vector has been corrupted, thinking it’s over a GB of data. It should be 64Ki.

Hopefully I’ll get this fixed, so for future reference this is when running the code in the ruwasm repo at commit 6088f711141fe566a8a02f9f40d5866ff6d8e82f.

Maybe it’s from some immature dependency. Maybe (more likely?) it’s a bug in my sloppily coded buffer handler. It does have a couple of unsafe.

GitHub Pages auto-deploy

You may have noticed that the URL to the live demo is hosted on github.io.

Because there’s no active server component, I thought I’d try out continuous deployment of this Wasm application. It was simpler than I expected. Just enable github pages, set them to be sourced from github actions, and add a simple github action, and upon every git push, the latest version gets hosted “in the cloud”.

That’s it. Now whenever I push a new version to github, it’s live on that URL a minute later. Pretty cool.

Future work is to trigger another github action on tagging a release, creating a permanent location where this tagged version is deployed. That way it’ll be possible to bisect looking for a bug, without even compiling.

I’ll also want to have it trigger on pull requests, so that it can be tested prior to merge, without the need to download and build locally.

Wasm, the end of Javascript?

Of course this is Betteridge’s law of headlines, but this video is still interesting and relevant.

WebUSB should provide direct access to SDR hardware like RTLSDR and USRPs. There’s no reason it shouldn’t be possible to run a large complex interactive SDR application directly against hardware, all inside the browser with no server components.

The strange webserver hot potato — sending file descriptors

Sun, 26 Oct 2025 00:00:00 +0000

I’ve previously mentioned my io-uring webserver tarweb. I’ve now added another interesting aspect to it.

As you may or may not be aware, on Linux it’s possible to send a file descriptor from one process to another over a unix domain socket. That’s actually pretty magic if you think about it.

You can also send unix credentials and SELinux security contexts, but that’s a story for another day.

My goal

I want to run some domains using my webserver “tarweb”. But not all. And I want to host them on a single IP address, on the normal HTTPS port 443.

Simple, right? Just use nginx’s proxy_pass?

Ah, but I don’t want nginx to stay in the path. After SNI (read: “browser saying which domain it wants”) has been identified I want the TCP connection to go directly from the browser to the correct backend.

I’m sure somewhere on the internet there’s already an SNI router that does this, but all the ones I found stay in line with the request path, adding a hop.

Why?

A few reasons:

Having all bytes bounce on the SNI router triples the number of total file descriptors for the connection. (one on the backend, then one each on the router for upstream and downstream). There are limits per process and system wide, and the more you have the more you need to juggle them in code.
It also wastes CPU and RAM.
I want the backend to know the real client IP address, via getpeername() or similar, on the socket itself.
I don’t want restarting nginx to cut existing connections to backends.
I’d like to use TLS keys that the nginx user doesn’t have access to.
I used proxy_pass for livecount, and last time I got blog posts on HackerNews nginx ran out of file descriptors, and started serving 500 for it serving just plain old static files on disk. For now I’ve moved livecount to a different port, but in the long run I want it back on port 443, and yet isolated from nginx so that the latter keeps working even if livecount is overloaded.

Livecount has an open websocket to every open browser tab in the world reading a given page, so they add up. (no, it doesn’t log. It just keeps count)

What I built

I built a proof of concept SNI router. It is a frontline server receiving TCP connections, on which it then snoops the SNI from the TLS ClientHello, and routes the connection according to its given rules.

Anything it reads from the socket is sent along to the real backend along with the file descriptor. So the backend (in my use that’s tarweb) needs to have code cooperating to receive the new connection.

It’s not the cleanest code, but it works. I got ChatGPT to write the boring “parse the TLS record / ClientHello” parts. Rust is a memory safe language, so “how bad could it be?”. :-)

It seems to work for all the currently used TLS versions.

It’s not plug and play

As I said, it requires the backend to be ready to receive “hey, here’s a file descriptor, and here’s the first few hundred bytes you should treat as if you’ve read them from the client”.

File descriptors don’t have an operation to “unread”. If they did then this would be easier. Then it would “just” be a matter of giving a backend webserver a file descriptor. For some use cases that could mean starting a new webserver process that reads and writes from stdin/stdout.

Not super efficient to go back to the fork-exec-per-connection model from the previous century, but it would achieve the direct connection.

But the details are academic. We do need to pass along the snooped bytes somehow, or the TLS handshake won’t succeed. Which means it does need cooperation from the backend.

But it is privacy preserving

Because the SNI router never writes to the client, and therefore doesn’t perform a TLS handshake, it doesn’t need any private keys or certificates.

The SNI router has no secrets, and sees no secrets.

I also added a mode that proxies the TCP connection, if some SNI should be routed to a different server. But of course then it’s not possible to pass the file descriptor. So encrypted bytes will bounce on the SNI router for that kind of flow. But still the SNI router is not able to decrypt anything.

A downside is of course that bouncing the connection around the world will slow it down, add latency, and waste resources. So pass the file descriptor where possible.

The hot potato

So now my setup has the SNI router accept the connection, and then throw the very file descriptor over to tarweb, saying “you deal with this TCP connection”. Tarweb does the TLS handshake, and then throws the TLS session keys over to the kernel, saying “I can’t be bothered doing encryption, you do it”, and then actually handles the HTTP requests.

Well actually, there’s another strange indirection. When tarweb receives a file descriptor, it uses io-uring “registered files” to turn it into a “fixed file handle”, and closes the original file descriptor. On the kernel side there’s still a file descriptor of course, but there’s nothing in /proc/<pid>/fd/:

$ ls /proc/699874/fd -l
total 0
lrwx------ 1 thomas thomas 64 Oct 26 21:47 0 -> /dev/pts/5
lrwx------ 1 thomas thomas 64 Oct 26 21:47 1 -> /dev/pts/5
lrwx------ 1 thomas thomas 64 Oct 26 21:47 2 -> /dev/pts/5
lrwx------ 1 thomas thomas 64 Oct 26 21:47 3 -> anon_inode:[io_uring]

This improves performance a bit on the linux kernel side.

The SNI router does not use io-uring. At least not yet. The SNI router’s job is much smaller (doesn’t even do a TLS handshake), much more brief (it almost immediately passes the file descriptor to tarweb), and much less concurrency (because of the connections being so short lived as far as it’s concerned), that it may not be worth it.

In normal use the SNI router only needs these syscalls per connection:

accept() for the new connection,
read() a few hundred bytes of ClientHello,
sendmsg() of same size to pass it on,
close() to forget the file descriptor.

HTTP/3 can redirect connections

At the risk of going off on an unrelated tangent, HTTP/3 (QUIC-based) has an interesting way of telling a client to “go over there”. A built in load balancer inside the protocol, you could say, sparing the load balancer needing to proxy everything.

This opens up opportunities to steer not just on SNI, and is much more flexible than DNS, all without needing the “proxy” to be inline.

E.g. say a browser is in Sweden, and you have servers in Norway and Italy. And say you have measured, and find that it would be best if the browser connected to your Norway server. But due to peering agreements and other fun stuff, Italy will be preferred on any BGP anycasted address.

You then have a few possible options, and I do mean they’re all possible:

Have the browser connect to norway.example.com, with Norway-specific IP addresses. Not great. People will start bookmarking these URLs, and what happens when you move your Norway servers to Denmark? norway.example.com now goes to servers in Denmark?
Use DNS based load balancing, giving Swedish browsers the Norway unicast IPs. Yes… but this is WAY more work than you probably think. And WAY less reliable at giving the best experience for the long tail. And sometimes your most important customer is in that long tail.
Try to traffic engineer the whole Internet with BGP announcement tweaks. Good luck with that, for the medium to long tail.
Install servers in Sweden, and any other place you may have users. Then you can anycast your addresses from there, and have full control of how you proxy (or packet by packet traffic engineer over tunnels) them. Expensive if you have many locations you need to do this in. Some traffic will still go to the wrong anycast entry point, but pretty feasible though expensive.

The two DNS-based ones also have the valid concern that screwing up DNS can have bad consequences. If you can leave DNS alone that’s better.

Back to HTTP/3. If you’ve set up HTTP/3 it may be because you care about latency. It’s then easier to act on information you have about every single connection. On an individual connection basis you can tell the browser in Sweden that it should now talk to the servers in Norway. All without DNS or anycast.

Which is nice, because running a webserver is hard enough. Also running a dynamic DNS service or anycast has even more opportunities to blow up fantastically.

Where was I? Oh yeah, file descriptors

I should add that HTTP/3 doesn’t have the “running out of file descriptors” problem. Being based on UDP you can run your entire service with just a single file descriptor. Connections are identified by IDs, not 5-tuples.

So why didn’t I just use HTTP/3?

HTTP/3 is complex. You can build a weird io-uring kTLS based webserver on a weekend, and control everything (except TLS handshakes). Implementing HTTP/3 from scratch, and controlling everything, is a different beast.
HTTP/1 needs to still work. Not all clients support HTTP/3, and HTTP/1 or 2 is even used to bootstrap HTTP/3 via its Alt-Svc header.
Preferred address in HTTP/3 is just a suggestion. Browsers don’t have to actually move.

What about encrypted SNI (ESNI), or encrypted ClientHello (ECH)

No support for that (yet). From some skimming ESNI should “just work”, with just a minor decryption operation in the SNI router.

ECH seems harder. It should still be doable, but the SNI router will need to do the full handshake, or close to it. And after taking its routing decision it needs to transfer the encryption state to the backend, along with the file descriptor.

This is not impossible, of course. It’s similar to how tarweb passes the TLS session keys to the kernel. But it likely does mean that the SNI router needs to have access to both the TLS session keys and maybe even the domain TLS private keys.

But that’s a problem for another day.

tarweb was first written in C++.
livecount keeps long lived connection.
tarweb rewritten in Rust, and using io-uring.
You can redirect with Javascript, but this still has the norway.example.com problem.
I passed file descriptors between processes in injcode, but it was only ever a proof of concept that only worked on 32bit x86, and the code doesn’t look like it actually does it? Anyway I can’t expect to remember code from 17 years ago.

Ideal programming language

Sun, 07 Sep 2025 00:00:00 +0000

My last post about Go got some attention.

In fact, two of my posts got attention that day, which broke my nginx since I was running livecount behind nginx, making me run out of file descriptors when thousands of people had the page opened.

It’s a shame that I had to turn off livecount, since it’d be cool to see the stats. But I was out of the country, with unreliable access to both Internet and even electricity in hotels, so I couldn’t implement the real fix until I got back, when it had already mostly died down.

I knew this was a problem with livecount, of course, and I even allude to it in its blog post.

Anyway, back to programming languages.

The reactions to my post can be summarized as:

Oh yes, these things are definite flaws in the language.
What you’re saying is true, but it’s not a problem. Your post is pointless.
You’re dumb. You don’t understand Go. Here let me explain your own blog post to you […]

I respect the first two. The last one has to be from people who are too emotionally invested with their tools, and take articles like this like a personal attack of some sort. They go out of their way to be offended, and then start screaming “but I don’t fucking want guitar lessons!”. They want to counter attack against another programming language, thinking I would take it personally too. Maybe this heretic is a Java programmer, and that’s why he’s stupid? (bad guess).

It also reminded me of PHP programmers back in the PHP 3.x days who would die on the hill of defending PHP as an awesome language, while admitting that they knew literally no other language. What should they know of England who only England know?

I’m not offended. Those replies are not offensive; They’re boring. There’s nothing to learn from the comments, and probably not from the people making such comments in general, either.

Also it seems that somebody managed to get my whole Blog comment database deleted from disqus. Either disqus itself was hacked, or just my account with them. Or someone tricked disqus into deleting it. They managed to restore it, though.

Keeping this third type of commenter in mind, I got an email a few days later asking what programming languages are “closer to ideal”, and if I maybe have a blog post in me about that.

I don’t know who’s asking, so I replied the long version, while still taking the question at face value.

My reply (after minor edits)

Ideal… Well, this is getting into the space of “what is the best programming language”, which doesn’t have a perfect answer. To do what?

To make an Android app (something I’m not an expert in. I’ve just made one simple one), I think Kotlin seems nice. But I don’t know it very well.

For web development, something I also don’t do much, it’s probably Typescript.

For maximum portability for systems programming, C or C++ (depending on if all your target platforms (e.g. embedded stuff) support C++) is probably best.

But these are practical answers. Some people like Lisp. Others like Haskell. Rust strikes a good position between practical, low level control, safe, and a high level type system. If the Rust compiler supports your platform, then it’s pretty much as portable as C/C++.

I’ve written about the deficiencies of Java and Go because the ways they are deficient are interesting. I don’t find the ways C++ is deficient to be interesting. C++ is what it is. I happen to enjoy coding C++.

I also have thoughts about the trap of accidentally writing too much in bash.

I have a draft of things wrong with Rust. But so far I think they are all fixable. (e.g. no placement syntax has been defined yet)

But I have no interest in writing a blog post about the lack of memory safety of C++. It is what it is.

Java’s deficiencies are interesting because they are the best guesses of the future that 1990 had to offer. And those guesses were almost all wrong.

Go’s deficiencies are interesting/frustrating because it (almost entirely) is the best that the 1980s-early 1990s had to offer. And yet it launched in 2009.

Enjoyment

I don’t enjoy coding Javascript. So I’m experimenting writing frontend stuff in Rust and compile to WASM. So far it works for me, but is not something I’d recommend for anyone who wants to get anything done.

But no, I won’t be writing up which programming language is “ideal”, because it’s one of those “it depends”.

Blargh

I fixed shell pipes

Problem summary

The previous version

wp — Wrap Pipe

Examples

Quick install, if you have cargo

Quake demos raytraced again

A new raytracer

Another AI detour

So let’s write a new raytracer

The process

The result

AI solving problems

AI coding detour

Could the AI solve it?

Could I have solved it?

The AI session

Prompt

Output

Prompt

Output

Final patch

Follow-up

RustRadio UI improved

LLM to the rescue

The demo, in all its glory

You can run the demo too.

Everything in C is undefined behavior

It’s not about optimizations

UB is everywhere

Accessing an object which is not correctly aligned

Actually, it was UB even before that

isxdigit() on char input

Casting from float to int

Object at address zero

Variable arguments and types (e.g. printf with %ld instead of %lld)

Divide by zero is UB

Bonus non-UB

LLMs are better than us at this

So what do we do now?

Related

Quantum safe amateur radio secure shell

The new API

axsh

The previous implementation

Requirements

Example

Dire Wolf setup

ICom 9700

AIOC

Why not just run this over TCP/IP?

Why not QUIC?

FFmpeg & Google

Footnotes

You cannot sell AI written software

A house analogy

Jurassic park phrased it well

Artisanally crafted custom solutions

AI can make useful custom stuff

Why can you not sell it?

Rustradio - SDR framework now also in the browser, with Wasm

Wasm, in short

Web workers

Computing and receiving messages on the same thread

async->sync->async

My example: An AX.25 1200bps decoder, in the browser

My first memory error in Rust

GitHub Pages auto-deploy

Wasm, the end of Javascript?

The strange webserver hot potato — sending file descriptors

My goal

Why?

What I built

It’s not plug and play

But it is privacy preserving

The hot potato

HTTP/3 can redirect connections

Where was I? Oh yeah, file descriptors

What about encrypted SNI (ESNI), or encrypted ClientHello (ECH)

Quick install, if you have `cargo`

`isxdigit()` on `char` input

Casting from `float` to `int`

Variable arguments and types (e.g. printf with `%ld` instead of `%lld`)