Jeff Epler's blog

27 March 2023, 0:01 UTC

Welcome to the Polity


(CW: Dysphoria, suicide)

Good news, Citizen!

Your connectome is among those archived from your ancestral homeworld, Earth. Galactic Polity \ufffd has chosen to instantiate it, and grants you full citizenship, with all the attendant rights and obligations.

read more…

15 March 2023, 19:31 UTC

How to fix pip in debian


Well meaning folks have broken pip: PEP 668 -- now, pip install whatever will fail with "error: externally-managed-environment" even if the intent is to install to the user's local "site-packages" area.

If it hasn't affected you yet, it'll affect you with the system python packages in debian testing.

The rationale is solid (-ish) but I prefer things to work as they do now, without changing EVERY pip install line in EVERYthing I cut and paste from the internet (to add --break-system-packages to the commandline).

The solution, at least until a "super PEP 668", is to remove the marker file EXTERNALLY-MANAGED. However, just doing a "sudo rm" won't stick when the package is updated by apt.

Here's one way to durably put aside the 'EXTERNALLY-MANAGED' marker of your system Python installation:

dpkg-divert --divert /usr/lib/python3.11/NOT-EXTERNALLY-MANAGED  --rename /usr/lib/python3.11/EXTERNALLY-MANAGED
This "diverts" (renames) the file, in a way that is respected by dpkg.

[permalink]

21 December 2022, 15:12 UTC

A quick example of transforming Python with libcst


I had occasion to encounter a Python library that used assert with a side effect:

assert initialize_hardware(), "hardware failed to initialize"
looking a bit more widely, this idiom was apparently used hundreds of times across a family of Python libraries.

"Aha", I said, "I bet I can fix this with an automated tool". In this round of investigation, I found LibCST and set about creating a program that would do what was needed, namely, to turn at assert into if not initialize_hardware(): raise RuntimeError("hardware failed to initialize").

While LibCST has an explicit facility for "codemodding", I didn't notice it at first and wrote in terms of transformers, with my own command-line driver program.

Unfortunately, while my transformer succeeded, attempting to format the CST back into code would result in an error without very many matches on my favorite search engine: Unexpected keyword argument 'default_semicolon'. That linked issue didn't provide an answer, but my further investigation did.

In the Python grammer as represented by LibCST, an assert statement is part of a SimpleStatementLine, while an if statement is not wrapped in a SimpleStatementLine. So if the transformation of an Assert node into an If node is done alone, the new If node lies inside a SimpleStatementLine node, and that is not valid. The problem is not detected until rendering the CST back into code. (It may be possible that using type checking would have found a problem, as this is essentially a type error)

The solution that I arrived at was to also transform any SimpleStatementLine which ended up containing an If node, by using the FlattenSentinel to do it. I think it might have been even more correct to directly perform the transformation within SimpleStatementLine, but what I ended up works now.

[permalink]

7 November 2022, 1:35 UTC

Local coordinate systems in OpenSCAD


Here's a small snippet for setting a local coordinate system. You need to know the location to treat as origin as well as the direction vectors to treat as X and Y.

// Take a position vector (p) and 3 direction vectors (uvw)
// and create a transformation matrix.  The origin is transformed to p,
// and xyz point along uvw.
// Normally, the caller will ensure that uvw are orthogonal unit vectors
function puvw(p, u, v, w) = [
    [u[0], v[0], w[0], p[0]],
    [u[1], v[1], w[1], p[1]],
    [u[2], v[2], w[2], p[2]]];

// Take a position vector (p) and 2 direction vectors (uv) which should be 
// orthogonal but need not be unit vectors. Calculate the unit vectors as well
// as the third vector and return the resulting matrix
function puv(p, u, v) =
    let(nu = u / norm(u), nv = v / norm(v), nw = cross(nu, nv))
    puvw(p, nu, nv, nw);

// Operate in a local coordinate system where the origin is the point p,
// x points along u, y points along v, and z points along the normal vector
// to uv. x and y should be orthogonal but need not be unit vectors.
module puv(p, u, v) {                         
    multmatrix(puv(p, u, v)) children();      
}                                             

For instance, suppose you want to lay out some feature relative to a cone 100mm in diameter and 500mm tall. It's convenient to set the origin to the intersection of the cone base with the +x axis, +x be the direction from the base of the cone to the tip, +y be the direction tangential to the cone, and +z be the direction 'out' from the cone. You might invoke puv like so (corr is a factor to make the flats of the faceted cone exactly touch the 100mm diameter circle):

effen = 16; // 16, 32, 64, ...
corr = 1 / cos(180/effen);

for(i=[0:45:360]) rotate(i)
puv([100,0,0], [-100, 0, 500], [0, -1, 0])
linear_extrude(height=1)
square([25, 4], center=true);

%rotate(45/4) cylinder(d1=200*corr, d2=0, h=500, $fn=effen);

[permalink]

4 September 2022, 2:11 UTC

Recent keyboard deeds


I made several USB HID keyboard conversions recently. They all use CircuitPython. The hardware setup is not documented as none are worthy of being called finished projects at this point, but the source code gives an idea what connections are needed. One or more of these may be the subject of a future Adafruit Learning System Guide.

Click a triangle to see the full code as an embedded github gist. (Note: if you're viewing this in a RSS reader, you may have to click through to get the embeds. Sorry!)

Commodore 16 Keyboard

Used an Adafruit KB2040. Gratuitous use of asyncio. The keymap needs work to be usable, but it's a good start. Direct gist link.

IBM XT "Model F" Keyboard

Used an Adafruit QT PY RP2040. This layout is AWFUL, the "beloved" iteration of the Model F was for AT computers, with a layout a lot more like a modern keyboard (aside from the placement of the Esc and Ctrl keys anyway) Direct gist link.

Tandy 1000 Keyboard

Used an Adafruit Feather RP2040. The trickiest, as caps lock and num lock are handled in the firmware; and there are some real oddities about the layout compared to the keyboards we're used to. This keyboard also needed the most clean-up but mechanically it's 100%. Direct gist link.

[permalink]

31 August 2022, 2:14 UTC

Don't wreck your system with miniconda/anaconda


I guess this software is a tolerable way to install those libraries and packages needed for so many machine learning things written in Python. But annoyingly, it wants to "go to the head of the line" in front of system Python. This is reallllyyyy not what I want.

I noticed that the little blob it deposits in ~/.bashrc can easily be surrounded with a function definition. So, now to activate anaconda in the current shell, but never replace/hide system python in a normal shell, I can just type "fml".

fml () {
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/jepler/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/jepler/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/home/jepler/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/jepler/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<
}

Now I don't feel quite so worried that having it present on the system is going to interfere with system software or with software I've installed to work with system software via pip.

[permalink]

15 April 2022, 16:23 UTC

Generate the 64 GCR values used by Apple DOS 3.3 and ProDos


for i in range(128):
    b = f"{i:07b}"                    # All these tests apply to the low 7 bits
    if '000' in b: continue           # 3 consecutive zeros disallowed
    if b.count('00') > 1: continue    # only one pair of consecutive 0s allowed
    if not '11' in b: continue        # one pair of consecutive 1s required 
    print(hex(i | 128), bin(i | 128)) # top bit is always a 1

[permalink]

15 April 2022, 2:14 UTC

An interesting finding about Apple DOS 3.3 diskettes


I can't be the first to have noticed this, but if somebody wrote about it years ago there's no way a search engine today will turn it up since they over-value recency and pay little attention to little sites with niche information. So here's my write-up. It feels a little inconclusive, but so be it. I imagine that these disks were mastered in an unsual way, rather than by duplicating a disk written on an Apple II using normal Apple II RWTS code.

I've been working on an open source tool to convert .a2r to .woz. While doing so, I got pretty low level with the Apple flux encoding.

This encoding varied over the years, and of course what really matters is what would work on real machines in the wild.

But one central claim about Apple floppy formats is that early DOS before 3.3 used the repeated bit sequence "1111 1111 0" to allow the floppy interace's state machine to synchronize with the stream of bits on the floppy. Starting with DOS 3.3 and continuing with ProDOS, and ProDOS, the bit sequence "1111 1111 00" is used instead. The newer sequence (also called FF40, because its hex value is FF and it takes 10 bits or 40 microseconds to transmit; the old code is also called FF36) is longer, but fewer repetitions are required to guarantee synchronization.

So I was surprised to see the following when I peered into "DOS 3.3 System Master [1983] - Disk 1, Side A.a2r" from CowgodA2R on archive.org (as are all the a2r files I inspected for this blog post):

111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
111111110    FF36
11111111     FF
11010101     D5
10101010     AA
10010110     96
11111111     FF
11111110     FE
10101010     AA
10101010     AA
10101111     AF
10101010     AA
11111010     FA
11111110     FE
11011110     DE
10101010     AA

That's right, the FF36 sequence appears in this DOS 3.3 floppy from 1983. Essentially the same sequence is seen on ProDOS User's Disk - Disk 1, Side A.a2r.

An even odder sequence is seen on the 1980 edition of DOS 3.3:

1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
111111110    FF36
1111111100   FF40
11111111     FF
11111111     FF
11001111     CF
11110011     F3
11111100     FC
11111111     FF
11010101     D5
10101010     AA
10010110     96
11111111     FF
11111110     FE
10101010     AA
10101010     AA
10101011     AB
10101110     AE
11111110     FE
11111010     FA
11011110     DE
10101010     AA

The CF/F3/FC/FF sequence would be an FF40 sequence if the earlier FF bytes hadn't already left you synchronized, but .. there's absolutely no reason for it!

Finally, a disk from the "non-originals" section, which I assume means it's cracked software, has the sequence I expect (more or less) based on reading documentation from the time as well as modern emulator source code:

1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
1111111100   FF40
11111111     FF
11010101     D5
10101010     AA
10010110     96
11111111     FF
11111110     FE
10101010     AA
10101010     AA
10101010     AA
10101011     AB
11111111     FF
11111111     FF
11111111     FF
11111111     FF
11101000     E8
10111111     BF
11111100     FC

[permalink]

29 January 2022, 19:38 UTC

Stuff I did in 2021...

3 November 2021, 21:27 UTC

80columns

31 October 2021, 14:17 UTC

Estimating WWVB Signal Health

30 October 2021, 21:12 UTC

Use printf() in Arduino programs

All older entries
Website Copyright © 2004-2021 Jeff Epler