Jeff Epler's blog

5 October 2021, 23:41 UTC

My experience adding type annotations to a 2.5k-line Python library


The wwvb package for Python has been a focus of my recent hobby-time programming. I've used it as a place to educate myself about the ins and outs of maintaining a Python package. In the past, I used it to learn about using pylint, black & code coverage to improve the quality of Python code. Most recently, I added type annotations through the whole package until mypy --strict was happy with the whole wwvb package and uwwvb module.

The annotations were added in two steps: See pull requests #7 and #8. Together, these PRs contained 320 insertions and 223 deletions across 14 python files, plus 6 insertions in 2 other files related to CI. I did the work during a part of a day, probably under 4 hours of time spent. Since the package currently contains exactly 2500 physical lines of Python code, adding type annotations touched or added over 10% of physical lines!

read more…

18 August 2021, 19:03 UTC

Using Adafruit Macropad as LinuxCNC Control Pendant


Update, 2021-09-25: For compatibility with CircuitPython 7.0.0


CircuitPython recently gained the power to have custom USB descriptors. With these, we can define a USB HID device that will work with LinuxCNC's hal_input.

For instance, the Adafruit Macropad has a (very coarse, just 20 detents/revolution) encoder, 12 keyswitch positions, and an OLED screen.

The two pieces of software below, when placed in the CIRCUITPY drive as boot.py and code.py configure it for use with hal_input, using a halcmd line similar to loadusr -W hal_input Macropad. I haven't actually done the work of hooking it all the way up to Touchy yet, but it causes all the buttons & the encoder to appear in halcmd show pin.

This is just the device I picked first; there's nothing to prevent you from hooking up more exotic things like voltage/temperature monitors through added CircuitPython code. Addition of output reports for status indicators is left for the motivated reader.

read more…

24 June 2021, 1:43 UTC

Quick CircuitPython Driver for ES100 WWVB Receiver


I picked up an ES100 WWVB receiver kit and wrote a quick & dirty library to interact with it in CircuitPython.

I'm not super thrilled with how the chip works; I imagined that the date & time registers would act like an RTC after a successful reception, but instead they just mark the second when reception & decoding completed and are cleared to zero as soon as a new reception attempt is kicked off.

Still, I'll have to figure out a clock to put it inside. I am still thinking of doing an edge-lit display version of the Roman Solar Clock, so maybe that's where it'll go.

The library is jepler_es100.py and the example is code_es100.py (rename to code.py). I ran it on a Feather nRF52840 Expess with CircuitPython 6.3, but it should work on a range of boards.

Because the ES100 just locks up the I2C bus if you "repeated-start" it, I had to use my custom rolled register library instead of adafruit_register. I did build it on top of adafruit_bus_device.

Files currently attached to this page:

code_es100.py1.1kB
jepler_es100.py3.3kB

[permalink]

16 August 2020, 18:05 UTC

Si5351 Frequency Planner in Python


The Si5351 and related clock generators are very flexible, but they are a bit cumbersome to "program".

The basic design of the Si5351 is that an incoming frequency is first multiplied to create a high internal frequency (nominally in the range from 600MHz to 900MHz), then divided to create an output frequency. The multipliers and dividers have restrictions on their ranges, there are just 2 PLLs but 3 output clocks, and certain types of configurations (e.g., "divisor is an integer multiple of 2") are said to give lower jitter outputs than others.

The datasheet advising users on how to create their own register values is very complicated and some of the parts resist comprehension even after multiple readings. Thie chip maker Silicon Labs provides a graphical closed source Windows program for generating register maps, though some internet commenters regard it as buggy.

This program represents my own effort to create a frequency planner. It neglects some datasheet rules, mostly related to output frequencies above 50MHz or so. It tries to

  • Favor lower-jitter configurations where possible
  • Favor making the first-listed frequency as accurate as possible
  • Try each combination of ways to allocate output clocks to the PLLs
  • Obey the datasheet restrictions as I've understood them
  • Do all arithmetic as infinite precision arithmetic, not floating point
It does not
  • Implement the divisor rules for extremely fast clocks
  • Implement the divisor rules for higher numbered outputs on 20QFN packages

For exact input clocks such as 25MHz, it is surprisingly hard to find a "plausible" frequency that is inexact, and even harder to find a "plausible" frequency that is less exact than your reference clock. I didn't actually find a case where the error is not much smaller than the frequency stability of any reference I'd plausibly have access to.

(Of course, if you measure your reference clock as 25MHz + 13.37Hz and plug that in as the --input-freq then almost everything is related to that clock inexactly, so the fact that the clocks will still be really, really close to the requested value is very nice.)

It does not directly create a register map, but the values shown are easy enough to convert to the form used in the CircuitPython adafruit_si5151 library. Sadly, as CircuitPython does not support the fractions module, it's not currently feasible to run this code on a CircuitPython board directly controlling the si5351 chip.

Example:

Generate 315M/88 (NTSC colorburst), 4.43361875M (PAL colour carrier), 25.8048M (UART clock) all exactly from a 25MHz reference clock or crystal. However, the PAL colour carrier will have more jitter since it uses a fractional divisor:

$ ./party.py 315M/88 4.43361875M 25.8048M
Input frequency: 25000000 (25000000.0)
Frequency plan score: 10.38

PLL A:
Frequency plan type: Fractional multiplier, double integer divisor (1)
Multiplier = 126/5 (25.2)
Intermediate frequency = 630000000 (630000000.0)

Desired output frequency: 39375000/11 (3579545.4545454546)
Divider = 176 (176.0)
Exact
r_divider = 0 (/ 1)


PLL B:
Frequency plan type: Fractional multiplier, fractional divisor (5)
Multiplier = 12059443/500000 (24.118886)
Intermediate frequency = 602972150 (602972150.0)

Desired output frequency: 17734475/4 (4433618.75)
Divider = 136 (136.0)
Exact
r_divider = 0 (/ 1)

Desired output frequency: 25804800 (25804800.0)
Divider = 12059443/516096 (23.366666279141864)
Exact
r_divider = 0 (/ 1)

Generate pi MHz from a 25MHz reference clock or crystal (error: 27.8 nano Hz)

$ ./party.py 3.1415926535898M
Input frequency: 25000000 (25000000.0)
Frequency plan score: 1.00

PLL A:
Frequency plan type: Fractional multiplier, fractional divisor (5)
Multiplier = 24 (24.0)
Intermediate frequency = 600000000 (600000000.0)

Desired output frequency: 15707963267949/5000000 (3141592.6535898)
Divider = 40306053/211042 (190.98593171027568)
Actual output frequency: 42208400000000/13435351 (3141592.653589772)
Relative Error: -8.83757e-15
Absolute Error: 2.78e-08Hz
r_divider = 0 (/ 1)

The source is available in a github gist:

[permalink]

3 June 2019, 1:17 UTC

Precision vs Accuracy: A Clock


I was inspired by this watch face design (I think that's the original version) and by the arrival of a "OCXO", a very reliable time keeping circuit, to finally make an electronic clock.

Accuracy: Between hardware and software tuning, the OCXO keeps time with an accuracy of possibly better than 100 microseconds per day (loses or gains well less than a half second per year) (Yes, I'm deliberately ignoring a lot about crystal aging here!)

Precision: The time displayed is to the nearest minute, and the touchscreen setting mechanism is (deliberately?) poor, making it hard to set the time closer than +- 2 minutes or so. Oh, and it takes a good fraction of a second to update the screen anytime it changes. (The best way to set it seems to be to wait until a few seconds before 6AM/6PM and plug it in, since it boots with that time showing)

The clock consists of: .. all in a 3d printed enclosure that's not quite the right size.


The dial as an animation (1 revolution = 12 hours)
Along the way, I added an even more accurate time source to a Raspberry PI (GPS with PPS) so that I could even measure the accuracy of the OCXO, and discovered I even have a GPS module which was negatively affected by the GPS rollover that occurred in April of this year (the second 1024-week rollover). This leads to a surprising sequence of clock arithmetic, and finally gpsd decides the GPS is returning a date sometime back in 1963.

[permalink]

24 February 2014, 13:10 UTC

How do you check a signature on a PGP/Mime message


…I'll answer in Python. First, put your whole message in a file in unix "mbox" format (this will be the problem if you use some brain-dead GUI mailer!), then in Python extract the signature and the signed message, and finally invoke gpg:

import email
import os
import sys

with open(sys.argv[1]) as mfile: message = email.message_from_file(mfile)

if message.get_content_type() != "multipart/signed":
    raise SystemExit, "message not signed"

# Really, you'd want safe temporary filenames, and you'd want to clean them up
with open("msg", "wb") as msg: msg.write(message.get_payload(0).as_string())
with open("sig", "wb") as sig: sig.write(message.get_payload(1).get_payload())

# Delegate the real business to gpg
os.execvp("gpg", ["gpg", "--verify", "sig", "msg"])

[permalink]

19 February 2014, 21:17 UTC

Red: the perfect unit of measurement?


If you're like me, when you write software that has to store distance information in floating point, you shudder because you know you have to pick either inches or millimeters, leading most common multiples of the other as numbers that are not exactly expressible in binary floating point.

So let me make a simple proposal: define red = 625nm, so called because light of 625nm is commonly perceived as red. Now, you can store all of the following as exact integers:

  • Any binary fraction of an inch down to 1/64inch (1/64in = 635 red)
  • Any 1/10 of an inch (1/10in = 4064 red)
  • Any multiple of 5um (5um = 8 red)
Unfortunately, the following are still not exact:
  • 1mil = 40.64 red
  • 1pspt = 1/72in ~= 564.44 red
  • 1/300in (common print resolution) ~= 135.46 red

Here are some other visible-light inspired fundamental distance units and the common distances they can express exactly as binary fractions:

Table of magic lengths
Color name Exact definition Wavelength (approximate) Exact binary fractions for multiples of
red 8192/11162109375m 733.91nm 1/64in mil pspt dpi nm
red 64/89296875m 716.71nm 1/64in mil pspt dpi 25nm
red 512/744140625m 688.04nm 1/64in mil dpi nm
red 4/5953125m 671.92nm 1/64in .01in dpi um
red 32/49609375m 645.04nm 1/64in mil 5nm
red 1/1587500m 629.92nm 1/64in .01in 5um
yellow 256/446484375m 573.37nm 1/64in mil pspt dpi 5nm
green 2/3571875m 559.93nm 1/64in .01in pspt 5um
green 2048/3720703125m 550.43nm 1/64in mil dpi nm
green 16/29765625m 537.53nm 1/64in mil dpi 25nm
green 128/248046875m 516.03nm 1/64in mil nm
green 1/1984375m 503.94nm 1/64in .01in um
blue 1024/2232421875m 458.69nm 1/64in mil pspt dpi nm
violet 8/17859375m 447.94nm 1/64in .01in pspt dpi um
violet 64/148828125m 430.03nm 1/64in mil dpi 5nm
violet 1/2381250m 419.95nm 1/64in .01in 5um
violet 512/1240234375m 412.83nm 1/64in mil nm
violet 4/9921875m 403.15nm 1/64in mil 25nm
computed by colors.py, frequency range for each colorname from wikipedia. Unlike the original 625nm "red" constant, common lengths are binary fractions rather than whole numbers. And of course these aren't nice integer multiples of nm either. I wonder why no oranges make the table.

Files currently attached to this page:

colors.py2.4kB

[permalink]

23 November 2013, 18:19 UTC

SNTP from Python: getting server's esimate of time quality


Anders Wallin recently discussed generating graphs of server time error using a timestamp from the Date: header of an http request.

NTP servers themselves have estimates of time error, and it's possible to get the local server's idea of the error by sending SNTP packets to the local machine. You can also see the last time this NTP server successfully communicated with a peer or upstream, and see whether the server considers itself desynchronized by checking the stratum and leap second fields.

Based on my tinkering, the root_dispersion field or the derived LAMBDA value is the easiest to look at for estimating how bad the NTP time is. Between communications with a peer, root_dispersion and LAMBA increase at a rate of 15µs/s, ntp's arbitrary estimate of the accuracy of the local undisciplined clock. (YMMV and you can appaerntly tweak CLOCK_PHI in your NTP configuration file)

The attached program is a basic Python SNTP client which by default polls the local server every 5 seconds and prints some information. License: GPLv2+

Typical output:

12:25:58  0.06233  228.95875 ST-SYNCH LI-SYNCH
12:26:03  0.06241  233.96404 ST-SYNCH LI-SYNCH
12:26:08  0.06247  238.96745 ST-SYNCH LI-SYNCH
12:26:13  0.06255  243.97141 ST-SYNCH LI-SYNCH
12:26:18  0.06262  248.97673 ST-SYNCH LI-SYNCH
12:26:23  0.06270  253.98213 ST-SYNCH LI-SYNCH
12:26:28  0.06277  258.98746 ST-SYNCH LI-SYNCH

Files currently attached to this page:

ntpsynch.py4.9kB

[permalink]

17 January 2013, 16:34 UTC

moinmoin cleanup script

1 December 2012, 1:12 UTC

Android "Birds of Australia" unpacker

9 October 2012, 2:30 UTC

FTL (un)packer for Linux

16 October 2011, 15:31 UTC

csql: perform sql queries on csv data

All older entries
Website Copyright © 2004-2021 Jeff Epler