CWWVB: Putting what I've learned about WWVB to use in a new decoder
It's time to write a new WWVB decoder from scratch. This one relies on regular sampling of the amplitude output of a low-cost WWVB receiver. Since these receivers already introduce up to 100ms of phase shift, trying to "home in on" the exact start of second isn't too useful, but sampling at 20ms is quite enough to tell the 0/1/Mark symbols apart.
Unlike other decoders I've read about (or written), this one is neither based
on simple pulse lengths (PulseIn) nor does it have a "start of second
acquisition" phase separate from the "receive & decode a full minute" phase.
Instead, the 'start of second' is continuously tracked by statistics over the
last 30-60 seconds of data, and then at the end of each second a symbol is
decoded.
The start of the second is the sample where the discrete derivative of the signal strength is greatest, at an offset of 0.16 here (based on a rather noisy set of input data):
Because the statistic is continuously (but efficiently) tracked, it doesn't matter if the local sampling clock has an error relative to WWVB. This just causes the offset to slowly shift, but doesn't affect decoding.
It's targeted at Cortex M microcontrollers, though it might fit on smaller micros like those on the classic Arduino. So far, I've only run it against logs from the WWVB Observatory, but it far outperforms my existing CircuitPython WWVB decoder (source code not online)---In an hour where my existing clock (using a PulseIn-like strategy) recieved 0 minutes successfully due to storms in the area, the new algorithm decoded 39 out of 59 minutes.
The C++ code is called CWWVB and it is up on github! It's not fully commented, but it does explain things more closely that this blog post does. My fresh receiver modules aren't coming before the end of the month, but I'm thinking of doing a very simple display, such as just showing MM:SS on a 4-digit 8-segment display, with an Adafruit Feather M4 for the microcontroller.
WWVB Observatory
A lot of my play coding lately has been related to WWVB, a 60kHz radio time signal broadcast from near Fort Collins, Colorado, USA.
I'm calling my latest work the "WWVB Observatory": I'm capturing the amplitude signal from an inexpensive "MAS6180C" receiver connected to a Raspberry Pi 50 times a second, and uploading the result to github hourly. The Pi is well-synchronized to the accurate time using NTP, and while it's not running as real-time software, the 20ms sample rate doesn't seem to pose any practical problems.
I'm mostly interested in using the data "ex post facto" to develop and measure the performance of different decoder algorithms, though I haven't started on that part yet. Others may have their own ideas.
In principle, this software infrastructure can also be used with other clock signals compatible with the MAS6180C: DCF77, HGB, MSF, JJY and BPC.
The code and data are on github. Right now I'm hoping to operate it for at least months, and if there's ever another leap second I fully intend to un-mothball it to try to record that very special moment that happens only rarely in the past few years.
One sub-part of the WWVB Observatory is an independent library for working with the leap second database known as "leap-seconds.list". I've uploaded it to github & pypi.
I still need to break out my "advanced Linux timekeeping APIs" library, erroneously called "clock_nanosleep.py". It wraps clock_nanosleep, clock_gettime, clock_settime and ntp_adjtime. clock_nanosleep is interesting because you can sleep until a particular deadline specified against a particular timesource (UTC, TAI, or monotonic being the useful options). Sleeping until a deadline is a fundamental building block of "realtime-ish" code like the WWVB Observatory.
My experience adding type annotations to a 2.5k-line Python library
The wwvb package for Python has been a focus of my recent hobby-time programming. I've used it as a place to educate myself about the ins and outs of maintaining a Python package. In the past, I used it to learn about using pylint, black & code coverage to improve the quality of Python code. Most recently, I added type annotations through the whole package until mypy --strict was happy with the whole wwvb package and uwwvb module.
The annotations were added in two steps: See pull requests #7 and #8. Together, these PRs contained 320 insertions and 223 deletions across 14 python files, plus 6 insertions in 2 other files related to CI. I did the work during a part of a day, probably under 4 hours of time spent. Since the package currently contains exactly 2500 physical lines of Python code, adding type annotations touched or added over 10% of physical lines!
Using Adafruit Macropad as LinuxCNC Control Pendant
Update, 2021-09-25: For compatibility with CircuitPython 7.0.0
CircuitPython recently gained the power to have custom USB descriptors.
With these, we can define a USB HID device that will work with LinuxCNC's
hal_input.
For instance, the Adafruit Macropad has a (very coarse, just 20 detents/revolution) encoder, 12 keyswitch positions, and an OLED screen.
The two pieces of software below, when placed in the CIRCUITPY drive as boot.py and code.py configure it for use with hal_input, using a halcmd line similar to loadusr -W hal_input Macropad. I haven't actually done the work of hooking it all the way up to Touchy yet, but it causes all the buttons & the encoder to appear in halcmd show pin.
This is just the device I picked first; there's nothing to prevent you from hooking up more exotic things like voltage/temperature monitors through added CircuitPython code. Addition of output reports for status indicators is left for the motivated reader.
OBS "insert current timestamp" hotkey
In the CircuitPython weekly meetings, one of the things the host does is record the timestamps of parts of the meeting into the notes document.
To facilitate this, I've created an OBS script that can be configured with a global hotkey. Then, just press the hotkey to insert OBS's idea of the current timestamp into any program.
It uses the program "xte" (debian package name: xautomation) on Linux to fake keypresses, and there's a spot where you would customize the script if you use another operating system-- Windows and Mac both have their own packages for making fake keypresses or using any other method that may be available to insert the text. (for instance, putting the timestamp in the operating system paste buffer and then synthesizing a press of the "paste" hotkey)
Collatz-bolge: A non-universal language, unless the Collatz Conjecture is false
Speculation on an esolang
I was reading recently about Malbolge, an esoteric language that was "designed to be as difficult to program in as possible". The language just has 8 operations. Those operations are "turing complete", (up to the limitation of memory size) but several complications are added for difficulty of creating working programs. In fact, while the language was specified in 1998, it took until 2021 for the announcement of a working LISP interpreter on a related language, Malbolge Unshackled.
Here I'm concerned with the "encypher operation": After each instruction is executed, the memory position indicated by the code pointer is "encyphered" according to a fixed permutation. Part of the proof of Malbolge's turing completeness was finding cycles under the encypher operation that would predictably alternate an instruction between a useful operation and a NOP.
In Collatz-bolge, the fixed permutation is replaced with the Collatz iteration function, n' = n * 2 + 1 (n odd), n' = n / 2 (n even), and the restriction that instructions be in the range 33-126 is relaxed to requiring instructions be 33 or above. It's widely believed that (as formalized in the Collatz Conjecture) every positive integer starting value eventually falls into the repeating sequence 4-2-1. If the Collatz Conjecture is true, every Collatz-bolge program's instructions would eventually decay into non-executable instructions, making the machine halt in a finite number of steps.
On the other hand, if there turn out to be other nontrivial cycles of the Collatz iteration function, it might be possible to prove the language was universal, by using those cycles to construct a translator from Normalized Malbolge to Collatz-bolge.
(Collatz-bolge would also need Malbolge Unshackled features like unbounded register sizes; as arbitrarily large numbers can probably be constructed by extremely long ASCII programs in Collatz-bolge we could also allow a training-wheels version where immense numbers could be directly specified)
Hybrid Keyboard Descriptor supports Boot & NKRO
Updated 2021-09-25: For compatibility with CircuitPython 7.0.0.
It has been suggested that it's possible to have both N-key rollover (NKRO) and be compatible with computer early boot screens (BIOS/UEFI/etc) by ensuring that the first 8 bytes of the report are compatible with the standard keyboard protocol.
On the heels of my earlier experience with how easy CircuitPython 7 made it to try a custom descriptor for NKRO, I tried implementing this too. And it works! On a sample of one Dell notebook computer, the hybrid descriptor can navigate in the boot menu but also enjoys full NKRO once Linux has booted.
You have to configure the descriptor in boot.py and then use the HybridKeyboard class in your code. It is compatible with the adafruit_hid.keyboard.Keyboard class, except that if you ask it to press more than 6 keys in a row, it doesn't throw an exception!
The original NKRO descriptor is also in this boot.py, in case you want to try both and compare how they work for you; code.py is intended to auto-select the correct interface class. Remember that you need to fully restart the Feather when modifying boot.py!
The first file below is boot.py and the second is code.py, and the code is designed for an Adafruit Feather RP2040 with custom 4x5 keypad, detailed in a Learning System guide.
CircuitPython NKRO Keyboard
Updated 2021-09-25: For compatibility with CircuitPython 7.0.0.
This is not a boot-compatible keyboard, but it does allow full NRKO. Based on my light testing, it works on Linux (Debian 11 & Android). Presumably it does not work as a boot keyboard device.
You'll need CircuitPython 7 (including the latest alpha release).
The descriptor was designed based on one from the qmk documentation.
In boot.py, set up a custom keyboard descriptor. After putting boot.py on CIRCUITPY, reset the device and make sure that boot-out.txt contains the line "enabled HID with custom keyboard device".
import usb_hid
BITMAP_KEYBOARD_DESCRIPTOR_REPORT_ID = 4
REPORT_BYTES = 16
bitmap_keyboard_descriptor = bytes((
0x05, 0x01, # Usage Page (Generic Desktop),
0x09, 0x06, # Usage (Keyboard),
0xA1, 0x01, # Collection (Application),
0x85, 0x04, # Report ID (4),
# bitmap of modifiers
0x75, 0x01, # Report Size (1),
0x95, 0x08, # Report Count (8),
0x05, 0x07, # Usage Page (Key Codes),
0x19, 0xE0, # Usage Minimum (224),
0x29, 0xE7, # Usage Maximum (231),
0x15, 0x00, # Logical Minimum (0),
0x25, 0x01, # Logical Maximum (1),
0x81, 0x02, # Input (Data, Variable, Absolute), ;Modifier byte
# LED output report
0x95, 0x05, # Report Count (5),
0x75, 0x01, # Report Size (1),
0x05, 0x08, # Usage Page (LEDs),
0x19, 0x01, # Usage Minimum (1),
0x29, 0x05, # Usage Maximum (5),
0x91, 0x02, # Output (Data, Variable, Absolute),
0x95, 0x01, # Report Count (1),
0x75, 0x03, # Report Size (3),
0x91, 0x03, # Output (Constant),
# bitmap of keys
0x95, (REPORT_BYTES-1)*8, # Report Count (),
0x75, 0x01, # Report Size (1),
0x15, 0x00, # Logical Minimum (0),
0x25, 0x01, # Logical Maximum(1),
0x05, 0x07, # Usage Page (Key Codes),
0x19, 0x00, # Usage Minimum (0),
0x29, (REPORT_BYTES-1)*8-1, # Usage Maximum (),
0x81, 0x02, # Input (Data, Variable, Absolute),
0xc0 # End Collection
))
bitmap_keyboard = usb_hid.Device(
report_descriptor = bitmap_keyboard_descriptor,
usage_page = 0x1,
usage = 0x6,
in_report_lengths = (16,)
out_report_lengths = (1,)
report_ids = (BITMAP_KEYBOARD_DESCRIPTOR_REPORT_ID,)
)
print(bitmap_keyboard)
devices = [
bitmap_keyboard
usb_hid.Device.CONSUMER_CONTROL,
]
usb_hid.enable(devices)
print("enabled HID with custom keyboard device")
In your code.py, use this subclass of adafruit_hid.keyboard.Keyboard:
class BitmapKeyboard(Keyboard):
def __init__(self, devices):
for device in devices:
if device.usage == 6 and device.usage_page == 1:
try:
device.send_report(b'\0' * 16)
except ValueError:
print("found device but could not send report")
continue
self._keyboard_device = device
break
else:
raise IOError("Could not find an HID keyboard device.")
# report[0] modifiers
# report[1:16] regular key presses bitmask
self.report = bytearray(16)
self.report_modifier = memoryview(self.report)[0:1]
self.report_keys = memoryview(self.report)[1:]
def _add_keycode_to_report(self, keycode):
modifier = Keycode.modifier_bit(keycode)
print (f"{keycode:02x} {modifier:02x}")
if modifier:
# Set bit for this modifier.
self.report_modifier[0] |= modifier
else:
self.report_keys[keycode >> 3] |= 1 << (keycode & 0x7)
def _remove_keycode_from_report(self, keycode):
modifier = Keycode.modifier_bit(keycode)
if modifier:
# Set bit for this modifier.
self.report_modifier[0] &= ~modifier
else:
self.report_keys[keycode >> 3] &= ~(1 << (keycode & 0x7))
def release_all(self):
for i in range(len(self.report)):
self.report[i] = 0
self._keyboard_device.send_report(self.report)
Quick CircuitPython Driver for ES100 WWVB Receiver
wwvb uploaded to pypi
7 1/2 years of Prius fuel economy
Semi-revived: Novelwriting
More Fibonacci?
Updated 'On the license of my Free Software'
Some exercises from Technical Drawing (Giesecke et al., 1949)
Calculating unreasonably large Fibonacci numbers quickly
All older entries
Website Copyright © 2004-2021 Jeff Epler