Where should CircuitPython go in 2022?
Looking back at 2021
Some of the goals I proposed for 2021 have been accomplished, yay!
- Support at least one new microcontroller family (I may have known the RP2040 was coming when I said that)
- Partially addressed the problem of the time required to build CircuitPython core with changes to GitHub CI
- circup and bundlefly greatly ease the problem of gathering items from the bundles for a particular application
- In the Adafruit Learn System, we got much more systematic about how programs and their assets are organized
- The bluetooth workflow is in the wild and people are using it, such as with the PyLeap app for Apple phones.
At a personal level, I accomplished one of the goals I listed, which was to improve my skills at 3D modeling and in particular I have become reasonably competent at part design using FreeCAD.
Some items we made progress on, but there's always room for further improvement:
- Continue to help people grow into the roles of reviewer and contributor
- JP's CircuitPython Parsec and Scott and FoamyGuy's livestreams are great, but I'm sure there's room for more "how to" video content around CircuitPython.
Some big pieces of progress we made that I didn't even anticipate (but wish I had) were
- Getting up to date with MicroPython, and then continuing to merge in their new releases
- Releasing asyncio support for CircuitPython
- Adding typing stubs to the core and types to many libraries in the bundle to work better in advanced IDEs
Looking forward to 2022
I'm not a big picture visionary about where the project should head. Lots of times it's modest or even invisible changes that give me the most satisfaction.
I look forward to implementing new drivers for hardware I wouldn't have dreamed of working on (reading from old DOS floppy drives, perfect example), adapting old algorithms into CircuitPython (let's do more image processing or maybe start doing audio processing), and finding small efficiency improvements and firmware size savings when they're needed.
I want to both learn more about and improve asyncio. A personal project goal would be to have an interactive display that continues to update & respond to input even while doing wifi requests. I don't think this works right now (I could be wrong), and I don't know how much is needed in order to make it work.
I look forward to hearing all about the projects you (yes, you) are planning to do in 2022; let's figure out if CircuitPython is the right coding language for those projects, and if it's not let's figure out if there are sensible additions that will make it work.
The Most Surprising Optimization I've Seen Lately
It seems that under some conditions, gcc (such as gcc 10.1 targeting amd64 linux with -O optimization) compiles the following bad C++ code into non-leaking code. In fact, the generated code is equivalent to return 3; without any call to an allocating function:
class T {
public:
T(int i) : i(i) {}
int i;
};
int should_leak_memory() { return (new T(3))->i;}
80columns
I've gathered up my set of improvements to mist64's 80columns program for Commdore 64 into the main branch of my fork.
Estimating WWVB Signal Health
Over in CWWVB, I've added a health estimate of the WWVB signal.
I define the health of the signal over one symbol as the number of samples that differ from the ideal for that symbol. The health of the signal is simply the health over a larger number of symbols. In the case of CWWVB, I've chosen to track 60 symbols (one minute) by default.
The signal health percentage is just the long-term health number divided by the total number of samples.
I passed a substantial number of logs from the WWVB Observatory to the standalone CWWVB decoder program, printing the health of each successfully decoded minute (the minutes were not checked for correctness against the known time, just for passing the basic tests: fixed mark and 0 symbols present, no BCD decode errors). Only 0.03% of minutes with health below 85% successfully decoded. I ended up picking 97% as the health cut-off number for CWWVB.
Use printf() in Arduino programs
Are you happiest with printf() debugging? Do circumstances lead to you writing "arduino code" because it's just about the easiest way to set up an embedded software project that's not to different from C(++)? I have something for you.
I've only tested it on a sample of 1 board (the Adafruit Feather M4 Express) but I suspect that it works on a wide variety of arm-based (or, actually, newlib-based) boards.
Long story short, put the following code in your .ino
file, call Serial.begin() as usual, then use
printf(...) and fprintf(stderr, ...) just
like you would anywhere. The only caveat I've discovered so far is that
printing floating-point numbers didn't work for me.
// This bridges from stdio output to Serial.write
#include <errno.h>
#undef errno
extern int errno;
extern "C" int _write(int file, char *ptr, int len);
int _write(int file, char *ptr, int len) {
if (file < 1 || file > 3) {
errno = EBADF;
return -1;
}
if (file == 3) { // File 3 does not do \n -> \r\n transformation
Serial.write(ptr, len);
return len;
}
// color stderr
static bool stderr_flag;
bool is_stderr = (file == 2);
if (is_stderr != stderr_flag) {
if (is_stderr) {
Serial.write("\033[95m");
} else {
Serial.write("\033[0m");
}
stderr_flag = is_stderr;
}
int result = len;
for (; len--; ptr++) {
int c = *ptr;
if (c == '\n')
Serial.write('\r');
Serial.write(c);
}
return result;
}
extern "C" int write(int file, char *ptr, int len);
int write(int file, char *ptr, int len) __attribute__((alias("_write")));
CWWVB: Putting what I've learned about WWVB to use in a new decoder
It's time to write a new WWVB decoder from scratch. This one relies on regular sampling of the amplitude output of a low-cost WWVB receiver. Since these receivers already introduce up to 100ms of phase shift, trying to "home in on" the exact start of second isn't too useful, but sampling at 20ms is quite enough to tell the 0/1/Mark symbols apart.
Unlike other decoders I've read about (or written), this one is neither based
on simple pulse lengths (PulseIn) nor does it have a "start of second
acquisition" phase separate from the "receive & decode a full minute" phase.
Instead, the 'start of second' is continuously tracked by statistics over the
last 30-60 seconds of data, and then at the end of each second a symbol is
decoded.
The start of the second is the sample where the discrete derivative of the signal strength is greatest, at an offset of 0.16 here (based on a rather noisy set of input data):
Because the statistic is continuously (but efficiently) tracked, it doesn't matter if the local sampling clock has an error relative to WWVB. This just causes the offset to slowly shift, but doesn't affect decoding.
It's targeted at Cortex M microcontrollers, though it might fit on smaller micros like those on the classic Arduino. So far, I've only run it against logs from the WWVB Observatory, but it far outperforms my existing CircuitPython WWVB decoder (source code not online)---In an hour where my existing clock (using a PulseIn-like strategy) recieved 0 minutes successfully due to storms in the area, the new algorithm decoded 39 out of 59 minutes.
The C++ code is called CWWVB and it is up on github! It's not fully commented, but it does explain things more closely that this blog post does. My fresh receiver modules aren't coming before the end of the month, but I'm thinking of doing a very simple display, such as just showing MM:SS on a 4-digit 8-segment display, with an Adafruit Feather M4 for the microcontroller.
WWVB Observatory
A lot of my play coding lately has been related to WWVB, a 60kHz radio time signal broadcast from near Fort Collins, Colorado, USA.
I'm calling my latest work the "WWVB Observatory": I'm capturing the amplitude signal from an inexpensive "MAS6180C" receiver connected to a Raspberry Pi 50 times a second, and uploading the result to github hourly. The Pi is well-synchronized to the accurate time using NTP, and while it's not running as real-time software, the 20ms sample rate doesn't seem to pose any practical problems.
I'm mostly interested in using the data "ex post facto" to develop and measure the performance of different decoder algorithms, though I haven't started on that part yet. Others may have their own ideas.
In principle, this software infrastructure can also be used with other clock signals compatible with the MAS6180C: DCF77, HGB, MSF, JJY and BPC.
The code and data are on github. Right now I'm hoping to operate it for at least months, and if there's ever another leap second I fully intend to un-mothball it to try to record that very special moment that happens only rarely in the past few years.
One sub-part of the WWVB Observatory is an independent library for working with the leap second database known as "leap-seconds.list". I've uploaded it to github & pypi.
I still need to break out my "advanced Linux timekeeping APIs" library, erroneously called "clock_nanosleep.py". It wraps clock_nanosleep, clock_gettime, clock_settime and ntp_adjtime. clock_nanosleep is interesting because you can sleep until a particular deadline specified against a particular timesource (UTC, TAI, or monotonic being the useful options). Sleeping until a deadline is a fundamental building block of "realtime-ish" code like the WWVB Observatory.
My experience adding type annotations to a 2.5k-line Python library
The wwvb package for Python has been a focus of my recent hobby-time programming. I've used it as a place to educate myself about the ins and outs of maintaining a Python package. In the past, I used it to learn about using pylint, black & code coverage to improve the quality of Python code. Most recently, I added type annotations through the whole package until mypy --strict was happy with the whole wwvb package and uwwvb module.
The annotations were added in two steps: See pull requests #7 and #8. Together, these PRs contained 320 insertions and 223 deletions across 14 python files, plus 6 insertions in 2 other files related to CI. I did the work during a part of a day, probably under 4 hours of time spent. Since the package currently contains exactly 2500 physical lines of Python code, adding type annotations touched or added over 10% of physical lines!
Using Adafruit Macropad as LinuxCNC Control Pendant
OBS "insert current timestamp" hotkey
Collatz-bolge: A non-universal language, unless the Collatz Conjecture is false
Hybrid Keyboard Descriptor supports Boot & NKRO
CircuitPython NKRO Keyboard
Quick CircuitPython Driver for ES100 WWVB Receiver
wwvb uploaded to pypi
7 1/2 years of Prius fuel economy
All older entries
Website Copyright © 2004-2021 Jeff Epler