Jeff Epler's blog

9 July 2024, 2:33 UTC

datetime in local time zone


ruff and flake8 consider that it is a mistake to ever use "naive" datetime objects.

However, this leaves no "obvious" way to do something like convert between local and UTC time using the datetime module!

I was not able to find a way in standard Python, but I was able to find it in dateutil (based on some stackoverflow where the top answer was a wrong answer, purposely not linked)

So: Use dateutil.tz.gettz() to get a timezone object that reflects the system's local time zone. This will pass ruff's checks.

>>> localtime = dateutil.tz.gettz()
>>> localnow
datetime.datetime(2024, 7, 8, 21, 33, 51, 897086, tzinfo=tzfile('/etc/localtime'))
>>> localnow.astimezone(datetime.timezone.utc)
datetime.datetime(2024, 7, 9, 2, 33, 51, 897086, tzinfo=datetime.timezone.utc)

You can also use a piece of code from the documentation called LocalTimeZone that makes a best effort to implement the local system time zone based on the items available in the "time" module. This has caveats, such as not working properly when a zone historically had different offsets than it does today, and you can't pip install it, you have to copy & paste it.

[permalink]

1 July 2024, 18:44 UTC

Sunflower Butter aka Sun Butter or Sunbutter (plus Chocolate Sun Butter)


These two bread toppings are part of our regular breakfasts and we hate to be without them.

Recipe makes two pint jars: one of regular sun butter and one of chocolate sun butter.

If you're used to nutella, this chocolate sun butter will be a surprise (but a good one, in my opinion)! It has much less sugar. If you like a sweeter product, you can use milk chocolate chips, or add additional refined sugar.

  • 3 quarts raw unsalted shelled sunflower seeds
  • olive oil as needed (around 2 tablespoons?)
  • salt according to taste (from a pinch to a teaspoon/5 grams)
  • generous 1/2 cup dark chocolate chips
  • up to 1/2 cup cocoa nibs for chunky chocolate sun butter

Special equipment needed:

  • "Half sheet" rimmed baking tray (US: 13x18 inches; I don't know how these are called in the rest of the world but that's about 330x450mm)
  • Food processor with 6 cup (1.5l) capacity

Directions:

  1. Preheat oven to 350°F (175°C)
  2. Place sunflower seeds on tray and place in oven
  3. Bake for a total of 25-30 minutes according to preference. Stir after 15 minutes
  4. Allow to cool for 10 minutes, then scoop all seeds into food processsor (or reserve 1/4 cup if you want a chunky sun butter)
  5. Allow to process in the food processor, regularly stopping to scoop the insides and bottom. At first this will be very powdery or grainy.
  6. When you feel you aren't making any progress, add olive oil in small increments and continue processing. At some point the mass will turn clumpy and finally become almost like a liquid.
  7. Process until the desired consistency is achieved. Taste for salt, oil, and texture; but be aware that the food processor has heated the mixture.
  8. If making chunky sun butter, add the reserved seeds and process briefly
  9. Scoop out a pint (470ml) of the paste: this is your finished sun butter.
  10. Add the chocolate chips to the food processor. It should be warm enough to melt the chips. Process long enough to thoroughly mix, scooping the inside at least once to ensure complete mixing.
  11. If making chunky chocolate sun butter, add the reserved seeds and/or cocoa nibs and process briefly
  12. Scoop out the remainder. This is your finished chocolate sun butter.

To make just a quart of sun butter, use 2 quarts sunflower seeds.

My food processor doesn't do a great job with just 1 quart of sunflower seeds, so if you want just the chocolate version I recommend using 2 quarts sunflower seeds & double the chocolate ingredeients.

We keep this in the cupboard and eat a batch over the course of a few weeks. It will also refrigerate for longer storage, but it's best to eat at room temperature.

[permalink]

2 June 2024, 12:31 UTC

An efficient pair of polynomials for approximating sincos


In CicuitPython, I was looking for an efficient polynomial approximation of sin(x) and cos(x) over the interval [0:π/2].

Being a simple person, I searched enough documentation to find that numpy was happy to make a polynomial approximation of any table of data:

>>> x = np.linspace(0, np.pi/2, 10000)
>>> Polynomial.fit(x, np.cos(x), deg=5)
Polynomial([ 0.70710188, -0.55535829, -0.21798633,  0.05707696,  0.01090002,
       -0.00171961], domain=[0.        , 1.57079633], window=[-1.,  1.], symbol='x')
>>> Polynomial.fit(x, np.sin(x), deg=5)
Polynomial([ 0.70710188,  0.55535829, -0.21798633, -0.05707696,  0.01090002,
        0.00171961], domain=[0.        , 1.57079633], window=[-1.,  1.], symbol='x')

But wait, the coefficients are the same, except for some of the signs? What's going on? That's not how sin and cos are related.

The design of numpy's polynomial fit routine has given us an unexpected gift: the original domain of 0 to π/2 has been changed (offset & scaled) to the window -1 to 1. This happens to make the shifted-sin and shifted-cos routines reflect around the line x=0. So it's no surprise that the even coefficients are the same (as (-x)^2k = x^2k for integers k) and the odd coefficients are negated.

This leads to an implementation of sincos that only has to do fewer multiplications than I expected, and will therefore be a bit quicker.

Oh, and the maximum absolute error for this polynomial seems to be about 5e-6, or little enough that it probably can't be noticed when processing 16-bit audio.

C implementation of the algorithm:

Compiled on godbolt with -Os -mcpu=cortex-m4 -mhard-float -fsingle-precision-constant, it gives some very tidy-looking code.

fast_offset_scaled_sincos(float, sincos_result*):
  vmul.f32 s15, s0, s0
  vldr.32 s12, .L2
  vmul.f32 s14, s15, s15
  vmul.f32 s13, s0, s15
  vmul.f32 s14, s14, s12
  vldr.32 s12, .L2+4
  vfma.f32 s14, s15, s12
  vldr.32 s12, .L2+8
  vmul.f32 s15, s15, s13
  vadd.f32 s14, s14, s12
  vldr.32 s12, .L2+12
  vmul.f32 s15, s15, s12
  vldr.32 s12, .L2+16
  vfma.f32 s15, s13, s12
  vldr.32 s13, .L2+20
  vfma.f32 s15, s0, s13
  vadd.f32 s13, s14, s15
  vsub.f32 s14, s14, s15
; function epilogue and table of constants (L2) omitted

Note: More optimal coefficients can come from better algorithms like Remez, as implemented by py-remezfit. The resulting worst error is a bit higher but the average error should be lower.

$ python3 remezfit.py -d single  -- "lambda x: np.cos((x+1)*np.pi/4)" -1 1 5
p = array([ 0.7071076   , -0.5553769   , -0.21797441  ,  0.05672453  ,
        0.011838001 , -0.0023185865], dtype=float32)
        
prec = 7.189810e-06

[permalink]

31 May 2024, 2:04 UTC

Leaving my roles in LinuxCNC


It's been a long time since I actively participated in this project and I want to let you all know that I have begun to remove myself from roles in various places including sourceforge & github. I'm in private discussions to ensure this happens without disrupting the project.

To the developers & community: Thank you so much for letting me play a part in this project. It was an important chapter of my life (that started around 20 years ago!) and I wish you all the best for the future. I hope and trust you'll continue to take this project in good directions. I wish you all the best.

[permalink]

13 January 2024, 15:04 UTC

Linux ThinkPad T16 Microphone "Muted" Indicator


This laptop's keyboard has an indicator on the F4 key, which also serves as the mic mute toggle key.

Frustratingly, in Debian 12 ("bookworm") with pipewire audio, the LED doesn't actually follow the mic mute state. This appears to be because pipewire doesn't mute the mic at the hardware level, so setting the corresponding LED's "trigger" to "audio-micmute" does nothing.

I don't know what the proper solution for this is, but I implemented a solution of my own and it seems to work.

First, the LED control file (in my case "/sys/devices/platform/thinkpad_acpi/leds/platform::micmute/brightness") has to be made writable by my user (I don't care about multi-user situations). I did this by making a boot-time cron job as root:

@reboot chown jepler.jepler  /sys/devices/platform/thinkpad_acpi/leds/platform::micmute/brightness

Second, I have to run a script that watches the PulseAudio mic mute status and updates the LED. It's shown at the bottom of this blog post. It requires python3 and the pulsectl library, installable via pip.

I start this script in the background at login time. In my case I do this via the ".xsession" script, but you will need to know the correct way to do it in your desktop environment.

Now, when I press the mute key, Fn-F4, the LED's state follows the mute state.

read more…

15 November 2023, 20:46 UTC

Notes on using skyui with Skyrim Anniversary Edition from GOG


1. install Lutris
2. enter GOG credentials
3. Install Elder Scrolls Special Edition
4. Install DLC
5. Run skyrim once and quit
6. manually download:
  - vortex mod from nexusmods
  - skse64 from https://skse.silverlock.org/
  - skyui from https://www.nexusmods.com/skyrimspecialedition/mods/12604
  - unofficial patch from nexusmods
7. In lutris, "run exe inside wine prefix", select vortex mod exe
   (uncheck the "run vortex" option in the last installer step)
8. In lutris, duplicate the skyrim launcher. Call the duplicate "vortex (skyrim se) and set its executable to .../Program Files/Black Tree Gaming Ltd/Vortex/Vortex.exe. Open the new program, and select the first option when prompted
9. On first run, agree to downloading microsoft .net
10. find "skyrim special edition" under unmanged (use "search for a game")
    and then select its location under My Computer / Drive C / GOG
11. Click to the mods pane.
12. drag & drop the downloaded skse64 and skyui 7z files to vortex
13. change the skyrim launcher path to .../GOG Games/Skyrim Anniversary Edition/skse64_loader.exe
14. Launch skyrim, choosing the first option

[permalink]

7 July 2023, 13:49 UTC

"Letter Boxed" puzzle statistics


You might wonder, for a fully random "letter boxed" puzzle, what proportion are unsolveable?

I modified my solver from the previous post to also generate random puzzles, and analyzed the results of 100,000 runs. For these runs, I used a dictionary from github that claimed to be "the scrabble dictionary", though I have no way of verifying it. It contained 178691 words which I had to convert to lowercase due to the way my search program is implemented.

To the statistics:

  • About 37% of puzzles did not result in a solution of up to 12 words
  • The most frequent number of letters in a solution was 18; the least was 13 and the most was 38
  • The median frequent number of letters in a solution was 19
  • The most frequent and median number of words in a solution was 4; the least was 2 and the most was 10
  • About 5% of puzzles with a solution had a 2-word solution; about 30% had a 3-word solution.

I haven't played enough official puzzles to know, but I suspect that there are additional constraints on the letters (For example, no Q without U; vowels on at least 3 edges), or puzzles are constrained by having at least one "relatively short" answer, or the puzzle is fully human curated.

Here's the worst puzzle I found, using that dictionary from above. It has a shortest solution of 10 words & 39 characters: fysvzhcplgim

puzzle answer for fys / vzh / cpl / gim ghi - ich - hip - phiz - zips - shiv - vivific - chili - ism - myc

Not all of those seem like words to me, but that is how scrabble dictionaries are.

[permalink]

6 July 2023, 19:04 UTC

NYT "Letter Boxed" solver in C++


My wife and I regularly solve the NYT crossword in the app.

Lately it's been trying to get me to solve a puzzle called "Letter Boxed".

In this puzzle, there are 12 letters arranged with 3 on each side of a square. Valid words have to be made from the letters, with the additional constraint that two consecutive letters may not be on the same side of the square. For example if the edges are "xoy", "tws", "apv" and "kri" then "air" is not a valid word because the consecutive letters "ir" come from the same side. "vow" is. The last constraint is that each consecutive word starts with the last letter of the previous word, so "vow" can be followed by "wok" but not "sat". (Words of any length are permitted, not just 3 letters)

There is no particular scoring to the game, though you're suggested to "try to solve it in # words"; you can also view the previous day's suggested solution.

To me, it seems that the best answer is in the fewest words, with ties broken by the fewest number of characters.

The following program (which assumes that there's a standard unix-style dictionary at /usr/share/dict/words) can find what appear to be "optimal" solutions in only a few milliseconds. Simply supply it with the 12 letters as the first commandline argument, and it will perform a breadth-first search (BFS) with up to 5 words in length. Each candidate printed is better scoring (shorter) than the previous one, so the last line is the best score.

$ ./a.out xoytwsavpkri
228 candidate words
proviso - oaks - sixty - yaw
provisos - sixty - yaw - wk
vow - warps - sixty - yak
Checked 3027570 sequences

I originally wrote a program in Python, but memory usage was high and speed was low. This version uses an efficient structure where each word is reduced to a 32-bit quantity that tracks the letters present, the word length, and the word's terminal character. A particular game play is characterized by a fixed-size data structure that includes the characters used so far, the number of words, the number of total characters, and an array of up to 5 words. The deque is initialized with one entry for each possible word. In the main loop, the item is taken from the front of the deque. Then, based on the terminal letter of the last word played it tries each word starting with that letter. If this potential solution is not lower-scoring than the best one, then it stops evaluating. Otherwise, if this word completes the puzzle, the solution is printed and the best known score values are updated. Otherwise, this puzzle state is added to the end of the deque. The program loops until all possibilities have been evaluated.

A dynamic-programming approach would probably beat the BFS but BFS is quite fast enough for the published puzzles I've solved. It might also be possible to work from both ends towards the middle.

I don't have enough experience with the puzzle to know if 5 words always suffice for published puzzles, but it seems likely. The two real puzzles I have tried have 2-word solutions from this program (13 characters, the minimum possible length for 2 words), while they were suggested to be solvable in 5 and 6 steps.

When it comes to randomly selected puzzles, there are possible boards for which I find a best answer of 7 words (using a modified version of the program)

knezcuvamybx
68 candidate words
zany - yak - kc - can - numb - beaux - xv 7/23
zany - ye - exec - cab - bevy - yuk - km 7/22
zany - yuk - km - me - eve - exec - cab 7/21
Checked 3516880 sequences

and other sequences which have no solutions up to 7 words and use a lot of RAM and time before giving no solution (11GB peak resident size, 16 seconds):

mtijacfshpwk
194 candidate words
Checked 636279365 sequences

I didn't go full dynamic-programming but I did track the best way to each each of the 12*4096 states and stop recursing if the new candidate doesn't reach a known state faster. This runs much faster and uses less memory; the 11GB & 16s example with no solution above is now <4MB and <.01s! Runs below are with that version. (and apparently using a different dictionary, some runs were on debian oldstable and some on debian stable)

mtijacfshpwk
214 candidate words
Checked 1944 sequences

I also found that there are 8-word puzzles:

kmujpocaziqh
69 candidate words
jam - ma - aqua - ah - ho - oz - zip - pick 8/22
Checked 854 sequences
and 9+-word but I don't like some of the words (and the app doesn't accept 2-letter words):
dnijzaexkyrc
153 candidate words
jerk - kc - ca - ax - xi - icky - yd - dz - zen 9/23
Checked 1038 sequences

Source code (GPL-3.0 license) (build with g++ -std=c++20 -O2):

Faster version:

Original version:

[permalink]

8 June 2023, 22:51 UTC

Conservation of Experience

11 May 2023, 12:27 UTC

Xerox 820 & CP/M

27 March 2023, 0:01 UTC

Welcome to the Polity

7 November 2022, 1:35 UTC

Local coordinate systems in OpenSCAD

4 September 2022, 2:11 UTC

Recent keyboard deeds

All older entries
Website Copyright © 2004-2024 Jeff Epler