Jeff Epler's blog

Cancellation error

I was recently reminded of the importance of choosing numeric algorithms that don't behave catastrophically for certain inputs. One example is the calculation of 1-cos(θ) for small θ. In this case, cos(θ) is very close to 1, leading to a large cancellation error in the subtraction step.

csql: perform sql queries on csv data

The websites I operate are flat-file based, so when I want to store data it's easiest to write it in csv format. But it's not particularly easy to make queries on this data without additional software.

I'd rather not fire up Open Office or try to figure out how to make awk parse a quoted excel-format file, so I wrote 'csql', which reads a csv file into an in-memory sql database, performs a query, and then writes the result also in csv format.

A short example:

rsvp.csv:

Buffy Summers,false,0
Xander Harris,true,2
Willow Rosenberg,true,1

rsvp.cols:

name text
attend boolean
count integer

Performing queries:

$ csql rsvp.csv 'select sum(count) from t where attend="true"'
3
$ csql rsvp.csv 'select name from t where attend="true"'
Xander Harris
Willow Rosenberg

Files currently attached to this page:

csql

3.4kB

Focus Stacking Success

I've finally had success with focus stacking!

Inspired by the focus stacking assistant, I wrote a Python program using pyptp that does something similar with a Canon EOS tethered to a Linux PC. (I intend to release this in a bit, but it's very much "under construction" code at this time)

I then aligned the images in hugin and stitched with enblend.

There's still some tinkering to do with tunable parameters, but it looks pretty good! In the thumbnail you can see that all of the table cloth is in focus; view the medium or large versions to see more detail.

tnef2mime: automatically extract parts from winmail.dat

In her professional life, my fiancée routinely gets emails with attachments in the 'winmail.dat' aka application/ms-tnef format. It does her no good to berate these prisoners of proprietary software, so I set out to provide her with an easy-to-use solution that would work on her end. Now, prompted by this blog entry I am posting it for others to use.

There are a number of programs that can rescue data from this proprietary attachment format, but I was not aware of a program that would automatically convert the winmail.dat attachment to (a series of) regular mime attachments. (it appears that ytnef_smtpd may be such a program)

So I wrote one, tnef2mime.py. tnef2mime reads a message on standard input, combs it for ms-tnef attachments, and uses the tnef(1) program to extract their contents. Then it takes the extracted files, adds them as new MIME parts, and prints the whole resulting message.

This is suitable for use in e.g. procmail:

:0fw
| tnef2mime

Several steps are taken in order to minimize the chances for data loss: Just in case it did an imperfect job, it leaves the existing ms-tnef attachment intact. If any problem is encountered, the unmodified message is written.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

Files currently attached to this page:

tnef2mime.py

3.1kB

Enabling caching of my blog...

A long time ago, I wrote but never deployed a filesystem-based cache for my aether blog. This weekend, I cleaned it up and it's now running live on my blog. I hope nothing weird pops up as a result.

The ultimate goal of this is to try to get the websites that are running on my home system onto a web hosting provider, where CPU and disk I/O are at much more of a premium. So far, it improves the time loading the front page of my blog by about a factor of 360 (yes—not 360%, 360 times) on the local network when a cached respones can be used , which should make it quite fast enough to run even on a shared hosting system like dreamhost.

URLs with query parts ("?…") are never cached, so if you read me via rss, may I suggest to use the URL http://emergent.unpythonic.net/_atom if you are not already.

If you spot any problems, drop me an e-mail.

Towards my GPS LED Light Clock

A few years ago, I made a CCFL light clock using an Arduino with a custom shield containing a transformer (to get a reliable 60Hz timebase) and a triac (for solid-state switching of the lamp). By having a simple 7-day alarm calendar (set at compile time), the clock seldom requires interaction except for the reading lamp function.

However, the design has two main problems:

Mandelbulber fractals

(M)(L)

Floater

After the video a few weeks ago, I've continued playing with mandelbulber. Here are a number of images I particularly liked.

hostmot2-firmware 0.6 released

The main improvement since 0.5 is a new experimental xml format for describing bit files. There are no functional changes in the firmwares since 0.5. Packages for Ubuntu Hardy are on the linuxcnc.org package server. The files attached to this page may be useful to people using other OS distributions.