Jeff Epler's blog

Don't wreck your system with miniconda/anaconda

I noticed that the little blob it deposits in ~/.bashrc can easily be surrounded with a function definition. So, now to activate anaconda in the current shell, but never replace/hide system python in a normal shell, I can just type "fml".

fml () {
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/jepler/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/home/jepler/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/home/jepler/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="/home/jepler/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<
}

Now I don't feel quite so worried that having it present on the system is going to interfere with system software or with software I've installed to work with system software via pip.

[permalink]

Hello, Allo

The good:

• 20, 300, and 1000Mbit/s rates available ($45 - $100/month base rate)
• Router/WiFi AP included in base rate
• Static IP is just $5/month
• symmetric connection speeds
• helpful staff
• No bandwidth caps, but "if you’re breaking the law, be assured you’ll be hearing from us."
• TV and telephone, if you're into that sort of thing

The bad:

• Except for static IP, you're behind "Carrier Grade NAT", so no "I'll cope with dynamic IP but still be able to SSH in"
• Your existing eqipment might not be able to keep up
• Given google's chat product also called "allo", these guys can be hard to search for
• No bandwidth caps, but "if you’re breaking the law, be assured you’ll be hearing from us."
• Small service area (9 Nebraska towns + Fort Morgan Colorado)
• No IPv6(!?), possibly deploying v6 in 2020.
• My static IP is listed in spamhaus "PBL" (but there is a self-service removal process, so it's fine)

The main headache I had was that first "the bad" bullet point: Despite having a phone app(!!) that acts like it can set up port forwarding, nothing I did could open up an incoming port. Staff were interested in helping me (including calling me back later to try one last idea), but ultimately the solution was just to add the Static IP option to my service.

The lesser headache, and the one which was totally my problem to solve, is that my firewall and NAT was being done by an older Buffalo wifi access point, WZR-HP-G300NH2 with an ancient version of DD-WRT. It simply couldn't get beyond about 160Mbit/s when doing NAT/forwarding. So I rejiggered my wires a little bit so that my i7 Linux desktop would take over those tasks. Additionally, all the "modern wifi" devices were already connecting to a newer Netgear R6220 in Access Point mode (routing functions disabled).

I had a second headache, which is apparently a decade-old bug in Linux's Intel e1000e driver. I was getting really poor rates on my internal network, and the kernel was logging "eth2: Detected Hardware Unit Hang". There are two main fixes that the internet suggests: modifying something about power saving modes, or disabling several advanced features of the NIC. In my case, I determined that using "ethtool --offload eth2 tso off" to disable tcp-segmentation-offload resolved the problem I was seeing. What's weird is that this NIC, eth2, is the one that I had been using all along; I had lots of network traffic on it for months. But the message never appeared in my local logs before I started also using "eth1" and doing NAT/packet forwarding yesterday.

Now from my desktop I get 960Mbit/s down, 720Mbit/s up (according to speedtest-cli), and 6ms pings to my office. My fastest wireless device gets somewhat over 200Mbit/s in each direction. Connecting to a VNC session at my office feels just as good as being there, which is primarily due to the extremely short packet turnaround; the bandwidth is a nice bonus though.

Right now it all feels pretty magical, and I'm looking forward to calling the cable company (spectrum) on Monday to cancel the service. I'm paying more (not quite 2x as much) but getting MUCH more service.

I'm contemplating buying one of these little embedded PCs with 2 NICs, they cost around $200 with RAM and a small SSD and it is claimed that they can forward at gigabit rates. They're literally just PCs inside (x86 CPU and BIOS booting), so all the headaches that attend little embedded ARM systems are nonexistent. But is an Intel Celeron "J1800" CPU actually up to pushing (including NAT) a quarter million packets a second?

I have a bittorrent client running with a bunch of Linux ISOs being seeded. I saw peak download rates of up to 92MB/s and typically 30-60MB/s, which is great. Right at the moment it's only clocking about 2MB/s of data "up"—the torrents seem to be pretty adequately seeded. I'm doing this primarily to see whether Allo treats "any traffic identifiable as bittorrent" as something that they'll tell you off about, or whether they are trying to distinguish "licit" from "illicit" when it comes to bittorrent traffic. I'm not sure which idea I like less.

[permalink]

Linux: don't wake display on mouse motion

There is not an explicit way to configure Linux (X11) so that it doesn't exit DPMS sleeps on mouse movement, but I found a tip on the internet to disable the mouse device at the XInput level when activating the screensaver, and reactivating it when the screensaver exits.

I don't run a "screensaver application" so wiring into things like dbus for notification doesn't work. Instead, I wrote an X program which polls the requested DPMS state and enables or disables the mouse device accordingly.

It remains to be seen whether this solves the problem which causes the display to turn on multiple times per night, but it might just fix it.

You'll need to customize the program by changing the "device_name" string in the source to match your own input device. If you have multiple input devices, then more extensive work will be required.

License: GPL v3+

Files currently attached to this page:

[permalink]

Literal copying of GPL code into ZFS on Linux

There are two main ways that zfsonlinux's zfs.ko could be a derivative work of linux: First, and the option that many commenters have concentrated on, is if the act of compiling zfs also includes copyrighted portions of linux in a way that activates the GPL's restrictions because the combined work is "a work based on [the Linux kernel]". However, there is a second way that zfsonlinux could be subject to the GPL and CDDL simultaneously: if source code licensed exclusively under the GPL has been copied into the zfsonlinux source tree.

In this article, I collect some samples of code from zfsonlinux and compare those samples to code committed earlier to linux. I present the samples using a customized word diff algorithm. To reproduce my results, you will need to manually find the associated refs in linux.git and zfsonlinux/zfs.git and use your own word-diff algorithm.

I don't know whether I've found the worst of the code copying; I found the two examples given below pretty quickly, imagined that I had found just the tip of the iceberg, and then decided to write this article before looking for more. But I haven't found any more that are as clear-cut as these two examples are in my mind, while also being over ten lines long. Beginner's luck? Or did I find the 50 copied lines out of 260,000 lines in a half hour and that's the whole story?

Because I cannot speak for the authors of the code committed to linux.git which appear to be duplicated in zfsonlinux, it is impossible for me to say whether they have some private arrangement with the code authors that permit the use of the code snippets under the CDDL. If not, it's a legal question whether the amount of code copied is relevant under copyright law. In this area, I imagine caution is best; after all, in the US, Google's copying of the 9-line "RangeCheck" function (out of how many million lines in the JRE?) has been determined not to be de minimis, though in a new trial Google will assert a fair use defense regarding that copying.

(note: some popular RSS readers do not show the color coding of insertions and deletions)

static inline bool dir_emit(struct dir_context *ctx,
                           const char *name, int namelen,
                           u64
    uint64_t ino, unsigned type)
{
       return ctx->actor(ctx->dirent, name, namelen, ctx->pos, ino, type) == 0;
}
static inline bool dir_emit_dot(struct file *file, struct dir_context *ctx)
{
       return ctx->actor(ctx->dirent, ".", 1, ctx->pos,
                         file->f_path.dentry->d_inode->i_ino, DT_DIR) == 0;
}
static inline bool dir_emit_dotdot(struct file *file, struct dir_context *ctx)
{
       return ctx->actor(ctx->dirent, "..", 2, ctx->pos,
                         parent_ino(file->f_path.dentry), DT_DIR) == 0;
}
static inline bool dir_emit_dots(struct file *file, struct dir_context *ctx)
{
       if (ctx->pos == 0) {
               if (!dir_emit_dot(file, ctx))
                       return false;
               ctx->pos = 1;
       }
       if (ctx->pos == 1) {
               if (!dir_emit_dotdot(file, ctx))
                       return false;
               ctx->pos = 2;
       }
       return true;
}

static inline int bdi_setup_and_register(struct backing_dev_info *bdi, char *name,
                           unsigned int cap)
{
        char tmp[32];
        int errerror;

        bdi->name = name;
        bdi->capabilities = cap;
        errerror = bdi_init(bdi);
        if (errerror)
                return err;(error);

        sprintf(tmp, "%.28s%s", name, "-%d");
        errerror = bdi_register(bdi, NULL, tmp, atomic_long_inc_return(&bdi_seqzfs_bdi_seq));
        if (errerror) {
                bdi_destroy(bdi);
                return err;(error);
        }

        return 0;(error);
}

[permalink]

DragonBoard™ 410c with rt-preempt failure

LinuxCNC latency test results on DragonBoard

First of all, it shipped with one terrible problem, making it useless for headless development: The onboard wireless doesn't receive multicast traffic, including ARP requests! And their kernel doesn't provide many modular drivers, so adding any old USB wireless dongle doesn't work until you rebuild the kernel. As of this writing, there is no useful workaround.

And that's when I hit the second show-stopper problem. I planned to build kernels for this thing anyway, because the goal is to run LinuxCNC on it with rt-preempt realtime. So I stuck in a large capacity micro-sd card, obtained the kernel source, built it, and rebooted with my new kernel.

(incidentally, testing kernels is a PITA. You have two choices: one, flash the one and only boot area with your new and untested kernel and hope for the best; two, tether to a PC (losing USB keyboard and mouse on the dragonboard!) and use fastboot every time you boot. ugh)

.. but the new kernel just didn't work. It scrolled kernel messages, but got stuck partway through booting. dmesg implicated the wireless card. This one at least has a workaround: cherry-pick a certain commit from their kernel git. As I understand it, this bug is triggered by using the compiler on the dragonboard, and not encountered with the linaro cross-compiler running on a traditional x86 desktop.

Anyway, having found out about this I was briefly celebratory. I booted with my own kernel and had a working USB dongle. Unfortunately, that celebration was short-lived. The USB dongle hasn't been terribly reliable either, particularly under high CPU load.

I soldiered on and patched the linaro 4.0 kernel with the 4.0-rt5 patchset. There were a few minor conflicts which I believe I resolved correctly. Unfortunately, the LinuxCNC latency-test readily encounters latencies above 15ms (15000us). LinuxCNC really would like sub-100us max latencies.

I briefly enabled tracers, and have traces that seem to implicate a variety of subsystems: networking, usb, and video *all* figure. More experienced developers than I will have to be the ones to sort out RT on 64-bit ARM.

[permalink]

Better pasting for irssi in X

read more…

Samsung ARM Chromebook: the portable I've always wanted

read more…

How do you check a signature on a PGP/Mime message

…I'll answer in Python. First, put your whole message in a file in unix "mbox" format (this will be the problem if you use some brain-dead GUI mailer!), then in Python extract the signature and the signed message, and finally invoke gpg:

import email
import os
import sys

with open(sys.argv[1]) as mfile: message = email.message_from_file(mfile)

if message.get_content_type() != "multipart/signed":
    raise SystemExit, "message not signed"

# Really, you'd want safe temporary filenames, and you'd want to clean them up
with open("msg", "wb") as msg: msg.write(message.get_payload(0).as_string())
with open("sig", "wb") as sig: sig.write(message.get_payload(1).get_payload())

# Delegate the real business to gpg
os.execvp("gpg", ["gpg", "--verify", "sig", "msg"])

[permalink]

Mosh automatic cleanup of dead sessions

Ctrl-digit to switch to tab on Linux Firefox

Got A Chromebook

pnacl vs Debian Jessie

First few weeks on Digital Ocean

Benchmarking ungeli on real data

Changing hosting providers

mailpie moves to github