Sufficiently Advanced Design
IMG_7592.jpeg

Blog

Thinking out loud

Posts tagged arduino
Witch Lights troubleshooting, this sensor is going to kill me edition

I think I figured out why this motion sensor keeps going haywire.

(TL;DR: The more lifelike trail animation causes low-frequency AC and induction on the long power harness. I think.)

Witch Lights 2's sensor2worked just fine in July 2017, with version 1 of the FastLED implementation, which brought multi-sprite automation, and the SpriteManager and SpriteVector framework to manage and create/store/delete Sprite objects in memory, respectively.

It only started malfunctioning in late June 2018, when I pulled the harness out of storage in the spare bedroom for the winter.

What changed?

This year, just the software.

The v.1 FastLED Witch Lights were completed at the last second: the last commit date on GitHub for that build was submitted after midnight on July 2nd. The priority had been fixing memory leaks: the Witch Lights run for 6-9 hours on a single battery in warm weather, and a slow memory leak could cause them to crash in hour 4.

Fortunately, Jim is an excellent developer, and did in fact build an object-oriented animation framework for the Witch Lights gratis, so I owe him a perpetual debt of gratitude. (He also walked me through the class inheritance and the way the state machine sprites were configured and such when I began work on v.2 in spring of 2018. He's good people.)

Anyway, the way we drew the sprite at the time was, we loaded a static CRGB struct into RAM, and then used memcpy to update the leds struct (also CRGB which is like an array of 3-byte arrays, each containing an R, G, and B value) with a function called strpcpy.

(We did it that way because we draw the sprites 3 pixels off-strip, and having an abstracted function that we could feed a location of -3 without accidentally overwriting the bootloader—again—helped a lot.)

Anyway, we just moved pre-rendered sprites around for the travel mode of the Sprite state machine. The idle animation of the time also was a set of pre-rendered animations loaded into RAM as series of CRGB structs that got stepped through like animation frames.

We did that because this was the first time we were going to try animating more than one sprite at a time. We knew the Arduino Due was a significant upgrade from the Duemilanove the first builds of the software was tested on. We didn't, however, have a feel for what the processor capabilities were.

We only knew that v.1 ran for hours without crashing in 1K of SRAM while running a 30-pixel test strip on the 8-bit Duemilanove, and the Due had 96K to run 600-750 pixels. So we threw RAM at the first draft implementation.

insert animation comparison video

In the first half of this (over-long, but forgive me, it was done in 30 minutes before leaving for a trip) documentation video, you can see the result.

Here's a detail.

Witch Lights test home.gif

Note that the length of the tail is exactly the same throughout its travel. It's a series of pre-rendered color values. So, even though the sprite starts off slowly and accelerates, the trail doesn't stretch to emphasize that naturalistic motion.

In version 2, I fixed that.

Turns out, the ARM chip in the Due provided us with excess processor capacity, and so I could afford to play around a bit. Instead of a long, pre-rendered set of fading pixels, each traveling sprite is now three pixels long. And when a sprite moves forwards one pixel, it calls a recursive DimTrail() method on the pixel it left shining behind itself.

DimTrail() uses the FastLED FadeToBlackBy function to fade the brightness of its pixel by a factor which scales with the current updateInterval (framerate) of the Sprite. In english, it fades a certain pixel more if the Sprite is moving slowly, and fades less the faster the Sprite moves.

And then it calls DimTrail() on the pixel behind it.

(There's logic to prevent the Sprite from eating recursion and dying, don't worry.)

As a result, the sprites now have stretchy, lifelike trails.

Witch Lights CO dark.gif

And that means that the total number of pixels we're powering changes unpredictably. And it stands to reason that the amount of current the NeoPixel strips pull from the power rail will also change unpredictably.

In english, the changing trail length creates alternating current (AC) on the power rail.

The same power rail that is powering sensor2.

The 85-foot-long power rail.

That one.

Everyone was telling me it sounded like I needed a low-pass filter on the power line for the sensor at the end of such a long power harness. And I agreed. But what was killing me was:

Why did it work in 2017, and not now? What changed?

The software changed.

I also didn't understand why sensor2 would seem to work fine for some random but short amount of time, and then malfunction. It's because, if there's nothing animating with DimTrail(), we don't have AC on the power rail. As soon as we get one or two sprites animating for a few seconds, AC along the 85-foot power lines build up, and sensor2 probably does not like that.

Several friends with more (real) electronics experience said it sounded like AC or inductance on the power pin was making the sensor behave unpredictably.

The Adafruit Discord Server suggested testing an RC filter. They also believed low-frequency AC on the power pin would definitely cause these symptoms. As did I.

I just didn't know why.

I still don't. Not yet. I have to look for the thing I've predicted will be there: AC on the power line.

My multimeter isn't the right tool to find this bug. Fortunately, a generous friend has mailed me an oscilloscope, and it arrives Wednesday.

Once I have that, I can measure the frequency of the oscillation on the power pin (assuming it exists), and design an RC filter, and fix this without installing a separate power supply for the sensor lines.

Faeries go both ways

This is a recap of all the work I did over the weekend. In theory, this stuff goes before the last journal entry. But I'm posting it now. For reasons.

DimTrail() function

Last year, we were seriously concerned about the performance of the Arduino while animating 4-20 sprites at a time, and so we pre-rendered as much as possible as a precaution. As it turns out, we're nowhere near taxing the performance of the processor: our bottleneck for performance is FastLED::Show.

So this year, instead of a pre-rendered pixel trail for the faerie sprites, I added a DimTrail() method.

When I move the pattern CRGB sprite one pixel fowards, it doesn't turn off the LED that, in the previous interval, was the rear-most pixel of the pattern. So I call DimTrail() on that pixel, and it gets faded by ~50% using a FastLED function. The DimTrail() method is recursive, so it then steps back one pixel and fades again, and so on, until it detects that it's reached a black pixel, at which point it exits.

// add int direction and make tailpixel += direction, and when direction is "backwards", offset the dimtrail by 3 pixels to prevent dimming the sprite
    void DimTrail(int tailPixel, int dimFactor, int direction) {
        if (tailPixel < 0) return;
        if (tailPixel > NUM_LEDS) return;
        if (! leds[tailPixel]) return;

        leds[tailPixel].fadeToBlackBy(dimFactor);
        tailPixel += direction;
        DimTrail(tailPixel, dimFactor, direction);
    }

The FastLED fadeToBlackBy function operates in 1/256ths, meaning that if you feed it 128, it fades 50%, if you feed it 64, it fades 25%, and so on. So I also wrote a function that maps updateInterval (the variable framerate) to dimFactor, so that now when faerie sprites move slowly, they start with a short trail, and when they accelerate to move quickly, they stretch out a long trail behind them.

That's great, but when the sprites stop for their idle animation, the trail fade doesn't look right.

So I created a FadeOutTrail() method, which itself calls DimTrail() recursively.

void FadeOutTrail(int tailPixel, int dimFactor, int direction) {
    if (tailPixel < 0) return;
    if (tailPixel > NUM_LEDS) return;
    if (! leds[tailPixel]) return;

    // Recursively run DimTrail() at tailPixel+1, so trail fades from the dim end
    DimTrail(tailPixel, dimFactor, direction);
    FadeOutTrail(tailPixel + direction, dimFactor, direction);
}

The result of this method is that, instead of staying the same length and fading out awkwardly, when the sprites stop and idle, the trails shrink from the far end inwards. It's a subtle improvement, but a nice one.

Algorithmic braking

Now that I've done some pixel animation of how I want the faerie sprites to move, I've worked out that I like to have them slow down and stop over a range of about 4-8 pixels, with the trails fading as they slow. I'm now working on slowing the sprites down by manipulating the framerate (which is how they accelerate).

I just put in some movement logic where the sprites calculate their distance from their destination pixel, and when that is within a certain percentage of the overall travel distance, it starts braking.

However, currently deceleration is handled by a single factor; how many milliseconds do you add or subtract to the updateInterval on each call of updateTravel()?

For acceleration from stop, it's simple enough to start at about 40ms, and subtract 1ms per interval as you travel. The result is a reasonably natural enough acceleration towards "cruising speed".

For making the sprite slow down to a stop, however, that doesn't really result in a pleasing deceleration curve.

After some work, I came up with this acceleration method:

int AccelerateTravel() {
    // Decide whether braking or accelerating
    if (currentDistance < totalTravelDistance * brakePercentage) {
        if (!isBraking) {
            isBraking = true;
            brakeDistance = abs(currentDistance - totalTravelDistance);
            brakePixel = currentPixel;
        }
    } else {
        isBraking = false;
    }
    // Accelerate or brake by returning positive or negative values to subtract from updateInterval
    if (! isBraking) {
        int x = abs(currentDistance - totalTravelDistance);
        return round(sqrt(x) * accelerationFactor);
    } else if (isBraking) {
        int x = abs(currentPixel - brakePixel);
        return -1 * round(sqrt(x) * brakeFactor);
    }
}

Whether braking or accelerating, the change to updateInterval is a function of the distance traveled from the point where you started accelerating. accelerationFactor and brakeFactor are semi-randomized values assigned on sprite creation, whose purpose is to make sure that, even if two sprites start at the same pixel headed in the same direction, they'll have slightly different acceleration and brake speeds, so they'll move a bit differently from each other. I'm still dialing in what those values should be.

Algorithmic idle animation

After the faerie sprite in the above video slows to a stop, it idles across 3 pixels, back and forth. I sketched this in terms of pixel values from the colorPalette array in my new notebook, and my friend Scott Longely translated it into a set of simple algorithms for each pixel.

That isn't the full desired result, though. What I need to do next is to write a colorFade() function to transition smoothly between pixel values.

I've found example code as gists on GitHub. Both use FastLED functions for the fade, which have so far resulted in good animation performance, so I'm game to see how they go.

I have so far not looked at the code at all to see how suitable it may be. So that'll happen soon.

Transition from idle to travel

There's a visual discontinuity at the moment where the animation transitions from isIdling to false; at that moment, it chooses a new destination, and draws the travel sprite. Unfortunately, the last frame of the idle animation and the travel sprite do not match perfectly.

So one of my next action items is to fix that.

Faeries go both ways

Last year's build of the Witch Lights concealed a dirty secret: sprites only had the capacity to move in one direction.

Basically, a sprite started at 0, and when in travel mode, it moved by adding 1 to CurrentPixel. When CurrentPixel was > NUM_LEDS, the sprite was marked done. (Or it started at NUM_LEDS and traveled towards 0.)

What I want, though, is for faeries to make one long travel move in the "forwards" direction (whichever way that happens to be for this particular sprite), and then make a bunch of small, random moves in either direction, basically flitting about while waiting for people to catch up along the path.

So that means that all the movement logic needed to be examined and re-written. Which I will not get into here. But basically, it happened.

So that's working, too.

Reverse the polarity of the neutron flow

The Witch Lights have two long-range motion sensors, one at each end. When someone on the "far" end triggers a sensor, we have to spawn a faerie sprite that starts at NUM_LEDS (the far end), and travels towards pixel 0.

There are lots of details involved in making sprites travel in the "other" direction; for example, when transitioning from travel mode to idle, the idle pixel animation has to run in reverse. And the logic to check whether the sprite is "done" (and can be deleted from memory) has to know the sprite's direction, and so on.

Or.

When we run an Update() method, when we take what we've animated and send it to the LEDs, what we actually do is call a function called stripcpy. That function takes the CRGB struct that we've been manipulating, looks up currentPixel, and then uses memcpy to write our pixels to the leds CRGB struct.

OK. So.

What if we told stripcpy to count from the other direction, and write the CRGB data backwards?

In theory, you'd get a perfect mirror image of the animation you're running. The sprite logic would always run from pixel 0 to NUM_LEDS, but it would draw to the pixels as though it was running in the other direction.

That simplifies the requirement for direction-checking logic all through the various methods of the sprite class, and only requires that sprites be created with a drawDirection value, which then gets passed to all stripcpy calls.

That's the theory. I haven't tried it out yet. But I'm eager to try it. If it works, that's a huge simplification, reducing the number of potential logic bugs in Travel mode.

It would very slightly complicate collision detection, but I'm pretty sure that's easily solved.

Up Next

Here's the todo list:

  • Add "flit" behavior, so that faeries make long travel moves towards NUM_LEDS, then a number of short, random moves in either direction, with short idles, and then move on with another long travel move
  • Make stripcpy run backwards
  • Add jumper-pin modes (set some pins to auto-pullup, and then jumper them to ground to activate various modes for setup and documentation)
  • Collision detection
  • Adjust accelerationFactor and brakeFactor randomization, maybe make them values passed to sprites on creation?
  • Create subclasses of the current sprite and change the idle animation
  • Fix the visual glitch when transitioning from idle to travel modes (FadeColor?)
  • Define zones of pixels where faerie sprites should not stop and idle
    • areas where the trail coils around a tree
    • zones at the very beginning and end of pixel strips
  • Use collision detection to make faeries more likely to stop and flit around near other faeries
  • Double-check memory usage and possible memory leaks caused by my messing with things I do not sufficiently understand

And other things. Secret things.

Minimum Viable Magic

Things accomplished:

Parabolic acceleration & braking curves for faerie sprites, implemented as a method

Recursive forced fade-from-dark-to-light algorithm when faeries stop and idle

New version of idle method which replaces CRGB pre-rendered pixel arrays with algorithm, one of a few experiments. This saves hugely on SRAM usage, allowing me to save that SRAM for pixel animation for special accents and looping travel animations.

There was much debugging.

I am pretty good at debugging. It’s basically how I approach the world.

Next up, what I want is for the sprites to travel long distances, idle, then do a random shorter move (like shown in the video below), idle, and repeat a few times.

It’s just adding another state bool to the class, and a bit of logic.

More concerning to me: when transitioning from idle to travel, there is a minor visual glitch, more obvious in person than in video.

It’s caused by the 3 pixels I’m manipulating suddenly changing state when UpdateIdle() calls StartTravel(). At that moment, 3 new entries from the same fixed colorArray are written to the CRGB that represent my pixel pattern.

I need a colorFade function to smoothly transition between two states. I’ve found example code. Fortunately, reading code and understanding what it does is also something I do decently.

Anyone reading this who might HAVE one already though for FastLED? Let me know please if so. 🙃

I have some secret ideas sketched in my lost notebook (🤬) that I was hoping to get to, but if I can just get this last faerie sprite checked in to GitHub I can turn my attention to other things that sorely need my attention.

Extra animations and things are luxuries, I’m going for minimum viable magic, so that my brain will relax and stop hyperfocusing on not having a viable product for the release date.

Considering making faeries more likely to stop and idle/flit around when they encounter other faeries. This requires collision detection. Have some sketches for that as well.