Eamonn O'Brien-Strain

I’ve been programming on Unix and Linux machines for many decades now, and one universal I’ve leaned on is the default availability of make, which has been labor-saving swiss-army knife for setting up my build and development flows, or for organizing any random commands that I execute repeatedly in a directory. It’s a great memory aid, reminding me when I come back later of how to work with the files in that directory.

Here are some examples of Makefiles I created a decade ago:

1. When I was doing Palm webOS development I wrote a Makefile that has an eclectic collection of commands that comprised my hacked-together Palm build system.
2. When I was doing Java web-server development, I wrote a blog post explaining how I used a Makefile to integrate the cool continuous-integration system of the day (Hudson) with the cool Java web framework of the day (Play Framework).

The things I like about make is that:

1. It has very little boilerplate.
2. It captures the commands, just as you type them in the command line.
3. It captures the dependencies between the commands.
4. The dependencies are simply the files generated by one command that are used by another command.
5. The files can be file patterns, so you can express how one type of file is generated from another in general terms.
6. It uses the timestamps of input and output files of the commands to determine if a command is out of date and needs to be run.
7. It has variables to factor out common strings.
8. It is ubiquitous and installed by default everywhere.

But I have been becoming increasingly dissatisfied with make. It was fine-tuned for the development environment of its day (compiling C on UNIX), and though it is remarkably versatile it still has lots of vestiges of special build-in support for that development flow. And over the decades it has accumulated a lot of complexity in how patterns and variables work.

However, I never found any replacement system that has all the positive features of make. The nearest is Bazel which is a multi-repo public version of Google’s internal monorepo build tool (which replaced an earlier make-driven build system in Google).

So I built my own tool, Bajel.

These days, most of my side projects use the npm ecosystem, so I built Bajel there. If you have npm installed, you can execute Bajel simply by doing

npx bajel

Bajel expects that the current directory contains a build file which in its most straightforward form is a JSON file. However Bajel also supports build files in other syntaxes, including YAML or TOML syntax, which make for cleaner files.

Here is an example build.yaml file, which is probably the closest of the available syntaxes to a classic Makefile:

TEST: ava test/contract_test.js

serve:
    deps:
        - dist/index.cjs
    exec: python -m SimpleHTTPServer 8888

test:
    deps:
        - test_default
        - test_contract_production
        - test_contract_development
        - test_contract_no_env

test_default:
    deps:
        - dist/index.cjs
    exec: ava

test_contract_production:
    exec: NODE_ENV=production $(TEST)

test_contract_development:
    exec: NODE_ENV=development $(TEST)

test_contract_no_env:
    exec: NODE_ENV=  $(TEST)

"perf.csv":
    deps:
        - src/node/perf.js", "src/common/optimizer.js"]
    exec: node $<

"dist/index.cjs":
    deps:
        - rollup.config.js
        - src/node/index.js
        - src/common/index.js
        - src/common/random.js
        - src/common/color.js
        - src/common/optimizer.js
        - src/common/contract.js
        - src/common/random.js
    exec: rollup --config $<

publish:
    deps:
        - dist/index.cjs
    exec: npm publish

clean:
    exec: rm -rf dist

However YAML as a syntax has fallen out of favor in some quarters, so you might prefer the following, which is semantically identical, but in the nice clean TOML format.

TEST="ava test/contract_test.js"

[serve]
deps = ["dist/index.cjs"]
exec = "python -m SimpleHTTPServer 8888"

[test]
deps = [
    "test_default",
    "test_contract_production",
    "test_contract_development",
    "test_contract_no_env",
]

[test_default]
deps = ["dist/index.cjs"]
exec = "ava"

[test_contract_production]
exec = "NODE_ENV=production $(TEST)"

[test_contract_development]
exec = "NODE_ENV=development $(TEST)"

[test_contract_no_env]
exec = "NODE_ENV=  $(TEST)"

["perf.csv"]
deps = ["src/node/perf.js", "src/common/optimizer.js"]
exec = "node $<"

["dist/index.cjs"]
deps = [
    "rollup.config.js",
    "src/node/index.js",
    "src/common/index.js",
    "src/common/random.js",
    "src/common/color.js",
    "src/common/optimizer.js",
    "src/common/contract.js",
    "src/common/random.js",
]
exec = "rollup --config $<"

[publish]
deps = ["dist/index.cjs"]
exec = "npm publish"

[clean]
exec = "rm -rf dist"

Some features to note in the above:

• TEST="ava test/contract_test.js" is setting a variable which is referred to later in the file as $(TEST). In TOML syntax all the variables have to be defined at the top of the file before anything else.
• [serve] is the first target, and is the one whose action is executed by default when you do npx bajel. You can execute any other target by adding it as an argument to the command line, for example npx bajel test.
• deps = ["dist/index.cjs"] says that serve has one dependency, the target dist/index.cjs. If that file does not exist or is older than any of its dependencies, then the action for dist/index.cjs is executed before serve is executed.
• exec = "python -m SimpleHTTPServer 8888" specifies the shell command that executes for serve. This is executed just as if you had typed python -m SimpleHTTPServer 8888 on the command line.
• exec = "rollup --config $<" contains $< which is replaced by the first dependency, in this case rollup.config.js. Also possible are $+ which is replaced by all the dependencies (blank-separated), and $@ which is replaced by the target.

Bajel implements all the features of make that I value (including % pattern patching not shown in this example), but keeps things as simple as possible.

For advanced use, there is also a JavaScript syntax, which allows the actions to be specified as JavaScript functions, making each target be like a cell in a spreadsheet. There’s also a markdown format, to allow for literate programming of build files.

For details, see the README on Github.

If you have any feedback, feel free to add a GitHub issue, or reach out to me on Twitter.

Naming note: “Bajel” is Bazel, but with a “j” for JavaScript. It is pronounced /ba-hel/ in the Spanish fashion, as it is a Spanish word for a sailing ship such the Santa Ana, which is pictured at the top of this post.

---

#tfr #fedi22

[Updated 5-May-2020 to change the examples to the new 1.0.0 API.]

I published a new npm module called smoothish that smooths out time-series data without some of the drawbacks of the usual moving-point average.

When working on the visualization of per-capita COVID-19 death rates I needed a way to smooth out the curves of some noisy and incomplete data, and I wanted the data to extend up to the most recent day.

Standard moving-average did not meet those requirements, so I ended up writing my own smoothing function which you can use like a moving average, but it does not drop the points at the beginning or (more importantly) at the end.

It works by, at every point, doing a least-squares linear interpolation of other points nearby. It is flexible enough to handle missing points or points at the boundaries.

It's easy to use. In your JavaScript project install it with:

npm install smoothish

Then using it is as simple as:

const smoothish = require('smoothish')

const daysPerMonth = [
    31, 28, undefined, 30, 31, null, 31, 31, null, 31, 30, 31]

smoothish(daysPerMonth)
// --> [ 30.0, 29.4, 29.8, 30.1, 30.5, 30.6, 30.8, 30.8, 30.8, 30.7, 30.6, 30.7 ]

Note, that not only does it produce a smoothed output, but it also interpolates missing values.

By default the function uses a radius of 2, indicating the width of the neighborhood of other points to be considered. All points are actually considered by default, but with the ones closer having more weight falling off exponentially with a time constant of the radius.

The smoothish functions also has options to do moving average and to have a step-function falloff, just like a normal moving average. In that case a radius of 2 would specify a five-point moving average.

The smoothish function always returns the same number of output values as in the input.

---

#tfr #fedi22

To make it easier to compare how COVID-19 is affecting different countries I created these population-adjusted graphs.

---

#tfr #fedi22

War is over, if you want it

War is over now

— Lennon

Have you been a belligerent in the code-formatting wars? Do you have strong opinions on tabs vs. spaces or on the merits of The One True Brace Style over other indentation styles?

Something has now changed that renders these wars obsolete.

When I joined Google one culture shock was discovering how picky Google was about seemingly trivial details of formatting. Time and again my code reviewers objected to what I thought was my creative expression of coding craft. They were unmoved by my assertion that b*b - 4*a*c was more readable than b * b - 4 * a * c or that FOO was a better name for a constant than kFoo.

Quickly I came to peace with the style guides and recognized their value. I realized that operating in Google's fabled monorepo I was reading more code than I was writing, making me appreciative of how a consistent style made the code much easier to read. Also as a code reviewer and tech lead I appreciated how just a link to a section of the quasi-legal style guides could cut short tedious bikeshedding on formatting issues and allow us to concentrate on the important issues.

Initially it was a bit tedious getting all the spacing right and remembering all the subtle line-breaking rules.

But over time it got better. First the linting tools improved and covered more of the style guide so that I generally could find and fix all the issues before sending code changes for review.

But then it got even better, with the linting tools supplemented by tools that automatically reformatted code according to the style guide for that language. Now I just basically no longer spend any brain cycles on formatting: I write arbitrary sloppily-formatted code and have it fixed up when I save the file.

The designers of the Go language were Googlers, and they bought into this philosophy. Pretty much all Go code is formatted by the gofmt program, which always reformats to the same standard style.

So when I started a personal JavaScript project I really wanted a JavaScript equivalent to gofmt which enforces a standard style.

In the JavaScript ecosystem, Eslint is the standard linter, which not only can warn against style violations, but can automatically fix them. And what's more it can go beyond simple indentation and spacing and fix more complex rules (all the ones marked with a 🔧 wrench in the list of rules).

Initially I assumed I would want to use the Google JavaScript style guide and the eslint rules that enforce it. I configured Eslint with

 {
    "extends": [
        "eslint:recommended",
        "google"
    ],
    "env": {
        "browser": true,
        "node": true
    },
    "parser": "esprima",
    "parserOptions": {
        "ecmaVersion": 6,
        "sourceType": "module"
    }
}

However, it was a bit unsatisfying to have a non-trivial config file with various things that need deciding and tweaking. It's also a bit annoying that it cannot not autofix all the problems it finds, so you still have to manually fix some of them.

Nevertheless it works fine, and produces nicely formatted code like below in standard Google format.

export const newCards = () => {
  const cards = [];
  for (let i = 0; i < 2; ++i) {
    const reversed = (i === 1);
    forEachPhrase((phrase) => {
      const responses = [];
      cards.push({phrase, reversed, responses});
    });
  }
  return cards;
};

But then I discovered standard JS. This has everything a auto style formatter should have. It is really easy to install, has good VS Code support, requires no configuration, and can fix all the problems it finds. It produces code like this:

export const newCards = () => {
  const cards = []
  for (let i = 0; i < 2; ++i) {
    const reversed = (i === 1)
    forEachPhrase((phrase) => {
      const responses = []
      cards.push({ phrase, reversed, responses })
    })
  }
  return cards
}

The most notable difference is that it omits semicolons.

Omitting semicolons is something the JavaScript language allows, but the exact rules for when you can do so have some subtle corner cases that can cause ambiguity and unexpected interpretation of code, so the conventional wisdom has long been to always include semicolons. That's the view of the Google style guide.

But the standard JS community have decided that the cleaner semicolonless look is preferred and have tackled the corner cases (which are rare) by making sure they are handled by other standard JS rules.

Personally, I don't care whether there are semicolons or not, but I'm going to stick with using standard JS for my projects as the tooling is better and it looks like it's gaining momentum in the JavaScript community.

Now of course pretty-printing has existed for a long time. The indent formatter dates from 1976, while text editors and IDEs have long been able to autoformat. But all these pretty-printers have one fatal flaw: they are too flexible, and can be configured with different style options.

The revolutionary novelty of Go's gofmt, standard JS, and Google's internal tool is that they have no options. They enforce a single standard style.

The wars are over! Everyone should just use a standard auto-formatter with no options.

---

#tfr #fedi22

So, I'll be going to Mexico in a few months and I wanted to brush up my Spanish vocabulary ...

So I built https://t.co/lNgilbwR3o with flashcards that have Spanish on one side, and images from a web search in Mexico on the other side.

Source code at: https://t.co/Phl3B2Dd1K pic.twitter.com/xNtR6WrmZz

— Eamonn O'Brien-Strain 👨‍💻🔍🌁🇮🇪🇪🇺🇺🇲🇺🇳⚛️ (@eob) November 6, 2019

There were some interesting aspects to building this web app.

1. The spaced repetition algorithm for displaying flash cards to maximise learning.
2. Using locale-specific web search to fetch the images
3. Architecting the app as a static web site that uses local storage to store user state.
4. Using vanilla JavaScript with no framework.
5. Using 3D animations for flipping cards.
6. Custom creating all the CSS styling.

In this post, I'll discuss the spaced-repetition algorithm, and defer the other aspects to future posts.

From Wikipedia

Spaced repetition is an evidence-based learning technique that is usually performed with flashcards. Newly introduced and more difficult flashcards are shown more frequently while older and less difficult flashcards are shown less frequently in order to exploit the psychological spacing effect. The use of spaced repetition has been shown to increase rate of learning.

There is a variety of published research on spaced-repetition, and different flashcard apps use a variety of different algorithms, some quite complicated.

I decided to start with something much simpler. Every time a user sees the card they say how accurately they predicted the reverse side of the card. We store a response object that has this correctness score along timestmp t of the response. For each card we calculate an average score, but weighted by time so that more recent scores count for more. Then we sort the list of cards so that the cards with the lowest scores come first.

The core of the algorithm is in common.js:

const TAO = 1000.0 * 60 * 60 * 24

export const score = (responses) => {
  if (responses.length === 0) {
    return -1
  }
  const now = Date.now()
  const weightedSum = responses
    .map((response) =>
         response.correctness * Math.exp((response.t - now) / TAO))
    .reduce((sum, x) => sum + x)
  return weightedSum / responses.length
}

The score function goes through the array of responses and sums up the correctness values weighted by an exponential decay function with a time constant TAO of 24 hours. This means, for example, that a day-old response is weighted at about 0.37 of a recent response.

This score is called from the sort function in index.js:

  const sort = () => {
    cards.sort((a, b) => score(a.responses) - score(b.responses))
  }

My hypothesis is that this simple algorithm with its single TAO parameter is as effective as the more complex algorithms. Hopefully by tuning TAO I can emulate those algorithms.

An observant reader will have noticed some significant algorithmic inefficiencies in the above naive implementation of the algorithm. In particular there are some superlinear big-O complexities that I'll probably need to mitigate as I add more cards to the app.

---

#tfr #fedi22

If you are reading this article on a phone or other touch device, did you know you can swipe left and right to get to other articles? Go on, try it now. I'll wait.

[Edit 2023-02-07: Actually you will need to try this on the old version of this blog.]

This turned out to be easy to implement thanks to Hammer.js. Here's how I did it.

1. I downloaded the Hammer.js library into my JavaScript directory:

   wget https://hammerjs.github.io/dist/hammer.min.js

and creates a new file called swipe.js in that directory. 2. I added the following to the HTML template of my article pages:

   <script src="/js/hammer.min.js" defer></script>
   <script src="/js/swipe.js" defer></script>

This loads the Hammer library and my application code into the web page. As an optimization I add the defer attribute so that the script loading does not block the initial page load: after all this is optional stuff that should not slow sown the normal functionality of the page. 3. I already had the following navigational structure in the article pages for going to the previous and next pages:

   <nav>
       <a id="prev" href="...">&laquo; ...</a>
       <a id="next" href="...">... &raquo;</a>
   </nav>

The JavaScript can pluck out the destination URLs from the href attributes of the #t and #prev elements. 4. I added the following to swipe.js:

   const mc = new Hammer(document.body)

   mc.on('swipe', (ev) => {
     const dir = (ev.deltaX < 0) ? 'next' : 'prev'
     const a = document.getElementById(dir)
     if (a) {
       window.location = a.href
     }
   })

This code initializes Hammer.js and sets up a swipe lister. When a swipe is detected the code looks at the deltaX event property to determine if this was a left or right swipe, pulls out the URL from the href of the corresponding link, and navigates to that link.

This simple solution works fine, but there is no feedback to the user to show that a swipe has happened. In the next article I'll show how to add animation to improve the UX.

---

#tfr #fedi22

In a follow-up to the last article which described Adding swipe actions to a web site, I'll show how to add simple animation to give some user feedback that the swipe has happened.

First of all, we'll modify the JavaScript to add a single line as follows:

mc.on('swipe', (ev) => {
  const dir = (ev.deltaX < 0) ? 'next' : 'prev'
  const a = document.getElementById(dir)
  if (a) {

    // Added line for animation:
    document.body.classList.add('animate-' + dir)

    window.location = a.href
  }
})

So, just before loading in the new page this code adds a CSS class animate-next or animate-prev to the body element of the page (i.e. to the entire page). This will trigger an animation that will run while waiting for the new page to load.

To actually cause the animation we add the following CSS:

  @keyframes slideLeft {
    to {
      transform: translateX(100%);
      opacity: 0;
    }
  }
  @keyframes slideRight {
    to {
      transform: translateX(-100%);
      opacity: 0;
    }
  }
  .animate-prev {
    animation: 0.2s ease-out 0s 1 slideLeft;
  }
  .animate-next {
    animation: 0.2s ease-out 0s 1 slideRight;
  }

Let's walk through this:

1. Each @keyframes includes a single to (100%) key-frame which is the state to animate to. Because there is no from (0%) key-frame the animation will default to using the current state as the starting state. It
   • translates 100% to the left or right (which for the body will translate it offscreen)
   • fades from opaque to transparent.
2. Each class selector has an animation that uses the corresponding key-frame, specifying that it should animate
   • over the given time
   • using ease-out (start fast then slow down)
   • with no delay
   • playing just once
   • the given #keyframes name

Note we are careful to only use High Performance Animations that can be efficiently implemented by a phone's GPU, namely translation, scaling, rotation, or opacity change. We could obviously use these four types of animations to further enhance the page-swiping animation to make it more effective, while still keeping it efficient.

---

#tfr #fedi22

---

I was intrigued enough by an article on hydration, Which drink is best for hydration? Hint: It isn't water, which appeared in my news feeds that I dug up the original paper, A randomized trial to assess the potential of different beverages to affect hydration status: development of a beverage hydration index, which seems to be from legit scientists, so I tweeted about it:

It turns out that water is not the best way to keep hydrated. Milk is better.

And beer or drip coffee are almost as good as water for hydration.

Good news for me as I seldom drink water, but drink lots of coffee all day, capped by a beer in the evening.https://t.co/KFaqQZtvZe pic.twitter.com/FTSd2VGgnJ

— Eamonn O'Brien-Strain 👨‍💻🔍🌁🇮🇪🇪🇺🇺🇲🇺🇳⚛️ (@eob) September 26, 2019

This seems to go against the conventional wisdom in the United States, where people feel the need to drink a lot of water to be healthy.

As someone who grew up in Ireland, I do find it strange how much value Americans put on water. In Dublin we had very high quality water from the Wicklow Mountains but we did not drink plain water that often. At family dinner we would not have water on the table, or indeed any drink. Instead we would drink copious amounts of tea after each meal and throughout the day.

Of course most of the United States gets much hotter in the Summer than Ireland, and indeed I'm suspicious that the kidney stones I developed within a few years of coming to the US were caused by me not upping my liquid intake to match the increased temperatures I was experiencing.

But even in hotter parts of Europe people don't drink as much water as Americans. Ubiquitous water is not the norm. In most restaurants you have to pay for bottled water if you want water with your meal.

But I've always been a bit dubious that there is a scientific basis for drinking quite as much water as many Americans believe is necessary to be healthy, and I've been suspicious of the assertion that coffee and beer “don't count” because of the diuretic effect of caffeine and alcohol.

The paper does seem to confirm my suspicions, and a quick Google search finds some confirmation. For example the New York Times UpShot column determined No, You Do Not Have to Drink 8 Glasses of Water a Day, which is confirmed by WebMD in The Quest for Hydration.

The “8 glasses of water” requirement actually includes all the water in food, and since the majority of most food is water, you can get a lot of your hydration needs just from eating. And if you consider water from coffee, tea, and beer you can easily get all your water needs without drinking actual water.

So, this emphasis on drinking water is a weird peculiarity of American culture, but it is a pretty harmless peculiarity. Even if the hydration benefits are greatly exaggerated, it is still good for the health if consumption of water displaces excessive consumption of alcohol or sugary drinks.

---

#tfr #fedi22

Well, it’s been about a year a year since I started this blog, and I’ve finally got fed up with my idiosyncratic static site templating framework that uses the venerable m4.

I was working on building a better templating language, inspired by the one we use in Google to power the main Google search page, but doing that was preventing me actually writing any content for the blog.

So, for now I have abandoned my non-standard approach and have converted to using the Jekyll static site generator. So now eamonn.org is generated by Jekyll, though I have kept the old version alive with links to the old versions of the pages (because Cool URIs don't change).

For now I’m just using the bare-bones default theme, but I’m thinking of working on making the a little nicer.

And now that it’s easier for me to add content, maybe I’ll get around to adding old content such as my Medium articles.

---

Image attribution Tom Preston-Werner CC BY 4.0

---

#tfr #fedi22