What is a JSON feed? Learn more

JSON Feed Viewer

Browse through the showcased feeds, or enter a feed URL below.

Now supporting RSS and Atom feeds thanks to Andrew Chilton's feed2json.org service


And now it’s all this

I just said what I said and it was wrong. Or was taken wrong.


Cutting the cord

Permalink - Posted on 2021-07-31 14:55

You may have seen Steve Mould’s recent video, called “The Spring Paradox.” If not, take a few minutes, and we’ll reconvene when you’re done.

As you might expect, I’m not interested in the traffic problem that takes up the second half of the video. But the spring problem that gives the video its name is cute, and Mould does a good job explaining how the behavior is related to the difference between springs in series and springs in parallel. But…

Surely the weight won’t rise under all conditions. If the two longer strings, the red and green ones, were much longer—or if the springs were much stiffer—the weight would drop. So what restrictions did Mould have to adhere to when he set up his demonstration?

Let’s start by drawing the two states next to one another. Before cutting is on the left and after cutting is on the right.

Springs and strings from Mould video

(The strings should be straight on the right, but I’ve drawn them curved so they don’t overlap.)

Our starting assumptions are:

  1. All the strings are inextensible.
  2. The long red and green strings are the same length, \(L_{\ell}\).
  3. The short blue string is of length \(L_s\).
  4. The springs have the same unstretched length, \(L_0\).
  5. The springs follow Hooke’s Law and have the same stiffness, \(k\).

Recall that Hooke’s Law says that the stretch in a spring, \(x\) is related to the force acting on it, \(F\), through the simple linear relationship

\[F = k x\]

Strictly speaking, we could relax the last condition and say only that the two springs must have the same force-deflection relationship. It doesn’t have to be a linear relationship, but that will make the algebra easier.1

In the “before” state, each spring is carrying a the full weight, \(W\), and is stretched a distance \(\frac{W}{k}\). Therefore, the distance between the ceiling and the weight is

\[2 \left( L_0 + \frac{W}{k} \right) + L_s\]

where \(L_s\) is the length of the short blue string.

In the “after” state, each spring is carrying half the weight, and is stretched a distance \(\frac{W}{2k}\). Therefore, the distance between the ceiling and the weight is

\[L_{\ell} + \left( L_0 + \frac{W}{2k} \right)\]

For the weight to go up when the blue string is cut, we must have

\[L_{\ell} + \left( L_0 + \frac{W}{2k} \right) < 2 \left( L_0 + \frac{W}{k} \right) + L_s\]

or, after some algebra,

\[L_{\ell} < L_0 + L_s + \frac{3}{2}\frac{W}{k}\]

This sets the upper bound on the lengths of the red and green strings if we want the weight to behave as it does in the video. If the strings are longer than this, the weight will go down when the blue string is cut.

There’s also a lower bound on \(L_{\ell}\). The problem starts with slack in the red and green strings, so

\[L_{\ell} > \left( L_0 + \frac{W}{k} \right) + L_s\]

Putting the two restrictions together, we have

\[L_0 + L_s + \frac{W}{k} < L_{\ell} < L_0 + L_s + \frac{3}{2}\frac{W}{k}\]

So to get the behavior we see in the video, the lengths of the red and green strings have to be within a range of


In the video, this is a relatively long range, because the springs are pretty stretchy. But if the strings were stiff, you’d have to be quite precise in setting their lengths. In the limit as \(k \rightarrow \infty\), which is what we’d see if we replaced the springs with inextensible strings, upward movement would be impossible—we can’t achieve both inequalities.

This, I suspect, is part of the reason the behavior seems weird to us. When we see the initial setup, our brains tend to think of the springs as being of a particular length that won’t change. We don’t expect them to shorten by more than the amount of slack in the red and green strings. Maybe if we had seen the springs at their unstretched length before the video started, we wouldn’t be as surprised to see them contract and the weight move up.

One last calculation: how much does the weight move up? Subtracting the “after” length from the “before” length gives us

\[y = \left[ 2 \left( L_0 + \frac{W}{k} \right) + L_s \right] - \left[ L_{\ell} + \left( L_0 + \frac{W}{2k} \right) \right]\]

which is

\[y = L_0 + L_s - L_{\ell} + \frac{3}{2}\frac{W}{k}\]

Solving this equation for \(L_{\ell}\) and substituting into the inequality above gives us

\[L_0 + L_s + \frac{W}{k} < L_0 + L_s + \frac{3}{2}\frac{W}{k} - y < L_0 + L_s + \frac{3}{2}\frac{W}{k}\]

After some cancellation, we get

\[-\frac{1}{2}\frac{W}{k} < -y < 0\]

which, after multiplying through by -1 and flipping the inequality signs, gives us

\[0 < y < \frac{1}{2}\frac{W}{k}\]

Is it a coincidence that the upper bound here is the same as the range of string length we calculated earlier? What do you think?

To get the most upward movement, we’d like the red and green strings to be at the short end of their range. But without much slack in the strings, the trick wouldn’t look as cool. The way to get a noticeable upward movement and significant slack in the strings is to use a stretchy spring, which is exactly what Mould does. Clever guy.

  1. In fact, as Mould points out, his springs are being stretched enough to act outside their linear range. The trick still works, but not exactly according to the calculations we do below. 

[If the formatting of equations looks odd in your feed reader, visit the original article]

Trains and tables

Permalink - Posted on 2021-07-28 02:24

For as long as I’ve had an iPhone, I’ve had a stripped-down version of the commuter train schedule between my town and Chicago. It started as a small set of web pages, then rotated through whatever text editing app was my current favorite, and finally went into Notes. Overall, the schedule has been pretty stable; I think I’ve had to change the actual data only once. But there were some pretty big changes made earlier this month, so I decided it was time to redo my copy.

Metra publishes the schedule of my line as a PDF with a set of tables: weekday, weekend, eastbound, and westbound. Here’s an image of one of the pages:

BNSF schedule PDF

To extract the data in the tables, I turned to a tool that I use fairly often in my job: Tabula. It’s built specifically to turn grids of information in a PDF into a nice CSV that you can import into all kinds of data analysis software. I didn’t need to do any analysis for the train schedules, but the conversion to CSV would make the table easy to edit into the form I wanted.

Tabula is an unusual app. It’s basically a web server that runs locally, but instead of forcing you to deal with the mess of server configuration, Tabula packages all that into a standard-looking Mac app. You double-click it, the server starts, and Safari (or whatever your default browser is) launches with a home page that asks you for a PDF to import. Very neat.

After Tabula has read the PDF, it presents it to you so you can select the table to extract.

Tabula table selection

After processing the stuff in the pink rectangle, it shows you what it found:

Tabula extracted data

If the form of the data isn’t what you expected, you can change some of the options in the left sidebar and try again. With the train schedule, which has a grid of lines between the entries, the Lattice option worked better than the default Stream method.

With the table saved as a CSV file, the editing was easy. I opened it in Numbers, deleted all the rows except those for Union Station and Naperville, deleted the trains that don’t stop in Naperville, and transposed the rows and columns to give me a table that was vertically oriented instead of horizontally.

This left me with all the parts of the schedule I wanted and none that I didn’t, but I still had a couple of tweaks to go before pasting the table into Notes. First, I wanted suffixes of a and p added to the times to distinguish between AM and PM. That would be time-consuming in Numbers but was a snap in BBEdit. I copied the table out of Numbers, pasted it into BBEdit, and used a combination of column selection and the Text‣Prefix/Suffix Lines… command to put the as and ps where they belong.

After pasting the edited data back into Numbers, I selected the rows for express trains and used ⌘B and ⌘I to make them bold and italic. That put everything in the format I wanted for pasting the table into Notes. Here’s what one of the schedules looks like on my iPhone:

Schedule in Notes

This is an example of what I call the “render unto Caesar” strategy. Although we all have a tendency to want to do every part of a task with just one tool, it’s often best to break the task up and use different tools for each part. In this case, it was Tabula for data extraction, Numbers for the big deletions, and BBEdit for the fine tuning. The keys to success in this strategy are being able to recognize which tool is best for each subtask and being able to move the results from one tool smoothly into the next.

[If the formatting of equations looks odd in your feed reader, visit the original article]

From TextExpander to Keyboard Maestro… again

Permalink - Posted on 2021-07-11 03:46

After a good bit of thinking, I canceled my TextExpander subscription today. This is not the first time I’ve left TextExpander—I dropped it when Smile first adopted a subscription payment model about five years ago and stayed away even when Smile listened to the complaints and lowered the subscription price.

Eventually, though, I returned. TextExpander was the only realistic snippet solution for iOS and iPadOS, and as I found myself writing more and more on my iPad, I couldn’t live without it. Also, I like making temporary snippets to handle common phrases—like the name of a product or a company—that appear often in my writing as I work on a particular project but will never be used after the project is finished. TextExpander has a very efficient way of adding new snippets.

Things have changed over the past few months. My M1 MacBook Air has brought me back to the Mac in a big way. I no longer write anything longer than a text or an email on my iPad, and I don’t expect that to change. So cross-platform expansion isn’t as important as it once was.1 And although Smile seems to have fixed the crashing problem I was having a month or two ago, I’m still leery of TextExpander’s reliance on a bespoke syncing service.

So I’m back to using Keyboard Maestro as my snippet expansion tool. It works well, and I didn’t have to do too much work to switch over. In a rare display of forethought, I didn’t delete my snippet macros. I had merely disabled them when I started using TextExpander again—now I just had to re-enable them.

Keyboard Maestro snippet groups

Yes, there were some snippets from TextExpander that I’d made in the past few years that needed to be moved over to Keyboard Maestro, but that didn’t take much time. Some were even improved in the translation.

And I decided to tackle the one big advantage TextExpander had over Keyboard Maestro: the ability to make a new snippet quickly. By combining AppleScript with Keyboard Maestro itself, I now have a way to make a KM snippet out of what’s on the clipboard.

For example, let’s say I’m writing a report about products made by Mxyzptlk Industries. To make a snippet for that name, I copy it to the clipboard and invoke my new Make Temporary Snippet from Clipboard macro. That brings up this window,

Make Snippet input window

where I can define the trigger (I chose “;mi”) and adjust the expansion if necessary. After clicking OK, I have a new snippet in my Snippet - Temporary group.

Mxyzptlk snippet

Here’s the macro that does it:

Make Snippet macro

The first step asks the user for the snippet information, prepopulating the expansion field with the contents of the clipboard. The second step does all the real work, running this AppleScript:

 1:  tell application "Keyboard Maestro Engine"
 2:    set trigger to getvariable "snippetTrigger"
 3:    set expansion to getvariable "snippetExpansion"
 4:  end tell
 6:  set triggerXML to "<dict>
 7:  <key>MacroTriggerType</key>
 8:  <string>TypedString</string>
 9:  <key>SimulateDeletes</key>
10:  <true/>
11:  <key>TypedString</key>
12:  <string>" & trigger & "</string>
13:  </dict>"
15:  set expansionXML to "<dict>
16:  <key>Action</key>
17:  <string>ByTyping</string>
18:  <key>MacroActionType</key>
19:  <string>InsertText</string>
20:  <key>TargetApplication</key>
21:  <dict/>
22:  <key>TargetingType</key>
23:  <string>Front</string>
24:  <key>Text</key>
25:  <string>" & expansion & "</string>
26:  </dict>"
28:  tell application "Keyboard Maestro"
29:    tell macro group "Snippet - Temporary"
30:      set m to make new macro with properties {name:expansion}
31:      tell m
32:        make new trigger with properties {xml:triggerXML}
33:        make new action with properties {xml:expansionXML}
34:      end tell
35:    end tell
36:  end tell

Lines 1–4 pull in the values of the variables set during the previous macro step. Lines 6–26 define the XML text that defines what will become the new macro’s trigger and action. Finally, Lines 28–36 create a new macro in the Snippet - Temporary group and define it according to the XML. I took the overall structure of this section of the script from the Keyboard Maestro Wiki.

How did I know the XML format? I created a test macro by hand, exported it, and opened the Test.kmmacros file in BBEdit. From there, it was easy to see the parts that defined the trigger and the action. I did a little editing to accommodate the inclusion of the trigger and expansion variables and pasted the result into the script.

Making a Keyboard Maestro macro that runs an AppleScript that creates a new Keyboard Maestro macro is a lot of fun. More important, though, is that it brings KM up to TE’s level when it comes to making new snippets. Now I’m not making any compromises in using Keyboard Maestro for text expansion.

Update 07/11/2021 8:47 AM
Unsurprisingly, there is prior art in the “quick creation of a Keyboard Maestro snippet” field. This tweet from @DasPretzels led me to two macros that do basically what mine does: this one from Tom and this one from Peter Lewis himself. They both have a significant improvement on mine, in that they start with a Copy step, which eliminates one keystroke. So I added that to the top of my macro and also added a Delete Current System Keyboard step to the end. This leaves the clipboard in the same state it was before the macro was invoked.

  1. Even when I was still writing on my iPad, TextExpander had become less useful. At some point, fill-ins just stopped working, and I had to weaken several snippets to accommodate that loss of functionality. 

[If the formatting of equations looks odd in your feed reader, visit the original article]

A stolen word count Quick Action

Permalink - Posted on 2021-07-04 14:50

I mentioned yesterday that installing NetNewsWire brought back some old RSS feeds that I had mistakenly dropped somewhere along the road from the old NetNewsWire to Google Reader to my homemade RSS reader. One of them is Erica Sadun’s blog. A lot of what she writes is too deep into Xcode and real programming for me to understand, but her more elementary stuff is at my level.1

A few months ago, she wrote about a simple, two-step word-counting Quick Action she built in Automator. It takes whatever text you have selected—in any app—and pops up a window with the word and character counts.2

Sadun Word count

The first step is this shell script:

echo `echo $1 | wc -w` words. `echo $1 | wc -c` characters.

And the second step is this AppleScript:

on run {input, parameters}
  display dialog input as string buttons {"OK"}
end run

After copying these steps outright, I decided to make a few changes. First, I noticed that the results window didn’t have a title, and the OK button wasn’t set to be the default—tapping the Return key wouldn’t dismiss it. So I made a couple of additions to the AppleScript:

on run {input, parameters}
  display dialog input as text buttons {"OK"} default button 1 with title "Word Count"
end run

As for the shell script, I felt a little nervous about passing a long stretch of text in as an argument, so I decided to change the script to this:

wc -wc | awk '{printf "%d words and %d characters", $1, $2}'

and have the selected text come in as standard input instead of as an argument. The output of wc -wc is a string with a pair of numbers separated by whitespace. Awk is perfect for handling text like this because it reads stdin automatically, splits it on whitespace, and assigns the resulting substrings to the variables $1, $2, $3, etc.

This worked well, and if I were smart I would’ve stopped there. But I thought about using this Quick Action on longer stretches of text and how it wouldn’t format the numbers with commas at the hundreds/thousands boundary. Large counts would be easier to read with commas.

As it happens, awk’s printf command inherits a formatting code from the system printf that puts commas at the appropriate places. The code, unfortunately, is %'d, and the single straight quotation mark is a pain in the ass when constructing a shell command. I’d like to be able to use this in the pipeline:

awk '{printf "%'d words and %'d characters", $1, $2}'

but that won’t work because the shell interprets all the single quotation marks as string delimiters—the ones I want to use as formatting codes never get to awk.

As is often the case in shell scripting, there’s a way around this, but it’s incredibly confusing and ugly:

awk '{printf "%'"'"'d words and %'"'"'d characters", $1, $2}'

I found this solution here, and it took me a while to figure out how it works. Basically, it’s concatenating five separate strings, which you can probably see better if I color-code them:

'{printf "%'"'"'d words and %'"'"'d characters", $1, $2}'

Two of the strings (the blue ones) are delimited by single quotes and contain a double quote. Two others (the yellow ones) are delimited by double quotes and consist entirely of a single quote. Also, it’s important that the variables $1 and $2 are in single quotes to keep the shell from interpreting them before awk gets a chance to. When all of these are put together, this is the command string that awk sees:

{printf "%'d words and %'d characters", $1, $2}


And that wasn’t the end of it. Even though this awk command worked at the command line, it didn’t work in Automator because Quick Actions don’t run in my normal command line environment.

The problem was with the locale, which isn’t set in the environment under which Quick Actions run.3 Luckily, the same web page that showed me the multiple quoting trick also showed me how to set the environment variable. Ultimately, the shell script step in my Quick Action was4

wc -wc | LC_ALL="en_US" awk '{printf "%'"'"'d words and %'"'"'d characters", $1, $2}'

Did I say “ultimately”? That was premature. The Quick Action worked fine with this shell script, but those five consecutive quotation marks bothered me. I knew I’d have trouble understanding them later (even if I had this blog post to explain them). I also knew that Python has a straightforward way to format numbers with commas. So I threw away the awk command and substituted in a longer, but easier to read, chunk of Python:

wc -wc | python3 -c 'import sys
w, c = map(int, sys.stdin.read().split())
print(f"{w:,d} words and {c:,d} characters")'

Python is a modular language and doesn’t automatically parse its input, so I needed to do some extra work on the front end. The second line reads in the standard input, splits it on whitespace, and converts the resulting strings into integers. That set up the variables w and c to be interpreted by the f-string in the print command. This is distinctly longer than the awk solution, but it’s also distinctly clearer.5

Here’s a screenshot of the Quick Action in Automator:

Word Count Quick Action

And here’s an example of its output:

Word count window

I hope Ms. Sadun forgives me for what I did to her simple automation.

  1. In this way, she’s a lot like Michael Tsai. I have to skip past many of his posts because they’re way over my head, but I stay subscribed for the ones I can follow. 

  2. For me, the value of this Quick Action isn’t for counting words I’m writing; BBEdit will tell me that in the status bar at the bottom of the window. This is for counting words in other people’s writing on (mainly) web pages. 

  3. You may have run into similar problems in which the Quick Action environment doesn’t set the PATH to what you expect. 

  4. Strictly speaking, setting LC_ALL is overkill. Just setting LC_NUMERIC to “en_US” would be sufficient to get the comma separators working. 

  5. Note that this script works only in Python 3. Apple has recently (starting with Catalina?) supplied Python 3 in addition to Python 2, but you have to install the Command Line Developer Tools to get it. 

[If the formatting of equations looks odd in your feed reader, visit the original article]

Back to NetNewsWire

Permalink - Posted on 2021-07-04 03:56

Ever since NetNewsWire came back from the dead, I’ve been thinking I’d eventually switch to it. I’ve been happy with my homemade RSS reading system, but it’s generally better to be in the hands of an expert, and you’d be hard-pressed to find someone more expert at RSS parsing than Brent Simmons. This morning, I switched over. It’s possible I’ll find something in NNW that’ll make me switch back, but it’s looking good so far.

I have a few unbreakable rules for an RSS reader:

  1. It must work on all my devices.
  2. It must sync between all my devices.
  3. It cannot force me to use a third-party RSS service to do the syncing.

Initially, the rejuvenated NNW broke all my rules; until recently, it broke Rule 3. But with the addition of iCloud syncing, it ticks all the boxes.

By the way, Rule 3 is not entirely due to my being a cheap bastard. I just don’t trust RSS services to stay in business. I keep thinking of Gabe Weatherhead’s wise assessment of the Google Reader apocalypse:

Bigger revelation: Google built a service that you configure with all your interests and biases. They couldn’t make it profitable.
macdrifter (@macdrifter) Mar 13 2013 7:45 PM

This is not a review of NetNewsWire. If you’re a Mac user interested in RSS readers, you’ve read the reviews and know how fast and “Mac-assed” it is. You’re probably already using it.

But I do want to tell you how I got the list of feeds from my homemade system into NNW. First, the list is built directly into the Python code:

jsonsubscriptions = [

xmlsubscriptions = [
    [and so on…]

As you can see, it’s really two lists, one for JSON feeds and one for regular RSS/Atom/XML feeds. I copied this code into a new BBEdit document, and used its Text▸Prefix/Suffix Lines… command to strip off the quotation marks and commas. Then a couple more deletions got me down to a simple list of feed URLs:

[and so on…]

Now it was time to convert this list to OPML format so NNW could import it. To see what kind of OPML NetNewsWire was expecting, I added two feeds “by hand” and exported them. The result was

<?xml version="1.0" encoding="UTF-8"?>
<!-- OPML generated by NetNewsWire -->
<opml version="1.1">
    <outline text="And Now It's All This" title="And Now It's All This" description="" type="rss" version="RSS" htmlUrl="https://leancrew.com/all-this/" xmlUrl="http://leancrew.com/all-this/feed.json"/>
    <outline text="Daring Fireball" title="Daring Fireball" description="" type="rss" version="RSS" htmlUrl="https://daringfireball.net/" xmlUrl="https://daringfireball.net/feeds/json"/>

I copied the top several lines and bottom two lines and pasted them directly into my new file (I did change the <title>). I guessed that I could get away with leaving out all the <outline> attributes except xmlUrl, so another go-round with Text▸Prefix/Suffix Lines…, this time Inserting rather than Removing, left me with

<?xml version="1.0" encoding="UTF-8"?>
<opml version="1.1">
    <outline xmlUrl="http://leancrew.com/all-this/feed.json"/>
    <outline xmlUrl="https://daringfireball.net/feeds/json"/>
    <outline xmlUrl="https://sixcolors.com/feed.json"/>
    <outline xmlUrl="https://www.robjwells.com/feed.json"/>
    <outline xmlUrl="http://inessential.com/feed.json"/>
    <outline xmlUrl="https://macstories.net/feed/json"/>
    <outline xmlUrl="http://automationorchard.com/resources.json"/>
    <outline xmlUrl="https://furbo.org/feed/json"/>
    <outline xmlUrl="http://feedpress.me/512pixels"/>
    <outline xmlUrl="http://alicublog.blogspot.com/feeds/posts/default"/>
    <outline xmlUrl="http://bitsplitting.org/feed/"/>
    <outline xmlUrl="https://kieranhealy.org/blog/index.xml"/>
    <outline xmlUrl="http://brett.trpstra.net/brettterpstra"/>
    <outline xmlUrl="http://www.libertypages.com/clarktech/?feed=rss2"/>
    <outline xmlUrl="https://david-smith.org/atom.xml"/>
    <outline xmlUrl="http://stratechery.com/feed/"/>
    <outline xmlUrl="http://feeds.feedburner.com/IgnoreTheCode"/>
    <outline xmlUrl="http://indiestack.com/feed/"/>
    <outline xmlUrl="http://feeds.feedburner.com/theendeavour"/>
    <outline xmlUrl="http://www.kungfugrippe.com/rss"/>
    <outline xmlUrl="http://www.caseyliss.com/rss"/>
    <outline xmlUrl="http://www.macdrifter.com/feeds/all.atom.xml"/>
    <outline xmlUrl="http://macsparky.com/blog?format=rss"/>
    <outline xmlUrl="http://www.marco.org/rss"/>
    <outline xmlUrl="http://merrillmarkoe.com/feed"/>
    <outline xmlUrl="http://mjtsai.com/blog/feed/"/>
    [and so on…]

My guess was right. NetNewsWire accepted my hastily made OPML file without complaint and figured out all the titles and other details on its own. I put in ten minutes of work at the most and had a fully functioning blog reader with over 40 subscriptions. A testament to BBEdit’s tooling and NetNewsWire’s adherence to Postel’s Law.

One last pleasant surprise: For reasons I don’t fully understand, when I first opened NetNewsWire, it populated the “On My Mac” account with several blogs I used to read. Presumably these subscriptions were saved in some preferences file in my Library folder, but I was doing this on my new MacBook Air, and I didn’t use Migration Assistant when setting it up. A similar thing happened in the “On My iPhone” and “On My iPad” accounts when I installed NNW on those devices.

Anyway, before deleting those subscriptions (remember, I want all of them synced through my iCloud account), I noticed a few blogs that I’d somehow lost track of over the years and which hadn’t made it into my homemade system. It was nice to see that they’re active, and I added them to my list of iCloud subscriptions.

Update Jul 4, 2021 11:16 PM
In a tweet, Brent Simmons told me that NetNewsWire populates the “On My Device” account with 16 subscriptions upon installation—and it’s the same 16 subscriptions for everyone. It just so happens that I have at some point subscribed to all 16 of them.

[If the formatting of equations looks odd in your feed reader, visit the original article]

Hey, I sped up Apple Mail Rules

Permalink - Posted on 2021-06-27 20:17

When the Hey email software/service came out last summer, I had two instant negative reactions:

  1. A proprietary layer on top of an open system? No, thank you.
  2. The headline feature is screening email? Like I did with procmail back in the 90s? For $99? No, thank you.

Since then, a lot has been written about Hey.1 It’s certainly more than just message screening, but very little about it is compelling to me. I’m lucky in that email has never been a big burden to me.

But it did get me thinking about procmail and Rules, which is the Apple Mail equivalent. Making new rules and extending old ones is easy, but it isn’t fast. Too much clicking and selecting and typing.

A particular type of spam inspired me to figure out an easier way to use Rules. It’s mail that starts like this:

Hi, Dr. Drang!

I just read your great article on [something I mentioned in passing in a post ten years ago]. We here at [whatever.com] are experts on [topic I don’t care about] and would love to write new articles for your site. Please get in touch…

What bothers me about these is not just the obvious insincerity, it’s the persistence of these bastards. Because if I don’t answer (which, of course, I never do), they’re back a week or two later, “checking in” to see what I think. And a week or two after that, they’re back again because maybe I’m not the right person for them to have contacted, and maybe I could pass their email on to someone else in the massive Leancrew empire.

For some reason, marking the first message as spam doesn’t prevent the later ones from appearing in my inbox. The only guaranteed way to block the followup emails from these pests is to make a Mail Rule for them. Like this:

Apple Mail Rules tab

Persistent Spam rule

But this is where my impatience and the fiddliness of Rules got in the way. I wanted to rid myself of this kind of email, but the click-click-click of adding a new sender to a rule like this took too long. The one thing that was definitely appealing about Hey was the speed with which you could block senders.

So I started thinking about how to automate Mail to simulate the ease of blocking senders with Hey. Because the various steps of adding a rule condition were in my head, I built a Keyboard Maestro macro that simulated all those steps:

Persistent Spam Keyboard Maestro macro

I’ve been using this macro for several months now, and it’s been quite reliable, but as I was writing this post, I began to have doubts. Could this be done better in AppleScript? Yes, and it was really easy:

 1:  tell application "Mail"
 2:    -- Start by getting the sender's address and the message's account
 3:    set selectedMsgs to selected messages of first message viewer
 4:    set thisMsg to first item of selectedMsgs
 5:    set acct to account of mailbox of thisMsg
 6:    set spamAddr to extract address from sender of thisMsg
 7:    get acct
 9:    -- Add that address to a new condition of the rule
10:    set psRule to rule "Persistent Spam"
11:    tell psRule
12:      set newCondition to make new rule condition at beginning of rule conditions
13:      tell newCondition
14:        set rule type to from header
15:        set qualifier to equal to value
16:        set expression to spamAddr
17:      end tell
18:    end tell
20:    -- Delete the message
21:    set mailbox of thisMsg to mailbox "Trash" of acct
23:  end tell

The most complicated part of this script is the center section in Lines 9–18. But even that’s pretty simple. It adds a new condition to the Persistent Spam rule and then sets the parameters for that rule. Boom.

I disabled the old Keyboard Maestro macro and created a new one: a single step that runs this AppleScript. Unlike the old macro, it runs instantaneously—no windows appearing and disappearing, no pauses.

I’m not sorry about the time I spent making the complicated Keyboard Maestro macro. It got me thinking about the logic of building and modifying a Mail Rule, and I probably wouldn’t have found the AppleScript commands for rules as easy to understand as I did if I hadn’t had that experience. I am embarrassed for falling into the most common automation trap: having the computer mimic what a person would do instead of working directly on the problem. But it’s not the first time I’ve made that mistake, and I’m sure it won’t be the last.

Regardless of how I got here, I now have a fast and simple way to do the one aspect of Hey I envied its users for.

  1. Some of it was even about Hey itself, instead of its developers

[If the formatting of equations looks odd in your feed reader, visit the original article]

Sometimes laziness isn’t a virtue

Permalink - Posted on 2021-06-26 14:42

A couple of weeks ago, I ordered a ceiling fan from Home Depot. This morning, I got a text message:

Home Depot text message

The “part of your order” language confused me, as I had ordered only one fan. Had I mistakenly set the quantity to two or more? That would mean some hassle to cancel the order or return the extras. So I went to Home Depot’s site to look up the order and found that the fan comes in four packages—there was no mistake with the order.1

Home Depot order summary

But the order summary also said that all four packages are ready for pickup, so why did the text message say “part”? I have a theory.

When the programmers who built this notification system for Home Depot realized there could be a handful of different pickup scenarios, they had to choose between two responses:

  1. Work out the logic of the various scenarios and program the system to send a clear message for each one.
  2. Bang out one or maybe two messages that imperfectly cover a range of pickup conditions and be done with it. After all, who cares?

Home Depot obviously didn’t care because it’s happily sending out puzzling messages to its customers. Customers might care, but we’ve had so much experience with poorly written instructions and confusing websites that we’re mostly numb to them. And after all, complaining never gets anywhere, does it?

Update Jun 26, 2021 4:35 PM
I went to the store this afternoon to pick up the order. It came in two packages.

  1. Why Hunter ships a simple ceiling fan in four boxes is a question for another time. 

[If the formatting of equations looks odd in your feed reader, visit the original article]

How Shortcuts for the Mac will fit in

Permalink - Posted on 2021-06-22 11:33

Two years ago, shortly before WWDC 2019, Guilherme Rambo told us that Shortcuts was coming to the Mac as a Marzipan app.1 I wrote a post about the levels of automation on the Mac and how a Marzipan Shortcuts would add a new level. I was concerned with whether Shortcuts would carry on the tradition of allowing communication between the Mac’s different levels. Since Shortcuts didn’t come with Catalina—or with Big Sur, for that matter—I haven’t had to revisit that post. But with WWDC 2021 and the official announcement of Shortcuts coming to Monterey, it time for an update.

Here’s how I characterized the Mac and its automation before Shortcuts:

Current Mac automation

The modern Mac OS—whether we call it OS X or macOS—sits on top of Darwin, a BSD-flavored Unix. The Darwin level can be automated through all the typical Unix scripting languages, and the Mac level can be automated through AppleScript or Automator. Significantly, though, Apple included ways for the levels to talk to one another:

  • The do shell script construct was added to AppleScript to allow automation at the Darwin level to be run within AppleScripts.
  • Automator included the Run Shell Script action to do the same thing there.2
  • The osascript and automator commands were added to /usr/bin to allow shell scripts to run AppleScripts and Automator workflows.

Had Shortcuts come to the Mac as a Catalyst app, I figured that would mean it would be restricted to automating other Catalyst apps, and there’d be a need for another set of automation tools to communicate between levels—something that Rambo’s article said nothing about.

Future Mac automation

But two things happened to make this graphic wrong:

  1. Shortcuts isn’t a Catalyst app and isn’t restricted to automating other Catalyst apps. So it sits nicely in the Macintosh level, just like AppleScript and Automator.
  2. Apple filled in communication gaps.

Over the past couple of weeks, I’ve read the posts and listened to the podcasts of those foolhardy brave souls who’ve installed the first developer beta of Monterey on their Macs and have tried out Shortcuts.3 I’ve also watched the “Meet Shortcuts for macOS” session video. Here’s how I think the state of Mac automation on Monterey can be characterized:4

Monterey automation levels

I’ve grayed out Automator because it’s clearly not long for the world. It hasn’t been officially deprecated, but I wouldn’t be surprised to get that announcement next year.

As you can see, Shortcuts has a Run Shell Script action, just like Automator’s. You can see it in the sidebar of this screenshot from the “Meet Shortcuts” session:

Shortcuts Scripting Actions

Going in the opposite direction, there’s now a shortcuts command that can run shortcuts from Terminal or a shell script:

Shortcuts command line

In writing the post two years ago, I forgot to include Automator’s ability to run AppleScripts with the Run AppleScript action. Shortcuts has an action with the same name, and there’s also a way to run Shortcuts from within AppleScript:

Shortcuts scripting interface

It looks like “Shortcuts Events” is like “System Events” and “Image Events”—an application that exists purely for its AppleScript dictionary. A word of warning: Jason Snell tells me that “Shortcuts Events” isn’t available in the current developer beta. But when it is, we’ll have ways to communicate in both directions between Shortcuts and AppleScript within the Macintosh level.

Shortcuts and AppleScript

Update Jun 29, 2021 3:25 PM
The second developer beta of Monterey added Shortcuts Events and the run the shortcut command.

All in all, this looks like everything I wanted in Mac Shortcuts. As I said in the post two years ago, the ability to run every kind of automation from every other kind of automation is key to making a fluid system, where you can use each tool for what it does best. Also, it means that third-party automation tools like Keyboard Maestro, which has a good AppleScript dictionary for running its macros, will fit in well with the new environment even before they incorporate Intents that are directly accessible from Shortcuts.

There is one oddity left unresolved. The emphasis Apple put on being able to run shell scripts from Shortcuts and vice versa seems to be at odds with its declaration with the release of Catalina that built-in scripting languages are being deprecated:

Scripting language runtimes such as Python, Ruby, and Perl are included in macOS for compatibility with legacy software. Future versions of macOS won’t include scripting language runtimes by default, and might require you to install additional packages.

I’m certainly not worried about this on my behalf—I know I’ll be able to install whatever languages I need. But people who are uneasy about using the command line but would like to use Shortcuts written by others may be left in the lurch. Maybe Apple will bundle scripting languages with Mac Shortcuts so the language runtimes will be there when you need them. Or maybe that deprecation will be deprecated.

  1. You remember Marzipan, don’t you? That’s what we called Catalyst before it was officially announced. 

  2. Sal Soghoian discusses how important this was to the Automator developers in this episode of the Mac Power Users

  3. If I had a spare Mac that would run Monterey, I’d’ve done it, too, despite my general avoidance of betas. Unfortunately, my spare Mac is a 2012 iMac, and it’s too old for Monterey

  4. I’ve added JavaScript for Automation (JXA) as an alternate to AppleScript because they’re based on the same technology. I forgot to include JXA in my previous post because I don’t think of it as a separate thing. Frankly, Apple has done so little with it, I generally don’t think of it at all. 

[If the formatting of equations looks odd in your feed reader, visit the original article]

Flowing Markdown reference links

Permalink - Posted on 2021-05-29 17:17

It’s not exactly Emacs vs. vi, but the two styles of linking in Markdown, reference and inline, have their adherents, and each makes their case. In the reference corner, John Gruber:

… I personally much prefer the reference style… the reference style is just so much more readable.

And in the inline corner, Jason Snell:

But when I’m writing, pasting in URLs as I’m typing not only matches the way it’s done in HTML, it also allows me to keep my writing flow as much as possible. I type the text, I paste the link, and I keep writing. Naming a link, moving down, adding a link line, and then returning up to where I was—it breaks the flow.

Both are correct. Reference links are more readable1, but they take more time to write—at least if you’re writing in a dumb editor. When I make a Markdown link in an online forum, like the Automators forum, I always use inline links because I’m writing in an HTML text field, which is the dumbest of text editors.

But when I’m writing a blog post, I’m doing it in BBEdit, which has an AppleScript dictionary that can handle all of the cursor movement and pasting that Jason (rightly) dislikes. I don’t waste time or lose my train of thought in a lot of mousing around.

In the typical case, the web page I want to link to is the front page in Safari. To link some text to that page, I select the anchor text

BBEdit with anchor text selected

and then type ⌃L to run my Safari Front Link script. It gets the URL of the front page in Safari, adds it to the bottom of the document with an incremented reference number, and surrounds the anchor text with brackets and the reference. The cursor is placed after the reference.

BBEdit with reference link added

If there’s no text selected, Safari Front Link does pretty much the same thing, but the anchor text is empty and the cursor is placed between the brackets for me to type it in.

Two scripts do the work. The first is a Python script called nextreflink, which reads its input and returns the number of the next reference link. In the example above, because 5 was the last reference link number when the script was called, nextreflink returned 6. Here’s the source code:

 1:  #!/usr/bin/python
 3:  import sys
 4:  import re
 6:  text = sys.stdin.read()
 7:  reflinks = re.findall(r'^\[(\d+)\]: ', text, re.MULTILINE)
 8:  if len(reflinks) == 0:
 9:    print 1
10:  else:
11:    print max(int(x) for x in reflinks) + 1

You can tell I’ve been using this for a while because it’s written in Python 2.

There’s not much to this script. It scans the text it’s given, finds all the bracketed numbers followed by a colon and a space, determines the maximum of those numbers, and prints out one more than that number. If there are no reference links in the text, it prints 1.

The nextreflink script is used by the Safari Front Link AppleScript, which I have bound to the ⌃L key combination. It does all the tedious cursor movement and text insertion.

 1:  -- Get the path to the Resources directory in the package.
 2:  tell application "Finder" to set cPath to container of container of (path to me) as text
 3:  set rPath to (quoted form of POSIX path of cPath) & "../Resources/"
 5:  tell application "System Events"
 6:    set pNames to name of every process
 7:    if "Safari" is in pNames then
 8:      try
 9:        tell application "Safari" to set myURL to the URL of the front document
10:      on error
11:        do shell script "afplay /System/Library/Sounds/Funk.aiff"
12:        return
13:      end try
14:    else
15:      do shell script "afplay /System/Library/Sounds/Funk.aiff"
16:      return
17:    end if
18:  end tell
20:  tell application "BBEdit"
21:    set hereDoc to linefeed & contents of front document & linefeed & "XXXXXXXXXX"
22:    set myRef to do shell script rPath & "nextreflink <<XXXXXXXXXX" & hereDoc
24:    if length of selection is 0 then
25:      -- Add link with empty text and set the cursor between the brackets.
26:      set curPt to characterOffset of selection
27:      select insertion point before character curPt of front document
28:      set selection to "[][" & myRef & "]"
29:      select insertion point after character curPt of front document
31:    else
32:      -- Turn selected text into link and put cursor after the reference.
33:      add prefix and suffix of selection prefix "[" suffix "]" & "[" & myRef & "]"
34:      select insertion point after last character of selection
35:    end if
37:    -- Add the reference at the bottom of the document and reset cursor.
38:    set savePt to selection
39:    select insertion point after last character of front document
40:    set selection to "[" & myRef & "]: " & myURL & return
41:    select savePt
42:    activate
43:  end tell

Lines 2–3 are a little tricky. I have both of these scripts saved in a BBEdit package (which is basically just a directory), and these two lines figure out the subdirectory where nextreflink is saved and put that path into the rPath variable. rPath gets used later when we call nextreflink. If you want to use this system, you’ll have to figure out your own way to set rPath.

The next chunk of code gets the URL of the front page in Safari. That’s Line 9. The rest of it is just error handling code. A warning sounds if Safari isn’t running or if it’s running but has no open window.

The rest of the code does all the cursor movement and text insertion. First, Lines 21–22 send the text of the current BBEdit document to nextreflink and save the number returned into the myRef variable.2 If there’s no text selected in the document, Lines 26–29 insert a blank pair of brackets followed by the reference number in brackets at the cursor location. If there is text selected, Lines 33–34 surround it with brackets and put the bracketed reference number after it. Lines 39–40 add the reference number and URL to the bottom of the document, and Line 41 moves the cursor back up to where we were writing.

I’ve had similar systems set up for adding reference links in every editor I’ve used for writing blog posts. If an editor isn’t programmable enough to do this, I won’t use it for blogging.

There is another option for getting the ease of writing with inline links while ending up with the readability of reference links: Brett Terpstra’s Markdown Service Tools include a script that converts inline links to reference links. I don’t use Brett’s tools because I’ve built up my own over the years, but they get good reviews from those who do.

One last thing: Jason refers to inline links as the “lazy” style. I think my way of making reference links is just as lazy.

  1. Jason argues that inline links are easier to edit because the URL is right next to the anchor. I would argue that link-checking in the raw Markdown source is holding it wrong. It’s much easier to check links by clicking them after the Markdown has been processed and opened in a web browser. 

  2. The text is passed to nextreflink through a shell construct called a here document

[If the formatting of equations looks odd in your feed reader, visit the original article]

Siri and numbers

Permalink - Posted on 2021-05-15 15:11

Several years ago, I wrote a post about using Siri to dictate measurements as I take them. I still do that, and it still works very well because Siri is fast and accurate at transcribing numbers. By using Siri as a sort of secretary, I don’t have to switch between my measuring tools and a notebook, and I don’t have to worry about mistyping a number on my phone’s keyboard. Over the past year, I’ve learned a new Siri trick with numbers: addition.

In my job, I often need to go through a set of drawings of a building or a machine, determine how many times a certain feature or part is used on each drawing, and then add them up. This obviously isn’t hard work, but it’s dull, and it’s easy to make mistakes when work is dull. But Siri doesn’t care if the work is dull, and because of the way it answers questions, it provides an automatic error-checking system.

Let’s say I have a list of numbers to add—all the Grade 5 bolts used in a machine housing, for example. I say, “Hey, Siri, what’s seven plus twelve plus fourteen plus four plus…” and wait for the answer. Siri responds, “Seven plus twelve plus fourteen plus four plus… is 73.” I not only get the result without a math error, I also get an echo of all the individual numbers, so I can make sure that Siri understood them all1 and that I didn’t skip or misread any of them. It’s faster and more accurate than I can do it with a calculator or spreadsheet.

There are limits. I’ve found that Siri tunes out and just won’t answer if the list of numbers is too long. Siri can handle lists of over fifteen numbers, but I generally don’t try more than about ten at a time. It’s easy enough to break a long list into sublists that Siri can then add later. Also, dictating out a long list of “A plus B plus C plus D plus…” is tiring and prone to error on my part.

(I do feel a twinge of embarrassment at using voice recognition and network access just to add a column of figures. That’s a lot of computing resources to throw at a such an elementary problem. But it’s a better way of adding.)

As you might expect, Siri can be weirdly persnickety in how you phrase the request. For example, “Hey, Siri, add one and two and three” will get the response “One plus two is three.” Not helpful. But if you ask, “Hey, Siri, what is the sum of one and two and three,” it will tell you what you want.2

I came across this way of getting sums in the past year because of the pandemic. I’ve been working at home, where there’s a HomePod near my desk, and one day it dawned on me that if it could understand my telling it to play “Sketches of Spain,” it could handle simple addition. I doubt I would have ever learned this at the office because

  1. I don’t use Siri on my Mac.
  2. If I pick up my phone to do a calculation, I’m going to open PCalc out of habit.

It was only because I’m used to talking to my HomePod that I thought of doing the sums this way. As things head back to normal and I spend more time at the office, I’ll try to take this lesson with me and do more by talking to my phone and watch.

  1. I’ve sometimes garbled my words badly when doing this, but I’ve never caught Siri in a transcription error. I think the word “plus” makes the context clear, and every number is interpreted as such. 

  2. I prefer saying “and” repeatedly to saying “plus” repeatedly, but I usually forget the proper preamble (I just had to test it to write this paragraph) so I typically go with the “plus” formulation. 

[If the formatting of equations looks odd in your feed reader, visit the original article]


Permalink - Posted on 2021-05-11 14:16

I suspect a lot of us who bought M1 MacBook Airs are thinking about the next generation low-end MacBooks (whether Apple calls them Airs or not) and wondering if maybe we should have waited a year for a step up in Apple Silicon power. That’s certainly the common regret in computer buying—that if we’d just held off a little longer, we’d have gotten something much better. But recently I’ve been thinking that regret may work the other way this time.

These are the things I’ve been putting together:

  • The current MacBook Air is an M1 system put into a shell made for larger and more power-hungry Intel guts.
  • As a result, I get actual all-day battery life from my Air. Not Apple’s idea of all day, but a full 16 hours with no concern about having to plug in. To me, this has been even more pleasing than the speed increase.
  • Apple’s tendency to go thinner and lighter—along with it’s traditional definition of “all day”—suggests that the next generation of low-end MacBooks will have less battery than today’s Air. Shaving three-quarters of a pound or more will feel good in the backpack, but when it’s late in the day, you’ll have to rummage around in that backpack for the charger.

Undoubtedly, there are people who will happily trade five of six hours of battery life for a significantly lighter machine. And there may come a time when I’ll be one of them. But not now. Because of Apple’s weird interim Air design, I may have stumbled into the ideal computer for me.

[If the formatting of equations looks odd in your feed reader, visit the original article]

What’s the diff?

Permalink - Posted on 2021-04-22 21:12

This morning’s post by John D. Cook brought up an interesting problem: what’s a good way to check for differences between files when the files consist of just a few (or maybe just one) very long lines?

The problem is that diff, the standard Unix tool for finding differences, tells you which lines are different. This is great for source code files, which are written by human beings and in which the lines tend to be short, but not so much for machine-generated files like JSON, XML, and some HTML files, in which line breaks may be few and far between.

Cook’s solution for comparing long-lined files is to pass them through fold before sending them to diff:

diff <(fold -s -w 20 temp1.txt) <(fold -s -w 20 temp2.txt)

The -s option tells fold to break at space characters instead of in the middle of a word. The -w 20 option tells it to make the new lines no more than 20 characters long. Breaking the text into lines of only 20 characters is overkill, but it certainly is easy to see differences between lines when you have only 20 characters to scan through.

The <() thing is a bit of clever shell scripting known as process substitution. It’s used instead of piping when you have more than one input that needs to be fed to a command.

I was unfamiliar with fold until today. Whenever I’ve needed to reformat a text file to a given line length, I’ve used fmt. What I like about fmt is that it defaults to breaking lines at spaces—no need for the equivalent of the -s option. So I’d do

diff <(fmt temp1.txt) <(fmt temp2.txt)

if I were OK with fmt’s default line length of 75 characters, or

diff <(fmt -20 temp1.txt) <(fmt -20 temp2.txt)

if I wanted to use Cook’s extremely short lines.

But I’d be even more likely to open both files in BBEdit and use its Search▸Find Differences▸Compare Two Front Windows command. That would give me this nice two-pane output:

BBEdit differences

In this example, there’s only one line, but it’s broken into easily digestible pieces by BBEdit’s soft wrapping, and the exact spot at which the difference occurs is highlighted by the darker purple background.1

There is a diffing app called Kaleidoscope that I’ve heard good things about, but I’ve never felt hampered by BBEdit.

  1. In his example text, Cook punned by changing Melville’s “hypos” to “typoes.” Not to be outpunned, I changed it to “typees.” 

[If the formatting of equations looks odd in your feed reader, visit the original article]