Shipping Swift code in workflows

[FireFingers21]

A big update is coming to my linkding Bookmarks workflow where I've rewritten all its jq code in Swift. The new code is measurably faster running as a compiled binary than running from the swift file directly. A decision I have to make is how to ship that binary. Presently, the $100/year developer cost to codesign a binary would only apply to this workflow, so that's not feasible. And with macOS Gatekeeper getting increasingly strict about non-notarized apps, I find myself hesitant to include a precompiled binary at all.

 

My two priorities are that the workflow be fast and not require the average user to jump through hoops to make it run. Running the swift code directly is noticeably slower than using jq, and macOS Sequoia, in its current state, makes opening unsigned binaries a cumbersome 5-step process. The solution I came up with is to have the user's machine compile a binary. The code runs in a script filter, shown below:

readonly exe_file="${alfred_workflow_data}/listBookmarks"

# Compile listBookmarks file
[[ -f "${exe_file}" ]] && [[ "$(date -r "main.swift" +%s)" -lt "$(date -r "${exe_file}" +%s)" ]] || (mkdir -p "${alfred_workflow_data}" && touch "${exe_file}" && xcrun swiftc -O "main.swift" -o "${exe_file}") &

# List Bookmarks w/ executable or main.swift file if unavailable
"${exe_file}" || "./main.swift"

 

This checks if a listBookmarks file exists in the workflow data folder and compares the modification dates of the original code with the binary. If the binary doesn't exist or the original code gets modified, the binary is (re)compiled. The compilation process creates the workflow data folder, makes an intermediary file with the same name as the binary, and then compiles the code, replacing the intermediary file when complete. xcrun is a blocking command, so the & at the end makes the whole line a background process. The intermediary file exists so script reruns or lack of Command Line Developer Tools aren't constantly trying to trigger recompilation.

 

For the last line, if there is no binary or only the non-executable intermediary file, the first part of the command will fail and fallback to running the swift file directly. We do this because running the original Swift is always faster than the binary compile process, so the user never has to wait for the binary to finish compiling. Once compiled though, the faster binary gets used.

 

While not as simple as having a codesigned binary, another upside with this method is that users can more easily customize the original Swift code since the workflow will automatically recompile it for them. I've seen forum comments wanting this for workflows that ship with precompiled binaries.

 

This post serves as both a learning opportunity and a sanity check, so I'd like to close with a question. Without paying $100/year for codesigning, is this a sensible way to handle workflows that run Swift code, or is there something better? I welcome any feedback or questions!

Edited July 31 by FireFingers21

[vitor]

1 hour ago, FireFingers21 said:

with macOS Gatekeeper getting increasingly strict about non-notarized apps

 

For what it’s worth, I tested in Sequoia Beta 4 and the old right-click → Open behaviour worked fine, so I’m not convinced that wasn’t a bug.

 

1 hour ago, FireFingers21 said:

not require the average user to jump through hoops to make it run.

 

I applaud the desire to reduce dependencies to zero, but remember the Dependencies Manager helps with that. I can count on half a hand the number of times I remember someone not being able to work with that, and even then it was because they completely skipped reading the note at the top of the Gallery page.

 

If the user doesn’t have the Xcode Command-Line Developer Tools installed, this behaviour to compile will pop that dialog, so it’s not completely transparent. The Dependencies Manager also does that, but it guides and verifies the steps along the way.

It’s worth noting you don’t need to use a compiled language to replace the whole of jq. JavaScript for Automation can handle JSON just fine (it is JavaScript, after all) and has an Objective-C to access macOS APIs. Furthermore, Swift is available as a language in Run Scripts, but that may not be viable for this specific use case.

In sum, for this particular case I’m not sure I’d go with a bespoke Swift tool, but I do see the sense in the approach. Compiling is fine, as the user can still inspect the original code. What would not be acceptable is shipping the unsigned binary within the workflow and then trying to bypass Gatekeeper on that (which you’re not doing, so this is purely informational).

 

See @zeitlings’s workflows too, as a few of them use a similar technique to what you’re proposing.

[zeitlings]

Yeah, that's also what I'm usually doing. Though, I usually don't mind blocking the thread for a moment to compile. Neat idea though!

 

if [[ "$(which swiftc)" =~ "not" ]]; then
    swift ./main.swift "${args[@]}" # crawl
else
    [[ -f ./binary ]] || $(swiftc -O ./main.swift -o ./binary) # compile
    ./binary "${args[@]}"
fi

 

However, I was wondering if the which command is localized, but so far I haven't heard any complaints 😄

Edited July 31 by zeitlings
typo

[vitor]

2 minutes ago, zeitlings said:

I was wondering if the which command is localized

 

Check the exit code instead:

 

if which swiftc
then
  echo 'Available'
else
  echo 'NOT available'
fi

 

[zeitlings]

Thanks, it's just that the result of the command being written to stdout will mess with Script Filters.

There's an easy fix though:

 

if which swiftc > /dev/null 2>&1; then
  # available
else
  # unavailable
fi

 

[vitor]

You can also replace > /dev/null 2>&1 with &> /dev/null. And I usually prefer hash to which: hash swiftc 2> /dev/null (no need to also redirect STDOUT in that case, though it wouldn’t hurt).

[FireFingers21]

5 hours ago, vitor said:

It’s worth noting you don’t need to use a compiled language to replace the whole of jq.

 

Being my first time using the language, rewriting in Swift was more of a personal challenge rather than finding the best way to reduce dependencies. I'll definitely keep JavaScript for Automation in mind as a future option though.

 

5 hours ago, vitor said:

If the user doesn’t have the Xcode Command-Line Developer Tools installed, this behaviour to compile will pop that dialog

 

That's good to know, I agree the Dependencies Manager is really good. The Gallery does wonders, so at this point I just have to justify this project being at least as good as jq in this case. If not, I'll continue using the Swift version myself anyway. And since all it does is remap JSON structures, maintaining both would be trivial.

 

I'll test on a clean install of macOS to see which user experience between Swift and jq seems better. As the developer, I really like the Swift implementation. But as far as publicly sharing the workflow goes, I'll see which one is better suited for a wider audience.

 

Thanks @vitor and @zeitlings for your help!