🏷 [cli golang utils]
🏷 [cli golang utils]
Cover photo - contextualelectronics.com
When I think of tooling, I think of my dad’s electronics shop, a big sprawling room, filled with random parts, power supplies and oscilloscopes. I used to spend my Saturdays with him as a kid. He would “put me to work” and I would help with small tasks, unsolder some old burnt-out capacitors, or log data on faulty power supplies. Nothing critical, but it was neat getting to learn and play at the same time. Anytime there was a big project around the house, my dad would rope me in. Fix a car, build a shed, my dad always had the right tool for the right job, form, fit and function.
… he would say. This was in stark contrast to an uncle I had, who would always go out and buy the absolute “best” and most expensive version of what my dad owned. Why buy a 14-inch chainsaw, when you can buy a 20-inch? I later learned that less is more, but god knows sometimes overkill is just fun. But here’s the thing, should work be “fun”? Does having that shiny new toy, make the process more enjoyable, or are you just showing off to the next person? It’s ok to want nice things, but how much, is too much?
The friends and family with “the knack” tend to like their cars and trucks set up the way they like them, and desks the way they like them. Keyboards of all shapes and sizes, to suit each person. Perpetual tinkers, the lot of us.
As developers, we sit on the shoulders of GIANTS. As a result of this, it’s easy to get bogged down in the latest JS framework, or Python library. When all you have to do is pip install antigravity
or npm
, it’s easy to throw in everything and the kitchen sink at projects. Exploration is healthy, pushing boundaries, and learning new things. I (at times) have been very fond of chasing after the latest and greatest libraries when working on hobby projects. But there comes a point where how much, is too much. Do you NEED that extension, is that plugin going to HELP you? It’s easier to not think TOO hard about it and sink into the mindset of “Just another library, it’s less code I have to write…” or “This plugin seems helpful* for X”.
Projects like CookieCutter are great for setting up template-based repositories, for consultants in particular. A nice repeatable structure, that follows a known pattern, can save time and energy. There is a catch, as with all good things. A lot of smaller projects I’ve seen on GitHub dump heaps of neat toys, on otherwise simple codebases. Do I need (…checks notes…) twen… TWENTY-FOUR (I’m looking at you hypermodern-python) .. test modules for writing a library to turn a light switch on and off? The answer is no, not for anything this simple. For larger, more complex projects, maybe. For example, I use quite a few dev helpers on my personal projects, not because they are essential, but because they automate annoying tasks and let me think about my app, and not about structure (Black & Flake8) or security (Bandit, Tartufo & Whispers). (Shh, it’s 14 🙃)
I’ve lost count of how many plugins I have installed in VSCode, I’ll admit. Each one was curated over many years of finding things that suit my taste. The same goes with my custom-tailored PyCharm, a bit less so, due to the sheer genius of JetBrains’s developers thinking of so many things that Microsoft left up to the community. Open source vs closed source, give the people what they WANT, vs. give the people what you think they NEED.
There is no kill, like overkill, but sometimes, we lose sight of the problem we are trying to solve…
People tend to be creatures of habit, finding comfort in the routine. This goes double (if not triple) for engineers of all trades. This idea is in stark contrast to the kitchen sink philosophy. I know what I know, and it gets the job done. Phrases like “I just want it done now” erode creativity. Tight deadlines, and angry clients for sure, make haste. Phrases like “it needed to get done yesterday”, can lower quality. There’s a phrase that comes to your mind.
“Jim Jarmusch once told me “Fast, Cheap, and Good…pick two. If it’s fast and cheap it won’t be good. If it’s cheap and good it won’t be fast. If it’s fast and good it won’t be cheap.” Fast, cheap, and good…pick (2) words to live by.”
-Tom Waits Spills the Beans to Tom Waits,” Sound Effects Blog, Boston Globe, May 22, 2008.
When we don’t give ourselves room to dabble, we can turn around three times and still be using COBOL… because it works (sorry if I’m being reductive). There is value sometimes to building a better mouse trap, but I’ve also seen folks fall prey to the “not invented here” syndrome. This battle of buy vs. build impacts us all. OpEx vs. CapEx, now vs. later, what’s the “best” answer?
We like what we like, and we like to do things the way we’ve always done them before because it works.
I’ve been asked at many a family gathering, “What do you do?”. I usually quip back, “I’m a software developer” or “I work with the cloud”, two easy-to-digest answers. The honest answer, convert coffee into code… which gets converted into money by folks savvy in the ways of business. We’re in the business of solving problems, usually business problems, sometimes people problems, but always human problems.
(Side note: I did go to school for business… but I also was going to be a social worker as well. What you’re good at, and what you went to school for, are sometimes, very different things -:wink:)
“The most dangerous phrase in the language is ‘we’ve always done it this way’.”
Admiral Grace Hopper
While I’m primarily a Python (and PowerShell) developer, I know many folks out there use a variety of different languages, each with its specific tooling. What works for Python, may not work for Java C# or Go.
I’ve compiled a short list from talking amongst friends over the last few years. The whole point of this list is to be short, NOT exhaustive, and list flexible practices. That would defeat the core theme of this blog post if they weren’t! All of these tools work on OSX and Linux, and SHOULD also work on WSL if you are working on a Windows dev box. Please keep in mind, that all of these items are current as of 2023, and should be around for a fairly long time, nothing bleeding edge. If you don’t like the tools of today, considering making your own, just please do your research online first. We have had enough of a problem with competing standards.
This versioning style allows for easily keeping track of how version updates will break builds during upgrades. Some vendors like to do date-style version numbers while some do not. Ex JetBrains IDEs vs VSCode.
Versions should be formatted like the below
v1.2.3-beta+684
Major.Minor.Patch-pre-release-type+build-number
A super handy way of putting diagrams in code. Many different IDEs and platforms support the format out of the box, and it is quite easy to read and write. PlantUML offers a more feature-rich alternative, a bit with less wide adoptions and harder (but more expressive) syntax.
Mermaid Example
stateDiagram-v2
[*] --> New_Idea
New_Idea --> Working_On_Project:Write down idea
Working_On_Project --> Bored:Life
Bored --> New_Idea:Inspiration
Project_Done --> [*] :Feeling of accomplishment
.editorconfig
file at the base of your repo and you’re done!root = true
[*]
charset = utf-8
end_of_line = lf
indent_size = 4
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
[*.md]
indent_size = 2
Great for scanning repositories locally for leaked keys and passwords. Works with several different languages.
Easy to use and deploy LOCALLY!
pip install tartufo
tartufo scan-local-repo . --progress
Can be integrated into your local Git pre-commit hooks, see below.
.github/workflows/
folder, inside your Git repo.name: "Generate TODO to Issue"
on: ["push"]
jobs:
build:
runs-on: "ubuntu-latest"
steps:
- uses: "actions/checkout@v3"
- name: "TODO to Issue"
uses: "alstr/todo-to-issue-action@v4"
with:
AUTO_ASSIGN: true
name: Release
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
on:
push:
tags:
- "v[0-9]+.[0-9]+.[0-9]+"
jobs:
release-gen:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v3
- uses: ncipollo/release-action@v1
with:
allowUpdates: true
generateReleaseNotes: true
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Docker meta
id: meta
uses: docker/metadata-action@v4
with:
images: ghcr.io/${{ env.IMAGE_NAME }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Build and push Docker image
uses: docker/build-push-action@v4
with:
context: .
platforms: linux/amd64 #,linux/arm64
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
This is more of a way of writing commit messages, than a tool per se.
Messages are written in the format of “fix: foo” or “feat: bar”.
git commit -m "fix: Thing no longer broken
How does this relate to SemVer?
| Type | Description |
| -------- | -------------------------------------------------------------- |
| feat! | Breaking change |
| feat | New feature |
| fix | A bug fix |
| revert | Rollback of a change |
| docs | Adding some documentation |
| ci | Changes made to an infrastructure job |
| chore | Upgrades, cleaning up... |
| perf | A performance improvement |
| refactor | A change in the code that does not affect the overall behavior |
| style | Formatting change |
| test | Adding some test cases |
This is a great extra included in the fantastic Git-Extras package (apt-get install git-extras OR brew install git-extras
).
Generating a nice change log is as easy as running git-changelog -p -a -x >> CHANGELOG.md
at the base of your Git directory.
As long as you have been tagging your releases with sane tag names, you should be good to go!
While GitHub releases are always beneficial, this keeps your change log close to the code, just like your README file.
Or do both with a little GitHub Actions love! Automate the boring stuff.
n.n.n / 2023-04-22
==================
* Added Auto change log generation
* Removed radon from suite, not being used in interactive test
* Added pause in test
v2.0.0 / 2023-04-08
===================
* Version bump
* Merge branch 'feat/cli-interface'
* Readme update
* Added keyring check
Great for testing and debugging API endpoints, a must-have for backend developers
For a simple CLI alternative, you might want to change out HTTPie (similar to curl)
A personal favorite of mine. Yes, I know we have Makefiles, but IMHO the syntax of this is easier to read and write. Before you get your pitchforks, please take a look, as I’ve said before, I try and keep things in this category both language-agnostic and cross-platform.
In this example, we can run task greet
and we will load and print the vars from our env file and get the directory contents of our home directory.
version: '3'
env:
ENV: testing
tasks:
greet:
dir: /home/user
dotenv: ['.env', '{{.ENV}}/.env.', '{{.HOME}}/.env']
cmds:
- echo "Using $KEYNAME and endpoint $ENDPOINT"
- ls
This is a fantastic tool for when you need your make files to think! This build is done via a Taskfile…
version: '3'
vars:
BUILD_DIR: pl_worker
tasks:
test:
silent: false
interactive: false
dotenv: [prod.env]
cmds:
- poetry run vulture --min-confidence 100 {{.BUILD_DIR}}
- poetry run xenon --max-absolute B --max-modules B --max-average B {{.BUILD_DIR}}
- poetry run pytest --cov --cov-fail-under=75
Can ALSO be done via PyInvoke
from dotenv import load_dotenv
load_dotenv("prod.env")
from invoke import task
BUILD_DIR= "pl_worker"
@task
def test(c):
"""Test Build"""
c.run(f"poetry run vulture --min-confidence 100 {BUILD_DIR}",echo=True)
c.run(f"poetry run xenon --max-absolute B --max-modules B --max-average B {BUILD_DIR}",echo=True)
c.run("poetry run pytest --cov --cov-fail-under=75",echo=True)
Will I redo all of my TaskFiles for my projects? No, what I have works for me and works well. But next time I need to write a CLI script runner for myself! (Side note: I’ve already written some neat stuff using Invoke for work… I like it 😄)
While I could list off dozens of neat little things I’ve found over the last year or so, I’ll keep this list very short, just a few select plugins and apps that have really helped me be more productive. Please keep in mind that these are either IDE or platform-specific, so YMMV.
Brings ctrl+click to the terminal Window, a-la VS Code
On the flip side, brings the joy of a quick search from JetBrains to VS Code
Depending on which platform you use, a quick desktop-wide search will save you hours over the months**. Really, check these out! (Yes OSX has the spotlight, and yes “it just works”, I would argue this is superior in every way to it)
While I know this blog post has been very VERY long, I do hope it helps those who come across it to not only improve their tooling but to also improve HOW they think about the tools they choose and choose NOT to use. I hope this blog post inspires those who read it to try some new toys… err tools!