ChrisHubbs.com

For a long time my podcast listening has been almost exclusively nerdy tech podcasts mixed up with nerdy theology podcasts, with an occasional news or true crime mixed in to liven things up. Somehow I have almost entirely bypassed any that were music-related. (I did listen to a couple episodes of Song Exploder right after it debuted, but it just didn’t hook me.)

Somewhere along the line, Rob Weinert-Kendt on Twitter started linking to A History of Rock Music in 500 Songs by British host Andrew Hickey. It took me only a couple episodes and I was hooked.

The format of the podcast is one song per 30-ish minute episode, but each episode covers far more than just the titular song. Hickey provides background on the artist, the influences that formed that artist, stories about the creation of the song, and so on. You come away from the episode having learned a lot not just about a particular song but also about the developing music scene in the Americas (and, once you get in a ways, in Europe). He starts with the first inklings of what would become rock music as they emerged in the big band scene. (Episode 1: “Flying Home” by the Benny Goodman Sextet.)

500 episodes is a significant feat for any podcast, and setting out that goal in the title of your show seems rather ambitious, but I’m willing to bet that Mr. Hickey has all 500 songs charted out, and the moxie to see it through. He’s currently up through Episode 157 (“See Emily Play” by Pink Floyd), and is publishing a lot of bonus material for Patreon subscribers. I’m learning a lot as I go, so even if some interruption keeps the series from completion, it’s still been an excellent investment of time.

So, if you’re interested in rock music, A History of Rock Music in 500 Songs is highly recommended.

I’ve had my Mastodon instance (iowadon.org) up and running for about a week and a half now. It’s not grown very large – 10 users or so last time I checked – but it’s been a great project to learn some more server admin type stuff.

Ten days and ten users in, my Linode instance seems to quite adequate from a processing standpoint. (I’m using the lowest-level Linode offering, which provides 1 CPU core and 2 GB of RAM.) However, the disk space usage is growing. The database itself isn’t overly large, but the cache directory (which stores profile pictures, preview cards for images) is up over 15 GB. I could work to get really aggressive about cache management, but realistically a larger cache is a reasonable thing. Mastodon provisions for this by providing easy hooks to use S3 buckets for the cache, so I figured I’d give that a shot.

I found Nolan Lawson’s excellent step-by-step instructions and followed them to a T. Well, almost to a T. I first went to set up an S3 bucket, kicked off the script to copy the cache over from Linode to S3, then went to bed. The next morning I did some more reading and decided that Linode’s very similar Object Storage service (it’s just their S3 clone) might be a better deal cost-wise. Amazon S3 charges a small amount per GB for storage and then a different rate for data access. Linode does it slightly differently – you pay a flat fee per month for a given bucket size, and then you get a large amount of transfer every month for free. Since my server is already on Linode, it was easier and simpler to just use the Linode buckets, so I tried again there.

One gotcha that’s not obvious when creating the bucket at Linode: if you’re going to put a custom domain name in front of the bucket, you need to name the bucket that domain name if you want their TLS/SSL stuff to work. In my case, I setup a CNAME record to point assets.iowadon.org at my bucket, so I needed to name my pocket assets.iowadon.org. There’s no renaming buckets, so I had to empty and delete my old one and then create a new one with the correct name. After that the certificate generation went smoothly enough and I once again kicked off the copy job. Then I went to the gym.

A couple hours and 68,000 file copies later, my cache is in the bucket and a quick restart of Mastodon via docker-compose pulled in the configuration updates that now point out to the cloud. It went amazingly smoothly.

Edit: I posted this a little too soon…

All the existing assets were working fine, but new assets weren’t loading properly. Commence some more googling. The correct answer was that in addition to the .env.production settings listed in the instructions above, you also need this one:

S3_ENDPOINT=https://<linode_region>.linodeobjects.com

In my instance, that looked like this:

S3_ENDPOINT=https://us-southeast-1.linodeobjects.com/

Now it seems to be fully working.

I was familiar with Carter Burwell’s name thanks to his score for the Coen brothers’ film True Grit, but I wasn’t aware of the full scope of his film compositions or of his backstory. A brilliant man who just picked up and learned lots of things. Just out of college and trying to make it as a musician while working a lousy warehouse job:

One day, Burwell saw a help-wanted ad in the Times for a computer programmer at Cold Spring Harbor Laboratory, a nonprofit research institution whose director, James D. Watson, had shared the Nobel Prize in 1962 for discovering the structure of DNA. Burwell wrote a jokey letter in which he said that, although he had none of the required skills, he would cost less to employ than someone with a Ph.D. would. Surprisingly, the letter got him the job, and he spent two years as the chief computer scientist on a protein-cataloguing project funded by a grant from the Muscular Dystrophy Association. “Watson let me live at the lab, and he would invite me to his house for breakfast with all these amazing people,” he said. When that job ended, Burwell worked on 3-D modelling and digital audio in the New York Institute of Technology’s Computer Graphics Lab, several of whose principal researchers had just left to start Pixar.

The Polymath Film Composer Known as “the Third Coen Brother” by David Owen in The New Yorker

His royalties from scoring Twilight funded a house on Long Island, where he lives and works from home, composing on a 1947 Steinway D that came from the Columbia Records studio in New York. “I still fret about having replaced the hammers, but they were worn almost to the wood—some say by Dave Brubeck.”

Worth reading the whole profile.

With the Muskian shenanigans happening over at Twitter, I’ve been looking into Mastodon as an alternative social media site. It’s an interesting concept – a distributed network of servers that all talk to each other through an open protocol. So, being the nerdy sort, I decided to try setting up my own server. Here’s how it went:

Registering a Domain Name

Just deciding on the domain name was the hardest part, I think. But once I settled on iowadon.org, I went over to Hover and registered it. Easy peasy.

SMTP email service

Mastodon needs an SMTP mail service to send notification emails. Not wanting to tie it to my personal Fastmail account, I looked around and determined that Amazon AWS Simple Email Service (SES) was a reasonable fit. The pricing is scalable and pretty cheap ($1 / 1000 emails at this time). It took a little bit of time to get configured, but the AWS website walked me through the steps pretty clearly. Basically, AWS will give you three pieces of info: an SMTP server name (available on the SMTP settings page, will be something like email-smtp.us-east-2.amazonaws.com) and then a username and password. For the latter two it generates a pair that look like guacamole, and it warns you it will only ever show you them once, so copy and paste them to a useful location. If you lose them you’ll have to regenerate a new pair.

Also note: if you’re a new AWS account, AWS will put you in a restricted mode until you request a review and a move into production. Once you verify a domain identity you can also verify individual emails. You need to do this before you setup Mastodon because Mastodon will want to send you an email or two.)

Linode

I decided I didn’t want to go with a hosted setup – most of the custom hosting providers are all slammed right now, too – so it felt like my best options were either DigitalOcean or Linode. DigitalOcean advertises a pre-configured “droplet” that will give you a Mastodon instance with a minimum of configuration; Linode provides instructions for installing Mastodon on Ubuntu 20.04. I went with Linode.

I started out with a 1 CPU core, 2 GB of RAM, 50 GB disk space Linode. Once I get past the initial credit they give you (go find a referral link… my account is still too new to be able to give you one of my own), I think they’ll charge me $10/mo for it. We’ll see how well it performs if I get more than a few users.

The instructions were good as far as they went, but a few key things seemed to be missing. I’m going to walk through the instruction sections here and comment on them.

“Before You Begin”

This section was pretty straightforward. I followed the instructions for Creating a Compute Instance, Setting up and Securing a Compute Instance, and Adding DNS Records.

“Install Docker and Docker Compose”

Followed the instructions for installing Docker, then for installing Docker Compose. When you clone the Mastodon git repository in the next step, it includes a docker-compose file. It took me a few minutes to realize that the docker-compose also includes the Postgres database, so there’s no separate database installation required on the Linode server.

“Download Mastodon”

OK, this bit was simple. It’s just a git clone command.

“Configure Docker Compose”

Editing the docker-compose file is a little bit of a pain. Pay attention to the details. For step #3, setting up the Postgres password, db, and user, you can set the password to whatever you want it to be. Write it down because you will need it down in the next section.

Now, step 6, generating SECRET_KEY_BASE and OTP_SECRET. There are 4 separate commands in the instruction box: 2 echos to create the values and two ‘sed’ lines that I mistakenly assumed would write the value into the config file. Don’t assume that! Follow the instructions directly and copy/paste the values for SECRET_KEY_BASE and OTP_SECRET into the .env.production file. Same story with the VAPID_PRIVATE_KEY and VAPID_PUBLIC_KEY.

“Complete the Docker Compose Setup”

The involved command here is the rake command that does the initial database setup. It will ask you a bunch of questions here, many of them duplicate info to what you just put into .env.production. Such is life.

“Initiate the Docker Compose Services”

Two commands. Easy peasy.

“Setup an HTTP/HTTPS Proxy”

Here we install nginx. There’s an nginx.conf file in the Mastodon distro that is a good start, but there are changes you will need to make to it.

There are two lines that say root /home/mastodon/live/public;. You have to change these lines. Having followed the instructions thus far, the actual location of my /public folder was /home/chris/mastodon/public. Update the root lines to point to the actual location of your public folder.

Next edit, and I don’t remember where I found it, but it seems to work: every instance in that file that says try_files $uri =404; needs to be updated to say try_files $uri @proxy;

I think that’s all I had to do to the nginx config.

“Get an SSL/TLS Certificate”

I followed the steps to install Certbot, but when I got to step #4 where you run certbot to generate a certificate, I got an error saying that there was no ssl in a listen section of my nginx configuration. It turns out there is a “snake oil” certificate you set up first to get things moving, then you can get and install the real certificate. For the snake oil thing, use the steps described here. I then had to cleanup a broken renewal with steps listed here, but that may have been because I was messing around w/ certificate stuff before I found out about the snake oil. If your renewal dry run works ok, you can ignore that second link.

“Using Mastodon”

The instructions would lead you to believe at this point that the site is ready to use. However, when I went to iowadon.org, I just got an nginx configuration error. After a bunch of debugging, the key issue I found was this: I had never run any command that did a bundle install, that step where ruby installs and builds all the gems listed in Mastodon’s gem file. Of course it’s not gonna work that way!

The command you will want to run is bundle exec rails assets:precompile.

When I ran that it then sent me through several searches to install other libraries on the linux server. To save yourself some time, run this:

sudo apt install ruby-bundler ruby-dev ruby-full build-essential libicu-dev zlib1g-dev openssl libgmp-dev libxml2-dev libssl-dev libpq-dev

That should give you all the libraries you need to complete the bundle install.

It works!

After all that, sure enough, it worked! My Linode CPU usage had been sitting solidly between 10% and 15% the whole time, apparently because it was in a loop continuously trying to restart my misconfigured Rails app. Once Rails started up cleanly, the CPU usage went down to about 2%.

Adding another admin user

I originally set up an admin account for iowadon.org during the setup steps. Then I realized I would want my own account (@cjhubbs@iowadon.org) to be an admin once I moved it over. How to do that, I wondered? Turns out Mastodon has a command-line tool called tootctl just for this purpose. But to run tootctl and get it to talk to the database running in Docker, you need to run a special version of the command. What you’re looking for is this:

docker-compose run --rm web bin/tootctl <tootctl command>

Magic!

Impressions

First off: it’s amazing how much snappier Mastodon is on this server than back on mastodon.social. The account migration process went smoothly enough, too. I will advertise it to a few friends once I get the Amazon AWS SES account moved to production so it can send emails to people other than me. Then we’ll see if it gets any traction. I’ll follow up later.

On my work trip to Brussels last week I was introduced to the liège waffle, a type of waffle that is denser and sweeter than the common “Belgian waffle” that I’m used to in the States. They were delicious! So when I got home there was no possible option but to find a recipe and try making them myself.

When you go search through the multitude of online recipies for liège waffles, you find that the secret ingredient for these things is pearl sugar — a compressed form of sugar bits smaller than sugar cubes, roughly the size of mini M&Ms. So, I ordered some from Amazon, printed off this recipe, and got to work.

With liège waffles you don’t really have a waffle batter – it’s more a waffle dough. It’s sort of a brioche, more similar to cinnamon roll dough that I’ve made than to pancake batter. Threw it all in to the KitchenAid mixer with the dough hook and kneaded it for 5 minutes, then threw it into a warm oven to let it rise for an hour.

Once the dough has risen you knead in the pearl sugar, which gives you a strange mound of dough that crunches when you cut it up into pieces.

Throw these guys into the waffle maker and they cook up very nicely! I used a medium heat that turned out to be just right to caramelize the sugar, giving the waffles a nice shine and a nice crisp surface.

The caramelization is, of course, a little more work to clean up…

Might scale the pearl sugar back by about 20% next time but otherwise this recipe was great! Not quite to the level I got from the pros in Belgium but for sure a good Saturday morning breakfast. These were a hit with the family and we will definitely be making them again.

Rich Mullins died on Sept 19, 1997 in a car accident. A songwriter, poet, and prophet, Rich spoke with a voice that had grabbed my heart as a teenager. Hearing that he had died then ripped my heart right out.

I still remember sitting in church on Sunday morning probably 36 hours later and hearing Pastor Jim (himself a great musician) mention Rich’s death from up front. I was stunned. Leaning forward, I put my face in my hands and tried to grapple with this news while my fiancée put a comforting hand on my back, unsure what to do next.

At some point later that fall I assembled a group of musician friends and put together three of Rich’s songs and got permission to play them in his memory at a college chapel service. I’m sure we didn’t really do justice to “Awesome God”, “Hold Me, Jesus”, or “Elijah” that day, but it felt like the least I could do to honor someone whose music had meant so much to me.

I’ve written about Rich’s music many times here on the blog. The concert summary of the tribute concert at the Ryman, the post where I realized that Rich wasn’t the one playing the piano on all those recordings, and the wistful alternate-universe press release (10 years old already!?!) are perhaps worth revisiting.

Today, though, on the 25th anniversary of his passing, this short clip (shared by Shane Claiborne on Twitter) reminds us Rich’s message remains relevant today.

Here’s a 1 min clip of Rich Mullins in honor of his death on Sept. 19, 1997. Classic. It’s from one of his last concerts before the car accident… he said this almost 30 years ago.? pic.twitter.com/2cP1nP7uai

— Shane Claiborne (@ShaneClaiborne) September 19, 2022

Rest in Peace, brother, awaiting the resurrection.

Eighteen years and seventeen days ago we brought our first child home for the first time. She was just a tiny newborn joining two new, naïve parents in the adventure of life.

Tomorrow, we load her packed boxes of belongings into the van and head out to launch her into the next phase of her adventure: college at SDSMT. I’m sure none of us really appreciate yet how much we will miss her, but we’re so excited for the opportunity she has and the path that lies ahead of her.

Some days have seemed long, but the time now seems to have been very short.

I have on occasion paraphrased something Fr. Robert Capon wrote about God’s love as ”if God is a bastard, we’re all in trouble”. I went digging for the original quote today and came up short of any such choice phrase, but for reference, here’s the longer quote from Between Noon and Three which makes that point, if just a bit less colorfully:

…I have, in this parable, been working only one side of the street — in my effort to do justice to grace, I have neglected justice itself. I am fully aware that in doing so, I have laid myself open to the charge of granting not only screwing licenses but also franchises for far worse things: for pride and prejudice, for torture and exploitation — in short, for getting away with murder.

In my defense, let me point out that Scripture lays itself open to the same charge — and that the other side of the street has been worked so long, so hard, and so often that most people don’t even know there is a sunny side…

But there’s more to it than that. I have expounded Saint Paul to you as saying that not only are we dead to sin but that God is dead to it too — that he has put himself out of commission on the whole subject of blame. And so, indeed, he has: ”I will forgive their iniquity, and I will remember their sin no more.”

I am fully aware that the Scriptures are paradoxical — that God speaks with a forked tongue — and that every lovely thing he says on the side of leniency can be matched by a dozen stringencies that will curl your hair. But I am also convinced that each of us has to make a decision about such utterances. When someone tells you many different things about his attitude toward you, you must first look at him long and long, and decide for yourself whether you care about him at all. But if you finally come to the conclusion that you do care, you must then decide which of his words you will take as his governing word. You ask me why I think God’s leniency governs his severity? Why grace is his sovereign attribute? Well, all I can say to you is that having been a father who has spoken out of both sides of his mouth to six children for twenty-six years — and having all those years believed in a heavenly Father who saves us not by sitting in his penthouse issuing edicts but by sending us the warm, furry body of a Son who drank the nights away with us and died obscurely of the foolishness of it all — all I can say is that I put my bet on the left fork of the tongue. It is my best hope that when my children think of everything I have said and done to them, they choose to remember the times of my severity when I just gave them a kiss on the cheek, poured myself a Scotch, and shut up. And it is my last hope that God hopes the same for himself.

So I really do make no apology for landing on the sunny side of the street. I am sorry if I have offended you; but to me there are some things that simply override everything that comes before or after. And I am sorrier still if you do not feel the same way. For without that ultimate cassation — without that final quashing of the subpoena, that throwing of the prosecution’s case out of court which is the only music there is for the ears of the hopelessly guilty — you and I, Virginia, are simply sunk.

— Fr. Robert F. Capon, Between Noon and Three, Chapter 17

June 2022 was quite a month for me. It started with the death of a long-time friend. It ended with my being on a phone call with my father while he experienced a stroke-like event. (Turns out: not a stroke. He’s home, and doing ok. Hallelujah.) In the middle was a thing called Pride Month, which had a little more visibility at our home than usual. Two of my kids identify as LGBTQIA+ and have embraced visibility more this year than ever before. (I got their permission to say this here.) So, one of these things is not like the other, but stick with me here and I’ll connect the dots.

Part 1: Online Community

Since Geof died last month, the online community he fostered for nearly 20 years (referred to as “RMFO” for reasons long since forgotten) has been renewed. I have hosted two Zoom “happy hour” calls, and both times 15-20 people have joined, chatted for 2-3 hours, and left with a request that we schedule another one. We have spent these hours catching up on life and recounting our own personal histories as they interact with the RMFO community: singles who met their future spouses in the group; couples once struggling to conceive who now have teenagers; marriages, divorces, job changes, moves, faith evolutions. Online friendships have led to “real-life” friendships, meet-ups, and job opportunities. At the end of the first call I realized that, while I had long considered many of these people meaningful to me, the call helped me realize that I was meaningful to them, too. My thought last week as I closed out the second call: this is the closest thing I’ve had to a healthy, functioning community in my adult life.

During last week’s call I talked about how my own personal views on some issues have evolved over the years, and how I have struggled to find an in-person community (specifically, a church community) where that evolution was welcome. Share those views too loudly and you will be welcome to find some other church to serve and worship at. Five years of quiet discomfort weren’t enough to dislodge me from my last church; their tepid COVID response was the straw that gave me the courage to break the proverbial camel’s back. Two years later I still haven’t found a new place to join. If I’m honest, I haven’t really looked too hard.

Part 2: My Dad

My dad is talking a little slower thanks to the medications he’s on after his health scare, but he’s not talking any less. When we talked last week after he got home from the hospital, we spent a while comparing current reading lists and discussing how some of the books I have given him over the years have helped guide his journey out of a fundamentalist faith into something much more open and gracious. That’s his story to tell, not mine. But what he said toward the end of the call stuck with me: that coming close to death now makes him unwilling to stay quiet on the topics important to him. And I thought to myself: that’s a lesson I should take to heart here in my 40s.

Part 3: Chuck Pearson

Chuck Pearson is a friend of one of the RMFO guys — someone I have never met but have followed on Twitter for years. Last week he posted a beautiful essay about his journey leaving a tenured professorship at a Southern Baptist university when he knew he would eventually be required to sign a “personal lifestyle statement” that would “[force] him to disown his LGBTQ+ friends and family”. Chuck ultimately found a post at another university, but still sat quiet about the topic of LGBTQ+ inclusion:

Ultimately, it stayed private because I didn’t want to burn those bridges. Even as I made that realization that I had to choose between two sides I cared about deeply, I couldn’t bring myself to take that final step of declaring my choice.

In his essay, Chuck quotes from an Alan Jacobs essay from 2014 that argues for Christians to stand firm against the cultural evolution of views on sexuality. Here is Jacobs’ conclusion [preserving the emphasis from the original post]:

Either throughout your history or at some significant point in your history you let your views on a massively important issue be shaped largely by what was acceptable in the cultural circles within which you hoped to be welcome. How do you plan to keep that from happening again?

Clearly Jacobs has in mind a call to resist the desire to be welcomed by “the world” by accepting “worldly” views on sexuality. But for me his question takes the exact opposite orientation. How long have I been convinced about full acceptance of LGBTQIA+ people in the church but been afraid to say so, knowing that it would make me unwelcome in the evangelical church circles I have run in my entire life? Far too long.

Conclusion

So let me make a long-overdue statement as clearly as I can. I am a Christian, and I affirm those who identify as lesbian, gay, bisexual, transgender, queer, intersex, and asexual as they are, unconditionally, including their full inclusion into the assembled body of Christ known as the local church. I know there are those who would say I cannot hold that view and be a Christian. I reject that assertion.

There are multiple serious approaches to the Scripture on the topic which I will not go into here. Suffice it for now to say that you cannot read Scripture as a flat text. It is the product of thousands of years of God’s progressive revelation as understood and recorded by humans. In the end you have to come to some conclusion on which texts capture most clearly the essence of who God is, and use those as the framework from which to understand the rest.

And so, to quote Brian Zahnd:

God is like Jesus.
God has always been like Jesus.
There has never been a time when God was not like Jesus.
We have not always known what God is like—
But now we do.

Jesus taught these two principles in summation of all the teaching: love God with everything you have, and love your neighbor as yourself. He embraced the outsiders and rebuked the self-righteous. He called us to follow Him in lives of self-sacrificial love.

In addition to Jesus’ teaching, we have the witness of the Holy Spirit in the lives of our LGBTQIA+ brothers and sisters — so many dear, tender souls who consistently model what it means to follow Jesus, even in the midst of a church that rarely welcomes them. And to that I can only respond as the Apostle Peter did after first seeing the conversion of Gentiles to Christ: “Can anyone object to their being baptized, now that they have received the Holy Spirit just as we did?” (Acts 10:47)

Rachel Held Evans summarized it this way: “The apostles remembered what many modern Christians tend to forget—that what makes the gospel offensive isn’t who it keeps out but who it lets in.”

To this I say: Amen. May it be so.

It had been a long time since I’d listened to this song. Then it came across my playlist this morning. Don Chaffer of Waterdeep in some strange alter ego called The Khrusty Brothers, with his own perspective on being angry at God…

I came stumbling into church with a hot gun in my hands
I was ready to talk to Jesus to tell him my demands
But Jesus ain’t no fool He’s seen this kinda thing before
And He had a couple angels stop me at the front door
I said “now come on that ain’t fair You should be accessible to all”
He said “everybody gets a secretary even just to take their calls”
“So address me to my face If you think you’ve got the balls
But I ain’t playin’ around  boy, at all”

This was not what I expected, so I stiffened in my stance
And I tried hard to remember every single shitty circumstance
Then I quivered like a victim with his predator in sight
I was ready now to vindicate, I was ready to start a fight
Now you can stand right there and judge me
Shoot, you can send me straight to hell
I know you got the power I know that fact full well
But before you do explain to me
Why suffering and why death?
And why did I pray all those years
And waste all that good breath?

Hallelujah, Hallelujah
Well the angels sang it under their breath by the door
Hallelujah, Hallelujah
I give up, I can’t go on like this any more

“Well I appreciate your kind”, he said and then Jesus poured a drink
My face musta looked funny cause he said, “It’s not like you think”
“I’m saddled with the job you know of interpreting my Dad
To a bunch of frightened people, frightened or just mad
And most of ’em think they got it right” (and then he threw some ice cubes in)
“But most of ’em are just dead wrong about life and death and sin
And then I got my fiancée, she’s supposed to speak my mind
But sometimes she’s just chicken and then she messes it up other times…”

Don Chaffer, Sympathy for Jesus