For the past few days, I have been getting an email that I hadn't seen before. It was from Let's Encrypt telling me that my SSL certificate needed renewed. I've been using Let's Encrypt for all of my pro-bono and personal websites for about a year now, and never have had an issue with renewing certs. I thought that perhaps, I had done something to my crontab, and removed the autoupdate.
So, on to the investigation. I was right, I had setup auto renew in my crontab with a simple certbot renew command. It had been working flawlessly every 90 days without failure until now. This problem seems to be happening to a lot of folks running Amazon Linux, but the solution doesn't appear to be very widely posted. So I am posting it here to help someone save the 2 days of banging their head on their desk that I just went through.
Don't spend days working through re-establishing links to crypto libraries, hand rolling updates, etc. That ultimately doesn't work, and just isn't feasible. Instead, move over to using PIP to control the installation. I wasn't even aware that PIP was a possibility until today.
So for those of you who are seeing certbot auto renew fail on your Amazon Linux AMI images, perform the following.
# rm -rf /opt/eff.org/*
# pip install -U certbot
# certbot renew --debug
That should do it! Dont' forget to update your crontab!