That was exciting! Yesterday morning I was having saturday morning coffee in bed doing a bit of prompt driven development to add weather predictions to my Remarkable sleep screen when suddenly weird things started happing on Ersa, a Raspberry Pi 4 Model B that hosts everything in my house including this blog. By weird I mean I was trying to create a new file but said it was a read only file system. I figured I had run out of space, but was not the case, so I opted for a “windows” solution: reboot. And that was all she wrote and Ersa did not come backup. The picture above is the open heart intervention on Ersa trying to figure out what was going on. Read on for the frantic recovery actions and a incident postmortem retrospective to ensure this learning experience leads to improvements!