This one is more a nudge to myself. On several occasions when building Infrastructure As Code (IAC), I split out a code sections into one or more files, for readability and reusability purposes. What I tended to do, and this was more apparent with the Linux builds than the Windows builds, was to forget to set the line terminator from CRLF to LF.
While this doesn’t really impact Windows builds too much (they’re kinda designed to support people being idiots with line endings now), Linux still really struggles with CRLF endings, and you’ll only see when you’ve broken this because you’ll completely fail to run any of the user-data script.
How do you determine this is your problem? Well, actually it’s a bit tricky, as neither cat, less, more or nano spot this issue. The only two things I found that identified it were file and vi.
So, how to fix this? Assuming you’re using Visual Studio Code;
You’ll notice this line showing “CRLF” in the status bar at the bottom of Code. Click on that, which brings up a discrete box near the top, as follows:
Selecting LF in that box changes the line feeds into LF for this file, but it’s not saved. Make sure you save this file before you re-run your terraform script!
Fantastic! It’s all worked!
In Nano, I’ve opened the part with the invalid line endings.
To fix this, we need to write the file out. Hit Ctrl+O. This tells us that we’re in DOS Format, and also gives us the keyboard combination to toggle “DOS Format” off – it’s Alt+D (In Unix/Linux world, the Alt key is referred to as the Meta key – hence M not A).
So, after hitting Alt+D, the “File Name to write” line changes, see below:
Using either editor (or any others, if you know how to solve line ending issues in other editors), you still need to combine your script back together before you can run it, so… do that, and your file will be fine to run! Good luck!
Featured image is “Fishing line and bobbin stuck on tree at Douthat State Park” by “Virginia State Parks” on Flickr and is released under a CC-BY license.
Very useful.
Came across the very problem recently when creating (copying and modifying really) a python script in VScode and then getting an error when the script was ran on a Linux server.
I ended up using the app (dos2unix) referenced in this post – https://askubuntu.com/questions/896860/usr-bin-env-python3-r-no-such-file-or-directory/896880#896880
And your post got me thinking that I should just set the default file ending to LF in VScode – it’s in settings ( ctrl + , ) – Text Editor – Files – Eol – set it to \n
Any issues doing this that you are aware of.
Cheers
Thanks Simon,
I’ve not seen any negative effects to having Windows Powershell scripts being LF-only terminated, but I have seen issues with Batch Files, so if you’re editing either of those, that might be an issue… but if you’re mostly working with files in the Linux world, then you should be fine :)
Hiya Jon – for what its worth, cat -A will show the giveaway ^M control characters of a CRLF!
(andyf@eff):/mnt/c/temp$ cat -A wintextfile-utf8.txt
this is a file^M$
with presumably windows line endings^M$
no empty line at the end
(andyf@eff):/mnt/c/temp$ cat -A wintextfile-utf8.txt
this is a file^M$
with presumably windows line endings^M$
no empty line at the end
Oh fab, thanks Andy, I didn’t realise that! :) Useful note!!