Bash
Bash
https://dwheeler.com/essays/filenames-in-shell.html
Commands I didn’t know
- du disk space used as opposed to df
- rename similar to sed eg
rename 's/ /_/g' *will replace all spaces with underscores. - split
- xargs
echo vs «<, or Useless Use of echo in Bash Award?
TDD frameworks
Extended test command
[[ ]]
All about {Curly Braces} in Bash
returns a value in variable $? where, conversely to the norm, 0 is true and anything else false.
Shell Parameter Expansion
String content of variable
${var}
Substring
${parameter:offset:length}
Eg first 4 characters
${var:0:4}
Length of string in variable
${#var}
Search and Replace
Note pattern is a glob, not a regular expression.
${parameter/pattern/string}
The above is akin to sed 's/pattern/string/' <<<"$parameter" in that it replaces the first occurence of pattern with string.
${parameter//pattern/string}
The above is akin to sed 's/pattern/string/g' <<<"$parameter" in that it replaces all occurences of pattern with string.
${parameter/#pattern/string}
The prefix must be pattern.
${parameter/%pattern/string}
The suffix must be pattern, handy for changing filename endings.
Here Strings
COMMAND <<< $WORD
Instead of
echo $WORD | COMMAND
Coreutils
https://www.gnu.org/software/coreutils/manual/html_node/index.html
https://github.com/step-/JSON.awk
tar
Packing:
tar -cvjf seatavern.tar.bz2 seatavern
Unpacking:
tar -xvjf seatavern.tar.bz2
tr
sed
Difference Between grep, sed, and awk
hq
https://github.com/coderobe/hq
curl -sSL https://www.quicket.co.za/events/1 | hq '[type="application/ld+json"]' text > quicket1.json
jq
https://stedolan.github.io/jq/
How To Transform JSON Data with jq
Pretty print a json file
jq '.' example.json
Create
Retrieve
Update
Delete
List
First item
jq '.[0]' example.json
less example.json | jq '.[2] |
{name: .name, url: .url, startDate: .startDate,
venue: .location.name, region: .location.address.addressRegion}'
https://github.com/itchyny/gojq
wget
Download a Directory and Subdirectories Using wget
Awk
In awk, the ‘^’ and ‘$’ anchor metacharacters match the beginning and end of a string, and not the beginning and end of a line.
Startup
BEGIN { action }
BEGIN { IGNORECASE = 1 }
Cleanup
END { action }
END {
close(curfile) # close the last one
for (f in filelist) # close all the rest
close(f)
}
Records
FNR the number of records that have been read so far from the current input file
NR the total number of input records read so far from all data files
RS record separator, default newline
RT term that matched RS
ORS output field separator
Fields
$1 first field, $2 second field…
FS field separator, default whitespace
OFS output field separator
Tools
https://github.com/koalaman/shellcheck
Date manipulation
+FORMAT
How to add days to date and get new date on Linux
https://www.shell-tips.com/linux/how-to-format-date-and-time-in-linux-macos-and-bash/#gsc.tab=0