Linux Best Practices and Tips
2017-03-06
5505 words
26 mins read
Linux is powerful, flexible, and can be adapted to a broad range of uses. While best practices for administrating Linux servers are not hard to find due the popularity of the operating system, there is always a need for up-to-date Linux advice, along with the best tips, from our experienced Toptal Linux administrators.
How to Avoid Frustration After Forgetting To Use Sudo Command
<p>
Have you ever typed a command in your terminal, only to find out you forgot to prefix it with the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
sudo
</div>
</td>
</tr>
</table>
</div>
<p>
command? You have to retype the whole command again just to add the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
sudo
</div>
</td>
</tr>
</table>
</div>
<p>
in front of it. Frustrating!
</p>
<p>
Well, you can add this simple alias to your
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
.bashrc
</div>
</td>
</tr>
</table>
</div>
<p>
to help you reduce the frustration:
</p>
<pre>
|
1
|
<p>
The command is pretty simple; <em>history</em> is a program that keeps track of your terminal command history. You can execute any command in your <em>history</em> by using various methods described in the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
man history
</div>
</td>
</tr>
</table>
</div>
<p>
page.
</p>
<p>
In our example, we use
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
history -p \!\!
</div>
</td>
</tr>
</table>
</div>
<p>
, which prints the most recent command in your history list. So, our alias
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
argh
</div>
</td>
</tr>
</table>
</div>
<p>
executes the last used command in the history with
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
sudo
</div>
</td>
</tr>
</table>
</div>
<p>
in front of it.
</p>
<p>
You can, of course, change the string
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
argh
</div>
</td>
</tr>
</table>
</div>
<p>
with your most favorite word. Here is an <a href="https://twitter.com/liamosaur/status/506975850596536320" target="_blank" rel="noopener noreferrer">example and the original source for the idea</a>.
</p>
<h2>
How to Analyze Log Files to Count the Number of Occurrences in a Time Span?
</h2>
<p>
Often, when analyzing log files, it is useful to count the number of occurrences of a particular event during some time span. Let’s say, for example, that we want to count the number of “new sessions”.
</p>
<p>
First, let’s verify the log format by finding the last two lines that contain the text “New session”:
</p>
<pre>
|
1
2 3 |
<p>
In the commands above,
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
is used to filter all log lines and show only those that have the text “New session” (case sensitive). Then,
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
tail -2
</div>
</td>
</tr>
</table>
</div>
<p>
is used to show only the last two lines filtered by
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
. The parameter
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
-2
</div>
</td>
</tr>
</table>
</div>
<p>
makes sure only the last two lines are shown, as the usual default of the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
tail
</div>
</td>
</tr>
</table>
</div>
<p>
command is to show the last ten lines. The results of the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
command are redirected to the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
tail
</div>
</td>
</tr>
</table>
</div>
<p>
command using the pipe
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
|
</div>
</td>
</tr>
</table>
</div>
<p>
. This makes all lines filtered by
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
serve as input to the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
tail
</div>
</td>
</tr>
</table>
</div>
<p>
command.
</p>
<p>
Now, let’s use the log lines we’ve extracted above to come up with a method to group the occurrences per hour. All log lines we obtain with
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep 'New session'
</div>
</td>
</tr>
</table>
</div>
<p>
will have the following format:
</p>
<pre>
|
1
|
<p>
We can use a combination of
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
cut
</div>
</td>
</tr>
</table>
</div>
<p>
and
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
uniq
</div>
</td>
</tr>
</table>
</div>
<p>
to count all occurrences per hour in the lines filtered by
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
. We’ll use
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
cut
</div>
</td>
</tr>
</table>
</div>
<p>
to extract just the <em>hour</em> from the log lines passed by
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
. The following comparison shows which characters we want from each log line:
</p>
<pre>
|
1
2 3 4 5 6 7 |
<p>
We can therefore see that we want to extract just the 12th and 13th characters from each of the log lines, since that represents the hour of each entry. With this, we can then group the results and verify the count of events for each hour of the day, which is what we do in the next step.
</p>
<p>
To group and count the events per hour, we use
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
uniq -c
</div>
</td>
</tr>
</table>
</div>
<p>
. Please note the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
uniq
</div>
</td>
</tr>
</table>
</div>
<p>
command depends on the sorted input to group correctly, meaning that there should never be hour 14 after hour 15, for example. This is rarely a problem with logs because the time should always increase as new lines are added.
</p>
<p>
Assembling the commands together and executing them, we get:
</p>
<pre>
|
1
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
<p>
Each line of the above output contains two numbers. The second represents the hour and the first represents the number of log entries (for new sessions) that hour. We can therefore quickly see from these results that something might have gone wrong between 4 and 7 AM, since there was such a drop in new sessions during those hours.
</p>
<p>
All this was easily accomplished with the combination of
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
grep
</div>
</td>
</tr>
</table>
</div>
<p>
,
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
cut
</div>
</td>
</tr>
</table>
</div>
<p>
and
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
uniq
</div>
</td>
</tr>
</table>
</div>
<p>
commands.
</p>
<h2>
How to Quickly Get Through Bash History?
</h2>
<p>
Let’s say some time ago you used a very long bash command that you would now like to reuse. However, you don’t want to scroll <em>up</em> by repeatedly clicking the arrow key to get to it.
</p>
<p>
There’s an easy trick to pull to access the Bash history, and it is called the
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
reverse-i-search
</div>
</td>
</tr>
</table>
</div>
<p>
mode. This mode will search through the history reverse-chronologically so that your last used command that has your input keyword will come up first.
</p>
<p>
So, to quickly get through you Bash history, press
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
CTRL+R
</div>
</td>
</tr>
</table>
</div>
<p>
on the keyboard to activate bash’s
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
reverse-i-search
</div>
</td>
</tr>
</table>
</div>
<p>
mode, then type in part of the command you’re looking for, and it will show it up. When it does, just press
</p>
<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
<table cellspacing="0" cellpadding="0">
<tr>
<td class="line-numbers">
<div>
1<br />
</div>
</td>
<td>
<div class="text codecolorer">
Enter
</div>
</td>
</tr>
</table>
</div>
<p>
to execute it.
</p>
<p>
Ninja tip: While searching, try to type in one or more unique words from the previous long command that you are looking for, as this will help you find it more quickly.
</p>
How to Add More Accountability to Command Line Work?
You’re not a machine. At least, not most of the time. Like any human, you might be fooled by your imperfection and commit what we call a human error.
The ensuing three short tips will help bring some peace of mind when you are not-so-sure about your guiltiness, or just let you know that you need to assume your error and think of a way to repair the damage.
Add a timestamp to your bash history
This small tip will allow you to look forward in your shell history, knowing when each of your commands was executed:
2
3
4
5
6
7
8
9
10
11
12
13
~$ history
<span class="hljs-number">1</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">03</span> clear
<span class="hljs-number">2</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">04</span> <span class="hljs-built_in">cd</span>
<span class="hljs-number">3</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">05</span> ls -ltr
<span class="hljs-number">4</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">07</span> clear
<span class="hljs-number">5</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">12</span> touch file
<span class="hljs-number">6</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">17</span> chmod +rw file
<span class="hljs-number">7</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">22</span> chmod +x file
<span class="hljs-number">8</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">28</span> cat > file
<span class="hljs-number">9</span> <span class="hljs-number">05</span>/<span class="hljs-number">05</span>/<span class="hljs-number">16</span> <span class="hljs-number">18</span>:<span class="hljs-number">07</span>:<span class="hljs-number">34</span> git commit <span class="hljs-operator">-a</span> -m ‘cool feature<span class="hljs-string">'
10 05/05/16 18:07:36 git push
</span>
If you want to make this permanent, add the following line to your
|
1
|
|
1
|
<span class="hljs-keyword">export</span> HISTTIMEFORMAT=<span class="hljs-string">"%d/%m/%y %T "</span>
|
Share your immediate bash history across multiple sessions
Nowadays, almost everyone uses more than one window or tab at the same time. Browser tabs, chats windows, and of course multiple console sessions. As you might have already noticed, your bash history is saved to the disk at the end of the session when you log out. This is perfectly fine most of the time, but for several reasons you might want to share that command history immediately with your other sessions.
To do that, you can just add this to your
|
1
|
|
1
2 3 |
<span class="hljs-keyword">export</span> HISTCONTROL=ignoredups:erasedups
<span class="hljs-built_in">shopt</span> <span class="hljs-operator">-s</span> histappend <span class="hljs-keyword">export</span> PROMPT_COMMAND=<span class="hljs-string">"<span class="hljs-variable">${PROMPT_COMMAND:+$PROMPT_COMMAND$'\n'}</span>history -a; history -c; history -r"</span> |
Now, let me explain how it works:
- The first statement will avoid duplicate entries in your command history.
- The second statement appends the history to your history file, instead of rewriting it.
-
The third statement will set the 1
<td> <div class="text codecolorer"> PROMPT_COMMAND </div> </td> </tr> </table> </div> <p> variable with the commands that will save the file, cleanup the in-memory command history stack, and reload the file from the disk again. The </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> PROMPT_COMMAND </div> </td> </tr> </table> </div> <p> variable contents are executed every time you execute a command in the shell session, or when you just press Enter, and then when the history stack is updated in the current session.</li> </ul> <h3 id="check-which-files-were-modified-by-a-command"> Check which files were modified by a command </h3> <p> There is an overall safety rule when working with the command line: if you don’t know what it does, don’t run it. However, sometimes <em>you want</em> to execute particular things, even your code in your test environment, to “see what happens”. </p> <p> Well, take that mysterious command and run it like this: </p> <pre>1
<td> <div class="text codecolorer"> D=<span class="hljs-string">"<span class="hljs-variable">$(date "+%F %T.%N")</span>"</span>; sleep <span class="hljs-number">0.5</span>; <command>; find . -newermt <span class="hljs-string">"<span class="hljs-variable">$D</span>"</span> </div> </td> </tr><p> Where </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> <command> </div> </td> </tr> </table> </div> <p> is what you want to execute. </p> <p> This example takes the current time and saves its value to the variable </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> D </div> </td> </tr> </table> </div> <p> , then waits for a half second and executes the command. Finally, search for any new file created since that moment in the current directory. It’s not perfect, as another process might have modified the files during that period, but it will give you an excellent idea about which files you need to inspect and look for changes. </p> <p> There are a few notes about the </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> sleep </div> </td> </tr> </table> </div> <p> command. Besides the fact that the </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> date </div> </td> </tr> </table> </div> <p> command is taking the time in nanoseconds resolution, your machine might be fast, and some of the files might not be caught by </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> find </div> </td> </tr> </table> </div> <p> . It’s always better to inspect false-positive results than miss some real occurrences. This time, I’m assuming that you’re using GNU </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> sleep </div> </td> </tr> </table> </div> <p> , which currently supports resolutions under 1 second. Otherwise, just use </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> 1 </div> </td> </tr> </table> </div> <p> keeping in mind that by extending the time to be evaluated by find, you might see some additional false-positive results. </p> <p> Now, check this example: </p> <pre>1
2
3
<td> <div class="text codecolorer"> <span class="hljs-comment"># D="$(date "+%F %T.%N")"; sleep 0.5; ./mysterious ; find /etc -newermt "$D";</span><br /> Dont<span class="hljs-string">'t worry, I'</span>m good<br /> /etc/passwd </div> </td> </tr><p> The </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> mysterious </div> </td> </tr> </table> </div> <p> command, despite the fact it prints “Don’t worry, I’m good”, modified your </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> /etc/passwd </div> </td> </tr> </table> </div> <p> file and you just found it by executing it in this way. </p> <h2> Which Desktop Linux Distribution Is Recommended for Developers? </h2> <p> Among developers, usually back-end developers who need to set-up their Ruby or NodeJS or any other language working environment, there is often a big dilemma: which Linux distribution should be used? Which Linux distribution is the best? Which Linux distribution is the easiest to setup? Which Linux distribution will run the best in a virtual environment, or on old hardware? All these questions are hard to answer, and whole series of articles could be written on the topic. The basic answer “pick the one you’re most comfortable using” does not apply when developers don’t have time nor resources to test all the different Linux distributions. Not to mention that whichever Linux distribution one recommends, there will be always two more that disagree. But our best <a href="https://www.toptal.com/linux">Toptal Linux developers</a> came up with a short list for people that are looking to pick the best, recommended by the best. </p> <p> Before the list, we need to mention desktop environments. In desktop Linux distributions, the main differentiating factor beside setup complexity for new users are desktop environments, like <a href="https://www.gnome.org/" target="_blank" rel="noopener noreferrer">Gnome</a>, <a href="http://unity.ubuntu.com/" target="_blank" rel="noopener noreferrer">Unity</a>, <a href="http://cinnamon.linuxmint.com/" target="_blank" rel="noopener noreferrer">Cinnamon</a>, and <a href="https://www.kde.org/" target="_blank" rel="noopener noreferrer">KDE</a>. If you were thinking that recommending a Linux distribution is a subjective and controversial topic, discussing favorite Linux desktop environments is even harder, because it can easily attract a lot of flame wars and it’s hard to make a technical discussion about it. Nevertheless, desktop environments are often playing important roles in the final decision, and we had to mention them, but we won’t go into details. Take a look at included links to learn about their philosophy, see how each one looks like, and pick the one you like the most. </p> <p> Here is the unranked list: </p> <ul> <li> <strong>Ubuntu</strong> – Consensus is that the first and the most general pick is Ubuntu. It is the easiest, comes with support for most hardware right out of the box, and it is very friendly to people who are new to Linux. It’s only downside is that it’s pretty heavy, because of the all the stuff that comes with. It is also worth noting that their Unity GUI is causing a lot of controversies between hard core Linux users. </li> <li> <strong>Xubuntu</strong> – If you are looking for simplicity, Xubuntu is the way to go. OS is lightweight, and it supports and runs well on the old hardware, but keep in mind that some things don’t work out of the box, especially if you have a very new computer with UHD (Ultra High Definition) display. </li> <li> <strong>Kubuntu</strong>. Popular Ubuntu based distribution for people who like KDE desktop environment, which has its own philosophy different to other environments. </li> <li> <strong>elementaryOS</strong> Ubuntu based, heavily influenced by OSX and Macintosh design. If you want Linux that seems like Macintosh, this is the distribution to go for. It has its own desktop “shell”, with very minimalistic and light-weight apps for daily, common usage. </li> <li> <strong>Mint</strong> – A bit heavy, but not as much as Ubuntu, nice Cinnamon desktop environment for people who like more classical desktops (Cinnamon is based on Gnome 2.x). On the other hand, it has out-of-the-box multimedia support, and if something is missing, it’s Ubuntu and Debian compatible. </li> <li> <strong>Fedora</strong> A user-friendly distribution from RedHat. It is recommended if you like the Gnome 3 user interface, which is arguably the “strangest” (it is not bad, just unusual). Fedora is also a good pick for developers that run RedHat Enterprise Linux or Centos on their servers because they share the same core, and <em>yum</em> and <em>rpm</em> knowledge transfers from RedHat Enterprise Linux and Centos. Out of the box, Fedora Linux distribution was always recognized as one supporting developers well, with a lot of IDEs and development build tools available in the scenarios. </li> <li> <strong>Arch Linux</strong> Popular “rolling-release” distribution, with no base “release”, but always the latest stable versions of individual packages. Developers looking to always be on bleeding edge should look into this distribution. </li> </ul> <p> In the end, avoid <strong>Debian</strong> and <strong>RedHat</strong>, especially if you are a new Linux user. They are more server-side oriented. However, while not very friendly to developers, they are still used by many professionals. </p></div> <h2> Which Server Linux Distribution Is Recommended for Back-end Developers? </h2> <p> We covered desktop Linux distributions, but what about server distributions? Desktop Linux distributions are focused on the GUI, desktop environments, and simplicity in order to attract as many new users to the platform. On the other hand, server Linux distributions are focused primary on stability and security. GUI is not an important factor, because often they are running in the “headless mode” (a server that has no monitor, keyboard or mouse), and users (developers) connect to the servers remotely via the terminal. Another reason is that GUI elements take up precious memory, and every bit of free memory is very valuable. Stability and security we don’t need to explain, everyone wants their application safe and available. </p> <p> So, which server Linux distribution should you pick? </p> <ul> <li> <strong>Debian</strong> is considered the most stable server OS by hardcode system administrators for its very stable release cycle and sturdy, robust base system setup. The install image is relatively small and can be customized to very specific needs. The software base is huge, with 56864 software packages as of this writing. There’s a caveat, though. These packages are shared with desktop versions of Debian. Many other distributions, both client and server are based on debian’s .deb packages. </li> <li> <strong>Ubuntu Server</strong> is not bad either. It’s built completely on top of Debian and it’s 100% binary compatible with it, and Canonical (the company behind Ubuntu) is investing more to make Ubuntu reliable server software. There is arguably more help about it online, and it has more up-to-date packages which is a mixed blessing in a server environment, but its LTS (long-term-support) releases are very popular. Developers working on Ubuntu and Ubuntu-based desktop distributions tend to prefer it due to the same software package management system, apt. </li> <li> <strong>RedHat Enterprise Linux</strong> or RHEL for short, is the other large stable server distribution, backed by RedHat. It is a commercial distribution, with base software available for free but paid support licenses. RedHat has many internal software tools and it’s working with several of the biggest enterprise software vendors, like Oracle, to make RedHat a perfect home for enterprise systems. Additionally, it’s at the heart of OpenShift, the RedHat platform-as-a-service initiative. RedHat Linux is popular with enterprise developers, as the support licenses can get a bit expensive for smaller projects. The software package system is based on rpm packages and yum update manager. It rivals Debian and Ubuntu for stability, longevity and software support. </li> <li> <strong>CentOS</strong> is the “free” version of RHEL. It’s built almost entirely out of RHEL, stripped of Red Hat branding and based on the same package system and same packages. It’s popular among developers who prefer to work with RPM and possibly ones using Fedora as their desktop system of choice. </li> <li> <strong>Scientific Linux</strong> “<em>is a Linux release put together by Fermilab, CERN, and various other labs and universities around the world ready tuned for experimenters</em>”. It’s a distribution focused more on computing and it’s suited for such purposes, and it’s based on RedHat/CentOS. </li> <li> <strong>CoreOS</strong> is very popular as a lightweight OS to run software containers on. Unlike the other distributions listed here, CoreOS comes with no package manager: the developer is expected to provide all software dependencies as a part of a lightweight “container”, a self-contained package of software. </li> </ul> <h2> How To Remove Color Codes From The Output Of A Command? </h2> <p> Many command line tools on Linux produce colored outputs. For example, the following command on a typical Linux and Git setup will produce lines of output where each will begin with a colored word: </p> <pre>1
2
3
4
5
<td> <div class="text codecolorer"> $ git log --oneline --color<br /> e785d0f Fix submit panel error<br /> d83ff21 Implement remote belt<br /> c0dfae9 Make login fields required<br /> 8f69a2a Initial commit </div> </td> </tr><p> These colored outputs, when read by a human, are convenient, however, they make it difficult to produce complex commands involving pipes and command substitutions. When the following command is executed in a terminal within a Git repository, it will yield the very first commit hash for the repository: </p> <pre>1
2
<td> <div class="text codecolorer"> $ git log --oneline --color | tail -n 1 | awk '{print $1}'<br /> 8f69a2a </div> </td> </tr><p> Since the output is colored, the actual content of the output contains some hidden characters. The following command, when executed within a Git repository, will end with a somewhat cryptic error: </p> <pre>1
2
3
<td> <div class="text codecolorer"> $ git show --pretty="format:" --name-only $(git log --oneline --color | tail -n 1 | awk '{print $1}')<br /> bash: $'\E[33ma1d1e20\E[m': command not found<br /> solver/solver.go </div> </td> </tr><p> Solving this issue is simple, and requires a simple change to the command above. Pipe the colored output through </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> sed </div> </td> </tr> </table> </div> <p> , a Unix utility that parses and transforms text, to remove all <a href="http://tldp.org/HOWTO/Bash-Prompt-HOWTO/x329.html" target="_blank" rel="noopener noreferrer">invisible color codes</a>: </p> <pre>1
<td> <div class="text codecolorer"> sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" </div> </td> </tr><pre>1
2
3
4
5
6
7
8
9
10
<td> <div class="text codecolorer"> $ git show --pretty="format:" --name-only $(git log --oneline --color | \<br /> tail -n 1 | \<br /> awk '{print $1}' | \<br /> sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g")<br /> README.md<br /> app.yaml<br /> index.html<br /> script.js<br /> solver/solver.go<br /> style.css </div> </td> </tr><h2> How to Search and Replace a String in All the File Names? </h2> <p> As a developer, you need to rename a file or more files in a directory from time to time, and replace part of the file name with a particular string. This can be achieved with the following single line in the Linux shell: </p> <pre>1
<td> <div class="text codecolorer"> for file in *oldname*; do mv $file $(echo $file| sed -e "s/oldname/newname/g"); done </div> </td> </tr><p> In the snippet above, </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> oldname </div> </td> </tr> </table> </div> <p> is the search string we need to replace, and </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> newname </div> </td> </tr> </table> </div> <p> is the new replacement string. The </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> for </div> </td> </tr> </table> </div> <p> loop iterates over all the files in the current directory and fills the </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> file </div> </td> </tr> </table> </div> <p> variable with every instance of the found files. The </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> sed </div> </td> </tr> </table> </div> <p> command replaces the found matching string with the new replacement string, and passes this as the second parameter to the </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> mv </div> </td> </tr> </table> </div> <p> command. Note that this snippet will not do string replacement recursively, meaning it will not walk into subfolders. </p> <p> Here is an example that renames all files having string </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> oldname </div> </td> </tr> </table> </div> <p> in their filename and replaces them with a </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> newname </div> </td> </tr> </table> </div> <p> : </p> <pre>1
2
3
4
5
6
<td> <div class="text codecolorer"> $ touch oldname.txt oldname.c oldname-test.c test-oldname.c<br /> $ ls<br /> oldname.c oldname-test.c oldname.txt test-oldname.c<br /> $ for file in *oldname*; do mv $file $(echo $file| sed -e "s/oldname/newname/g"); done<br /> $ ls<br /> newname.c newname-test.c newname.txt test-newname.c </div> </td> </tr><p> If we want to do the same thing, but this time including the files in all the subfolders (nested or not), one way to do it in Bash (version 4+) would be like this: </p> <pre>1
2
<td> <div class="text codecolorer"> $ shopt -s globstar<br /> $ for file in ./**/*oldname*; do mv $file $(echo $file| sed -e "s/oldname/newname/g"); done </div> </td> </tr><p> Enabling </p> <div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;"> <table cellspacing="0" cellpadding="0"> <tr> <td class="line-numbers"> <div> 1<br /> </div> </td> <td> <div class="text codecolorer"> globstar </div> </td> </tr> </table> </div> <p> and using the double star notation, we ask the shell to walk recursively through the folder tree to find all the matching files. The rest of the snippet remains the same. </p> <p> Source: <a href="https://www.toptal.com/linux/tips-and-practices">Toptal</a> </p> <pre></pre> <p> </p>Related Articles:
- 2016/07/10 Get your local IP address like pro
- 2015/11/30 Disk usage by file type
- 2015/10/15 Get count of lines in scripts (shell)
- 2015/08/17 bash – expand shell variables ( bash bug resolved )
- 2015/07/09 evvsubst – substitute variables in text in shell
Authored By Amit Agarwal
Amit Agarwal, Linux and Photography are my hobbies.Creative Commons Attribution 4.0 International License.