cool sed/grep magic to convert output to csv format

I generallly keep doing this a lot, so thought will share with you. Lets assume we are capturing free ouput every min/hour/or whatever. The output looks like this:

Time: Mon Jan 21 23:59:10 AEDT 2019
——————-

total        used        free      shared  buff/cache   available
Mem:          32014        8656        1735        1697       21621       21308
Swap: 51195 75 51120

then we can use some grep and sed to convert this to something like this:

Mon Jan 21 23:59:10 AEDT 2019,32014,8656,1735,1697,21621,21308

Continue reading

bash function for rpm whatprovides

Sometimes some simple one-liner function can save you a lot of time, like-

wps ()
{
    rpm -q --whatprovides $(which $1 )
}   # ----------  end of function wps  ----------

Directories with maximum number of files

Lot of times, I want to find the directories with maximum number of files and so I wrote this quick function to do exactly the same

function count_lines ()
{
    oldIFS=$IFS
    count=0
    IFS=$'\n'
    dir=${1:-.}
    cd $dir
    find . -type d |while read line
    do
        echo -n "$(find $line -type f |wc -l) $line"
        echo 
        printf "Directories :: %8d\r" $count >&2
        ((count++))
    done|sort -n
    IFS=$oldIFS
}   # ----------  end of function count_lines  ----------

mv command with progress

When moving large files/directories, I would like to see the progress.

Idea for this is to use rsync with progress and remove source files. But that option does not remove the empty directories left behind so find command to delete that.

So, here is function for that:

mv-progress () 
{ 
    rsync -ah --progress --remove-source-files "$1" "$2";
    find "$1" -empty -delete
}

Highest disk usage of directory in subdirectories

I find myself doing this lot of times so thought will share this with you all. Basically, once I want to clear out the directory, I first want to find out the sub-directory using the maximum disk space so I wrote a function for that and here it is:

disk_usage_dirs () 
{ 
    find . -maxdepth 1 -type d -not -name '.' | while read line; do
        du -s "$line";
    done | sort -n | tail -${1:-5}
}

Some other posts you might find useful on this :

Linux Best Practices and Tips

<p>
  Have you ever typed a command in your terminal, only to find out you forgot to prefix it with the
</p>

<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
  <table cellspacing="0" cellpadding="0">
    <tr>
      <td class="line-numbers">
        <div>
          1<br />
        </div>
      </td>
      
      <td>
        <div class="text codecolorer">
          sudo
        </div>
      </td>
    </tr>
  </table>
</div>

<p>
  command? You have to retype the whole command again just to add the
</p>

<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
  <table cellspacing="0" cellpadding="0">
    <tr>
      <td class="line-numbers">
        <div>
          1<br />
        </div>
      </td>
      
      <td>
        <div class="text codecolorer">
          sudo
        </div>
      </td>
    </tr>
  </table>
</div>

<p>
  in front of it. Frustrating!
</p>

<p>
  Well, you can add this simple alias to your
</p>

<div class="codecolorer-container text solarized-light" style="overflow:auto;white-space:nowrap;width:550px;">
  <table cellspacing="0" cellpadding="0">
    <tr>
      <td class="line-numbers">
        <div>
          1<br />
        </div>
      </td>
      
      <td>
        <div class="text codecolorer">
          .bashrc
        </div>
      </td>
    </tr>
  </table>
</div>

<p>
  to help you reduce the frustration:
</p>

<pre>

  <td>
    <div class="text codecolorer">
      alias argh='sudo $(history -p \!\!)'
    </div>
  </td>
</tr>

Trello – backup to your linux machine

Just in case, you are looking for backing up your trello account boards, you can use the following bash script to do so:

#!/bin/bash -
#===============================================================================
#
#          FILE: backup-trello.sh
#
#         USAGE: ./backup-trello.sh
#
#   DESCRIPTION:
#
#       OPTIONS: ---
#  REQUIREMENTS: ---
#          BUGS: ---
#         NOTES: ---
#        AUTHOR: Amit Agarwal (aka)
#  ORGANIZATION: Mobileum
# Last modified: Thu Dec 22, 2016  01:14PM
#       CREATED: 08/12/2016 09:41:08 AM IST
#      REVISION: $Revision: 1.0 $$
#===============================================================================

# Your backup directory
BDIR=/backup

# Your trello api token and key goes here :)
token=<>
key=<>


# IDs of the boards go here. This is easy to get, just go to your 
# board and check the last part of URL
BOARDS=( a b )


URL="https://trello.com/b/"
POST='&actions=all&actions_limit=1000&cards=all&lists=all&members=all&member_fields=all&checklists=all&fields=all'

for i in ${BOARDS[*]}
do
    /usr/bin/curl -H 'Accept-Encoding:gzip, deflate, br' -H 'Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' "$URL$i.json?&key=$key&token=$token$POST" > "$BDIR/$i.json.gz"
    # gzip -f "$BDIR/$i.json"
done