Here\’s something that I had been planning to do for sometime now. I have my blog on wordpress and have Statpress plugin installed. The plugin does give me a good idea of the visits to my site. Now, I have to keep logging in every now and then to check the stats. Also since I did not want to spend some more amount 🙂 so I do not have the shell access to the server, too bad 🙁
But I want to monitor the stats of my site. So I was looking for options and to see if I could get it from my desktop and mail that to myself :). This would be pretty neat to do.
Problem: I cannot access the page without logging in.
Ah, problems is what makes Linux worth using. I know for quite sometime now that curl can be used with cookies and to submit forms and can really do a lot of things but had never tried it. So I spent the night searching for some docs and examples of curl and finally managed to do what I had to do 🙂
1) From the shell script get the web-page of the statpress
2) mail the web page.
3) Put it in cron.
How to do it:
First use curl with cookies and post data to get to the page. This might take some time depending on your requirement. But here\’s what I needed:
curl -v -b ~/cookies.txt -c ~/cookies.txt -d \’username=raj&password=raj\’ \”http://blog.amit-agarwal.co.in/admin.php?page=statpresscn/statpresscn.php\” > /tmp/ak.html
Here, the -b and -c, options sets the cookies that will be used intially and the update cookie file. and then -d is used to submit the form with the parameters that you require: username and password. For the security of my website, I have changed the data above, but you can get the parameter name by runnig the curl command once on your site. You will see the login page with the name and id of the boxes for username and password.
And then finally we have the URL to fetch with the redirection to desired file.
So to put it all together. Here\’s the script to put in the /etc/cron.hourly directory:
curl -b ~/cookies.txt -c ~/cookies.txt -d \’username=raj&password=raj\’ \”http://blog.amit-agarwal.co.in/admin.php?page=statpresscn/statpresscn.php\” > /tmp/ak.html
cat /tmp/ak.html |mailx -s \”Stats for my site at `date`\” email@example.com
BTW, after you have got the page for the first time and you have the cookie, you don\’t need to use the username and password, if your server supports cookies. Just remove the -c and -d part and run curl to get your page.
Hope you find it useful. I found that the same approach can be used to do a lot of other stuff also. Let me see how that would go.
Update (Jan 29) :Just a couple of days after posting this, I found that Linux Joural posted a video on using curl to loging to website\’s :). Its definately with more details but Video, check it out here.