All posts by Ruturaj Vartak

MySQL

MySQL DB Pie Graph

1 Comment

Its same as what http://blog.olindata.com/2009/02/using-the-google-graph-api-with-mysql-stored-functions/ had posted, I’ve just updated a little. And there are no smart quotes here. So u can copy paste 😉

DELIMITER $$

DROP FUNCTION IF EXISTS `test`.`FNC_GOOGRAPH_DB_SIZE`$$
CREATE FUNCTION `test`.`FNC_GOOGRAPH_DB_SIZE` (
  p_chart_type CHAR,
  p_height INT,
  p_width INT) RETURNS varchar(3000) CHARSET latin1
  READS SQL DATA
BEGIN
  /* Author:    Walter Heck - OlinData */
  /* Date:      20090216 */
  /* Note:      After an idea by Alex Gorbachev - Pythian */
  /*            http://www.pythian.com/blogs/1490/google-charts-for-dba-tablespaces-allocation */

  /* variable declaration */
  DECLARE v_done BOOLEAN default false;
  DECLARE v_url varchar(3000);

  DECLARE v_schema_name varchar(3000);
  DECLARE v_data_length_sum int;
  DECLARE v_data_length_total int;
    
  DECLARE v_legend_labels varchar(3000);
  DECLARE v_chart_labels varchar(3000);
  DECLARE v_chart_data varchar(3000);

  /* Cursor declaration */
  DECLARE c_schema_sizes cursor for
      select
        t.table_schema,
        round(sum(t.data_length + t.index_length) / 1024 / 1024) as data_length_schema
      from
        information_schema.tables t
      group by
        t.table_schema
      order by
        t.table_schema;

  /* Handler declaration */
  DECLARE CONTINUE HANDLER FOR NOT FOUND SET v_done = true;

  /* Initialize the variables */
  SET v_legend_labels = '';
  SET v_chart_labels = '';
  SET v_chart_data = '';

  /* Get the total data length + index_length for all tables */
  select
      round(sum(t.data_length + t.index_length) / 1024 / 1024) as data_length_total
  into
      v_data_length_total
  from
      information_schema.tables t;
    
  /* Open the cursor */
  OPEN c_schema_sizes;

  /* Loop through the cursor */
  get_data: LOOP

      /* Fetch the next row of data into our variables */
      FETCH c_schema_sizes INTO v_schema_name, v_data_length_sum;

      /* if there is no more data, v_done will be true */
      IF v_done THEN
        /* Exit the loop */
        LEAVE get_data;
      END IF;

      /* Add the schema name to the labels for the legend */
      IF v_legend_labels = '' THEN
        SET v_legend_labels = v_schema_name;
      ELSE
        SET v_legend_labels = concat(v_legend_labels, '|', v_schema_name);
      END IF;

      /* Add the total size of the schema to the labels */
      IF v_chart_labels = '' THEN
        SET v_chart_labels = v_data_length_sum;
      ELSE
        SET v_chart_labels = concat(v_chart_labels, '|', v_data_length_sum);
      END IF;

      /* Get the percentage of the total size as the graph's data */
      IF v_chart_data = '' THEN
        SET v_chart_data = ROUND(v_data_length_sum / v_data_length_total, 2) * 100;
      ELSE
        SET v_chart_data = concat(v_chart_data, ',', ROUND(v_data_length_sum / v_data_length_total, 2) * 100);
      END IF;

  END LOOP get_data;

  /* Close the cursor */
  CLOSE c_schema_sizes;
    
  /* Build up the google graph url */
  SET v_url = 'http://chart.apis.google.com/chart?';
  SET v_url = CONCAT(v_url, 'cht=', p_chart_type);
  SET v_url = CONCAT(v_url, '&chs=', p_width , 'x', p_height);
  SET v_url = CONCAT(v_url, '&chtt=Database Sizes (MB)');
  SET v_url = CONCAT(v_url, '&chl=', v_chart_labels);
  SET v_url = CONCAT(v_url, '&chd=t:', v_chart_data);
  SET v_url = CONCAT(v_url, '&chdl=', v_legend_labels);

  /* return the url as the function's result */
  RETURN v_url;
END$$

DELIMITER ;

Web

Scribe PHP logging

1 Comment

I’d put some efforts to make scribed logging work with PHP, what I did was follow python’s example script “scribe_cat”. And made a similar PHP Script out of it, I’d to create many PHP scripts out of n number of .thrift files. Anyways I’ve got a working example. Here it is.

<?php
/*
 * As found on http://highscalability.com/product-scribe-facebooks-scalable-logging-system
        $messages = array();
        $entry = new LogEntry;
        $entry->category = "buckettest";
        $entry->message = "something very interesting happened";
        $messages []= $entry;
        $result = $conn->Log($messages);
*/

$GLOBALS['THRIFT_ROOT'] = './includes';

include_once $GLOBALS['THRIFT_ROOT'] . '/scribe.php';
include_once $GLOBALS['THRIFT_ROOT'] . '/transport/TSocket.php';
include_once $GLOBALS['THRIFT_ROOT'] . '/transport/TFramedTransport.php';
include_once $GLOBALS['THRIFT_ROOT'] . '/protocol/TBinaryProtocol.php';
//include_once '/usr/local/src/releases/scribe-2.0/src/gen-php/scribe.php';

$msg1['category'] = 'keyword';
$msg1['message'] = "This is some message for the category\n";
$msg2['category'] = 'keyword';
$msg2['message'] = "Some other message for the category\n";
$entry1 = new LogEntry($msg1);
$entry2 = new LogEntry($msg2);
$messages = array($entry1, $entry2);

$socket = new TSocket('localhost', 1464, true);
$transport = new TFramedTransport($socket);
$protocol = new TBinaryProtocol($transport, false, false);
$scribe_client = new scribeClient($protocol, $protocol);

$transport->open();
$scribe_client->Log($messages);
$transport->close();

You can have as many messages or entries into one log, as I’ve demonstrated or tried above, please change the corresponding scribed’s host and port values. I’ve attached a working file and all the required includes generated by scribe. Except for the above script everything is generated by Scribe/Thrift.

Web, MySQL

PHP, Python Consistent Hashing

2 Comments

I found out the hashing algorithm used in PHP-Memcache is different from that of Python-Memcache. The keys went to different servers as the hash created by python and php were different.

I posted a question on the memcache groups and was lucky to find this wonderful reply.

import memcache
import binascii
m = memcache.Client(['192.168.28.7:11211', '192.168.28.8:11211
', '192.168.28.9:11211'])

def php_hash(key):
    return (binascii.crc32(key) >> 16) & 0x7fff

for i in range(30):
       key = 'key' + str(i)
       a = m.get((php_hash(key), key))
       print i, a

This is the only thing that has to be done on Python’s end, change the way the hash is calculated. The coding on PHP end remains same. All you guys using PHP for web based front-end with MySQL and Python for back-end scripts shall find this helpful.

Thanks Brian Rue.

Reference: http://groups.google.com/group/memcached/msg/7bb75a026c44ec43

Biking, Trek

Dodital Trek and Uttaranchal: Day 12 & 13 (19, 20 Jun 08)

1 Comment
This entry is part 13 of 13 in the series Himalayan Trip: Dodital Trek and Uttaranchal

Day 12, 19 Jun 08

to Haridwar

5.30 am, Its 5.30 and I cant resist my bowel movement. I need to shit… aagggh!!! my stomach needs an overhaul. By 6.00 Vj’s alarm is ringing, but we decide to cut it off, my head is aching and I don’t feel very healthy, my head is aching a bit, we decide to go back to sleep. Finally around 7.30 we wake up, but it is dark outside, I wonder if its a morning or an evening. Its raining, heavy clouds over the Hrishikesh. We prepare for our ‘check out’. At 9.00am we are at the ashram office, Vj finalizing the details, its raining. I hope it does not rain on the Ram jhulla where we would have no cover to run for. 15 minutes later, ‘Parmarth Niketan’ settlement is over and we move for the Vikram stand on the other side of the river. Things weren’t lucky for us on this trip, and my fear comes true, we were just at the halfway of the bridge and the clouds break lose. The heavy drizzle turns into a torrential downpour. We are left with no option but to run with the big bags till we find ourselves with other fellow tourists under a shop shelter.

 

We wait well over 30 minutes and the rain has gone from bad to worse. Few more minutes and luckily its back to the ‘heavy drizzle’ and we move on. By 10.15am we get a Vikram for Haridwar which is approximately 28kms from Hrishikesh, just a hour plus journey. Around 10.45 and we are in the Motichur forest zone close to Rajaji National park and the rain is now insane. It continues well till the borders of Haridwar. As if heavy bags and wet clothes were not enough, a huge traffic jam at one of the national highway junction is added to our misery. It takes about quarter of a hour for that to clear and we reach ‘Har ki pauri’ by noon.

Haridwar

Noon, We were lucky enough to find a cycle-rickshaw to port us to our local stay at the Gujarati Ashram. The poor fellow waded uphill amongst bustling streets full with cattle, people, cars, pulling around 170kgs of payload just for Rs. 20. At the ashram, luck seemed to be on ourside, when we got a decent room (for a trekking nomad) with an attached toilet/bathroom. We dry ourselves and rest. I’m for sure not feeling normal. Seems like I have temperature. Mallik calls, with bad news, his train is cancelled and now has to fly back to Mumbai. We move out to find food, same old ‘bhojanalayas’ with big fat tandoori rotis and spicy curries, we didn’t want those but weren’t spared. At food, we wondered if we should stick to original plan of the train or book the flight tickets, finally we too decided on shortcut route, flight. We tried to find a cybercafe on the main road, but couldn’t some of the vendors didn’t even understand what we were searching for. Haridwar in this context is quite backward to Hrishikesh, which due to its more foreign influx has somehow kept itself updated. Luckily we find a cafe just close to our lodging. We were lucky to find some tickets vacant for the Mumbai bound flights and book it.

It was almost 4.00pm when we reached back to our room, with nothing to do, we decided to rest. We had planned a evening aarti at Har ki pauri had it been our original plan, but now after booking for a 10.00pm bus to Delhi from Haridwar, that idea was out. My head was still aching and some body was tired, we decide to sleep. 7.00pm, I can’t sleep anymore, perhaps I’m uncomfortable for a sound sleep. In the balcony its cool breezy evening with beautiful colors at the horizon of Haridwar. We click our final snaps of our tour from the balcony of the Gujarati ashram. Go get back on the streets around 7.30pm to have tea and cancel our rail tickets. Vj gets a shave from a local saloon and feels better. By 8.00pm we were packing our bags for the final time, it was more than 12 days back when we had packed our bags for the next day flight for the very first time, wondering how we were gonna make the trek with the 10kg + bags. The chapter was finally on its last few pages.

9.00pm, After checking out of the dharamshala, we were at the travel agent’s shop waiting for a cycle rickshaw to pick us for the bus stand. We wait for almost 30 minutes, getting restless that the travel agent is just not willing to arrange for the cycle rickshaw. Finally he arranges one by 9.30 and we were on our way to the bus stand, in pitch black darkness somewhere around the corner there was a pack of buses, where the fellow asks us to get down, the cycle-rickshaw fellow checks with other fellow who gets a sheet of paper and probably ticked at our bus seat numbers. The bus is full of Gujaratis, Vj jokes he has no respite from them. It feels like being in a bus from Rajkot to Junagarh. Around 10.30 the engine whines and the tyres roll taking us into a new day for the last journey in our trip, back to Mumbai!

Day 13, 20 Jun 08

The seats of the bus are not ‘ergonomic’ a word we really didn’t care at the begining of the trip, but with the last moments with a tired body it made a lot of difference. My neck was aching, the bus stood still, I woke up, the bus was waiting at a restaurant for the tourists to break for food/washroom. It was 2.30am in the morning. After a few minutes its rolling again. I make a head rest out of my wind cheater and luckily have some sound sleep till 4.30, when the conductor of the bus calls for ‘Last Stop, ISBT !’.

That meant we were in Delhi, the bus was circling around some big area lighted with tall lamp posts, somewhere I saw ‘ISBT’. We were circling around the Delhi bus stop, ISBT. Few minutes later the bus stopped and our bags were yet again on our backs, for the last time. There were many auto-rickshaws offering a ride to the airport, with ranges of Rs. 150 – 300. We decided to check at the bus stop. Luck couldn’t favour us more, in front of us we saw slick bus with electronic signboard ‘IGI Airport’

5.00am, The bus moves and we move through the Delhi’s posh and wide roads, circling many government buildings, the Red Fort till we reached the Airport by 5.45. We have some biscuits to soothe our burning stomachs and wait. With us there was group of foreign models probably from Eastern Europe, opposed to us they were very well dressed, with their make ups and energetic. We at the other end were tired, dirty from night long bus ride. I decided to brush my teeth and take a dump at the Airport. Felt a little better, but my body ached, fever for sure, we waited restlessly for our call of ‘Indigo’ flight for Mumbai.

Around 8.30, we checked in our luggage and waited for boarding. By 9.15 we were in the fuselage. As I took my seat my memories returned when all 6 of us were in a similar plane all enthusiastic for the 2 week trip. Around 10.15 the plane took off. We didn’t have much to chat and discuss, both of us tired. As I put my head on the head rest, every single moment of the Dodital trek raced through, Mussoorie, Dehradun, Hrishikesh and all. By 12.15 the plane landed and it took us almost 1.30 before we ventured on the roads in the rickshaw, Vj’s friend came to meet at the airport and I hugged him thanks and all that for the trip and we to split into our own directions. The streets of Mumbai felt familiar but somewhat strange, from the lovely hillocks of the north, crisp clear skies, cool winds and snowy caps, I was back into Cement jungles and tar roads, nevertheless felt very happy to be back Home!!!

Trek

Dodital Trek and Uttaranchal: Day 11 (18 Jun 08)

Leave a comment
This entry is part 12 of 13 in the series Himalayan Trip: Dodital Trek and Uttaranchal

Day 11, 18 Jun 08

Hrishikesh

7.00 am, In a ‘sleepy’ mode, we are bid goodbye by Mallik. He is still interested in another small trek to Neelkanth. It would be around 2 hrs of a simple trek, but none of us are interested, or rather should I say have the energy. Mallik is still enthusiastic to head back into the woods. I wake almost an hour and half later, a crisp morning in Hrishkesh feels splendid. I wake up Vj and ask him if he is interested in bath at the ghats. Rajo too agrees when I knock at his room. We wait for an hour till Amey and Nikhil get ready to be with us.

The Hrishikesh Ganga is cold as well !! But extremely refreshing. It felt like diving into a river of chilled Soda water, the rich silt brushing, with fizzling cool water running over the skin. Bathing with soap is not allowed so after 15 minutes or later (the sane period we thought would be good enough in the water without troubling our health !) we head back to our rooms for ‘proper baths’.

 

10.00 am, After a clean bath in warmer water we are hungry and wonder about our day’s breakfast. Vj has some Maggi packets left and we decide to prepare it. In a cool religious place, in an ashram you don’t realize how time flies by swiftly. It was almost noon when we finished our ‘breakfast’ and wait for Mallik to come back. Today is our rest day. So nothing more than rest and we are back on our beds.

2.00 pm, Yet again in our sleeps, its Mallik who is waking us up. This time he is coming back from the trek ! We all get back on our bums and listen to Mallik’s experiences about the small trek to Neelkanth. hmm.. this day is good, relaxing, chatting, easing out, one of our first days after the long trek and travelling. TPing.. its almost 4 when we realize we still have to have our lunch. Back on the streets move into the local food-joint trying to search for some appeasing menu item. We all were sick of the paneers, daals and big tandoori rotis. But that was not to leave us yet. After alu sabzi, daal and roti, we roamed in the adjoining street full of travel agents, middle-men, cyber cafes, etc.

Mallik check’s that his train has yet again flip-floped to ‘running’ mode. He has to leave today. Rajo gets home sick and wants to return to Sangli instead of Mumbai. Amey too doesn’t find the trip anymore interesting and wants to leave early. So as tickets get cancelled and booked, me and Vj wonder if sticking to the original plan is the right idea. However we stick to it. Amey and Rajo too are to leave today evening with Mallik and Nikhil. Mallik and Nikhil have got to leave by 7.00 to reach Haridwar for the next days early morning train to Delhi @ 6.00 am. Amey and Rajo book a bus ticket around 9.00 with the cyber cafe / travel agent for Delhi. All of them unknowingly having a flight from Delhi the next day.

6.00 pm, Mallik, me and Vj come back to the ashram. Mallik has to start packing, while others head off for marketing. By 7 Mallik has completed his packing and wondering why hasn’t Nikhil yet come back, who is to leave with him. We three venture out on the banks for the famous Ganga aarti. Over there the scene is captivating, just behind the big Shiva idol, is the west where the sun is setting behind a small hill, its golden rays piercing through the scattered clouds in the blue sky. A golden hue merging into the dark clouds. On the left the temple skyline is lit with a lovely pink/golden tone as the background.

Sunset at hrishikesh
Sunset at hrishikesh		 Hrishikesh temple skyline at sunset
Hrishikesh temple skyline at sunset

After capturing the aarti, Mallik decides its time to leave and yet Nikhil hasn’t come back, sticking to military-like plan he leaves us with a hug, the long journey with Mallik on this trip had finally come to an end. We watch him with his backpack till we lose him in the crowds. Around 8pm the trio appears. Nikhil is in a hurry, obviously he has lost the company of resourceful Mallik, and now has to manage on his own. He decides to leave with Rajo and Amey for the Delhi bus and would get down at Haridwar station, unsure, but now his only option. By 8.30pm we are just two of us as even Delhi bus trio leaves. We decide to go back to the Ganga ghat to enjoy the last moments in Hrishikesh’s banks.

9.00 pm, We come back and decide that we cant have more of tandoori rotis, etc. Bread and butter would satisfy us for dinner. I remember I still have my ‘cup noodles’, so thats on the platter as well. We share a laugh the mess reminding us of ‘Mukti’s cousin’. Its almost 10pm and we pack our big bags for yet another journey to Haridwar for the next morning. Its almost after 10 when finally the lights go out for the last time in Hrishikesh.

Trek

Kalavantindurg Trek

1 Comment

Kalavantin Durg, the small steep hill adjoining to Prabalgad was our motto of the day, Dec 6, 2008.

With Mallik, his cousin, Raksha, Vj, Rajo, Janki and Neeraj adventured the steep, kindof scary trek.

Kalavantindurg
View the Gallery

Travelling Directions

  • Get to Panvel, Bus or Train (Central Harbour)
  • From Panvel ST Bus Depot, take a bus for Thakurwadi, and get @ the last stop, the base village

Approximate time of Trek: 4-5 hrs (including the return to the base village)
Difficulty: easy to medium, if you are not scared of heights.