There was once when we needed a LAN chat server. After hunting located Openfire which is an XMPP implementation. I did have the knowledge of it being an Open stardard but thats all i knew then. That was more then a year back. Now we have a dozen servers across the city and all running OSS software on a medium grade desktop machine hardware. We use the same server to chat with all the staff and archive messages.

After installing the second server, I knew, it needed a self health reporting system so we sitting in the moon can see if a problem is developing or even the reason for a problem. There are all sorts of problems that come on servers that are not in a protected space and without a stable internet or network connections. The first idea was simply to make a webpage and dump vital stats. But who the hell will go to a dozen IPs for checking if it is fine. Thats where I knew, my Openfire XMPP was the key for a fantastic new way of realtime monitoring and much more.

Just like any chat servers, it also supports making rooms. These rooms can be joined by anybody and can simply see the activity going within. Thats the key. I created a health-check room. I developed by entire stats gathering core and piped it using XMPPHP library into the targetted room. Then scheduled a script to run every 5 minute. Now the result is that my room can be joined by any person who wants to monitor the current stats. The stats gathering core has been scheduled in every installation with a gap of 30 seconds. Therefore, all servers send their reports in a gap of 30 seconds ina  continous stream within the room.

The above practise was used perfectly till I realized something more. Openfire has inbuilt support to archive messages. Therefore the stats was not parsed with details instead left as is within its Mysql DB. The same DB later was later also the point where remote comman output stream was sent to. Today we have 3 rooms for realtime stats, health stats and one exclusive realtime room for test cases, runtime errors in PHP/Mysql server. This is far better than RSS fees since it refreshes itself automatically in a stream and sourced from the servers itself. Far easier to use. An RSS feed was also made just for my mobile which parsed data from XMPP archive DB.

My favorite XMPP client is PSI. Its sleek, java-less (which means loads faster with less RAM) and with better UI. XMPPHP is a great library to access and conduct all operations using the standard. It happens that its docs don’t have everything it is capable of. I can share some of my code on request, specially the groupchat code which was not easy to locate.

This party includes only me. Accidentally went over Calcutta OSM map today and realized that it is totally missing! Where are the mappers? Really, some cities get totally lost like nobody in that town is a G33K! Anyway, just some standard streets were visible authored mostly by non-locals. So took this as the next mission. Myself not a local from Kolkata, West Bengal, IN. But having lived in Southern Part of that city for 7 years, I think my memory can supply enough POIs.

Me using Potlatch for many days now. Anyway, since i don’t have any trails, thats the only way I can map that town sitting a thousand kilometres away. Just mapped my first home beside the Golf Course to start with - The Golf Greens. Then Jodhpur Park where I lived for the second part. Noticed that Google Earth on Ubuntu won’t load this area. Strange, I was at 50 km altitude but still getting hopeless tiles. Anyway, since potlatch runs on Yahoo Ariel Maps, I was kinda lucky. Finally concluded todays session with Gariahat Road and Gol Park. The tile links may not show much if your calender shows June 13. It’ll take a day for renderers to finish painting these tiles. Try changing to T@H and  you might just see it faster.

The ride through the virtual world activated lots of memories of my school days. Have not POIed any of my homes or college or school or that cinema hall where I watched my first Hollywood flick - The Jurasic Park. Ah, those days. Will keep mapping for next few days till as long I get find it interesting. central areas like Esplanade or Park Street would most definately be mapped I think. Also noticed that India’s First real underground metro railway is totally missing! Shame! I would be able to help there cause I only know the first six stations. So its done for tonight.

The experiment as of now is a failure just so you would want to before you read this. I here share how I did do what I did to try to succeed.

I have been running a Windows 2003 Server  on a real machine past few months. Since its on the internet, virus attacks attacks are far too common. Eventually got very irritated and had to start on this experiment. Read docs of VirtualBox and locate on trick on creating a VKDK image from a disk itself. So physically connected my win2k3 disk on a existing ubuntu 9 with latest VirtualBox. Collected some disk clonning utilities include an ISO of ultimate boot CD.

Fired the commands and made a VMDK. Then, created a new Virtual Machine and made a new virtual disk on 100GB. Also hooked my VMDK on the same machine. started the machine using XXCopy Cloner and asked it to clone the real disk on a virtual disk. That took a some half hour and got done.

Then unplugged the vmdk and iso from the machine, restarted it and tried to boot. That is the problem. I have set the disk type to SATA and allowed also options on Virtualbox machine config. The best I can do is get a OS selector menu (pressing F8) and also getting a Windows 2003 Server starting logo screen. It never goes past this point after all efforts. Even  tried safe mode or VGA mode boot but all in vain.

I doubt there is any VirtualMachine detector to halt the system. I have repaird the system with a Windows 2003 ISO too. Done fixboot and fixmbr on the disk. But have not been able to go beyond this.

Sharing this blog so someone can go past this point and share on the fix.

Tata Sky has recently launched its Plus service which allows recording and replaying live TV. That is very nice sounding feature and I was almost going to upgrade my regular subscription connection to a Plus. What stopped me was that there is no information of when will HDTV be available in India. Since HDTV will anyway require a new set-top box so therefore, I hesited to upgrade to Plus and waste the money on this midway box. Im also aware that HDTV will least take least 5 years for indian release. My perspective.

Now the technology of Plus is to record TV as save in a proprietory format on its Hard drive. All that kit is part of the new set-top box. Hard disk will always have a space contrain. Therefore, to get maximum length of recording, the quality has to be low. Regular TV runs on 320×240 with 25fps. Transmissions are always analogue therefore, you won’t ever notice rough endges. Now since, Tata Sky wanted to offer Plus service with good recording duration, it certainly would reduce its transmission quality or bandwidth. Transmission of Tata Sky uses some MPEG like encoding protocol since its being transmitted on a digital connection.

What I have noticed in past about 3 months is that I see small patches similar to those you notice when watching low quality MPEG or a you-tube movie on a large resolution. These patches definately won’t be visible on a Cable TV type analougue connection. These patches destroy least 25% of the original quality of any channel that wish to see. Since there is no talk around about this topic, I assume the reduction is negligible. Nevertheless, the reduction is TV quality is not what a TV subscriber asked to pay 300Rs. a month would accept. I will be digging more about this issue and would update this web log in next few days.

Feb 26: When it launched it was just as any other TV broadcast so I could not have been a DVD Quality TV even though they keep claiming so. Now, with their reduced quality its worse than a standard MPEG VCD quality.

The problem comes for various reasons. The one that attacted me was since I never added the subdirectories of the /app/tmp directory into my subversion repository. The name tmp suggested me that it would be recreate the entire required directory  structure. But I guessed it wrong. After hunting the solution and not able to find it, I simply realized that the default install of cake has content below the tmp. and, sooner I placed it back, cakephp home page was happy. The directories structure is as:

• app
  • tmp
    • cache
    • logs
    • sessions
    • tests

Simply make sure tmp directory structure remains the same. As suggested by the cakephp error, core.php won’t actually help in this particular case.

Just few minutes back I finished resizing an LVM partition. The history if you must read starts like this, I installed Fedora 9 using default settings (i was crazy that day!). Then I liked what I saw as this was my first personal Fedora install. Then fedora said to update, so i did. Then I said, I wanted to install VirtualBox for few other OSs. Getting virtualBox to run is another story. Anyway, after enjoying openSUSE for 2 days, I wanted to install it in real partition. Now was the question, where? I had no space!

Every site around said you can easily resize  and get space. My fedora was taking under 10G on a 100G partition. I installed GParted and tried. It reported /dev/sda3 as a unknown type and so won’t resize it. Damm! Then after digging I realize that Fedora installed LVM2 on my partition. I never know what was that!  Then some other site confidently said, Gparted was very good and even could handle LVM partitions. So I download ISO and burned to a bootable CD. It started and reported the same error. My target partition was still unknown! Now, I had given up of any easy way out.

So I start to learn the LVM funda. Redhat had docs and lots of links about resizing LVMs. Everyone said, LVM was made to be easy. It was made to handle changs without even reboot! Sounded great. Then I came to Mr. resize2fs. This guy said he was expert in rewsizing many partitions including ext3. He also said, he could resize on the fly which is called on-line resize. Any kernel past 2.6 can handle on-line resize. So I tried. And tried. and failed! resize2fs kept reporting that “/dev/sda3″ is in use therefore it should try on-line resize. Then the next line said, on-line resize from NNN to NNN not supported! I went crazy for those 15 mins hunting why did it fail! Anyway, everything love to fail. Finally, I reach a good blog which told the process in slow motion. I understood more about LVM and what do each of the steps to in a LVM resize job.

1. LVM is simple a seperate partition table system that linux internally supports. That is also why it allows chaning LVM without reboot.
2. LVM creates a main single partition in the disk with a filesystem flag as lvm or lvm2. These are no filesystems so they are only parsed by LVM itself. We know fat32, ntfs,ext,ext2,ext3 etc are actual filesystems while lvm is a wrapper file system.
3. LVM resides on a real disk partition and could contain multiple Volume Groups.
4. Volume Group itself could contain Volumes.
5. Volumes are the ones which are made up of a readable filesystem such as ext3.
6. LVM at boot time will scan for lvm partitions on the disk and later parse everything inside to create new devices such as /dev/VolGroup00/LogVol00. These will then be mounted just like any normal partition.
7. So now we know its not very easy to resize a basic disk parition which itself holds a Volume Group and which then holds multiple Volume which are real filesystems.

Now you know little bit about LVM itself. Everything about LVM is here. My explanation will have minor bugs but the LVM system. As I understood and explained here is more or less correct. How did I resize is now been shown in the following steps.

1. Boot using the Fedora 9 ISO cd what was used for installing, choose resume mode
2. Do not mount any filesystems when prompted by setup, and jump directory to the prompt.
3. start vlm
4. here, do a vgscan do verify that vlm can read all your volumes.
5. Now to work on volumes you need to activate that volume. This is NOT the same as mounting. Activation will call LVM core to parse lvm partitions and create a virtual devices. These devices themselves won’t be mounted but made available to any utility who needs them in an exclusive mode. To activate, call

   vgchange -a y my_volume_group

6. Now the volume device will be ready to resize2fs.
7. some explaination: resize2fs kept reporting last time when I ran it within the session of my target fedora that the disk was in use therefore it will do a on-line resize. Therefore, I knew I have to give him the disk when nobody will be using it. The hurdle is that the disk itself is a virtual lvm volume. Therefore I had to struggle more on understanding how to activate a volume without mounting any filesystem and that too in a rescue mode.
8. Now we run resize2fs –physicalvolumesize NNG /dev/VolGroup00/Logvol01
9. Amazingly it ran without any complain.
10. explanation: Resize is a two stage process. Its the same even in the LVM context. You can’t just go to and resize the partition. The data units will also have to placed within the new boundries before the resize is done. Otherwise, you will simple have many  bad files reported from your disk check utility. therefore In windows, filesystems are defragmented before being shrunk. Defragmentation moves all the chunks of the file to the start of the disk and the tail of the partition becomes clean. Now after resize, no data is lost. In linux ext2/3 context, as far as I know, linux keeps its file chunks spread all over the available space. This trick saves the requirements of frequent defragmentations requests from the disk. Since all chunks in a file are contigous a file itself gives good read/write performance. But files themselves are never contiguous  and have large chunks of free space when written on disk. This property allows the filesystem space on the tail of every file which may expand. At the last point when no further expansion is possible the chunk is broken and continued some place else. In percentage terms the fragmentation of ext2 file system is under 5% after a very long use compared to ntfs or fat file system.
11. So resize2fs moves all the chunks within the limits of the suggested boundry but unlike windows, does not defragment everything.
12. Once, resize2fs is done, call lvresize to change the volume itself. It will report that data might get lost. Since we alreadt called resize2fs, it has already done the shifting job to prevent a data loss.
13. Then you need to resize the entire volume group using vgresize.
14. In my case, it failed cause there was already a volume at the end of the group. Since it was swap space, I simply vlremove it and went on.
15. The next step is to resize the physical volume using pvmove.
16. Finally, to use fdisk and delete old parition, create a new one with smaller size and save changes. Make sure the system id is 8E for lvm flag.
    
17. I did a reboot and saw my freespace.
    

Phew!

MVC model is what I never tried my hands on. The ModileTrailExplorer is also based on the same framework but I did not work on the core. Ruby on Rails in entire on MVC and then again I did not work on that either. Me a PHP guy who creates his class libraries for the UI of the code. So, how can I experiment on MVC was the long standing question. Finally learned about CakePHP from many guys on OSSCamp and other places. And there start the story.

The installation is easy. But based on the configuration that I have understood about cakephp, I had to give it a seperate VirtualHost pon my apache. The reason also being that my trunk root is already using Joomla. Joomla means it comes with its own mod_rewrite rules. CakePHP also has its own set of rules. Therefore Im still confused of mixing the two. Currently the application Im working runs on a single domain. And therefore, Ill have to understand to configure both Joomla and CakePHP togerher on the same DocumtnRoot.

Then comes the fact to actually make a small form which saves datta to my table. CakePHP runs on its own database naming convention. That is a hurdle since my application is old and already has its own unique convention. So I had to learn about AppModel configurations which will allow me to give custom table names, primaryKeys and even foreignKeys. Actually, till now (my fourth day of cooking) I have spend least 60%  of my time configuring AppModels. Feels like to get cakephp on an existing application as well as on an existing database means loads of homework in AppModels. I believe once my required Models are done, my application coding can take its own pace. But for now, its very slow. api.cakephp.org is my reference storehouse for now. For a newbie, every 5 mins I have to hunt how-to-do-the-foo-bar thing in cakephp. This is not a point of discouraging, you need to learn as well as port. Both things as the same time will be very very slow. All depends on your experiance level and speed to learn.

Another point to share is that I had my exclusive class library in my generic PHP application. That class cannot and will not be ported to cakephp cause its become the most complicated engine. Importing an existing class library and include files was done using the the bootstrap.php. The config part is still not done in a neat way but somehow its running without making me to copy the entire class library into cake. I would prefer it that way so that we don’t start maintaining two codes of the same files.

more updates as time goes. the project is doing good and i like many things already. will list them soon.

I have been searching for a very good and usable routing engine on my OpenStreetMap. We have a few beta code but all were very young. Infact, I yet could not find a single which did routing on my New Delhi-India roads. And now finally, reaching the RSS of OSM today morning, I reached yournavigation.org and my search was over.

The primary reason I needed a routing engine was for Quality Testing my uploaded maps. I have more less made all the map of Gurgaon OSM and also made major contribution to South Delhi maps. Setting the correct roads and marking the correct one way turns will now be verified using this routing engine. Im so very excited to play with it. Since most OSM routing engines are using the same algorithm, an improvement suggestion on yournavigation will also cause improvement on all hand held implemantations (beta). \

I now enter the OpenStreetMap addiction Stage 3.

Updated - November 20, 2008

I just found some more routing engine implements. This time I found the engine of yournavigation. It is called gosmore. It runs on windows/winCE/GTK. We also have AOSM which is a J2ME routing engine (project). Im checking gosmore right now and will update the fun in a while.

Updated - November 22, 2008

The great news is that gosmore ran in a fantastic way on my Fedora 9. I first checkout from the repository and make’ed it. Rebuild the PAK file using the New Delhi OSM file and the routing was doing fine. The objective to locate bugs in maps was met in a fantastic way. The next thing is to experiment the CGI mode of gosmore which outputs a CSV style output of the route based on paramters. As told me in the OSM-talk list, YOURS is converting this CSV into KML and rendering with another tool. Its lovely to see getting an output using different tools who just do their job in a innocent way. That is what I call hacking. Twisting the software without disturbing it and changing the output. My mid term plan is to

1. buy a good server hardware, AMD QuadCore most likely
2. get a static IP from my ISP
3. plant a OSM mapnik INDIA server,
4. YOURS based INDIA routing service and
5. Buy a OpenMoko device
6. Run gosmore routing on it.

The plan is very expensive but the joy to do the job is all that counts. The encouraging factor is that most of the code is open therefore configuration is the only thing needed.

Amazing morning it was. My bro calls at 630am to connect to Doordarshan 1, our National Television channel for a live broadcast of India’s first moon mission Chandrayaan I from the Sriharikota, INDIA . After USA , Russia, Japan, China and now its the turn of India! We are reaching there with our own big bang. The technology used for the mission is completely indigenous. I have always known the PSLV lauch vehicle used in the project. It was first launched in 1993 with a payload. Later it also started servicing other countries with their payloads. The Chandrayaan project asked much more perfection and as was delivered.  We will all be excited and watching the next 15 days of our journey to the moon. Planned for a month orbit aound the moon. Equipped with hitech infra, X-Ray and topography sensors to map the entire of the moon (i have no clue of whatever that is :p).  Also to be noted, PSLV also carries 6 payloads from NASA and ESA and this time for free. And even better, a Moon Impact Probe will carry another set of sensors along with our National Flag. We will become the fourth country after USA, Russia and Japan to place a flag on the Moon. Check out the the links and keep reading on. Congratulations India!

November 18, 2008: Chandrayan 1 had sent the Moon Impact Probe a few days back and now is busy orbiting the scanning the moon terrian 24 hours of the earth day. As per the today’s morning news, the second unmanned moon mission, Chandrayaan 2 has been approved by the Government just yesterday. So we now know, the fun will grow lot more.! There is very little official ISRO matter about the next moon rover mission at this point.

The trick that is working great for me particularly is mentioned here. I have a SonyErricson K790i with J2ME. So I installed mgmaps on it. Then came creating an offline cache so that it does not need to download each and every time as I move my GPS along the undocumented roads. To create cache I select zoom 0-17, use Openstreetmap (Maplink) renderer. After cache map is created using the creator tool, I download all the OSM tiles using the gMapMaker which can read and create full offline cache directory structure for mGMaps. Thats the homew0rk part of it.

Now as I go along GPS Tracing using my Holux m241, I keep checking ways on the already OSM tile on mgmaps. As long as I can’t find a way on screen, I going the right way!.  The target is to hunt unmapped way therefore have the marked ways with you and do the turns which are missing.

Later in the evening, I load the GPX file itself and then download the OSM data for that zoom level. Now I have partly marked ways partly just GPX lines. Which I convert to real ways and name them as much I can. The other problem of Gurgaon, HY, India is that we have not named our streets, therefore most often, the names are either guess work of just human readable rather than in sync with the policatal/official data.

The modes of transportis either a bycycle(almost like this model) or my classic Kinetic Honda (which is no longer manufactured) and m241 set of 1sec log autolog mode.