Did my first heavy address mapping. Had very few streets. The got WMS plugin to get sat view. Drew as many streets as visible. Already had the information map from the sector. Job started. Have mapped 25% of the sector in about 3 hours. No clue why I “wasted” so much of my sunday evening but was enjoying every bit of it. Specially when mapnik has these days started rendering so quickly. Adjusted the spacing a few times for a few lines. Finally got the job done. Actually since the another 3 quaters still remain, will be spending another few days in completing them too.

Seems like OSM has more new things faster than you can get mapping your neighbourhood. Had mapped most of the streets around where I live and it was beconing either a power=tower or a POI flooding, and then came the JOSM with address tool. fan-tas-tic! Even better is the pic layer plugin. The only problem is sector maps are most often not to scale so overlay does not always help so much. Anyway, piclayer can atleast help in address mapping nevertheless.

Have spend this saturday evening in collecting such sector maps (6 pictures total using a 12mp cam handheld cam) from their enterances. The maps for new sectors are though drawn but are without any actuall streets. So those maps can’t be used at all. While other maps are merely just streets without plot numbers so they help in mere mapping the blocks in the sector. From all the maps I have seen till date, only one was digitally map (Nirvana Country). Every map seen is always hand oil-painted. The times when these boards get made are many years before the first house comes up on these sectors. So, they loose their accuracy after about 3 years. All HUDA sectors never change the map for more than a decade and are outdated by more than 10 percent. Have not located errors on them yet. Will take time for OSM users here to use the map and then later find problems.

The show was yesterday evening - Ice Age 3 - 3D at PVR Ambience Mall on NH8. I booked the tix on Wednesday for this show and got relaxed. Its always very comfortable to book tix on the net and reach the theatre seats directly from the parking lot.

So I reach the theatre 10n minutes before the showtime and hunt for that kiosk machine which swipes my card to print my booked tickets. there were six tickets and to be printed. I reported that I cannot print tickets here, please use the counter. WHAT! I just managed to walked pass that counter crowd and now its telling me to go back there! Ok ok.. not the first time. I go to the Gold members counter and ask for tix. They say, since its not Gol tix, get back in line of other counters! Dude! show starts in 4 mins, and I have to get in the line!!!? Ans :- I can’t help you in this. Nightmare is started already! I wait for the line to crawl and watch my clock tick every second! Damm! Firstly they are selling 3D tix at 250 each! is already a huge premium and now this queue!

By the time I reach the counter, Im already aware that most of the people are debating with the counter guy for why the had to stand in the queue though they did an advance booking. Ok, so im not the only one. So now I get the 6 tickets and rush for the entrance as its already T-1 minute left!

Oh boy! The counter is a No-Entry. The counter guy says the show will start after 15 mins. And there are like a hundred people standing outside just to enter PVR lobby. such a crowd was only for the spidey flicks I remember. But not like this! after another 10 mins they allow entry to the lobby and now, even a dozen of people are expressing anger to any one with a PVR batch! Same point. How come is this a online booking or a convenience charge that you tag along to your tickets when people are to wait standing for 10 minutes outside and another 15 mins in the small lobby! Showtime was 1805 while it was 1820 when entry to the theatre started. It takes another 10 mins before you reach your seats though.

Anyway, now everyone is seated and pain in a let-go mode just as we see the first 3D. Disaster! They even screwed up here! The whole movie has started and its still blurred with the polarised glasses! This is nuts! Its not like I enjoyed my last flick in this same 3D theatre but now I’m pretty sure I WILL NOT WATCH ANY OTHER 3D MOVIE AT PVR - EVER!. Half an hour is gone and eyes are painting. You can’t make sense without the polarised glasses and with then you can’t understand much. this was the peak of my chaos theory.
After 15 mins the projector staff probably realized a projection blunder so they do their first adjustments on a running movie. No change after they try aligning 3 times. At about 30mins on the movie the screen is blacked and the hall light glow. Everyone wondering how come its a intermission already! It wasn’t. The alignment exercise is tried again but off-screen and now, finally, I can see 3D. Its looks lot better and now its definitely 3D.

I’m a regular to PVR theatres. I always enjoy the movie, the cleanliness, the generous staff. When bookmyshow started and with PVR tickets, I use that site. But 2 months and PVR come out of that alliance . Their site those days was completely hopeless. Very complicated for me. I tried movies on other theatres then and finally when the latest version of their site came, I was back being a regular. They always charged a convenience charge to tickets.

So makes me I wonder. Probably the crowd was far too much for them to be prepared for -  a weekend 3D flick. Probably a primary staff member was MIA – these were answers given by a staff members to the angry crowd. Probably this hall has a history with chaos – no clue. Anyway, movie did go fine after that and there were no more troubles.

PVR is known to learn from their mistakes and therefore they should make sure they have better planning and handling in place.

There was once when we needed a LAN chat server. After hunting located Openfire which is an XMPP implementation. I did have the knowledge of it being an Open stardard but thats all i knew then. That was more then a year back. Now we have a dozen servers across the city and all running OSS software on a medium grade desktop machine hardware. We use the same server to chat with all the staff and archive messages.

After installing the second server, I knew, it needed a self health reporting system so we sitting in the moon can see if a problem is developing or even the reason for a problem. There are all sorts of problems that come on servers that are not in a protected space and without a stable internet or network connections. The first idea was simply to make a webpage and dump vital stats. But who the hell will go to a dozen IPs for checking if it is fine. Thats where I knew, my Openfire XMPP was the key for a fantastic new way of realtime monitoring and much more.

Just like any chat servers, it also supports making rooms. These rooms can be joined by anybody and can simply see the activity going within. Thats the key. I created a health-check room. I developed by entire stats gathering core and piped it using XMPPHP library into the targetted room. Then scheduled a script to run every 5 minute. Now the result is that my room can be joined by any person who wants to monitor the current stats. The stats gathering core has been scheduled in every installation with a gap of 30 seconds. Therefore, all servers send their reports in a gap of 30 seconds ina  continous stream within the room.

The above practise was used perfectly till I realized something more. Openfire has inbuilt support to archive messages. Therefore the stats was not parsed with details instead left as is within its Mysql DB. The same DB later was later also the point where remote comman output stream was sent to. Today we have 3 rooms for realtime stats, health stats and one exclusive realtime room for test cases, runtime errors in PHP/Mysql server. This is far better than RSS fees since it refreshes itself automatically in a stream and sourced from the servers itself. Far easier to use. An RSS feed was also made just for my mobile which parsed data from XMPP archive DB.

My favorite XMPP client is PSI. Its sleek, java-less (which means loads faster with less RAM) and with better UI. XMPPHP is a great library to access and conduct all operations using the standard. It happens that its docs don’t have everything it is capable of. I can share some of my code on request, specially the groupchat code which was not easy to locate.

This party includes only me. Accidentally went over Calcutta OSM map today and realized that it is totally missing! Where are the mappers? Really, some cities get totally lost like nobody in that town is a G33K! Anyway, just some standard streets were visible authored mostly by non-locals. So took this as the next mission. Myself not a local from Kolkata, West Bengal, IN. But having lived in Southern Part of that city for 7 years, I think my memory can supply enough POIs.

Me using Potlatch for many days now. Anyway, since i don’t have any trails, thats the only way I can map that town sitting a thousand kilometres away. Just mapped my first home beside the Golf Course to start with - The Golf Greens. Then Jodhpur Park where I lived for the second part. Noticed that Google Earth on Ubuntu won’t load this area. Strange, I was at 50 km altitude but still getting hopeless tiles. Anyway, since potlatch runs on Yahoo Ariel Maps, I was kinda lucky. Finally concluded todays session with Gariahat Road and Gol Park. The tile links may not show much if your calender shows June 13. It’ll take a day for renderers to finish painting these tiles. Try changing to T@H and  you might just see it faster.

The ride through the virtual world activated lots of memories of my school days. Have not POIed any of my homes or college or school or that cinema hall where I watched my first Hollywood flick - The Jurasic Park. Ah, those days. Will keep mapping for next few days till as long I get find it interesting. central areas like Esplanade or Park Street would most definately be mapped I think. Also noticed that India’s First real underground metro railway is totally missing! Shame! I would be able to help there cause I only know the first six stations. So its done for tonight.

The experiment as of now is a failure just so you would want to before you read this. I here share how I did do what I did to try to succeed.

I have been running a Windows 2003 Server  on a real machine past few months. Since its on the internet, virus attacks attacks are far too common. Eventually got very irritated and had to start on this experiment. Read docs of VirtualBox and locate on trick on creating a VKDK image from a disk itself. So physically connected my win2k3 disk on a existing ubuntu 9 with latest VirtualBox. Collected some disk clonning utilities include an ISO of ultimate boot CD.

Fired the commands and made a VMDK. Then, created a new Virtual Machine and made a new virtual disk on 100GB. Also hooked my VMDK on the same machine. started the machine using XXCopy Cloner and asked it to clone the real disk on a virtual disk. That took a some half hour and got done.

Then unplugged the vmdk and iso from the machine, restarted it and tried to boot. That is the problem. I have set the disk type to SATA and allowed also options on Virtualbox machine config. The best I can do is get a OS selector menu (pressing F8) and also getting a Windows 2003 Server starting logo screen. It never goes past this point after all efforts. Even  tried safe mode or VGA mode boot but all in vain.

I doubt there is any VirtualMachine detector to halt the system. I have repaird the system with a Windows 2003 ISO too. Done fixboot and fixmbr on the disk. But have not been able to go beyond this.

Sharing this blog so someone can go past this point and share on the fix.

Tata Sky has recently launched its Plus service which allows recording and replaying live TV. That is very nice sounding feature and I was almost going to upgrade my regular subscription connection to a Plus. What stopped me was that there is no information of when will HDTV be available in India. Since HDTV will anyway require a new set-top box so therefore, I hesited to upgrade to Plus and waste the money on this midway box. Im also aware that HDTV will least take least 5 years for indian release. My perspective.

Now the technology of Plus is to record TV as save in a proprietory format on its Hard drive. All that kit is part of the new set-top box. Hard disk will always have a space contrain. Therefore, to get maximum length of recording, the quality has to be low. Regular TV runs on 320×240 with 25fps. Transmissions are always analogue therefore, you won’t ever notice rough endges. Now since, Tata Sky wanted to offer Plus service with good recording duration, it certainly would reduce its transmission quality or bandwidth. Transmission of Tata Sky uses some MPEG like encoding protocol since its being transmitted on a digital connection.

What I have noticed in past about 3 months is that I see small patches similar to those you notice when watching low quality MPEG or a you-tube movie on a large resolution. These patches definately won’t be visible on a Cable TV type analougue connection. These patches destroy least 25% of the original quality of any channel that wish to see. Since there is no talk around about this topic, I assume the reduction is negligible. Nevertheless, the reduction is TV quality is not what a TV subscriber asked to pay 300Rs. a month would accept. I will be digging more about this issue and would update this web log in next few days.

Feb 26: When it launched it was just as any other TV broadcast so I could not have been a DVD Quality TV even though they keep claiming so. Now, with their reduced quality its worse than a standard MPEG VCD quality.

The problem comes for various reasons. The one that attacted me was since I never added the subdirectories of the /app/tmp directory into my subversion repository. The name tmp suggested me that it would be recreate the entire required directory  structure. But I guessed it wrong. After hunting the solution and not able to find it, I simply realized that the default install of cake has content below the tmp. and, sooner I placed it back, cakephp home page was happy. The directories structure is as:

• app
  • tmp
    • cache
    • logs
    • sessions
    • tests

Simply make sure tmp directory structure remains the same. As suggested by the cakephp error, core.php won’t actually help in this particular case.

Just few minutes back I finished resizing an LVM partition. The history if you must read starts like this, I installed Fedora 9 using default settings (i was crazy that day!). Then I liked what I saw as this was my first personal Fedora install. Then fedora said to update, so i did. Then I said, I wanted to install VirtualBox for few other OSs. Getting virtualBox to run is another story. Anyway, after enjoying openSUSE for 2 days, I wanted to install it in real partition. Now was the question, where? I had no space!

Every site around said you can easily resize  and get space. My fedora was taking under 10G on a 100G partition. I installed GParted and tried. It reported /dev/sda3 as a unknown type and so won’t resize it. Damm! Then after digging I realize that Fedora installed LVM2 on my partition. I never know what was that!  Then some other site confidently said, Gparted was very good and even could handle LVM partitions. So I download ISO and burned to a bootable CD. It started and reported the same error. My target partition was still unknown! Now, I had given up of any easy way out.

So I start to learn the LVM funda. Redhat had docs and lots of links about resizing LVMs. Everyone said, LVM was made to be easy. It was made to handle changs without even reboot! Sounded great. Then I came to Mr. resize2fs. This guy said he was expert in rewsizing many partitions including ext3. He also said, he could resize on the fly which is called on-line resize. Any kernel past 2.6 can handle on-line resize. So I tried. And tried. and failed! resize2fs kept reporting that “/dev/sda3″ is in use therefore it should try on-line resize. Then the next line said, on-line resize from NNN to NNN not supported! I went crazy for those 15 mins hunting why did it fail! Anyway, everything love to fail. Finally, I reach a good blog which told the process in slow motion. I understood more about LVM and what do each of the steps to in a LVM resize job.

1. LVM is simple a seperate partition table system that linux internally supports. That is also why it allows chaning LVM without reboot.
2. LVM creates a main single partition in the disk with a filesystem flag as lvm or lvm2. These are no filesystems so they are only parsed by LVM itself. We know fat32, ntfs,ext,ext2,ext3 etc are actual filesystems while lvm is a wrapper file system.
3. LVM resides on a real disk partition and could contain multiple Volume Groups.
4. Volume Group itself could contain Volumes.
5. Volumes are the ones which are made up of a readable filesystem such as ext3.
6. LVM at boot time will scan for lvm partitions on the disk and later parse everything inside to create new devices such as /dev/VolGroup00/LogVol00. These will then be mounted just like any normal partition.
7. So now we know its not very easy to resize a basic disk parition which itself holds a Volume Group and which then holds multiple Volume which are real filesystems.

Now you know little bit about LVM itself. Everything about LVM is here. My explanation will have minor bugs but the LVM system. As I understood and explained here is more or less correct. How did I resize is now been shown in the following steps.

1. Boot using the Fedora 9 ISO cd what was used for installing, choose resume mode
2. Do not mount any filesystems when prompted by setup, and jump directory to the prompt.
3. start vlm
4. here, do a vgscan do verify that vlm can read all your volumes.
5. Now to work on volumes you need to activate that volume. This is NOT the same as mounting. Activation will call LVM core to parse lvm partitions and create a virtual devices. These devices themselves won’t be mounted but made available to any utility who needs them in an exclusive mode. To activate, call

   vgchange -a y my_volume_group

6. Now the volume device will be ready to resize2fs.
7. some explaination: resize2fs kept reporting last time when I ran it within the session of my target fedora that the disk was in use therefore it will do a on-line resize. Therefore, I knew I have to give him the disk when nobody will be using it. The hurdle is that the disk itself is a virtual lvm volume. Therefore I had to struggle more on understanding how to activate a volume without mounting any filesystem and that too in a rescue mode.
8. Now we run resize2fs –physicalvolumesize NNG /dev/VolGroup00/Logvol01
9. Amazingly it ran without any complain.
10. explanation: Resize is a two stage process. Its the same even in the LVM context. You can’t just go to and resize the partition. The data units will also have to placed within the new boundries before the resize is done. Otherwise, you will simple have many  bad files reported from your disk check utility. therefore In windows, filesystems are defragmented before being shrunk. Defragmentation moves all the chunks of the file to the start of the disk and the tail of the partition becomes clean. Now after resize, no data is lost. In linux ext2/3 context, as far as I know, linux keeps its file chunks spread all over the available space. This trick saves the requirements of frequent defragmentations requests from the disk. Since all chunks in a file are contigous a file itself gives good read/write performance. But files themselves are never contiguous  and have large chunks of free space when written on disk. This property allows the filesystem space on the tail of every file which may expand. At the last point when no further expansion is possible the chunk is broken and continued some place else. In percentage terms the fragmentation of ext2 file system is under 5% after a very long use compared to ntfs or fat file system.
11. So resize2fs moves all the chunks within the limits of the suggested boundry but unlike windows, does not defragment everything.
12. Once, resize2fs is done, call lvresize to change the volume itself. It will report that data might get lost. Since we alreadt called resize2fs, it has already done the shifting job to prevent a data loss.
13. Then you need to resize the entire volume group using vgresize.
14. In my case, it failed cause there was already a volume at the end of the group. Since it was swap space, I simply vlremove it and went on.
15. The next step is to resize the physical volume using pvmove.
16. Finally, to use fdisk and delete old parition, create a new one with smaller size and save changes. Make sure the system id is 8E for lvm flag.
    
17. I did a reboot and saw my freespace.
    

Phew!

MVC model is what I never tried my hands on. The ModileTrailExplorer is also based on the same framework but I did not work on the core. Ruby on Rails in entire on MVC and then again I did not work on that either. Me a PHP guy who creates his class libraries for the UI of the code. So, how can I experiment on MVC was the long standing question. Finally learned about CakePHP from many guys on OSSCamp and other places. And there start the story.

The installation is easy. But based on the configuration that I have understood about cakephp, I had to give it a seperate VirtualHost pon my apache. The reason also being that my trunk root is already using Joomla. Joomla means it comes with its own mod_rewrite rules. CakePHP also has its own set of rules. Therefore Im still confused of mixing the two. Currently the application Im working runs on a single domain. And therefore, Ill have to understand to configure both Joomla and CakePHP togerher on the same DocumtnRoot.

Then comes the fact to actually make a small form which saves datta to my table. CakePHP runs on its own database naming convention. That is a hurdle since my application is old and already has its own unique convention. So I had to learn about AppModel configurations which will allow me to give custom table names, primaryKeys and even foreignKeys. Actually, till now (my fourth day of cooking) I have spend least 60%  of my time configuring AppModels. Feels like to get cakephp on an existing application as well as on an existing database means loads of homework in AppModels. I believe once my required Models are done, my application coding can take its own pace. But for now, its very slow. api.cakephp.org is my reference storehouse for now. For a newbie, every 5 mins I have to hunt how-to-do-the-foo-bar thing in cakephp. This is not a point of discouraging, you need to learn as well as port. Both things as the same time will be very very slow. All depends on your experiance level and speed to learn.

Another point to share is that I had my exclusive class library in my generic PHP application. That class cannot and will not be ported to cakephp cause its become the most complicated engine. Importing an existing class library and include files was done using the the bootstrap.php. The config part is still not done in a neat way but somehow its running without making me to copy the entire class library into cake. I would prefer it that way so that we don’t start maintaining two codes of the same files.

more updates as time goes. the project is doing good and i like many things already. will list them soon.

I have been searching for a very good and usable routing engine on my OpenStreetMap. We have a few beta code but all were very young. Infact, I yet could not find a single which did routing on my New Delhi-India roads. And now finally, reaching the RSS of OSM today morning, I reached yournavigation.org and my search was over.

The primary reason I needed a routing engine was for Quality Testing my uploaded maps. I have more less made all the map of Gurgaon OSM and also made major contribution to South Delhi maps. Setting the correct roads and marking the correct one way turns will now be verified using this routing engine. Im so very excited to play with it. Since most OSM routing engines are using the same algorithm, an improvement suggestion on yournavigation will also cause improvement on all hand held implemantations (beta). \

I now enter the OpenStreetMap addiction Stage 3.

Updated - November 20, 2008

I just found some more routing engine implements. This time I found the engine of yournavigation. It is called gosmore. It runs on windows/winCE/GTK. We also have AOSM which is a J2ME routing engine (project). Im checking gosmore right now and will update the fun in a while.

Updated - November 22, 2008

The great news is that gosmore ran in a fantastic way on my Fedora 9. I first checkout from the repository and make’ed it. Rebuild the PAK file using the New Delhi OSM file and the routing was doing fine. The objective to locate bugs in maps was met in a fantastic way. The next thing is to experiment the CGI mode of gosmore which outputs a CSV style output of the route based on paramters. As told me in the OSM-talk list, YOURS is converting this CSV into KML and rendering with another tool. Its lovely to see getting an output using different tools who just do their job in a innocent way. That is what I call hacking. Twisting the software without disturbing it and changing the output. My mid term plan is to

1. buy a good server hardware, AMD QuadCore most likely
2. get a static IP from my ISP
3. plant a OSM mapnik INDIA server,
4. YOURS based INDIA routing service and
5. Buy a OpenMoko device
6. Run gosmore routing on it.

The plan is very expensive but the joy to do the job is all that counts. The encouraging factor is that most of the code is open therefore configuration is the only thing needed.